Lee Naylor opened the scoring for the Hoops before Kenny Miller levelled for Rangers. Marc Antoine Fortune restored the advantage on the stroke of half-time with what proved to be the winner before Lee McCulloch saw red in the last minute.
Naylor said: “Neil’s put me straight back in and he knows what I can do. I hope he gets the job because he’s done a terrific job.”
Lennon made five changes for his first Old Firm derby as interim boss.
Mark Wilson appeared to be preferred to Josh Thompson in the heart of defence after Glenn Loovens failed to recover from injury.
Robbie Keane, Landry NGuemo, Lee Naylor and Diomansy Kamara were also drafted into the side.
The Scottish champions had a more familiar look after the weekend with Allan McGregor back in goal and Miller, McCulloch, Steven Whittaker, Kyle Lafferty and Steven Davis also starting.
But Rangers were dealt a blow early on when McGregor was injured.
Unfortunately for replacement keeper Neil Alexander, his first task was to pick the ball out of the net when Celtic took the lead three minutes later.
Naylor stepped up for the free-kick after a Steven Naismith foul on Aiden McGeady and his low shot was helped into the bottom corner off David Weir.
Rangers could have hit back but Miller’s cross from the left was stabbed wide by Lafferty.
Rangers pulled a goal back three minutes from the break when Miller threw himself in front of a Whittaker cross and nodded past Artur Boruc.
Celtic were back in front on the stroke of half-time when Fortune met Naylor’s cross and Alexander allowed the header to slip through his hands.
Rangers could have been level again when Miller cut inside from the left but he dragged his shot wide with Whittaker just a yard too slow to connect and prod home.
Then, with 53 minutes gone, Celtic were also forced to change goalkeepers. Boruc had picked up a hand injury late in the first half which appeared to be causing a problem and he was replaced by Lukasz Zaluska.
Rangers were more dangerous now and Miller was perfectly placed to rifle home at the back post but a wonderful block from Wilson prevented Naismith’s pass reaching Miller.
Zaluska then came to the rescue again with a great save to deny Davis, before Miller’s shot crashed off the post. And the match ended in controversy when McCulloch was dismissed in the final minute for a second yellow for a foul on McGeady.
CELTIC: Boruc (Zaluska 53), Hinkel, Wilson, O’Dea, Naylor (Braafheid 78), McGeady, Brown, N’Guemo, Kamara, Keane, Fortune (Samaras 74).
RANGERS: McGregor (Alexander 5), Whittaker, Weir, McCulloch, Smith, Davis, Edu, Thomson, Naismith, Lafferty (Novo 77), Miller.
Referee: C Murray (Scotland).