Caley got off to a dream start in the seventh minute when striker Adam Rooney had his flick deflected past Fraser Forster by Hoops’ defender Charlie Mulgrew, before Kris Commons levelled less than two minutes later.
It looked like the visitors would reassert themselves to win their game in hand over the Light Blues and leapfrog them at the top of the table.
However, in a remarkable second-half, goals from Grant Munro and Shane Sutherland – before Commons scored an injury-time penalty – gave the Highlanders an unlikely victory to keep Rangers one point clear of Celtic with three games remaining.
Celtic boss Neil Lennon, who revealed that midfielder Beram Kayal came off in the second-half with suspected concussion and a suspected broken wrist, pointed the finger mostly at his defence.
“We are better than that,” said Lennon. “We have been very good defensively all season but the back four were really poor the second-half.
“I might have to change the back four after that performance, I will assess it on Thursday and Friday.
“Some players picked today of all player to have their poorest game of the season. The two full backs were poor. The two centre halves were poor and the two centre forwards were eachy-peachy (both as bad as each other).
“You can’t have that many off form and get a result and we didn’t need to win the game. If you play like that you don’t deserve to win any game”.
Lennon continued: “There are no excuses. We gave them all the warnings and told them what might happen.”
Lennon admitted the pendulum has swung towards Ibrox but is hoping one of Rangers’ opponents can do Celtic a favour in the remaining three games.
INVERNESS CT: Esson, Tokely, Munro, Innes, Gillet, Duff, Cox, Ross, Sutherland (Odhiambo 82), Rooney (Morrison 90), Doran.
CELTIC: Forster, Izaguirre, Majstorovic, Wilson, Mulgrew, Brown, Kayal (Ki 71), Commons, Maloney, Stokes (Murphy 58), Hooper (McCourt 66).
Referee: Stevie O’Reilly.