Alex McLeish’s men had been cruising at 3-1 up thanks to two first half goals from Gregory Vignal and a third soon after the restart from Dado Prso.
When Kevin McBride pulled one back from the penalty spot it looked like being only a consolation effort for the home side but, when Barry Ferguson was sent off and Martyn Corrigan made it 3-2, the stage was set for a nerve-jangling finale.
Rangers hung on to take the win they needed to climb one point above Celtic, who lost 2-0 at home to Hearts on Saturday, at the summit of the Premier League.
: Marshall, Paul Quinn, Craigan, Partridge, Hammell, McBride, Leitch, Corrigan, Paterson (Foran 59), Clarkson (Britton 81), Scott McDonald.
: Waterreus, Ricksen, Kyrgiakos (Khizanishvili 71), Malcolm, Ball, Namouchi (Arveladze 66), Ferguson, Alex Rae, Vignal (Lovenkrands 61), Prso, Novo.
: D McDonald (Scotland).