Despite a gutsy show, the Scots looked like being denied that elusive first win but Leinster were rocked by a late Steve Jones drop goal to settle a rugged contest.
Leinster took just two minutes to break the deadlock when Borders strayed offside and Felipe Contepomi slotted the penalty.
However, fears of a rout were quickly eased as Calum MacRae launched a thrilling counter-attack from which the Reivers eventually took the lead.
The full back sidestepped two tackles before Scott Gray’s grubber kick was brilliantly picked up by Craig Hamilton, who plunged over near the posts.
MacRae added the conversion before increasing the home side’s lead with an easy penalty.
Borders belied their basement position as they continued to trouble the Irish province but the visitors reduced the leeway when Ross Beattie’s challenge on Guy Easterby was ruled illegal and Contepomi again bisected the posts.
The Reivers hit back with thrilling breaks from Kelly Brown and Chris Cusiter but they were denied a second try when Opeta Palepoi spilled a pass at the crucial moment.
Instead, they had to be content with a second MacRae penalty before Leinster turned the screw ahead of the break.
First, Stephen Keogh was driven over for a try, converted by Contepomi, before the Argentinian’s injury-time penalty gave the visitors an undeserved half-time lead.
Contepomi appeared to have put Leinster in control against a tiring Reivers outfit with his fourth penalty soon after the restart.
However, the Borders refused to buckle under the mounting pressure and instead launched another counter-attack to put themselves firmly in contention for an upset.
MacRae reduced the deficit following another infringement by Leinster before Cusiter sprinted 40 metres to take his side within metres of a second try.
But superb Leinster tackling kept them out and instead it took another MacRae penalty to level the scores.
Then, in a dramatic finale, Jones dribbled over the decisive drop goal before Borders withstood heavy injury-time pressure.
Borders scorers: Tries: Hamilton. Cons: MacRae. Pens: MacRae 4. Drop Goals: Jones.
Leinster: Tries: Keogh. Cons: Contepomi. Pens: Contepomi 4.
BORDERS: MacRae, Robertson, MacDougall, Law, De Luca, Jones, C. Cusiter, Kerr, Ford, Kalman, Palepoi, Hamilton, Beattie, Brown, Gray.
Replacements: Scott for Ford (74), Douglas for Kalman (58), Dalziel for Beattie (40), Miller for Gray (75).
LEINSTER: Kearney, Fitzgerald, O’Driscoll, D’Arcy, Hickie, Contepomi, Easterby, Corrigan, Blamey, Green, Hogan, O’Kelly, Jowitt, Heaslip, Gleeson.
Replacements: Lewis for Hickie (48), Willis for Easterby (21), Vermaas for Corrigan (58), Finnigan for Hogan (65), Keogh for Jowitt (26).
Ref: Neil Ballard (WRFU).