Murphy pounced three minutes after the restart to reflect the Barrowsiders’ dominance and despite John Murphy missing a penalty, Smith’s late effort ensured victory.
Daniel St Ledger put Carlow in front for the first time on 27 minutes and they led 0-4 to 0-2 at the break.
Carlow put the game beyond London when an unmarked Ed Finnegan found Brian Murphy for an easy finish. Shane Redmond then won a penalty but John Murphy’s spot kick was straight at Declan Traynor, whose clearing kick clocked referee Richard Moloney on the back of the head.
Smith grabbed a second goal three minutes from time.
Scorers for Carlow: Brian Murphy 1-2 (1f), JJ Smith 1-0, P Reid 0-2, D Hayden, D St Ledger 0-1f each.
Scorers for London: M Gottsche, L Colfer, T Gaughan, E O’Neill (f) 0-1 each.
CARLOW: T O’Reilly; BJ Molloy, P McElligott, P Murphy; B Kavanagh, S Redmond, T Bolger; Brendan Murphy, J Murphy; Brian Murphy, C Murphy, E Finnegan; JJ Smith, S Gannon, D Hayden.
Subs: D St Ledger for Hayden (23), P Reid for C Murphy (ht), E Ruth for Brian Murphy (49), C Walshe for Finnegan (65), C Mullins for Smith (67).
LONDON: D Traynor; S Bradshaw, S Browne, D McGreevy; S Hannon, S Mulligan, J Scanlon; M Gottsche, A Raftery; C McCallion, S Doolan, S McVeigh; L Colfer, M Mullins, L Mulvey.
Subs: T Gaughan for Bradshaw (ht), E O’Neill for Colfer (ht), C Hands for Browne (51), E Mageean for McCallion (61), S Kelly for Doolan (63).
Referee: R Moloney (Limerick).