Shelbourne had an early chance when winger Kavanagh slotted through a perfect ball to striker Philip Gorman, who rounded keeper Darren Quigley but then shot into the side-netting.
John Mulroy went close for Bray when he outpaced Andy Boyle but then sent his delicate chip shot just over the bar.
Debutant Pat Seery should have had the Seagulls in front on 21 minutes when, unmarked, he headed inches wide from eight yards after an Adam Hanlon corner.
Shelbourne opened their account on 28 minutes when winger David Cassidy neatly lobbed advancing keeper Quigley, after a free-kick by Lorcan Fitzgerald.
Kavanagh went close to stretching Shels’ lead only for midfielder Dean Zambra to get back and clear his lines, with Quigley beaten.
Bray were level on 36 minutes when Marty Waters took advantage of a slip by Brian Shortall to fire into the net from 16 yards.
Shelbourne had their noses in front again on 45 minutes when Philip Hughes’ low cross was squeezed home by Philip Gorman.
Waters moved from hero to villain when being shown the red card by Dublin referee Neil Doyle in first-half added time after a foul on Shortall and then raising an arm to push Mathews to the ground.
After Gorman had a shot rebound off the crossbar early in the second half, 10-man Bray equalised on 48 minutes. A Kevin Knight cross was missed by keeper Bennion, under pressure from Jason Byrne, and Mulroy fired home from close range.
Byrne should have had Bray ahead on 61 minutes when a cross by Adam Hanlon landed at this feet, but he shot straight at Bennion from five yards.
And Shelbourne’s numerical advantage eventually told on 77 minutes when a Mathews cross was punched out by Quigley straight to Kavanagh, who made no mistake from eight yards.
BRAY WANDERERS: Quigley; S. O’Connor, Seery, Massey, Knight (Doyle 68); Hanlon (Kelty 85), K. O’Connor, Zambra, Mulroy, Waters; J. Byrne (Dixon 75).
SHELBOURNE: Bennion; Mathews, Boyle, Shortall (Paisley 64), Fitzgerald; Cassidy, Dawson, S. Hurley 6 (McGill 74), Kavanagh; Hughes, Gorman (P. Byrne 86).
Referee: Neil Doyle (Dublin).