Murray nodded in a free-kick from Darren Ambrose, the scorer of the opening goal, as Palace overturned the odds against a much-changed United team at Old Trafford.
Ambrose had ended shot-shy Palace’s long goal drought with a stunning goal after 65 minutes but United forced extra time with a Federico Macheda penalty.
Murray may have been close to straying offside as he got forward for the winner but Palace deserved their success for their endeavour throughout a tight contest.
United boss Alex Ferguson fumed afterwards: “My apologies are for the fans because that was not a Man United performance.”
United may have made 10 changes but were hardly lacking in senior experience with the likes of Dimitar Berbatov, Park Ji-sung, Jonny Evans and Antonio Valencia starting.
Full-back Fabio da Silva was the only survivor from the weekend draw against Newcastle but Chris Smalling returned for his first start since October after overcoming a foot injury.
Palace, backed by a sizeable following in a below-capacity 52,624 crowd, went into the game with confidence despite failing to score in their previous five games.
Manager Dougie Freedman made five changes himself but his team looked the brighter at the onset. Wilfried Zaha blasted a couple of chances over and had a penalty appeal turned down, before Kagisho Dikgacoi dragged a shot wide.
United were forced to make a change when Fabio limped off after 37 minutes and 19-year-old Ezekiel Fryers replaced him at left-back. The hosts had their best chance of the first half when Valencia crossed from the right and Mame Biram Diouf volleyed over with an overhead kick.
Palace also had injury problems as Sean Scannell was forced off for Jonathan Parr and Dean Moxey was receiving treatment when the first half ended.
Moxey was unable to continue and was replaced by Ambrose after the break while United boss Sir Alex Ferguson gave Ravel Morrison a chance in place of Berbatov. United almost took the lead soon after the restart as Valencia robbed Anthony Gardner and hit a low shot past Lewis Price but it rolled narrowly wide.
Just as it seemed United were taking charge Palace took the lead with a stunning goal against the run of play. There seemed little danger as Ambrose received the ball from Zaha but he launched a vicious shot from 35 yards which curled into the top corner.
It was a brilliant way to end a goal famine which had lasted 550 minutes but the London side’s lead was wiped out just four minutes later — Macheda netting from the spot after his shirt was grabbed by Patrick McCarthy.
Palace refused to let their heads drop and had the better chances as normal time ran out. Murray hit the side-netting in the last minute and Zaha shot straight at Ben Amos.
Gibson went close early in extra time when a shot from the right flew agonisingly wide after taking a slight deflection.
Yet Palace proved again that they were made of stern stuff as they reclaimed the lead in the 98th minute. Ambrose clipped in a free-kick from the left and Murray shook off Evans to head home from close range.
Gibson led the United response with another long-range effort which narrowly missed and Macheda poked a chance wide. Ambrose almost hit a second screamer when he drilled a 40-yard free-kick at goal but Amos made a fine save. The second period of extra time was a much cagier affair with Palace defending deeply but they survived a couple of heart-stopping moments to prevail.
MAN UTD: Amos, Rafael Da Silva (Pogba 64), Smalling, Evans, Fabio Da Silva (Fryers 37), Valencia, Gibson, Park, Diouf, Berbatov (Morrison 46), Macheda.
CRYSTAL PALACE: Price, Clyne, McCarthy, Gardner, Moxey (Ambrose 46), Dikgacoi, O’Keefe, Wright, Zaha, Scannell (Parr 45), Easter (Murray 74).
Referee: Chris Foy (Merseyside).
* Carling Cup semi-final draw (2 legs, ties to be played between January 9 and January 23): Manchester City v Liverpool; Crystal Palace v Cardiff