Leading 3-2 from the first leg of their play-off clash in Glasgow, the visitors fell behind to a goal from skipper Markus Rosenberg who was banned for the first leg.
However, just before the interval Parkhead midfielder Nir Bitton had the ball in the net from a corner only for Serbian referee Milorad Mazic to award a foul to the home side.
It appeared a harsh decision and there was more woe in the 54th minute when Malmo grabbed their second goal with a header from substitute Felipe Carvalho which Hoops defender Dedryck Boyata could only divert over the line.
Celtic manager Ronny Deila had passed up on two chances to qualify for European football’s elite club competition last season and yet another failure is set to cost the club upwards of £15million.
“We looked scared, uncomfortable and stressed,” Deila said last night. “And that is my responsibility. I have to go through what happened and learn from this, that is so important.
“We need to get better. We were close but when we have a last performance like this we don’t deserve to get there. It is the whole team. It is big games for Celtic - everyone wants to get through, to play well, but we didn;t play as a team and looked very uncomfortable.
“You learn through experience and this experience is very hard. The opportunity was there for us tonight but int heend we didn’t deserve it. We are very disappointed.
“We felt that we had a big chance. Had the performance been fantastic and you lose then we could deal with it but today we had a disappointing performance and that makes it even tougher.”
That will be partially offset by the Europa League into which the Scottish champions will fall, although the Norwegian’s reputation will take a dent.
It did not need to be this way.
Former Celtic loan player Jo Inge Berget had made the return game in Sweden more fraught than it ought to have been with the second of his two goals scored in the last seconds at Celtic Park.
And with no goals conceded in their last nine qualifying home games, and having overturned a 2-0 first-leg deficit against Salzburg in the previous qualifier to win 3-2 on aggregate, confidence was high in the Malmo ranks.
Celtic right-back Mikael Lustig failed to recover from a hamstring injury which meant 19-year-old Saidy Janko, signed from Manchester United in the summer, was handed his first start in Europe, while Charlie Mulgrew replaced the dropped Emilio Izaguirre at left-back.
The match started and continued at breakneck pace.
In the 20th minute Celtic defender Virgil van Dijk, linked with a £10million move to Southampton, was booked after kicking the ball away on his way back from a Hoops attack.
Three minutes later there was much more for the Dutchman and the visitors to worry about when Rosenberg leapt the highest to knock in Yotun’s corner with his shoulder, with Celtic Craig Gordon unable to get to the ball.
Griffiths then picked up a yellow card along with Vladimir Rodic, after Anton Tinnerholm went to the ground claiming foul play with his knee by the Hoops striker, backed up by television replays.
Moments later came the match’s most controversial moment. Malmo defender Kari Arnason appeared to handle Stuart Armstrong’s corner before Bitton turned and hooked the ball into the net – only for referee Mazic to signal an infringement in the home side’s favour. “That was terrible,” Deila said last night.
Seconds before the interval Gordon made a great save from Nikola Djurdjic’s drive after the Celtic defence was opened up and the Scottish champions went in at the break both relieved and frustrated.
Kris Commons replaced Armstrong for the start of the second half while Carvalho came on for Rasmus Bengtsson.
Only a minute had elapsed before Rosenberg fired wide from 14 yards, Celtic defender Boyata then getting himself booked for a foul on Berget.
Gordon made a fine double save, first from Rodic and then from Rosenberg for a corner.
However, the Swedish side’s second goal was only seconds away, coming when Rodic’s cross was met by the head of Carvalho, with Boyata appearing to get the last touch amid some chaos in the Parkhead defence.
Bitton then entered the book for a foul on Djurdjic and in the 72nd minute he was replaced by Nadir Ciftci, as Deila looked for another way back into the tie, while also protecting his midfielder from a possible second card.
Gary Mackay-Steven then came on for James Forrest, but try as they might Celtic could not get back into the tie and will now have to content themselves with Europa League football again.
Wiland, Tinnerholm, Arnason, Bengtsson (Carvalho 46), Yotun, Rodic (Rakip 82), Lewicki, Adu, Berget (Mehmeti 90), Rosenberg, Djurdjic.
Gordon, Janko, Boyata, van Dijk, Mulgrew, Brown, Bitton (Ciftci 72), Forrest (Mackay-Steven 79), Johansen, Armstrong (Commons 46), Griffiths. Ref: Milorad Mazic (Serbia).