The Hoops went into the game buoyed by their 3-3 draw with Manchester City on match day two last month. However, keeper Craig Gordon had to make several crucial saves in a first half largely controlled by the slick-moving Bundesliga outfit who picked up their first points in Group C.
Captain Lars Stindl got the breakthrough in the 57th minute and then Andre Hahn made it 2-0 on the break in the 77th minute - both times Celtic defender Kolo Toure was found wanting - to leave Brendan Rodgers’ side with one point from a possible nine, ahead of the return fixture in Germany in two weeks’ time.
The Scottish champions have still to play Barcelona at home and City away which meant it was always going to be a key match.
The visitors arrived without a point after two matches and with a squad hit by injury and illness.
Hoops boss Rodgers made four changes to his side with Toure, Mikael Lustig, Nir Bitton and Tom Rogic returning to form the same starting 11 which tested City to the limit.
And without a domestic defeat this season, there was no lack of confidence ahead of the game.
Another packed house at Celtic Park, however, saw the visitors start the game with pace and purpose.
In the first couple of minutes Toure got back to block an angled drive from Hahn after the German striker raced on to a pass from Stindl.
The corner came to nothing but it was an early warning for Celtic. A Bitton free-kick in the 15th minute was pushed round the post by Gladbach keeper Yann Sommer for a corner but aside from that, the Bundesliga side looked in control.
The game zipped from end to end.
Gordon made a great save from Ibrahima Traore after the Gladbach wide man had been set up by Jonas Hofmann, the Parkhead keeper gathering the low drive at the second attempt with Hahn lurking.
The Scotland keeper made another good save in the 26th minute, this time from Hofmann’s stinging drive after the visitors again exploited space behind left-back Kieran Tierney.
The home supporters grew increasingly agitated as Gordon tipped a drive from Hahn over the bar after another swift Gladbach break.
But after all that, the Hoops should have gone into the interval with a goal.
Rogic’s defence-splitting pass found Scott Sinclair but with plenty of time and only Sommer to beat, he scooped his shot over the bar.
Borussia defender Tobias Strobl came close with a curling shot from 25 yards early in the second half but the opener was only moments away.
There looked little danger as Toure tried to shepherd a loose ball out of play.
However, Hahn refused to give it up and hooked the ball back to Stindl who, from an angle, drilled through Gordon’s legs from 10 yards before taking the acclaim of the away fans.
And the points were heading back to Germany when Hahn burst through towards the Celtic penalty area chased by Toure, who had been dispossessed, and Lustig, before he knocked it high past Gordon to seal a good night’s work for Andre Schubert’s men.
Gordon, Lustig, Toure, Sviatchenko, Tierney, Forrest (Roberts 72), Brown, Bitton (McGregor 64), Sinclair, Rogic (Griffiths 71), Dembele.
Sommer, Korb, Vestergaard, Elvedi, Wendt, Traore (Nico Schulz 86), Kramer, Strobl, Hofmann, Stindl, Hahn (Herrmann 89).
Tasos Sidiropoulos (Greece).