Arsenal face two matches away to Liverpool next week after this deserved and determined victory.
This League Cup win has earned Mikel Arteta a potential fourth round tie at Anfield – should Liverpool beat Lincoln tonight - which is to be arranged just days after Monday night's trip there in the Premier League.
Eddie Nketiah booked Arsenal's place in the last 16 with a last minute goal after a Nicolas Pepe effort had forced an own goal out of Christian Fuchs.
Nketiah was leading the line with Alex Lacazette on the bench and Pierre-Emerick Aubameyang rested and took full advantage by following up his weekend strike against West Ham with another impressive performance.
Arteta's side did not have it all their way, but they play with a sense of confidence and resilience not seen in recent years in Arsenal teams.
Arteta made seven changes from that narrow London derby weekend win and Arsenal struggled initially with Leicester's pace.
Kelechi Iheanacho had a great chance to put Brendan Rodgers' side into a second minute lead. The former Manchester City forward was in again soon after and keeper Bernd Leno had to react fast to save.
Arteta's side eventually settled and gradually had the most of the possession without forcing Leicester keeper Danny Ward into many serious saves.
The recalled Reiss Nelson whistled one shot inches over the bar, aided by a finger tip Ward save.
The England under-21 international, who is expected to be loaned out for the season before next month's transfer window closes, squandered a much better chance with half an hour gone when he shot well wide with the goal at his mercy.
A much better move featured a great run down the left wing by Bukayo Saka and he pinged a low delivery across the face of goal and inches away from a finishing touch from both Eddie Nketiah and Joe Willock.
Saka was unfortunate to be denied a penalty when he was bundled over from behind by Daniel Amartey just as he was about to shoot from close range. Referee Peter Bankes was not interested and this match was not even being televised in England, let alone having VAR to make amends.
Arsenal had pace in their attack and should have made more of that advantage against Leicester's centre back pairing of Wes Morgan and Christian Fuchs, combined age of 70 years old.
Arsenal nearly paid for not making the most of their opportunities shortly before the break, however. Madison, Leicester's best player on his return from injury, spotted Leno slightly off his line and clipped a curling right foot shot over his head and off a post.
There was no let up from Leicester or the driving rain when play resumed and Madison had another shot which flew high and wide to mark a warning of sorts.
It proved to be a false alarm as Arsenal were in front after 57 minutes thanks to a mixture of brilliance and good fortune. Pepe jinked his way past a couple of Leicester defenders and forced a strong save from Ward with his initial shot. The follow up cannoned off a post at an off balance Fuchs and straight into the back of the net.
Pepe, making his first start of the season, was almost too embarrassed to celebrate. Fuchs looked mortified.
Nketiah had an effort headed off the line before his stoppage time decider which he shot over the line as he was falling from a series of desperate challenges.