The Gunners, who beat Bayern Munich in the Champions League and Everton to briefly go top of the Premier League in the last seven days, were brought back down to earth by the Championship outfit, who deservedly progressed to the quarter-finals for the first time since 2002 thanks to goals from Ross Wallace, Lucas Joao and Sam Hutchinson.
Arsenal boss Arsene Wenger admitted his side were well beaten.
He said: “It was a very bad night for us. We lost two important players in the first part of the game and then we were not at the level we needed to be.
“Congratulations to them, they were faster, sharper and more decisive than us. We could not create anything. We were very naive when defending.
“We cannot master how people will react. I left six players at home and I still lost two to injury.
“The level was too high for the young players tonight. They are not ready to play at this level.” Defeat means Wenger’s wait for a first League Cup of his lengthy reign goes on and the Gunners have not won this competition since 1993, when they were victors over the Owls at Wembley.
But what may come as a bigger cost to Wenger on a damaging night is injuries suffered to Alex Oxlade-Chamberlain and Theo Walcott, who both went off in the opening stages of the match.
The fortunes of the two sides in those 22 years could not have been more different, but Wednesday were every inch the worthy winners and they claim a second successive Premier League scalp following a third-round victory at Newcastle.
Most of the action in the opening 20 minutes was in the Arsenal dug-out as they lost both Oxlade-Chamberlain and Walcott, which could have implications for their visit to Swansea on Saturday.
Those changes meant the Gunners struggled for any fluency and — after neutralising any early threat — Wednesday took a 27th-minute lead with a fine goal.
The Owls worked the ball superbly down the left and Daniel Pudil, set free by Barry Bannan, pulled back a superb cross to Wallace on the edge of the area and the former Scotland international drilled a first-time effort into the bottom corner.
Hillsborough was rocking like it used to when fixtures like this were a regular occurrence in the mid-90s and the hosts and Wallace were inches away from a second minutes later as the left-footer curled a 20-yard free-kick just wide, with goalkeeper Petr Cech only able to look on.
After making an impressive save to keep out Jeremy Helan’s effort, Cech was rooted to the spot again as Wednesday did make it two in the 40th minute.
The resulting corner from Helan’s effort was played short to Wallace and he sent in a cross which an unmarked Lucas Joao headed into the corner.
The Gunners were back out after the interval well before their hosts, clearly keen to stage a second-half comeback, but any such hopes were crushed six minutes after the restart as Hutchinson made it three.
Bannan clipped a free-kick to the far post where Tom Lees was unmarked and his ball back across goal was too easy for Hutchinson to scoop home.
Arsenal enjoyed plenty of the ball, but inexplicably they rarely tested rookie Wednesday goalkeeper Joe Wildsmith, who was a spectator for much of the final 30 minutes.
The one time Wildsmith was called into action, he superbly turned Per Mertesacker’s header over the bar as Wednesday saw out a famous win and a ninth game in all competitions without defeat.
Wildsmith, Hunt, Lees, Loovens, Pudil, Hutchinson (McGugan 80), Wallace, Lee, Bannan (Semedo 75), Helan, Lucas Joao (Nuhiu 85).
Cech, Debuchy, Chambers, Mertesacker, Gibbs, Flamini, Kamara (Bielik 60), Campbell, Oxlade-Chamberlain (Walcott 5, Bennacer 19), Iwobi, Giroud.