The Addicks suffered a 1-0 loss at The Valley, squandering the opportunity to progress to the semi-final stage against the Coca-Cola League Two side.
The feelings of some Charlton fans were made perfectly clear both during and after the game with chants of “you’re not fit to wear the shirt” as well as calls for new manager Les Reed to be sacked and replaced by former West Ham boss Alan Pardew.
The defeat represented a new low for the south London club who are second bottom of the Barclays Premiership, five points adrift of safety.
Varney did not make any excuses and admitted he could understand the fans’ frustration. He said: “It was very unsatisfactory and I think a lot of our players have to take a good look at themselves.
“We need to apologise to the fans. They were very vociferous during the game and there was a protest outside afterwards. You cannot blame them for that.
“That type of performance was not acceptable for Premiership players. It was a quarter-final of a major cup competition and we were playing a side from the second division. You would expect the players to be up for reaching the semi-final but the hard proof is that they were not.”