Stoke had been 3-1 up on Tuesday, following the first-half dismissal of Birmingham’s Wade Elliott, but two late Peter Lovenkrands goals took the game to extra time.
Kenwyne Jones then put Stoke back in front but substitute Olly Lee got a 118th-minute equaliser, his first senior goal, to take the game to penalties.
Birmingham missed their first two spot kicks, Mitch Hancox and Callum Reilly off target, and though they converted their third and fourth, Stoke scored each of their first three, leaving Steven N'Zonzi to step up and fire home the winner, sending the visitors through to just their second League Cup quarter-final since 1978-79.
Stoke boss Mark Hughes made five changes from Saturday’s defeat at Old Trafford and two of his inclusions - Marc Muniesa and Ousama Assaidi - combined to put the Premier League outfit in front after nine minutes.
Muniesa ran onto Marko Arnautovic's perfectly weighted throughball, before back-heeling into the path of Assaidi who fired an excellent 20-yard strike straight into the top corner for his first goal for the club.
The hosts drew level 28 minutes in when Tom Adeyemi got onto a Nikola Zigic knockdown and finished well past Thomas Sorensen.
But, in first-half injury time, a clash between former Burnley midfielder Elliott and Assaidi left the Moroccan winger in a heap and his Stoke team-mates incensed.
Referee Simon Hooper subsequently showed Elliott a red card.
Stoke took full advantage of the dismissal when Peter Crouch met Muniesa's whipped cross, steering his header back across Colin Doyle and into the far corner of the net.
Arnautovic looked to have put some gloss on the result with a superb strike from just inside the box after latching onto N'Zonzi's pass with 19 minutes remaining.
Stoke should have been out of sight by that point, but the hosts pulled one back through Lovenkrands in the 85th minute and Lee Novak drew a great save from Sorensen a minute later as they pushed for a leveller.
Hughes' men looked to have done enough but, with three of the five minutes of injury time gone, Lovenkrands latched onto Demarai Gray's pass and fired home a sensational equaliser.
Stoke again went ahead in extra time as Jones turned well and rounded Doyle before finishing into the empty net from a tight angle.
But, unbelievably, the 10-men of Birmingham came back again and equalised through Lee with just two minutes of extra time remaining, setting up the dramatic shootout which saw Stoke finally claim a hard-fought win.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
