Liverpool's disappointing start to the season looked set to take another turn for the worse when Marvin Emnes' second-half volley put Swansea on course for the quarter-finals.
Brendan Rodgers' much-changed side lacked a cutting edge and appeared to be on course to complete a third consecutive game without scoring until much-maligned striker Balotelli struck four minutes from time, just seven minutes after coming off the bench.
It was the Italian's first goal in nine games and prompted a stunning finale, which saw Swansea defender Federico Fernandez harshly shown a straight red card for a challenge on Philippe Coutinho.
To add insult to injury, Coutinho swung over a free-kick five minutes into time added on and Dejan Lovren headed home his first Liverpool goal after Gerhard Tremmel had misjudged the Brazil midfielder's delivery.
There was no time for Swansea to respond as they crashed out and Liverpool moved into the last eight.
And there was also late drama at Dean Court, where substitute Callum Wilson fired Bournemouth into the quarter-finals for the first time with a 2-1 win over Premier League side West Brom.
Eunan O'Kane's 49th-minute strike broke the deadlock, but Bournemouth had to defend resolutely to keep Albion out.
The introduction of Georgios Samaras eventually paid off for the visitors as the Greece international's effort deflected in off Tommy Elphick after 85 minutes.
However, Bournemouth immediately went down the other end and Wilson netted for the fifth game running to extend Albion's winless streak to four games.
Premier League leaders Chelsea needed a Jermaine Grandison own goal to beat Shrewsbury Town 2-1.
The League Two side had drawn level 13 minutes from time when Andy Mangan cancelled out Didier Drogba's opener, but Grandison's mishap ensured Jose Mourinho's side edged through.
Derby County produced a rousing fightback to hammer Fulham 5-2 in a thrilling all-Championship clash at Craven Cottage.
A Moussa Dembele double put Fulham in charge, but the prolific Chris Martin converted a penalty just before half-time to give Derby a lifeline before goals from Johnny Russell, Simon Dawkins [two] and Jeff Hendrick stunned the London club.
Former Derby manager Nigel Clough was indebted to a late Michael Higdon double as his Sheffield United side won 2-1 at League One rivals MK Dons, who had led courtesy of a Benik Afobe penalty.
