Michael Laudrup's side took advantage of United being reduced to 10 men with 10 minutes remaining to record a maiden victory at Old Trafford courtesy of Wilfried Bony's last-minute winner.
Swansea went ahead on 12 minutes with a superb lofted effort from Wayne Routledge, but United responded quickly through Javier Hernandez's close-range finish.
United full-back Fabio came on as a 76th-minute substitute for Rio Ferdinand, but the Brazilian lasted just four minutes before receiving a straight red card for a lunge on Jose Canas.
And Laudrup's men made the most of the extra man, as Bony headed home Routledge's cross in the 90th minute to earn a tie at Birmingham City, Bristol City or Crawley Town and pile the misery on United manager David Moyes.
Pressure also continued to mount on West Ham boss Sam Allardyce, as his side's Premier League form continued with a 5-0 thrashing by Championship outfit Nottingham Forest at the City Ground.
Jamie Paterson starred for Billy Davies' men with a second-half hat-trick - his first goals for the club.
Djamel Abdoun opened the scoring on 12 minutes from the penalty spot, and Forest's lead was doubled midway through the second half by Paterson.
The 22-year-old confirmed the hosts' superiority on 71 minutes with a deflected effort, before completing his treble with 11 minutes remaining.
Andy Reid then netted a fifth in injury time to set up a fourth-round encounter with Ipswich Town or Preston North End.
Liverpool face a visit to Bournemouth or Burton Albion in after avenging last year's shock fourth-round defeat to Oldham Athletic with a laboured 2-0 triumph over the same opponents at Anfield.
Iago Aspas volleyed home the opener in the 54th minute, his first goal for the club, and Brendan Rodgers' men ensured their progression when Raheem Sterling's strike deflected in off James Tarkowski in the 82nd minute.
Jose Mourinho's Chelsea, meanwhile, overcame a tricky encounter with Derby County at Pride Park by battling to a 2-0 success.
After keeping Chelsea at bay for the opening 45 minutes, Derby's resistance was broken when John Obi Mikel marked his 300th appearance for Chelsea with the opener on 66 minutes, the Nigerian rising highest to head home Willian's free-kick.
Four minutes later, the tie was essentially put to bed courtesy of Oscar's powerful drive from the edge of the penalty area, as Chelsea set up a meeting with fellow Premier League outfit Stoke City in round four.
In other matches, Sunderland put their Premier League woes behind them to defeat League One side Carlisle United 3-1 at the Stadium of Light, while Plymouth Argyle overcame a two-goal deficit to earn a replay with a battling 2-2 draw at Port Vale.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
