Ten-man United, who lost their opening game of 2014 to Tottenham on Wednesday, recovered from going a goal down early on as Javier Hernandez cancelled out Wayne Routledge's opener, but failed to break down a stern Swansea back line on Sunday.
And the 11-time FA Cup winners were punished for their lack of cutting edge when Wilfried Bony headed home in the 90th minute to pile the pressure on United manager David Moyes and inflict a fifth home defeat of the season and fourth in the side's last six Old Trafford outings.
Things began badly for Moyes in his first FA Cup clash as United manager, when Routledge scooped an effort over a helpless Anders Lindegaard 12 minutes in.
However, the visitors' joy was short-lived as Hernandez restored parity from close range four minutes later.
United had Fabio sent off 10 minutes from time for a rash tackle on Jose Canas and Bony then struck a dramatic winner when he buried Routledge's cross in the dying stages.
Rio Ferdinand also hobbled off injured during another disappointing afternoon for the Premier League champions, while Swansea can now look forward to a fourth-round tie against either Birmingham City, Crawley Town or Bristol Rovers.
Moyes handed starting spots to goalkeeper Lindegaard and defenders Ferdinand and Alexander Buttner, while Danny Welbeck and Hernandez led the line in the absence of injured pair Wayne Rooney and Robin van Persie.
There were five changes from the Swansea side that lost to Manchester City in the Premier League on Wednesday. Captain Ashley Williams was on the bench with manager Michael Laudrup forced to contend with an injury list featuring the likes of Michu and Michel Vorm.
The hosts began the brighter and should have gone ahead after 11 minutes, but Hernandez failed to connect with an effort from inside the area after good link-up play between Welbeck and Chris Smalling.
United were made to pay soon after, when Alejandro Pozuelo played in Routledge, who produced a delightful lob over Lindegaard to put Swansea ahead.
Four minutes later, United were level, Hernandez making amends for his earlier miss to prod home Buttner's cross inside the six-yard box.
After the early goals, chances dried up for both sides before Jonathan de Guzman and Buttner threatened from set-pieces for either side.
Hernandez wasted another opportunity a minute before the break, volleying Jonny Evans' lofted ball wide of Gerhard Tremmel's right-hand post.
The sides cancelled one another out in the early stages of the second half, with Swansea perhaps enjoying greater possession, without causing any problems.
Bony's swerving effort had Lindegaard worried on the hour before it whistled wide, while Swansea's resolute defending at the other end stifled United's forward line.
United's cause was not helped by the loss of Ferdinand to a suspected knee injury and the dismissal of his replacement Fabio, who was shown a red card for a late challenge on Canas 10 minutes from time.
And Bony compounding Moyes' misery to secure Laudrup a first win in eight.
