Northampton, 17th in the fourth tier of English football, forced a 2-2 draw after extra time before winning 4-2 in the shootout to continue Liverpool's poor start to the season under new coach Roy Hodgson.
GEAR:Liverpool home, away and third kits available here. Go get 'em
The other big upset came at Stamford Bridge where Newcastle United became the first team to beat champions Chelsea this season, striker Shola Ameobi scoring with a glancing header in the final minute to secure a thrilling 4-3 win.
Big-spending Manchester City also went out, losing 2-1 at West Bromwich Albion after taking the lead, but holders Manchester United survived an early scare to crush Scunthorpe United 5-2.
Liverpool, fielding a largely second-string team, took an led through Milan Jovanovic but Northampton, who spent one season in the top flight in the mid-1960s, took the match to extra time through a 56th-minute strike by Billy McKay.
Northampton silenced Anfield by taking the lead eight minutes into the extra period through Michael Jacobs before a goal by striker David N'gog four minutes from time levelled the scores at 2-2 and sent the game to penalties.
The fourth division team controlled the shootout and Abdul Osman converted his spot-kick to secure a memorable 4-2 victory.
"Quite frankly, when they took the lead in extra time it was going to be an uphill battle," Hodgson told Sky Sports.
"We all know penalty shootouts are a little bit of a lottery but on the other hand when you are at home and you are kicking them in front of the Kop, you do harbour hopes at least that it's going to be your team that comes out on top," he added.
FIRST DEFEAT
Chelsea, also playing a much-changed side, lost a competitive match for the first time since April when Newcastle came from behind to claim their first victory at Stamford Bridge since 1986.
Chelsea went ahead after six minutes through defender Patrick van Aanholt, but Newcastle roared into a 3-1 lead soon after half-time with goals from Nile Ranger, Ryan Taylor and Ameobi.
The hosts, reduced to 10 men after using all their substitutes, struck back with two Nicolas Anelka goals before Ameobi headed the winner.
"Nobody thought that we could win coming here, hopefully it will give us momentum for the games coming up. We showed great charater and some good finishing to get back in the game" Newcastle manager Chris Hughton told reporters.
Chelsea manager Carlo Ancelotti said he was pleased with his young team's display.
"We are disappointed about the result, but absolutely not about the performance," he said. "We had a fantastic reaction and showed good spirit when it was 11 against 10.
"We now know we can count in the future on these young players, but tonight was not our priority for the season."
OWEN DOUBLE
Former England striker Michael Owen struck twice to send holders Manchester United through to the last 16 after coming from a goal down at Scunthorpe United.
Josh Wright put Scunthorpe ahead in the 19th minute before Darron Gibson equalised three minutes later and opened the floodgates.
Defender Chris Smalling made it 2-1, former England striker Owen grabbed his first after halftime and Park Ji-sung scored with a low shot
