Raheem Sterling and Daniel Sturridge put Brendan Rodgers' side in control at half-time before Yoann Arquin and Adam Coombes brought County level to take the game into extra time.
However, Sturridge's second and a Jordan Henderson strike five minutes from the end ensured the eight-time League Cup winners remain in the running this year.
There was no such luck for Premier League newcomers Crystal Palace, who were sent crashing out, with a late Owen Garvan goal not enough to stop them losing 2-1 at Bristol City.
Fulham were pushed all the way by League 2 side Burton Albion with Jack Dyer and Michael Symes putting the hosts ahead after Adel Taarabt had scored his first goal for Fulham.
However, Hugo Rodallega equalised three minutes from the end of extra time before Martin Jol's men won through 5-4 on penalties.
Sunderland were almost on the end of a giantkilling against League 1 outfit MK Dons, as they came from two goals down to edge into the third round 4-2.
Patrick Bamford and Izale McLeod gave Karl Robinson's side a two-goal lead after 55 minutes before Jozy Altidore pulled a goal back 12 minutes from time.
Connor Wickham then struck twice and Adam Johnson added another to spare Paolo Di Canio's blushes.
Fellow Premier League sides Southampton and West Brom enjoyed a more comfortable evening as they dispatched Barnsley and Newport County respectively.
Mauricio Pochettino's men were 5-1 winners at Oakwell while Saido Berahino's hat-trick ensured progression for West Brom at The Hawthorns 3-0.
Norwich City ran out 6-3 winners against Bury in a nine-goal thriller with Johan Elmander getting his first two goals for the club while West Ham saw off Cheltenham Town 2-1.
Hull City edged through against League 1 leaders Leyton Orient thanks to a Robert Brady goal in extra time.
Reading were on the wrong end of a 6-0 thumping against Peterborough United while goals from Nile Ranger and Alex Pritchard gave Swindown Town a 2-0 victory over QPR at Loftus Road.
Tranmere Rovers dumped out Bolton Wanderers at Prenton Park on penalties, Jermaine Beckford having equalised after Cole Stockton had given Tranmere the lead.
However, Ronnie Moore's men kept their cool from the spot to win 4-2 in the shootout.
The all-Championship clash between Yeovil Town and Birmingham City saw Lee Clark's men progress on penalties while two late goals from Jonathan Hogg and Adam Hammill gave Huddersfield Town a slender 3-2 victory over Charlton Athletic.
Two first-half goals gave Burnley bragging rights in the Lancashire derby against Preston North End while Leeds United overcame Yorkshire neighbours Doncaster Rovers 3-1.
Derby County and Leicester City are both into the next round as well after seeing off Brentford and Carlisle United respectively.
