Up 2-0 at half-time, Liverpool lost focus in the second half at Anfield and the visitors were able to tie things up with just over six minutes to play, with the match heading to extra-time.
Goals from Daniel Sturridge and Jordan Henderson finally secured Liverpool's progress into the next round, much to Rodgers' relief.
"I think you have to give a big credit to Notts County because at 2-0 down here at Anfield, we're in good form at the minute," Rodgers told Sky Sports.
"It was a great fightback for them and that will bode well for their season because they showed great character and quality.
"We've seen it in football before - 2-0 is always a dangerous scoreline no matter who you're against.
"We're obviously disappointed with the goals we conceded and our management of the game, but across the board, we're glad to get through."
The match was a taxing one for Liverpool, with Joe Allen, Kolo Toure and Aly Cissokho succumbing to injury during the match.
"Young Cissokho was devastated," Rodgers said.
"He shouldn't be too long, he's just got a nick on his ankle, but obviously it was early on in the game and he'd actually made a good run that created the space for the goal so he was distraught to come off in his first start here.
"Joe Allen may well be a hamstring so we'll have a look at that and obviously Kolo Toure, we'll need to assess it but his groin has opened up a wee bit.
"We'll know more in the morning but as I said, it wasn't a good night injury-wise."
