The competition holders went in front through Ben Watson - the man who scored the winner in the final last year - in the 36th minute but were pegged back by Aaron Wilbraham 20 minutes from time.
James McClean then netted the winner in the 78th minute to send his side into the draw for the fifth round as Wigan's good run of form continued.
Wigan have now lost just twice in 11 matches in all competitions under Rosler, and the German felt his team were worthy victors on Saturday.
"I think the win was deserved," he said.
"The players did a very good job against a strong Crystal Palace side.
"It is fantastic to be in the last 16 and who knows what can happen if we get a good draw at home in the next round.
"In the first 10-15 minutes we looked a bit nervous but then Callum McManaman got on song and looked very good. Both wide players (McManaman and McClean) did very well.
"We had a lot of opportunities to put the game to bed but we didn't do that. That's why it was nervous near the end."
