The south-coast outfit emerged from a recent slump to win three out of four matches over an unbeaten festive period, returning the fourth place in the Premier League.
Last weekend's spirited 1-1 draw against leaders Chelsea was followed by a 2-0 victory over Arsenal on New Year's Day.
Southampton also overcame Arsenal in the League Cup before Koeman experienced the sharp end of English cup football's habit of throwing up shock results.
League One Sheffield United beat their top-flight opponents 1-0 to secure a semi-final place and Koeman is demanding complete focus from his players against Championship high-flyers Ipswich.
"Every game we take it seriously," he said. "We did that in the League Cup and we will do that in the FA Cup. I don't like to change a lot because I believe in the spirit of the team.
"I don't believe in a lot of rotation of the team. Maybe one or two changes from the Arsenal game but not much more."
Having enjoyed a highly decorated playing career with the likes of Ajax, PSV, Barcelona and Netherlands, Koeman has admired the FA Cup largely from afar and believes the famous old competition has lost none of its magic in the modern era.
"I think [there is] always a lot of attention to the FA Cup," he said. "If I have one of the dreams, it's a dream about playing the FA Cup final.
"It's a long way to Wembley, you have to beat a lot of teams."
Sadio Mane scored for the third game in succession against Arsenal, but the Senegal forward is set to miss out at the weekend having limped out of the action before half-time against Arsene Wenger's men with a knock.
Mane joins ankle-injury victims Nathaniel Clyne and Jack Cork among the absentees, although Morgan Schneiderlin returns after serving a one-match suspension last time out.
Defender Maya Yoshida, who signed a three-year contract extension on Friday, is unavailable having joined up with Japan's Asian Cup squad.
An 11-match unbeaten run means Ipswich lie second in the Championships, a point behind leaders Bournemouth, and manager Mick McCarthy is ready to make changes to keep his squad refreshed for their promotion push.
Former Southampton trainee Tyrone Mings and fellow defender Luke Hyam are back in contention following respective toe and Achilles injuries.
Mings is not the only Southampton connection on Ipswich's books, with goalkeeper Bartosz Bialkowski and forward David McGoldrick available to face their former club.
