Steve Clarke's side have not lost any of their last five matches against Villa and Mulumbu knows how important it is to the fans that they make it six after the international break.
The midfielder – who played in four of those five matches – scored the winning goal in their match at The Hawthorns in April 2011 and is hotly anticipating another home clash.
"I'm looking forward to this game," he told TheBirmingham Mail. "I've played in derbies against Wolves and Villa and the atmosphere is unbelievable.
"These games are really important to our fans.
"It's a really interesting game and a huge deal – like a Champions League game.
"I remember when I scored against Aston Villa – the fans were singing my name even outside the stadium. I'd love to do that again."
West Brom and Villa sit 10th and 11th respectively in the Premier League table after similar starts to the season, with both sides on 14 points from their opening 11 matches.
‘Even though Kevin Keegan was my boyhood hero, we never really saw eye-to-eye. That was an underwhelming time for me – I felt on the outside’: Newcastle cult hero reveals disappointment over how things went under King Kev
Alexander Isak puts uncertainty over his Newcastle United future to bed amid fresh PSG and Arsenal links
