Kiwomya was dismissed from his role as manager at Meadow Lane on Sunday, but caretaker boss Steve Hodge got his temporary reign under way with a crucial three points.
Notts County went in front on the 20-minute mark through Danny Haynes, but Adam Rooney drew Oldham level on the stroke of half-time.
Yoann Arquin put Hodge's team back in the lead 15 minutes after the restart, before midfielder Callum McGregor looked to have secured the win with his sixth goal of the season in the 68th minute.
Teenage striker Jonson Clarke-Harris ensured it was a nervy last 20 minutes for struggling County by reducing the visitors' arrears immediately, but Hodge's men held on for just their third League One win of the season.
Meanwhile, in Tuesday's only League Two fixture, York City ended a four-game winless run in all competitions with a comprehensive 4-1 home victory over Scunthorpe United.
Nigel Worthington's men went into the match having won only twice at Bootham Crescent in the league this season, but produced a first-half blitz to go in four goals to the good at the break.
Young midfielder Josh Carson got the hosts off to a flyer by opening the scoring after just three minutes, before Wes Fletcher doubled their advantage nine minutes later.
Ryan Brobbel then got in on the act in the 21st minute, before the on-loan Middlesbrough winger got his second of the game with a stunning volley five minutes before half-time.
Scunthorpe's top scorer Sam Winnall gave his team an ideal start to the second period by pulling one back in the 47th minute with his seventh of the campaign.
However, the visitors were unable to mount a comeback and slipped to just their second defeat in their last 10 League Two fixtures.
