Utrecht could have climbed to fourth with victory at Stadion Woudestein ahead of the Eredivisie weekend, but they instead failed to win for the fourth time in six games on the road.
Tom van Weert set Excelsior on their way 23 minutes in, tapping into an unguarded net after Jeff Stans' effort hit the post and fell kindly, but Utrecht hit back in even more bizarre circumstances.
Kristoffer Peterson cut in from the right flank and badly skewed a left-footed effort, but Mark van der Maarel was aware and smartly flicked the ball home with his shoulder.
Chris Kum looked to have snatched a win as he flicked home a deep free-kick with just six minutes to play.
But De Reuver stepped up for the hosts in stoppage time when he was left unmarked to lash Ninos Gouriye's left-wing cross home and snatch a draw, which lifts Excelsior five points above the relegation zone.
