Claudio Ranieri's side looked set for a gut-wrenching draw, having twice taken the lead through Valere Germain and Jeremy Toulalan, only for Prince Oniangue's double to peg them back.
However, Aissa Mandi was dismissed in the closing stages for a high tackle on Kurzawa, and the left-back popped up to turn home after James Rodriguez had hit the post to move Monaco within two points of table-topping Paris-Saint Germain ahead of their trip to Toulouse on Sunday.
The game looked set for a regulation home win just eight minutes in when Germain stole in at the near post to flick Joao Moutinho's corner home.
But Reims maintained an attacking threat and, though they could not find an equaliser in the first half, it came eight minutes after the interval.
Odair Fortes slipped a fine throughball to Oniangue, who raced through the home defence before slotting past Danijel Subasic.
Reims pushed on as they smelt blood but were hit on the break 10 minutes after scoring as Toulalan cut in from the left and thundered a low shot that clipped the heels of Lucas Ocampos on its way in.
Once again, Monaco could not hold out and they only had themselves to blame for coughing up the lead for a second time as Subasic and Kurzawa collided under a high ball, leaving Oniangue the simple task of scoring his second of the night.
Rodriguez curled wide and Germain saw an effort well saved as Monaco looked to be running out of time, but Mandi's dismissal for a rash chest-high challenge on Kurzawa gave the hosts renewed hope.
And the 21-year-old dusted himself down to clip home after Rodriguez's effort landed kindly at his feet to keep Monaco well in the title hunt.
