Goals from Tom Beugelsdijk and Danny Bakker put Den Haag - bottom before kick-off - 2-0 up at the break, and a first league win in four games seemed assured.
However, a Feyenoord fightback nearly spoiled the celebrations.
Jordy Clasie and Graziano Pelle brought Ronald Koeman's side level with 16 minutes to play, but Roland Alberg's first goal for the club secured the three points and lifted Den Haag out of the bottom three.
Meanwhile, Jetro Willems denied former PSV coach Dick Advocaat a dream return to the Philips Stadion as his AZ side were beaten 1-0 in Eindhoven.
Park Ji-Sung's shot was blocked by the AZ defence, but the full-back latched onto the loose ball and fired home his fourth goal of the season from 20 yards after just three minutes.
The win moves Philip Cocu's side into sixth place, above AZ and Groningen, as their campaign continues to recover from a dreadful start.
PEC Zwolle ended a run of nine Eredivisie games without a win with a rousing comeback against NAC Breda, Ron Jans' side scoring twice in the second half to earn a 2-1 success.
Joey Suk gave Breda the lead in the 19th minute, but Guyon Fernandez levelled five minutes into the second half from the penalty spot before Mustafa Saymak sealed the win with 15 minutes to play.
Elsewhere, Roda eased their relegation worries with a 1-0 victory over Utrecht in Jon Dahl Tomasson's first home league game in charge.
Striker Guus Hupperts notched his fifth goal of the season in the 34th minute to seal the win and see his side climb out of the relegation places.
Utrecht's fall down the table continues, though, as the defeat was their fourth in a row and sees them slip back into the bottom half.
