The Dutch had started Group A with a shock 2-1 defeat to the Czech Republic last month and coach Guus Hiddink expressed a desire to see his side produce a victory with a performance to match in the build-up to this fixture.
However, Renat Abdulin's headed goal in the 17th minute threatened to subject the hosts to a humiliating defeat.
Netherlands tried their utmost to find a leveller, but it did not look to be their night until Klaas-Jan Huntelaar got his head to an Ibrahim Afellay cross in the 62nd minute.
Baurzhan Dzholchiev saw red for a reckless challenge soon after, and Hiddink's men eventually made their visitors pay through the influential Afellay, who found the net in the 82nd minute on his first international appearance since November 2012.
Robin van Persie completed the scoring in the closing stages from the penalty spot after being brought down by Dmitri Shomko.
The late effort helped extend Netherlands' run of consecutive victories in home qualifiers to 18.
Hiddink switched to a 4-3-3 formation, having stuck with Louis van Gaal's favoured 3-5-2 for the defeat to the Czech Republic, as Gregory van der Wiel, Jeremain Lens, Afellay and Arjen Robben returned to the first-team fold.
And it was Lens who got the first real sight of goal in the 12th minute - his shot parried to Robin van Persie but the Manchester United forward's effort was ruled out for offside.
Kazakhstan took everyone by surprise five minutes later when Abdulin powered in a header off the crossbar from an Andrei Karpovich corner, although Bruno Martins Indi will feel he could have done better in defending the set-piece.
Van Persie headed over from a chipped Wesley Sneijder ball into the box as Netherlands desperately hunted a leveller, before Robben fired high and wide of the target on the half-hour mark.
The hosts continued to press and Van der Wiel's low ball across the six-yard box caused alarm among the Kazakhstan defence, but Lens' eventual shot went harmlessly wide of the left-hand post.
Afellay fired over as the beginning of the second half proved just as frustrating as the first for the home side, before Van Persie fired wide from a tight angle when substitute Huntelaar was primed to receive the ball in the six-yard box.
The hosts finally pulled level in the 62nd minute when Huntelaar, who looked to be in an offside position, got his head to an Afellay cross from the right.
What looked like being a dream evening for Kazakhstan threatened to go downhill quickly when Dzholchiev was dismissed for a studs-up challenge on Afellay's calf.
Netherlands tried to make the most of the man advantage and hit the crossbar through Robben and Huntelaar saw a shot saved from point-blank range soon after.
But it was Afellay who came up trumps for the home side - firing home a deflected effort from 25 yards – before Van Persie made absolutely sure of the points in the 89th minute.
Huntelaar saw an effort chalked off for offside in stoppage time, but everyone in Orange at the Amsterdam Arena breathed a sizeable sigh of relief at the final whistle.
