Victory lifted the Croatians to the top of Group A on 13 points from five games, three more than Belgium who are away to Macedonia later on Friday.
Serbia have four points from five matches and face a tough task to qualify for next year's finals.
"We knew this game was Serbia's last chance to haul themselves back into the race for one of the top two spots in the group and that's why we had to be patient," Croatia coach Igor Stimac told a news conference.
"We are overjoyed and I can only congratulate my players and the staff for doing an outstanding job but this result is history as of now, because we have a tough visit to Wales on Tuesday and anything less than a win would devalue this success."
The match at Dinamo Zagreb's Maksimir stadium was played under tight security amid fears of crowd trouble but apart from sporadic volleys of abuse that pockets of home fans directed at Serbia's players, it was an incident-free event.
Both federations had agreed not to take away fans to either fixture with the reverse fixture set for September 6 in Belgrade.
While Stimac fielded an adventurous 4-4-2 formation, Serbia counterpart Sinisa Mihajlovic surprisingly left out several regular starters and his experiment backfired as the visitors were punished for a cagey approach.
Croatia took the first meeting between the two sides as independent nations by the scruff of the neck after a cautious opening and the Serbians were lucky not to have trailed by a bigger margin at half-time.
Mandzukic fired Croatia ahead midway through the first half when Olic capitalised on a calamitous error by Serbia's left-back Aleksandar Kolarov and squared the ball back to his strike partner, who prodded it home past two defenders from close range.
The 33-year-old Olic doubled the home team's advantage in the 37th minute, chesting in from the edge of the five-yard box a teasing Darijo Srna free-kick after Kolarov was booked for a rash tackle on Mandzukic.
Serbia missed a good chance to pull one back when Croatia keeper Stipe Pletikosa parried Kolarov's stinging shot from 16 metres and Zoran Tosic side-footed the rebound wide with the goal at his mercy.
Croatia then twice went close as Luka Modric's shot from 20 metres sailed just wide and Ivan Rakitic's low cross flashed across the face of goal.
Substitute Filip Djordjevic wasted a golden opportunity for Serbia in the closing stages, directing his shot from six metres straight at Pletikosa.
Mihajlovic took the blame for a tepid Serbian performance.
"We conceded two very cheap goals and it was very difficult to come back into the match after that, although we improved in the second half and created some good chances," he said.
"We lost to a better team and I take full responsibility for the defeat because the lads played as well as they could and this is the best they can do at the moment."
