The defeat is likely to pile more pressure on Partizan coach Avram Grant after the former Chelsea manager had come under fire from the fans and media following two league draws against modest opposition.
Champions and cup holders Partizan have since won two league games to stay six points clear of Red Star in the 16-team first division but losing to their bitter foes in the first real test of Grant's credentials will have done the Israeli no favours.
"It was a strange game because we created many good chances but when you don't score you get punished and we paid the price for the mistakes we made at the back," a grim-looking Grant told a news conference.
"I congratulate them for their victory but it's not over yet, we still have a chance to overcome the deficit in the return leg on our turf," he said.
Grant's 4-5-1 formation stifled Red Star's midfield but Partizan were punished for missing a hatful of opportunities as the 1991 European Cup winners scored two opportunist goals to delight their half of a raucous 45,000 crowd.
Midfielder Milunovic, who joined Red Star from Belgian side Zulte Waregem in January, fired the home side ahead in the 19th minute with his third goal in as many games for the club when he threaded the ball past goalkeeper Vladimir Stojkovic.
The 19-year old finished with some aplomb after fellow signing Luka Milivojevic pounced on a defensive blunder and delivered a defence-splitting pass through the middle.
POOR FINISHING
Partizan dominated after falling behind but striker Lamine Diarra and captain Sasa Ilic failed to convert gilt-edged chances.
Diarra volleyed over the bar and then hit the post from close range before he and Vukic left Ilic with only the keeper to beat in the 54th minute, but the 34-year old playmaker scuffed his shot from 10 metres wide of the far post.
Diarra failed to get a clean connection on Nemanja Tomic's teasing cross from the right before Red Star took advantage of another defensive error in the 76th minute, when substitute Kasalica was on target.
The striker, who is yet to start a game for Red Star after joining them in January from first division rivals Sloboda Sevojno, shook off a defender tugging his shirt and gave former Red Star keeper Stojkovic no chance with his clinical finish.
The match was played under tight security amid fears of crowd trouble but apart from dozens of flares lit by both sets of fans on the terraces, the fixture which has a history of violence was incident-free.
"We had a fair share of luck and we still have a lot of room for improvement but this is a morale-boosting win which will also instill a new level of confidence ahead of the title run-in," said Red Star coach Robert Prosinecki.
"Partizan dominated long spells of the match but credit to my team which included five derby debutants because we showed character and resilience," he added.
Earlier on Wednesday, Vojvodina Novi Sad were held to a 0-0 draw at Borac Cacak
