The 1991 European Cup winners parted company with 43-year-old Prosinecki following a 2-2 draw with neighbours Rad over the weekend and a row with the board.
Jankovic, who will remain in charge of the under-21 side, had a brief spell at Red Star during the 2007/08 season. His new club host Bordeaux in the first leg of a Europa League play-off on Thursday.
"It is an emotional moment for me to return to Red Star after four years and the fact that the club called me up in a difficult moment is very gratifying," Jankovic told a news conference on Tuesday.
"I am here because the Serbian Football Association showed plenty of understanding and allowed me to carry on with the Under-21 team, a group which I just couldn't abandon.
"Prosinecki is a close friend and it will take a lot of hard work to match his energy, dedication and commitment to Red Star."
Prosinecki, who won the European Cup with Red Star as a player and finished third with Croatia at the 1998 World Cup, endured a torrid 20 months at the club and fell out with president Vladan Lukic after a string of patchy performances.
Having failed to end the dominance of bitter city rivals Partizan, who have won five league titles in a row, Prosinecki hung on to his job last season when he won the Serbian Cup.
Red Star scraped past Naftan Novopolotsk of Belarus and Omonia Nicosia in the Europa League preliminary rounds, but a tepid performance against unfancied Rad cost Prosinecki his job despite widespread fan support.
Jankovic acknowledged that qualifying for the group stage of Europe's second-tier competition would be a tall order for Red Star.
"It's a very delicate moment and a massive task but I can't wait to get started," he said.
