Mihajlovic, famed as a tough taskmaster, issued a set of rules signed by all players and staff when he took over last week and the first item on the agenda was that everyone had to sing the national anthem.
"Ljajic has been sent home from the team's European tour and the decision is based on Mihajlovic's rulebook stipulating a code of conduct which Ljajic has breached," the FSS said on its website.
"Ljajic has told Mihajlovic he refused to sing the national anthem out of personal beliefs and that there would be no change in his position regarding the matter.
"The player will only be allowed to return to the national team if he changes his attitude and officially notifies Mihajlovic that he has done so."
Ljajic, a Slav Muslim from southern Serbia's ethnically mixed region of Sandzak bordering Bosnia and Montenegro, also clashed with his former Fiorentina coach Delio Rossi earlier this month during a Serie A match in Italy.
Rossi was sacked for punching Ljajic after the forward sarcastically applauded him having been substituted midway through the first half of Fiorentina's 2-2 draw with relegated Novara.
The Serbian FA then defended Ljajic and called Rossi a "disgrace to all football professionals" but this time had no sympathy for the 20-year old, who has struggled since he joined Fiorentina from Partizan Belgrade in 2010.
Serbia, who failed to reach Euro 2012, lost to European champions Spain 2-0 in the Swiss city of Saint Gallen on Saturday in the first of three games on their tour designed as a build-up for the 2014 World Cup qualifiers.
They are away to France on Thursday and Sweden on June 5.
Serbia's World Cup qualifying group includes neighbours Croatia and Macedonia as well as Belgium, Scotland and Wales.
‘The pure simplicity of the way Slot has managed the squad is probably the biggest thing I could say about him. It’s not broken, so let’s get on with it’: Liverpool legend full of admiration for Jurgen Klopp's successor at Anfield
‘I trained at Spurs and thought they’d give me a chance. But I received a letter thanking me but saying they didn’t think I was good enough – I was gutted’: How Tottenham missed out on signing Wales legend John Toshack
