Kovac's men had led Brazil in the 11th minute at the Arena Corinthians in Sao Paulo thanks to Marcelo's own goal, although the hosts equalised before half-time through Neymar.
Croatia had been widely expected to be brushed aside by Brazil in the opening game of the 2014 World Cup but gave the home side plenty to think about before Japanese referee Yuichi Nishimura awarded a penalty in the 69th minute.
Dejan Lovren was judged to have held Fred in the box, although replays indicated there was minimal contact, and Kovac was furious with the decision after the game, as it had allowed Brazil to take the lead through Neymar.
"If that was penalty, than we should play basketball, not football," Croatia's coach told HTV.
"This is a disgrace.
"If you continue like this you will have 100 penalties. I think 2.5 billion people watching on TV saw this was not a penalty.
"This was ridiculous and if we continue in this way we will have a circus."
But the 42-year-old former Croatia international did not blame Fred for going down in the penalty area.
"The referee was well placed, he was not unsighted. He saw everything," Kovac said.
"I don't blame Fred at all. I blame the referee."
Kovac continued by arguing the referee had treated his team harshly throughout the 90 minutes.
"If that's how we start the World Cup, we better give it up now and go home," he said.
"We talk about respect, that wasn't respect, Croatia didn't get any."
