Kovac's men opened their FIFA World Cup campaign with a controversial 3-1 loss to host nation Brazil in Sao Paulo on Thursday.
Brazil were awarded a second-half penalty when Dejan Lovren was adjudged to have held Fred inside the area, leading to Kovac labelling the decision as a 'disgrace' after replays indicated there was minimal contact.
Despite walking away from the game empty handed, the performance provided Kovac with cause for optimism ahead of Wednesday's clash at the Arena Amazonia in Manaus.
"We are allowed to feel optimistic, we have shown an excellent performance against Brazil," Kovac said.
"Unfortunately, our tactics and our football knowledge was not in the spotlight. We are looking ahead and I believe we can make it to the second round.
"Everybody could see that Croatia played a fantastic match and all will be well should we repeat that.
"I can see the desire, the fire in my players' eyes, they want to make amends."
Another reason for Kovac to be optimistic of reaching the next round is the return of star striker Mario Mandzukic.
Mandzukic was suspended for the opening match after receiving a one-game ban for a red card he received against Iceland in Croatia's World Cup qualifying play-off in November.
"With Mandzukic we gain extra quality, even though (Nikica) Jelavic was brilliant," the 42-year-old said.
"Mandzukic could be a crucial factor in the next two matches."
