Clint Dempsey gave the USA the lead in the first minute on Monday before Ghana dominated the rest of the match, eventually equalising with eight minutes remaining through Andre Ayew.
But while Ghana looked to have all the momentum after Ayew's goal, Klinsmann insisted he never lost belief in his side's chances of victory, which was secured by John Brooks' header in the 86th minute in Natal.
"We've gone through a very difficult game, which we always expected," the German coach said.
"We knew it would be a grind and go down to the wire against a very strong Ghana side, but I was still convinced that we would win this game even after Ghana's equaliser.
"I had the feeling that another two or three opportunities would come and we just needed to take one, which we did."
Klinsmann added: "We trained over and over on set-pieces and it (the winning goal) was well-deserved."
Ghana had 62 per cent possession at the Arena das Dunas and took 21 shots to the USA's eight but only tested American goalkeeper Tim Howard on three occasions.
Klinsmann conceded after the game that the USA had not been at their best but was thrilled with how they fought until the last moment.
"We have a great spirit," he told the host broadcaster after the final whistle.
"The US team always has a great spirit and fights until the last second … it was a grind but it's wonderful at the end of the day."
The USA lost striker Jozy Altidore to an apparent hamstring injury in the first half but Klinsmann refused to discuss any diagnosis before scans.
"It was a little bit of a shock that Jozy got injured," the 49-year-old said.
"We'll double check after this whole thing."
