USA led 2-1 thanks to goals from Aron Johannsson and Jozy Altidore sandwiched in between Nicklas Bendtner's first-half opener at NRGi Park in Arhus.
But Bendtner scored two goals during the final seven minutes to condemn USA to their fourth defeat in five matches.
It was the third time USA have lost a match from a winning position recently, after also leading in the second half of friendly losses to Colombia and Chile.
"Obviously, what we don't like are late goals because of losing focus for a second or losing your marker or heavy legs," Klinsmann said post-game.
"You never want to give a game away where you're up 2-1. Overall, it's a bit frustrating in the end, especially when you lose last minute."
Klinsmann added: "There's still stuff to improve, but overall I think they did a lot of really great work tonight.
"There was a lot of good stuff taken from this game, definitely. What we need to do is step it up in terms of managing the game all 90 minutes."
Manchester United eyeing surprise move for AC Milan star for whom tension in Italy is growing: report
‘Even though Kevin Keegan was my boyhood hero, we never really saw eye-to-eye. That was an underwhelming time for me – I felt on the outside’: Newcastle cult hero reveals disappointment over how things went under King Kev
