The two nations are on a journey towards the Copa America in Chile starting in June, and Paraguay took the most out of the contest - after Angel Romero's deflected goal was cancelled out by a Paolo Guerrero header in the second half, before Gonzalez stepped up to convert late on.
Victor Yotun threw his arm at Oscar Ruiz's goal-bound header in second-half injury time, and referee Ricardo Marques Ribeiro rightly showed the defender a red card - and pointed to the spot.
Basel man Gonzalez - a late substitute - finished clinically into the bottom left corner past Peru goalkeeper Pedro Gallese in the 92nd minute.
Earlier, Paraguay took the lead when one of Victor Genes' half-time substitutes, Celso Ortiz, broke free on the left flank - and after his initial cross was blocked, he squared a pass for Romero.
Ortiz's fellow substitute did the rest, firing a powerful shot on the target - which deflected off a Peru defender and past stranded Gallese in the 70th minute for the 22-year-old's first international goal.
Peru's response was emphatic, Guerrero picked out by Andre Carrillo with a pin-point cross - and the skipper duly nodded home his 21st international goal with consummate ease.
However, it would be in vain as Paraguay were able to capitalise on their late pressure .
Paraguay's winless run was ended at four matches, ahead of their return clash with the Peruvians in Lima on Monday while Peru were denied their first draw in 2014 - losing their fourth match to go with their five wins.
Peru suffered their third straight away loss to Paraguay, and were unable to break their winning duck against the hosts on Paraguayan soil.
Earlier, Roma man Tonny Sanabria - a Barcelona youth product - was wreaking havoc for the hosts, as he had Paraguay's two best chances of the opening period inside the first eight minutes.
Sanabria had a free header from Victor Ayala's set-piece, but could not connect in the fifth minute, before Edgar Benitez picked out the teenager on the penalty spot with a chipped pass - but Sanabria blazed his effort over the bar.
The first quarter-hour was all Paraguay, but Pablo Bengoechea's visitors eventually found their feet in the game.
Despite causing some problems in their attacking area, Peru were lacking the final polish.
Immediately after the restart, Paraguay set out to break the deadlock - Benitez earning the hosts a corner within 30 seconds after his shot from an acute angle was blocked.
Captain Roque Santa Cruz was having an influence for Paraguay as he threatened aerially on occasion, while Jorge Moreira was also a threat for Genes' men on the right.
The game seemed destined for a draw after goals were traded three minutes apart.
But Paraguay prevailed after the late drama, as Gonzalez scored his first goal for his country to seal the win for the home nation.
