Jonathan Fabbro had equalised for Cerro Porteno with 12 minutes left on the clock and Ortigoza completed the dramatic comeback in the second minute of added time as the visitors emerged triumphant at the Estadio Defensores del Chaco.
Leonardo Astrada's Cerro Porteno - who have won three matches in succession - are one point clear atop the table with three rounds remaining following Libertad's 1-1 draw at home to 3 de Febrero on Monday.
Guarani made the perfect start in Asuncion when Jorge Mendoza broke the deadlock in the 14th minute.
Cerro Porteno hit back, however, with former Bayern Munich striker Julio Dos Santos equalising 13 minutes later and Sergio Diaz finding the back of the net in the 34th minute as the visitors took a one-goal lead into the break.
Guarani quickly erased the deficit after half-time thanks to Federico Santander and Luis Cabral.
Santander restored parity in the 62nd minute, while Cabral put the home side ahead just three minutes later.
But that was as good as it got for Guarani, with Fabbro and Ortigoza both on target in the final stages.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
