Deportivo had previous only beaten Eibar away from home in the league this season, but Borges' 72nd-minute penalty ultimately proved decisive in an even encounter.
The 26-year-old, who impressed for his country at the 2014 FIFA World Cup, arrived on loan from Swedish side AIK Stockholm this month and made an instant impact after waiting over two weeks to make his first appearance.
Borges' first came in fortuitous circumstances, though, as goalkeeper Tono parried Ivan Cavaleiro's free-kick into his path to bundle in from close range.
Alberto Bueno equalised soon after in the 22nd minute, prodding home from eight yards after Alberto Lopo's headed clearance fell right at the feet of the Real Madrid youth product.
Borges clinched the win 18 minutes from time, however, stroking home a penalty after Abdoulaye Ba wrestled Sidnei to the ground at a corner, moving Deportivo to within two points of 10th-placed Rayo.
'People will remember us because we got to No.1 in the world rankings and achieved great things - but our biggest regret is that we didn’t win anything': Ex-Everton star Kevin Mirallas opens up on 'honour' of being part of nation's golden generation
‘I’m sure we’d have finished in the top four or five if he hadn’t joined Chelsea in January. Him leaving damaged the team, but we couldn’t stand in his way’: Alan Curbishley reflects on failed Champions League dream after selling key player
