Deportivo had previous only beaten Eibar away from home in the league this season, but Borges' 72nd-minute penalty ultimately proved decisive in an even encounter.
The 26-year-old, who impressed for his country at the 2014 FIFA World Cup, arrived on loan from Swedish side AIK Stockholm this month and made an instant impact after waiting over two weeks to make his first appearance.
Borges' first came in fortuitous circumstances, though, as goalkeeper Tono parried Ivan Cavaleiro's free-kick into his path to bundle in from close range.
Alberto Bueno equalised soon after in the 22nd minute, prodding home from eight yards after Alberto Lopo's headed clearance fell right at the feet of the Real Madrid youth product.
Borges clinched the win 18 minutes from time, however, stroking home a penalty after Abdoulaye Ba wrestled Sidnei to the ground at a corner, moving Deportivo to within two points of 10th-placed Rayo.
