Substitute Gouffran struck 17 minutes from time to secure victory for Alan Pardew's men after Christian Benteke had cancelled out Hatem Ben Arfa's early opener.
Newcastle headed into the contest after an international break that has seen supporters voice their anger at the lack of signings by owner Mike Ashley and director of football Joe Kinnear.
But the old guard produced the goods at Villa Park as the visitors extended Villa's streak without a Premier League clean sheet to a staggering 27 matches.
Villa manager Paul Lambert named an attacking line-up, with Benteke, Andreas Weimann and Gabriel Agbonlahor all included from the start.
Loic Remy made his first start for Newcastle after coming off the bench in the 1-0 win over Fulham prior to the international break, while Yohan Cabaye, who had been linked with a move to Arsenal in the window, was named in the starting XI for the first time this campaign.
Newcastle started brightly, going close to taking the lead as Papiss Cisse fired just off target with an acrobatic backheel after good work down the left-flank from Davide Santon.
And the visitors were rewarded for their early dominance after 18 minutes when Ben Arfa slammed home from point-blank range after Cisse had flicked Remy's left-wing cross into his path.
Both sides continued to push forward following the opener, with Remy at the fore for Pardew's men as his pace caused plenty of problems for the Villa defence.
Lambert was forced to adjust his side just before the half-hour mark when Jores Okore limped off to be replaced by Ciaran Clark.
As they reshuffled, they were almost caught out by Newcastle again on the counter-attack, with the visitors wasting the chance to double their lead when Remy fired over from the edge of the area.
The hosts found their feet shortly before the interval and they were unfortunate not to level, Weimann hitting the side-netting after Tim Krul had failed to deal with a Matt Lowton cross, before Agbonalhor broke through the Newcastle defence only to shoot well wide.
Still, it was Pardew’s side who should have scored the game’s second goal on the stroke of half-time when Cabaye lashed a volley just past the post after collecting a superb pass from Moussa Sissoko.
The second half started in frenetic fashion as Villa pressed in search of an equaliser.
Newcastle were struggling defensively and they were lucky not to surrender their lead when Agbonlahor inexplicably missed with the goal at his mercy following a pin-point cross from Weimann.
The pace of Remy and Ben Arfa was continuing to trouble Lambert's side, the latter denied a second with a curling effort by Brad Guzan’s strong save.
Remy was brought off shortly afterwards - Gouffran coming on in his place - and Newcastle soon conceded the initiative as Benteke levelled matters with a bullet header. The Belgian international met Ashley Westwood’s excellent delivery to fire home.
However, Newcastle regained the advantage six minutes later – and Ben Arfa was heavily involved once again. He produced a fierce left-footed shot that Guzan could only parry and Gouffran was on hand to slot into the empty net.
That goal stretched the game further, with Cisse and Ben Arfa going close for Newcastle.
Fabian Delph almost equalised for Villa late on, but the visitors stood firm.
