Luke Brattan gave the visitors the lead six minutes before the interval, but Newcastle flew out of the traps in the second half and Taggart drew them level on 51 minutes.
And the Australia international struck again with 16 minutes remaining to lift his side up to third in the A-League table - one place behind Brisbane, who slipped to second courtesy of Western Sydney Wanderers' 1-0 win over Melbourne Victory on Saturday.
After a dour opening to the game, Brattan broke the deadlock with a powerful shot through a crowded penalty area.
However, Taggart equalised with a chip shortly after the break before Nathan Burns then looked to have spurned the chance to take all three points for the hosts when he missed a penalty.
But Taggart was on hand to spare his team-mate's blushes with a superbly taken winner in the 74th minute.
The only negative for Newcastle came when defender Kew Jaliens was sent off for dissent late on, meaning he will miss next weekend’s home clash with the bottom-placed Melbourne Heart through suspension.
‘Newcastle’s board said I was over the hill, so I left and played 108 consecutive games and won the European Cup – not a bad way to answer that accusation!’: Ex-Magpies defender reflects on career highlight after proving former side wrong
‘Parma viewed him as the perfect replacement for Adrian Mutu, who’d left to sign for Chelsea. But Manchester United’s interest was much more concrete’: Cristiano Ronaldo's 2003 transfer details revealed by former Sporting director of football
