The Serie A champions looked well on their way when Pirlo - making his 100th Champions League appearance - curled in a trademark free-kick after 20 minutes.
But the Greek side stunned the hosts through Alberto Botia and Delvin N'Dinga's to lead by the hour mark.
Juve went out at the group stage of last year's tournament but turned the game on its head in 46 seconds. First, goalkeeper Roberto was unfortunate to put through his own net, but Paul Pogba smashed home the winner just a minute later on his 100th appearance for the club.
Arturo Vidal saw a late penalty tipped onto the crossbar to keep the Italians third in the pool behind Olympiacos on head-to-head record.
The nature of the win breathed new life into Juve's Champions League campaign though, and Pirlo was happy with how the team played.
He told Sky Sport Italia: "It is a fundamental victory and we resolved it with the calm and strength to turn it around. It's a shame about the penalty, but we are satisfied with the performance.
"A win was crucial and we knew that it depended on us. Now there are another two games we have to make the most of."
Pogba - who scored his first Champions League goal - chose not to dwell on Vidal's miss.
"We missed a penalty and football is like that, but we know that we have to keep going like we did today and try to win every game," he added.
"We know that you can't joke around in the Champions League and we have to learn that we cannot let go even for a second."
