Julian Palmieri's outstanding volley from the edge of the penalty area – his first goal since November 2012 – gave Bastia the lead for the first time in the second half, before he added a second in the final minute as they climbed out of the bottom three.
The first-half goal flurry began early as Lucas Moura and Adrien Rabiot both took advantage of slack Bastia defending to put the visitors two goals ahead.
Any hopes of a routine triumph were thwarted by two lapses in concentration as Ryad Boudebouz converted a penalty after Gregory van der Wiel had handled, and Francois Modesto headed home a late corner.
Palmieri's sensational 25-yard strike completed an unlikely turnaround after 56 minutes, Nicolas Douchez given no chance as the ball arrowed into the top corner.
Thiago Silva had PSG's best chance to get back into the game with 21 minutes remaining, but his effort struck the crossbar and - after Palmieri had extended Bastia's lead - Laurent Blanc's side were left to reflect on a failure to capitalise on Marseille's 2-1 loss at Montpellier on Friday.
PSG looked threatening from the kick-off and were rewarded inside the opening 10 minutes as Lucas broke the deadlock with a composed finish.
Yohan Cabaye played a superb long pass over the top of Bastia's defence and Lucas held off the challenge of Florian Marange before lifting the ball over the onrushing Alphonse Areola.
Lucas should have doubled his tally three minutes later having been released by Zlatan Ibrahimovic, but he dragged his shot wide of Areola's goal.
PSG did not have to wait long for a second goal, though, as Rabiot netted in the 20th minute.
The midfielder was the beneficiary of some assiduous work from Blaise Matuidi and Maxwell – one of three changes for the visitors – who combined to steal possession and set up Rabiot to fire home first time from 12 yards.
Despite their dominance PSG then gifted the hosts a way back into the game after Van der Wiel handled a right-wing cross, with Boudebouz coolly slotting the spot-kick past Douchez.
And, on the brink of half-time, Bastia were level as three PSG defenders failed to clear a corner, allowing Modesto a free header from close range.
No doubt frustrated at throwing their lead away, PSG came out in full attack mode in the second half, with Ibrahimovic the first to test Areola from a 20-yard free kick.
However, a stunning volley from Palmieri gave the hosts the lead four minutes before the hour, the 28-year-old firing into the top corner after PSG poorly cleared a left-wing corner.
To add to PSG's frustrations, they were denied an equaliser after 69 minutes as Silva's half-volley thundered against the crossbar from a right-wing free kick.
Still Blanc's side pushed for a leveller, but it was Bastia who got the final goal as Palmieri met Abdoulaye Keita's cross to volley past Douchez and give the Corsicans a memorable victory.
