In Bielsa's first competitive match in charge of Marseille, the south coast club recovered from conceding early to storm into a 3-1 lead just after the hour mark, before Bastia surged in the final 30 minutes, scoring twice to secure a draw in the opening weekend of the new Ligue 1 season.
The match on the island of Corsica was the battle of two new coaches with Claude Makelele making his competitive debut on the bench for Bastia.
Makelele was thrilled with how his team adapted to Bielsa's preferred, yet unconventional, 3-1-3-3 formation after going two goals down but his opposing number was adamant Marseille should have won.
"This is a bad result. [Marseille] should have won this match but we didn't because we couldn't impose the difference in quality between the two sides," the Argentinian said.
"I have the feeling we were below our best and that Bastia played close to their full potential."
Bielsa felt Marseille lost control of the game after Andre-Pierre Gignac made it 3-1 in the 62nd minute.
"When we were leading 3-1, we should have dictated how the match was to be played," he said.
"It was a football problem, not a tactical problem, or a mental problem. We should have neutralized our opponents' reaction.
"It was during this period that I have the most things to discuss with my team."
Makelele argued, however, that his introduction of a second striker - Gadji Tallo - for attacking midfielder Ryad Boudebouz in the 58th minute provided the platform for Bastia to launch their comeback.
"I had to re-explain some things because we were in trouble," the former Chelsea and Real Madrid midfielder said.
"I switched to two strikers because we left Marseille space; it allowed us to roll them back."
Tallo scored within just eight minutes of coming off the bench, converting a penalty after Benjamin Mendy had wrestled Christopher Maboulou to the ground.
Maboulou then scored his second goal of the game in the 73rd minute to secure a share of the spoils for Bastia.
Makelele started five new players at the Stade Armand Cesari with Brazilian striker Brandao only completing his transfer from Saint-Etienne last week but the 41-year-old coach was impressed with how his team came together so quickly.
"I had a difficult preparation with late arrivals but the players who arrived were able to adapt quickly, there was a reward at the end of this game," Makelele said.
Maboulou was another of the new signings and the 24-year-old started brilliantly, unleashing a 25-yard shot that looped over Marseille goalkeeper Steve Mandanda into the net in just the eighth minute.
Marseille responded quickly, however, with Gignac heading Mendy's cross home three minutes later before the visitors hit the front in the 17th minute thanks to Christian Romaric's own goal.
Gignac then scored from the spot in the second half after Sebastien Squillaci had handled the ball in the penalty area, before Bastia recovered to earn a point.
"We played better in the first half than in the second," Bielsa said.
"The way Bastia attacked caused us more problems than anticipated. We had the space and the opportunity to cause them more problems than we did.
"We finished the match on top, which means that between 1-3 and 3-3, we lost control of the game."
