Bielsa's men led twice in regulation away to Grenoble, while the visitors also hit the front in extra time, before their fourth-tier hosts equalised in the 120th minute to force a penalty shoot-out.
Grenoble went on to win 5-4 in the shootout with Paul Cattier tipping Florian Thauvin's spot-kick over the bar.
While Grenoble's speed and tenacity pulled Marseille's defence apart on numerous occasions - the home side had 10 shots for the match - Bielsa argued "there is no way to explain a defeat like this".
"This is not a game to analyse. Defeat is unjustifiable," the Argentinian said.
"Any explanation will not prevent us to be responsible. It makes no sense to review the game.
"Merit is one of the parts of the game. There is no way to explain a defeat like this."
Bielsa went on to claim Marseille's loss was his fault.
"No, the blame is for me because of the way we played," he said.
"The differences between the two teams were very large. I led the team that had all the advantages to win. The fact we did not win gives me more responsibility than my players."
Marseille captain Andre Ayew, who scored the visitors' extra-time goal before Selim Bengriba bobbed up in the 121st minute to head home Grenoble's third equaliser, was also furious with his team's performance.
"We have not done enough. This is unacceptable, unacceptable," he said.
"We carry the lead three times … I must think of something else and focus on the championship."
'Obviously Messi is the best player in history and being compared to him means that you're doing things right, but I try to be myself': Barcelona star responds to comparisons with Argentine legend
'Darwin Nunez would be gold for Arsenal': Liverpool legend claims Gunners should have signed Uruguayan striker in January
