Bielsa's men led twice in regulation away to Grenoble, while the visitors also hit the front in extra time, before their fourth-tier hosts equalised in the 120th minute to force a penalty shoot-out.
Grenoble went on to win 5-4 in the shootout with Paul Cattier tipping Florian Thauvin's spot-kick over the bar.
While Grenoble's speed and tenacity pulled Marseille's defence apart on numerous occasions - the home side had 10 shots for the match - Bielsa argued "there is no way to explain a defeat like this".
"This is not a game to analyse. Defeat is unjustifiable," the Argentinian said.
"Any explanation will not prevent us to be responsible. It makes no sense to review the game.
"Merit is one of the parts of the game. There is no way to explain a defeat like this."
Bielsa went on to claim Marseille's loss was his fault.
"No, the blame is for me because of the way we played," he said.
"The differences between the two teams were very large. I led the team that had all the advantages to win. The fact we did not win gives me more responsibility than my players."
Marseille captain Andre Ayew, who scored the visitors' extra-time goal before Selim Bengriba bobbed up in the 121st minute to head home Grenoble's third equaliser, was also furious with his team's performance.
"We have not done enough. This is unacceptable, unacceptable," he said.
"We carry the lead three times … I must think of something else and focus on the championship."
‘Every time Trent Alexander-Arnold has a bad game now, questions are going to be asked about his commitment – it baffles me that Liverpool have let his contract run down’ Former Anfield star bemused by lack of resolution over full-back’s future
‘To go from League Two to scoring that goal for England in 10 years, you don’t get that without riding the wave perfectly. He’s optimised every opportunity’: Ollie Watkins' rise from Exeter City to England heroics relied on one crucial change
