Sporting director Jose Anigo will take charge of the Ligue 1 team on a temporary basis until the winter break.
Baup earned plaudits after taking the reins from France boss Didier Deschamps and leading an unfancied Marseille to a second-placed finish last season but it has been a different story this time around.
They will bow out of the UEFA Champions League after Wednesday's match against Borussia Dortmund having failed to collect a single point so far in this season’s competition, while Friday’s loss was their fourth at home this term and saw them tumble to fifth in the Ligue 1 table, 13 points off table-toppers Paris Saint-Germain.
"It's the most painful and difficult decision I have had to make since I joined the club," said Marseille’s executive president Vincent Labrune.
"Elie Baup was unanimously appreciated for his human qualities. No one could forget the effort he gave at Olympique de Marseille.
"This season though, he didn't get the expected results. A psychological shock was needed for us to revive our ambitions.
"The team has not shown the quality and determination expected, especially for a club like Olympique de Marseille."
Former Marseille player Anigo has previously enjoyed a spell in the dugout at the Stade Veledrome, overseeing a run to the 2004 UEFA Cup final.
