"Our approach and that of Joern Andersen did not agree anymore as the coach was developing in a different direction," club manager Christian Heidel told reporters.
The club also sacked assistant coach Juergen Kramny and appointed under-19 coach Thomas Tuchel as Andersen's replacement.
Andersen joined Mainz in 2008, steering them to promotion in his first season in charge.
"I regret the decision of the club but our visions were different," Andersen told reporters.
Mainz lost 2-1 to lowly Luebeck and were eliminated in the first round of the Cup.
‘Chelsea have to qualify for the Champions League for this season to be a success – they showed that they have the players to perform at a high level, they need a win quickly now’ Blues legend’s fears over recent bad run
'I’d probably pick Cole Palmer alongside Bellingham and Rice in there at the moment. For me, Declan Rice is good enough to play that central midfield role on his own' Thomas Tuchel advised to DROP Manchester City star by England legend
