Zola, who finished his playing career at the Stadio Saint'Elia, replaced the experienced Zdenek Zeman in December following a poor start to the season for the Sardinian outfit.
However, a 5-0 defeat in his first game in charge offered little hope of a quick turnaround, and it has now been decided that the club will look at alternative options.
Saturday's 2-0 loss at Sampdoria was the final straw and Zola's departure was confirmed on Monday with the club languishing four points from safety.
A statement read: "A decision made reluctantly against a great man who wrote unforgettable pages in the history of Cagliari, giving luster and international prestige to the land of Sardinia.
"He and his staff go with thanks for the work done in recent months with dedication, professionalism and passion, and the heartfelt good luck for the rest of his career."
Veteran Czech coach Zeman has been linked with a swift return to the club as Zola's successor.
‘He instantly popped into our ratings as one of the best U21 midfielders in Europe. The impressive thing is Slot has made him into more of a no.6' Inside Ryan Gravenberch's transformation at Liverpool
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
