Zola, who finished his playing career at the Stadio Saint'Elia, replaced the experienced Zdenek Zeman in December following a poor start to the season for the Sardinian outfit.
However, a 5-0 defeat in his first game in charge offered little hope of a quick turnaround, and it has now been decided that the club will look at alternative options.
Saturday's 2-0 loss at Sampdoria was the final straw and Zola's departure was confirmed on Monday with the club languishing four points from safety.
A statement read: "A decision made reluctantly against a great man who wrote unforgettable pages in the history of Cagliari, giving luster and international prestige to the land of Sardinia.
"He and his staff go with thanks for the work done in recent months with dedication, professionalism and passion, and the heartfelt good luck for the rest of his career."
Veteran Czech coach Zeman has been linked with a swift return to the club as Zola's successor.
Manchester United eyeing surprise move for AC Milan star for whom tension in Italy is growing: report
‘Even though Kevin Keegan was my boyhood hero, we never really saw eye-to-eye. That was an underwhelming time for me – I felt on the outside’: Newcastle cult hero reveals disappointment over how things went under King Kev
