The Italian signed a two-year deal to replace Fernando Santos in July, but has seen his reign ended after only four months in charge.
Ranieri's position had already been called into question after Euro 2016 qualifying defeats to Romania and Northern Ireland, as well as a 1-1 draw in Finland.
And a humiliating defeat against the Faroes, ranked 187th in the world, at the Karaiskakis Stadium proved to be the final straw.
Greece Football Federation president Giorgos Sarris said in a statement released on Saturday: "Following this devastating result for the national team, I take full responsibility for the most unfortunate choice of coach, which has resulted in such a poor image of the national team being put before the fans.
"The governing board will convene and take responsibility among all the members to make the necessary changes needed to avoid a repeat of such an embarrassing night."
Greece are languishing at the bottom of Euro 2016 qualifying Group F following their shock defeat to the Faroes.
