Levein, a former international who became manager in December 2009, has departed with Scotland bottom of their 2014 World Cup qualifying group.
Scotland have collected two points from four matches after opening draws with Serbia and Macedonia before defeats in Wales and Belgium last month.
"The Board of the Scottish FA has taken the decision primarily due to the disappointing results in the FIFA World Cup Brazil 2014 Qualifiers," the Scottish FA said in a statement.
Levein oversaw three wins in 12 competitive fixtures in charge of Scotland who are seeking to qualify for a first major tournament since the 1998 World Cup.
Scottish FA chief executive Stewart Regan said Levein had paid the price for the increasing importance of results in modern-day football and Scotland's drop in the world rankings.
"We are taking this decision with real sadness. Craig has worked hard to bring success, and has been thoroughly professional in his approach to the job," said Regan.
"However, he would be the first to agree that football is a results-driven business. The challenge for the squad now is to improve our position in Group A and, subsequently, improve our position in the FIFA rankings."
Regan said the Scottish FA would honour Levein's contract, due to run until the end of the 2014 World Cup qualifying campaign next October.
Levein, who won 16 caps for Scotland during a playing career spent mostly at Hearts, took charge of the national side when they were 46th in the rankings but they now lie 10 places lower.
Under-21 coach Billy Stark will manage the team for the November 14 friendly in Luxembourg.
