The former Real Madrid coach, who also had a one-game spell as manager of Wales in 1994, announced his decision at a televised news conference in Cardiff.
GEAR:Retro Wales tops available here, with a 10% discount courtesy of Kitbag
Toshack, 61, had been at the centre of speculation about his future after Wales lost their opening Euro 2012 Group G qualifier in Montenegro last week and said on Thursday he was standing down with immediate effect.
He said he had been "a little disappointed with one or two things" after a May friendly defeat in Croatia but initially wanted to oversee the first three Euro qualifiers before reviewing the situation.
"I discussed things with the president after the Montenegro game, I gave my point of view and he gave his, and we came to the agreement that it may be better for everybody concerned if a change is made now," he said.
"As regards the next two games I think Wales are still very much in this group. Now the decision has been made, hopefully they will be able to get some positive results in the next two games.
"Although there is not too much time for the new man I still think the group is open and I will be there just like every other fan at Cardiff's new stadium rooting for them to get a result."
Wales next host Bulgaria on October 10 and visit Switzerland four days later in Group G. They host England in March.
Toshack, a prolific striker with Liverpool and Wales in his playing days from the late 1960s into the 1980s, made his name as a manager by taking Swansea City on an incredible run from the fourth division to the old first division in four years.
He went on to manage all over Europe, including two spells at Real Madrid and three at Real Sociedad.
His first spell as Wales boss though lasted only one game in 1994 and though he had some promising moments during his second after succeeding Mark Hughes, he never came close to getting his country into their first major tournament finals since 1958.
Toshack was not helped by the premature international retirement of some of Wales's leading players, most notably Ryan Giggs, who continues to ply his trade with aplomb at Manchester United while being touted as a future Wales manager.
