Goals from Marco Fabian, Oribe Peralta and Javier Cortes, who had only been on the pitch two minutes when he scored, led to wild scenes of delight as Mexico clinched a place in the gold medal match back at Wembley.
Their next opponents Brazil swept South Korea aside 3-0 in front of 69,389 at Manchester United's Old Trafford stadium later on Tuesday.
Fabian summed up Mexico's feelings, saying: "This is an unforgettable moment and we are making history. But this is not over yet and we want to go for everything. We want to win the gold now."
The victory continues a trend started in 2005 when Mexico won the Under-17 World Cup and repeated that success last year.
With the Olympics being primarily an Under-23 competition with three overage players allowed, the future is looking bright for the country's next generation of footballers.
BRIGHT START
Japan, who beat Mexico in a friendly just before the Olympics, started brightly and went ahead with an arrowed shot from Yuki Otsu from outside the area after 12 minutes that gave Mexico skipper and keeper Jose Corona no chance of saving.
But their hopes of emulating the Japan's women's team, who reached their final by beating France at Wembley on Monday, faded as Mexico gradually took control.
The equaliser came in the 31st minute when Fabian nodded in a flicked-on corner from Giovani dos Santos from close range - the first goal Japan had conceded in five games at the tournament.
Dos Santos, usually Mexico's key playmaker with three goals at the London Games so far, was strangely off-key for once and, after missing three good scoring chances in the first half, was replaced at the interval.
Mexico, though, maintained control and, with Jorge Einriquez commanding in the middle, looked the better team and deservedly went ahead with a sizzling shot from Peralta after 65 minutes following a defensive mistake by Takihiro Ohgihara.
Substitute Cortes then completed the scoring with an angled shot after a solo run in the third minute of added time.
Japan coach Takashi Sekizuka must now lift his players for before the bronze medal playoff at Cardiff's Millennium Stadium on Friday against South Korea.
"I am very disappointed and even though we scored first we allowed Mexico to profit from our mistakes. We really wanted to go to the final, but now we will have to concentrate on the other medal," he said.
