Van Gaal's men led 2-0 after 40 minutes in Genk thanks to goals from midfielder Rafael van der Vaart and winger Arjen Robben.
However, Japan produced an excellent comeback, attacker Yuya Osako pulling one back one minute before the interval before Keisuke Honda levelled the match on the hour mark.
And Van Gaal was quick to criticise his side for their collapse.
"We let Japan back into the game, and that was completely unnecessary," he said.
"If you end the first half at 2-0, you know you can play the same game after the break. During the second half we were not the better team. We lost every second ball.
"The team defended poorly. Therefore the centre-back pairing made mistakes. This makes sense.
"The decline was much too big. This should never happen to us."
‘Even though Kevin Keegan was my boyhood hero, we never really saw eye-to-eye. That was an underwhelming time for me – I felt on the outside’: Newcastle cult hero reveals disappointment over how things went under King Kev
Alexander Isak puts uncertainty over his Newcastle United future to bed amid fresh PSG and Arsenal links
