After routing world champions Spain 5-1 in their opening clash, the Netherlands duly followed up with an action-packed 3-2 win in Porto Alegre on Wednesday.
Arjen Robben and Robin van Persie had starred in the Spain demolition, and the duo were once again pivotal figures, both notching their third goal of the tournament.
However, the Dutch did not have it all their own way, Tim Cahill's sensational volley and a penalty from Mile Jedinak threatening an upset.
Van Persie ultimately restored parity and, although Australia's Mathew Leckie missed the chance to put his side back in front from close-range, Memphis Depay's effort from distance - which found the net via a clanger from Australia goalkeeper Mat Ryan - proved decisive.
Van Gaal said: "I'm not relieved, because I believe in the second half we played better than Australia. In the first half they played better, but in the second half we had five, six, seven chances.
"I expected us to win. When (Leckie) missed that chance, I said to (assistant coach) Patrick Kluivert, 'We're going to win this match'.
"The way my players scored their goals was excellent, and it was very important for the spectators. And I don't like singling out players, but Nigel de Jong was inspirational."
‘He instantly popped into our ratings as one of the best U21 midfielders in Europe. The impressive thing is Slot has made him into more of a no.6' Inside Ryan Gravenberch's transformation at Liverpool
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
