Some questionable refereeing in the tournament's opening game against Croatia, which the five-time champions won 3-1, led to suggestions that the governing body were giving preferential treatment.
And now Van Gaal has claimed that the schedule is unfair, with Brazil - first to play in each of the opening two rounds of games - concluding their group campaign after the Netherlands.
He said: "FIFA have advertisements at every match and publicity around the theme of 'Fair Play' and things like that.
"But FIFA play these tricks. It's not a good thing, of course. It's not fair play."
With the winner of Group B - which will be determined by the outcome of the Netherlands' meeting with Chile on Tuesday - set to face the side that finishes second in Group A in the last 16, Van Gaal fears his side's outcome could impact how Brazil approach their final first-round game later that day.
He added: "Well, we're going to focus on getting a victory and I don't think that will be affected by the fact that Brazil will be playing after us.
"I'm just assuming that they will do their sporting duty, but it's always a matter of discussion when they do that, so why on earth would FIFA create that situation?
"That's why you're asking that and it's a justified question."
