Sports Minister Aldo Rebelo told a news conference that Valcke's words on Friday had been "offensive" and "unacceptable."
"The government will no longer accept... [Valcke] as an interlocutor," Rebelo said. "The FIFA secretary's declarations... make it difficult to have an environment of cooperation and understanding."
His comments marked a further escalation of a dispute with FIFA that has simmered for years as several Brazilian stadiums, hotels, roads and other basic infrastructure for the 2014 Cup run badly behind schedule.
Valcke, who was due to visit Brazil in coming days, had said "not a lot is moving" in the country's preparations. He dismissed Rebelo's reaction as 'puerile'.
"Why doesn't he deal with the issue?," he replied when informed of the comments during a news conference in England where he was attending the annual meeting of FIFA's International Football Association Board.
"I'm fine. If that's the problem because nothing has happened over the five years the result of one remark I made, then, wow.
"One comment saying things are not working well, and I for once say exactly what is happening in Brazil. If the result is they don't want to talk to me any more, I'm not the guy they want to work with - that's a bit puerile."
BEHIND SCHEDULE
Stadiums in smaller Brazilian cities such as Porto Alegre and Cuiaba are notably behind schedule.
Brazil is experiencing skilled labour shortages due to its booming economy, which has made construction difficult. A lack of financial guarantees and constant legal disputes have also often caused building to be put on hold.
Brazil's Congress has angered FIFA by dragging its feet in passing laws relating to the sale of alcohol at the games, as well as which groups will receive discounted tickets.
Rebelo said in separate comments on Friday that Brazil's government still wants native Indians and low-income families to be able to afford the games, accusing FIFA of wanting only to make a profit.
Making matters more complex, the disputes have come at a time of turmoil for Brazilian football. The head of the country's football confederation, Ricardo Teixeira, faces corruption allegations and constant rumors that he may be forced out.
The stakes for Brazil are enormous. President Dilma Rousseff has made public-works projects for the Cup a personal priority, and Olympic officials are also watching Brazil's progress closely ahead of the 2016 Games in Rio de Janeiro.
Valcke said Brazil should have passed the World Cup Bill in 2007 as part of guarantees given by the government for being awarded the tournament.
"We have compromised on a number of things such as ticket price discounts for elderly people and a number of things on the wording of these guarantees but some are key," he said.
"There is no way we cannot receive a guarantee on security because action against terrorism is a responsibility of the state: We have no leverage on this so this World Cup
