The 25-year-old Atletico Madrid striker caused controversy this week when he voiced his desire to play for Spain instead of the country of his birth.
However Scolari, who is preparing his Brazil team for friendlies against Honduras and Chile, denied he was upset by the striker's choice, as had been widely reported.
"I ask you not to put words in my mouth because I've already seen a lot of people saying a lot of words that I never said," Scolari said.
"I said (Costa's) not selected and that's my only job.
"Now, if I express while smiling, laughing or frowning or whatever, it is just how I look.
"And you can't judge how I look - you can only judge my words."
Brazil travel to Florida for their match against Honduras on November 16 before going to Toronto to play Chile four days later.
The friendlies are a chance for Scolari to test out players he will be selecting for the World Cup in Brazil next year.
"We have to give it our all because the opposition have nothing to lose," he said.
"We're going to keep playing within our spirit, the spirit the players have, and think that this game is worth three points."
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
