Reacting to questions in the light of John Terry's four-match ban for racially abusing Anton Ferdinand and the ugly events during and after England's under-21 match in Serbia on Tuesday, Wenger said skin colour was not the only issue.
"It is not only racism, black and white, it is against all kinds of insults we still have in the stadiums. We must fight more against it," the Frenchman, who has suffered his fair share of taunts over the years, told reporters.
"You look at some faces when you walk around the pitch, what they shout at you is scary. That is, for me, racism."
England defender Danny Rose said he was racially abused and pelted with stones in the Serbian city of Krusevac at Tuesday's match, prompting the British government to call for tough sanctions against Serbia.
Players will wear Kick It Out T-shirts at league matches this weekend, supporting a campaign to eradicate racism from English football - an issue which has been under the microscope in the wake of the Terry case and Liverpool's Luis Suarez serving an eight-match ban last season for racist insults.
Some black players have threatened not to support the campaign this weekend, among them Reading striker Jason Roberts, saying not enough is being done to punish offenders.
Roberts's stance has been criticised by Manchester United manager Sir Alex Ferguson and Wenger also said that all players should be unified in their support of the Kick It Out campaign.
"I feel that [black players] are the main targets and if they do not join in it makes the whole thing not efficient and not credible. We need all to fight together against that," Wenger said. "They have faced the most abuse, so I think it is important they are on board.
"If they feel the punishments are not hard enough they have to express that in a different way but I think that it would be sad if they do not join in," he added.
Ferguson said Roberts's threatened protest was the wrong way to get his message across.
"I don't know what point he is trying to make," Ferguson told reporters at his Friday news conference.
"I don't know if he is trying to put himself on a different pedestal from everyone but he really should be supporting all the rest of the players who are doing it."
Chelsea captain Terry decided on Thursday not to appeal against a four-match FA ban after being found guilty of racially insulting Ferdinand in a Premier League match last October.
