Chelsea suspended five fans from attending games at Stamford Bridge following their involvement in alleged racial abuse on the Paris Metro last week.
Tottenham then called for strong action to be taken after footage of West Ham fans signing an anti-Semitic song emerged online following the 2-2 London derby draw at White Hart Lane.
Lord Ouseley is eager to see a strong presence of police and security at Wembley to "root out" fans looking to cause trouble.
He told reporters: "Clearly we want to see all authorities come together, to be prepared for Sunday's game at Wembley, to swamp the place with security and police, to root out those who are coming and looking for trouble, to be on those trains and stopping the chanting and to make sure we are going to rid football of that element.
"It is a game in which we always receive a large number of complaints from Jewish people that have been offended by the chanting, from both sides because they're equally offended by the songs that Spurs fans sing [containing the word 'yid'].
"We're caught in between of trying to educate fans about the sensitivity they must apply, because it is offensive to others, but we're equally trying to encourage opposing fans not to respond in an abusive way. It's hard and uphill but we've got to keep trying."
