Former Chelsea player Paul Elliott, the club's first black captain, also cast doubt on the club's claims that have put racism on the back pages again.
Chelsea lodged an official complaint to the Premier League season after last Sunday's 3-2 home defeat by United, saying Clattenburg, who sent off two of their players in, had used "inappropriate language" to two of their players.
Later in the week they issued another statement saying they did not have evidence to support a second complaint of verbal abuse but said they had lodged a complaint with the Football Association regarding comments to Nigerian John Obi Mikel.
The Football Association, which has had to deal with high-profile racism cases involving Chelsea's former England skipper John Terry and Liverpool striker Luis Suarez this year, has opened an investigation into the claims.
Ferguson, however, refuses to believe Chelsea's version of events.
"I don't believe Mark Clattenburg would make comments like that, I refuse to believe it," Ferguson told reporters at his Friday news conference ahead of Saturday's Premier League clash at home to Arsenal.
"I think it's unthinkable in the modern climate and I just don't believe it, there is no way that a referee would stoop to that. I'm convinced of that.
"I think in the modern game its completely changed. I know myself that the banter that went on between players and the referee 25 years ago is different than today.
"I've never heard of a player coming to me in the last 15 years saying the referee swore in a game. That's where I stand, I don't believe it."
Talking to Reuters, Elliott, an ambassador for the anti-racism charity Kick it Out, said: "To me, what happened at Chelsea last Sunday, seemed like a car out of control, I thought with the John Terry thing, this had stopped.
"I thought we were moving forward but its like we've been hit by a two-footed challenge from the side and you've not seen it coming.
"Mark Clattenburg is a very capable and competent referee and I would be astonished if an authority figure in our game like him, an elite referee, could be found guilty of racism.
"I think we have to keep it in the balance and say "they are nothing more than allegations at the moment. There has to be a robust and transparent process to gather the information. Football does not need this at the moment.
"Our referees, in general terms are among the best in the world. And Mark Clattenburg has a fundamental right to defend those allegations."
FIFA referee Clattenburg, who will not be officiating at any Premier League games this weekend in the wake of Chelsea's claims, was at the centre of a tumultuous clash at Stamford Bridge last week.
He sent off Chelsea's Branislav Ivanovic and then showed a red card to the home side's Fernando Torres after believing the Spaniard had dived to try and win a free kick.
He later allowed United's 75th-minute winner, scored by Javier Hernandez, despite it looking suspiciously offside.
Some Chelsea players and
