The Sweden international was shown a second yellow card in the 60th minute of Sunday's 1-0 defeat at Molineux, after hauling down Rajiv van La Parra as the winger sped towards the right-hand side of the area.
As referee Simon Hooper brandished the card in Olsson's direction, the defender laid a hand on the official and gave him a slight shove, before walking from the field visibly angry.
On Monday, the FA issued a statement announcing that Olsson had been charged.
"Norwich City's Martin Olsson has been charged by the FA in relation to an incident which occurred during the game against Wolverhampton Wanderers on Sunday August 10, 2014," it read.
"It is alleged that Olsson's action in making physical contact with the referee immediately prior to being shown a red card and for his behaviour as he left the field of play amounted to improper conduct, which occurred outside the jurisdiction of the match officials.
"Olsson has until 6pm on August 14, 2014 to respond to the charge."
'You couldn’t blame Trent Alexander-Arnold if he wanted to play for Real Madrid – I hope he stays, but even with Mo Salah, I would understand if he decided to leave': Former Liverpool defender explains why he’s braced for key Anfield exits
‘Sven asked if I’d like to play for England – even to be considered made me proud, but I don’t think they really wanted a foreign player in the national team’: Former Premier League star reveals conversations over converting nationality
