Southampton launched a complaint against the official following the 2-1 defeat to Everton at Goodison Park on December 29.
An initial complaint was dismissed by the Professional Game Match Officials Ltd. (PGMOL), but the Premier League club were not content with that decision and took the case to the FA, seeking an apology from Clattenburg.
But the governing body supported PGMOL's stance by confirming that Clattenburg had not breached their code of conduct.
A statement on the FA website read: "The FA can confirm it received a complaint from Southampton concerning referee Mark Clattenburg and his alleged comments made to their player Adam Lallana.
"The FA has given its full consideration to this matter and it is our view that the comments, as disclosed in Southampton's letter of complaint, do not constitute misconduct under The FA's Rules.
"Consequently, The FA will not be taking any further action in relation to this matter and now considers this case to be closed."
