The game was abandoned after less than half an hour.
"Milan left the pitch following racist insults on the part of a minority of the Pro Patria fans directed at [Kevin-Prince] Boateng, [M'Baye] Niang, [Urby] Emanuelson and [Sulley] Muntari," the club said on their website.
A video posted on YouTube showed Boateng in possession near the penalty area when jeering was heard from fans nearby.
He stopped playing, collected the ball and kicked it towards the stands. He then took his shirt off and walked off the pitch, followed by his team-mates.
"I'm disappointed and saddened but I think that it was the right choice to not return to the pitch, out of respect to our players and all other coloured players in any other league," Milan coach Massimiliano Allegri told reporters.
"We have to stop these uncivilised gestures. Italy has to become a little more civilised and a little more intelligent.
"It's unfortunate because of the families and children who had come here to enjoy the beautiful day of sport and sun.
"We hope we have given a signal that will be followed if such acts occur again in any league. We apologise but we hope that these uncivilised acts will not happen again."
Italian football federation (FIGC) president Giancarlo Abete said the incident was "unspeakable and intolerable".
"We must react with force and without silence to isolate the few criminals that transformed a friendly match into an uproar that offends all of Italian football," he said on the federation's website (www.ficg.it).
Captain Massimo Ambrosini said the jeering had started before the walkoff.
"We were annoyed from the beginning," he said. "We gave a strong signal, although we regret it because a few have spoiled it for everyone else."
Based in the town of Busto Arsizio, Pro Patria play in the fourth tier of Italian football.
AC Milan director Umberto Gandini said on Twitter: "Very proud of the Milan players who decided to walk off the pitch today for racist abuse from few idiots! No racism, no stupidity."
'People will remember us because we got to No.1 in the world rankings and achieved great things - but our biggest regret is that we didn’t win anything': Ex-Everton star Kevin Mirallas opens up on 'honour' of being part of nation's golden generation
‘I’m sure we’d have finished in the top four or five if he hadn’t joined Chelsea in January. Him leaving damaged the team, but we couldn’t stand in his way’: Alan Curbishley reflects on failed Champions League dream after selling key player
