The Swedish champions frustrated their hosts for an hour in Turin on Tuesday before a bit of magic from Kwadwo Asamoah and Carlos Tevez broke the deadlock.
Having received a pass from Tevez, Asamoah produced a sumptuous back heel to return the ball to his team-mate and the former Manchester City striker netted.
A second goal came in the final minute as Tevez again beat Robin Olsen, this time a perfectly-placed free-kick that the goalkeeper could do nothing about.
Hareide sang the praises of his hard-working team and expects them to keep on improving in their new surroundings.
"We did an hour of very good work," said the Malmo coach.
"We laid a good foundation the first 60 minutes, we can improve on it.
"The players were very disciplined.
"[But] there was a man on the pitch that became the difference. It was Carlos Tevez, we never got on top of him."
Malmo's first home game in Group A will see Olympiacos travel to Sweden on October 1.
