Stanislav Manolev had given the hosts the lead before Osvaldo's double put Italy ahead and it was left to Bulgaria's lively midfielder Milanov to earn a share of the points with a goal midway through the second half of the Group B clash.
"I didn't expect Bulgaria to play like this but you see that all teams have a difficult periods and Italy are no exception," said Italy coach Cesare Prandelli.
Bulgaria coach Luboslav Penev said: "We totally outplayed Italy, one of the best teams in the world, not [just] in Europe but they scored two lucky goals.
Expressing his frustration with the decisions of English referee Martin Atkinson he added: "There were three or four handballs in their penalty area. It was very difficult for the referee to award us free-kicks."
The Bulgarians, who failed to score in their four home Euro 2012 qualifiers, almost went ahead when defender Ivan Ivanov's curling shot from Emil Gargorov's free-kick forced a spectacular save by Italy goalkeeper Gianluigi Buffon.
The hosts continued to press forward and took a deserved lead when Manolev gave Buffon no chance with a stunning 30-metre drive on the half-hour at the half-empty Vasil Levski stadium.
But Italy, who switched to a 3-5-2 formation, responded with Argentine-born striker Osvaldo scoring twice before half-time.
The AS Roma player, who was not part of the Italy squad at Euro 2012 where they lost in the final, netted from close range in the 36th minute from the unmarked Claudio Marchisio's clever assist to level the scores with Italy's first shot on goal.
MEMORABLE EVENING
Osvaldo, who got his chance with Antonio Cassano off form and Mario Balotelli suffering from an eye complaint, gave the Azzurri the lead when his header deflected off Ivan Ivanov to leave keeper Nikolay Mihaylov stranded before the break.
Bulgaria captain Ivelin Popov blasted a diagonal shot just wide two minutes before half-time while Mihaylov parried a thunderous Osvaldo drive from the edge of the area at the other end, denying the Italian his first international hat-trick.
"It's a memorable evening for me and I'm happy or at least half-happy with my two goals because the result was not satisfactory," said Osvaldo.
"I usually score beautiful goals while those were scruffy, but they are worth as much as the others.
"It was a difficult game, at times we were a bit slow and we didn't play the way we normally play."
The visitors began the second half with more confidence and Sebastian Giovinco forced another superb save from Mihaylov.
But the home side responded and Milanov made it 2-2 with a fine shot from 10 metres following a right-wing cross from substitute Iliyan Mitsanski after 66 minutes.
Bulgaria could have snatched victory in the 74th minute but Buffon pulled off a great save, instinctively palming a Manolev effort over the bar.
Italy held on after midfielder Daniele De Rossi, who was doubtful with an ankle injury before the game, limped off with 10 minut
