After a disappointing draw with the Republic of Ireland last weekend, the 2006 world champions were again far from impressive as the European minnows grabbed a late equaliser.
They looked to be comfortable when Claudio Marchisio opened the scoring in the ninth minute, but Italy promptly wasted a host of chances to extend their lead.
And they were punished with five minutes to play when Maxime Chanot rose to head home a Mario Mutsch corner, earning Luxembourg a shock draw.
Cesare Prandelli made eight changes to the side that was held by the Republic of Ireland on Saturday, with Gianluigi Buffon returning in goal and Mario Balotelli starting up front.
And it was the Milan striker who created the opening goal in the ninth minute.
Having drifted to the right, Balotelli delivered a perfect cross into the penalty area where Marchisio headed home from close range after getting in front of his marker.
Rather than use the momentum of that early goal, Italy sat back and the crowd had to wait until the 28th minute for the next chance.
Marco Verratti found enough room on the edge of the Luxembourg penalty area to play a through ball to Balotelli, the 23-year-old timed his run perfectly but fired straight at the goalkeeper from five yards.
Antonio Candreva and Andrea Pirlo both went close to doubling the lead in the closing five minutes of the half but Italy had to settle for a one-goal lead.
The introduction of Antonio Cassano in 54th minute breathed new life into the match and within two minutes of coming on, he had created a chance for Balotelli, but the Milan forward struck the crossbar from 12 yards.
Worryingly for Prandelli, the visitors were able to create some good openings after the hour mark, but Aurelien Joachim was unable to meet crosses from Mathias Janisch and Dwayn Holter.
With five minutes remaining the minnows did shock their hosts, though, as Chanot rose above Alberto Aquilani to head beyond Buffon.
Italy now travel to Brazil for the World Cup, with their opening game of the tournament coming against England on June 14 - and Prandelli will know there is much work to do.
