The Real Madrid midfielder picked up the problem in his right knee in a collision with Andrea Pirlo, and was forced from the field after 67 minutes.
Khedira’s involvement in next year’s FIFA World Cup is now in some doubt, though German team doctor Hans-Wilhelm Müller-Wohlfahrt is hopeful the 26-year-old could recover in time.
"Sami has to be treated surgically. We have the hope that he will start the World Cup," Müller-Wohlfahrt told the German Football Association (DFB) website.
And Madrid confirmed on their website that Khedira was set to undergo the procedure in Augsburg on Saturday "after the initial diagnosis of a break in the medial collateral ligament of his right knee".
Germany coach Joachim Low added: "This is a bitter blow for Sami and all of us. He is, on and off the field, a very great fighter and personality.
"He always thinks positive - this will help him, and so I am optimistic that he is, in time for the kick-off of the World Cup in Brazil, fit again.
"We wish him all the best and keep our fingers crossed that the surgery goes well and he can quickly be put into rehab."
The outlook had looked somewhat gloomy for Khedira when Jerome Boateng described the atmosphere in the dressing room as "sad" following the match, predicting that his team-mate had been "injured severely".
Khedira has made 15 appearances in all competitions for Madrid this season, scoring one goal.
Germany are scheduled to meet England in a friendly at Wembley on Tuesday.
‘He instantly popped into our ratings as one of the best U21 midfielders in Europe. The impressive thing is Slot has made him into more of a no.6' Inside Ryan Gravenberch's transformation at Liverpool
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
