The Young Boys Berne defender was forced to come off just nine minutes into his country's 5-2 Group E thrashing at Arena Fonte Nova after being accidentally kicked in the face by France striker Olivier Giroud.
Von Bergen, who suffered a nasty cut just under his left eye, was taken to hospital in Salvador and it was confirmed by the Swiss Football Federation on Saturday that his tournament is over due to a fractured eye socket.
The 31-year-old will fly home from Brazil and must watch from afar as Switzerland attempt to put their hammering at the hands of France behind them and qualify for the last 16.
Ottmar Hitzfeld's side face Honduras in their final group game next Wednesday, with second-placed Ecuador - level on points with Switzerland, but with a superior goal difference - up against group leaders France.
