Marseille's two late goals in a 3-2 victory at Borussia Dortmund in the other final Group F game meant Didier Deschamps' team qualified for February's first knockout round and Olympiakos had to settle for third place and the Europa League.
The Greeks ended the group stage on nine points, just one behind the French. Arsenal had secured top spot and progression before Tuesday's game.
"We're out of the Champions League but we're proud of the way we played today," said Olympiakos coach Ernesto Valverde.
The Spaniard, as well as visiting manager Arsene Wenger, identified Olympiakos' first group game which ended in a 1-0 home defeat by Marseille as the main reason the Greeks failed to clinch a rare qualification to the knockout stage.
Olympiakos had not played a competitive game before facing the French side in September because the Greek league had not started following legal rows.
"Our first defeat at home to Marseille cost us a lot," said Valverde. "Football is about doing the right things at the right time.
"Despite losing our first two games, we managed to push for qualification right up to the last game, so that's a success for us. Now we'll see how far we can go in the Europa League."
FAILED CLEARANCE
After a bright opening by both sides, Olympiakos took the lead in the 16th minute when the under-strength Arsenal defence failed to cut out a through ball from David Fuster and Rafik Djebbour latched on to the pass to round Lukasz Fabianski and score.
Fabianski was injured a few minutes later in a collision with Djebbour and Vito Mannone took over in goal but was soon picking the ball out of the net after his failed clearance in the 36th minute landed at the feet of Fuster, whose weak shot trickled over the line.
Arsenal reduced the arrears in the second half when Marouane Chamakh chested the ball down for Yossi Benayoun to fire in a volley in the 57th minute.
The weakened Londoners were unable to find a second goal against the injury-ravaged Greeks and Olympiakos clinched a third in the 89th minute when Francois Modesto scrambled home from a corner.
It seemed like the Greeks were going through until news came from Dortmund, leaving Olympiakos players shaking their heads in a suddenly quiet stadium.
They will now have to settle for a place in the Europa League last 32 with the draw due on December 16.
"I thought overall Olympiakos deserved to win the game but, unfortunately for them, it's not enough because they will not qualify for the Champions League. Marseille made a super result," Arsenal manager Wenger told reporters.
"They missed the first game at home against Marseille and it's very difficult to get that back. Olympiakos deserved a better place. They paid a heavy price for not starting the championship before their first game.
"Offensively and defensively, we were not convincing tonight. It was a fantastic experience [for the
