With their fate already sealed after defeats to Argentina and Nigeria, Bosnia played with freedom and made history as goals to Edin Dzeko, Miralem Pjanic and Avdija Vrsajevic saw them collect their first points at the showpiece.
The result also ended Iran's stay in Brazil, as they finished last in Group F with just one point from three matches.
Carlos Queiroz's side needed to beat Bosnia and hope Argentina defeated Nigeria, but saw their slim chances of progression end in Salvador on Wednesday.
Dzeko broke the deadlock midway through the first half with a well-placed low strike from 25 yards, and Pjanic doubled their lead a minute before the hour.
Reza Ghoochannejad gave Iran a lifeline with eight minutes left, tapping in from close range, but Vrsajevic responded within 60 seconds for Bosnia, scoring an excellent angled finish.
The result means Nigeria – who were beaten 3-2 by Argentina – progress to the last 16 with Alejandro Sabella's side.
Bosnia coach Safet Susic rang the changes following their defeat to Nigeria, making five, as Vedad Ibisevic lined up alongside Dzeko in attack, while, in stark contrast, Iran were unchanged after their impressive, albeit fruitless, performance against Argentina.
Bosnia – who had been on the front foot since kick-off – had their pressure rewarded in the 23rd minute when Dzeko, who proved an early threat, broke his World Cup duck.
The Manchester City forward won the ball 35 yards from goal and charged forward, evading two challenges before firing a low left-footed strike past Iran goalkeeper Alireza Haghighi, into the inside of the post, and in.
Iran were denied an immediate response when Masoud Shojaei's deflected effort from just inside the penalty area hit the crossbar after good work from Ashkan Dejagah on the right.
Ghoochannejad was next to go close for a rejuvenated Iran, but was denied by Bosnia goalkeeper Asmir Begovic in the 31st minute, while after the break, Dejagah completely missed the ball when presented with a good chance to equalise.
And Pjanic made Iran pay with 59 minutes played, as he netted a composed finish from seven yards after Tino Susic found him with a perfect pass.
Iran gave Bosnia a scare though, when after failing to net in their first two games, they finally got themselves on the scoresheet.
Ghoochannejad was the man responsible, applying the finishing touches to Javad Nekounam's dangerous low cross.
It mattered little, though, as Bosnia sealed their victory when Vrsajevic fired in off the post to end a swift counter-attack.
