A 63rd-minute spot-kick from Jordi Gomez put the Championship side ahead after Mertesacker fouled Callum McManaman but the Germany international pulled his team level with eight minutes of normal time remaining.
Alex Oxlade-Chamberlain – Arsenal's major attacking threat throughout – rattled the crossbar during extra time but Arsene Wenger's men would ultimately hold their nerve to keep hopes of a first major trophy in nine seasons alive.
Patrick Vieira famously sealed victory over Manchester United on penalties in the 2005 FA Cup final and on this occasion the decisive contribution came from Lukasz Fabianski.
Arsenal's back-up goalkeeper saved the first two Wigan penalties from Gary Caldwell and Jack Collison, while his team-mates Mikel Arteta, Kim Kallstrom and Olivier Giroud were all on target.
It was left to Santi Cazorla to hit the winning penalty, as Arsenal sealed their spot in the May 17 showpiece against either Hull City or Sheffield United, who contest the second semi-final on Sunday.
Influential Arsenal midfielder Aaron Ramsey made his first start since suffering a thigh injury in December, while Wigan made seven changes.
McManaman – man-of-the-match in last season's final – was one of Wigan's inclusions, with manager Uwe Rosler having overseen a 1-0 loss at Millwall on Tuesday with one eye on this showpiece occasion.
Oxlade-Chamberlain was Arsenal's leading light during the opening exchanges and it required a fine save from Wigan goalkeeper Scott Carson to deny Yaya Sanogo – preferred up front to Giroud - from the England man's sixth-minute cross.
Arsenal struggled for rhythm and McManaman was again impressing at Wembley, shooting over from an acute angle after a driving run.
Lukas Podolski curled a free-kick wide and Carson was quickly off his line to thwart Sanogo as Arsenal belatedly lifted a listless display.
Wigan were on their way to another incredible FA Cup win when Mertesacker brought down McManaman inside the penalty area before the hour-mark.
An injury sustained by Arsenal full-back Nacho Monreal as the winger bustled his way towards goal resulted in a four-minute delay, but Gomez kept his cool to dispatch the spot-kick impeccably.
Sanogo then prodded over from a Ramsey cross as Wenger's men sought a response, and the French forward was on the receiving end of a superb last-ditch tackle from Stephen Crainey shortly after.
Arsenal ramped up the pressure as Sagna headed against the post from Oxlade-Chamberlain's centre, while Crainey desperately cleared when a header from Kieran Gibbs - on for Monreal - threatened to squirm beyond Carson.
And they finally levelled proceedings in the 82nd minute when Mertesacker turned the ball home following a bout of set-piece pressure.
Wigan substitute Nick Powell drilled a speculative effort off target as extra time began but Arsenal remained on the front foot.
Sanogo displayed fine technique to collect Oxlade-Chamberlain's 97th-minute pass and get a shot away but Carson was equal to the task.
Oxlade-Chamberlain was rarely far away from Arsenal's best moments and thudded a drive against the angle of post and bar with nine minutes of extra time remaining, while Collison wasted a header at the other end.
Unfortunately for West Ham loanee Collison, he would soon suffer greater disappointment in the shootout, rendering the spot-kicks put away by Jean Beausejour and James McArthur meaningless.
