Arsene Wenger's team had failed to collect a Premier League win on their own turf since the opening day against Crystal Palace and they appeared set for another afternoon of frustration against the bottom side in the division.
Burnley winger George Boyd was fortunate not to concede a penalty before half-time, while Alexis – who scored both goals in the 2-0 triumph at Sunderland - drew two fine saves from Tom Heaton.
But it was the former Barcelona man who sparked relief around the Emirates Stadium in the 70th minute.
Impressive right-back Calum Chambers crossed for Alexis to score with a header, and the teenager added a second himself two minutes later with his first goal in senior football.
Alexis fashioned space from Kieran Gibbs' low centre to net a 10th goal in 12 appearances in the final minute and add gloss to the scoreline, as Arsenal moved above West Ham into fourth, while Burnley are three points adrift at the bottom and five from safety.
Arsenal were unchanged from the team that beat Sunderland last weekend, while Burnley recalled midfielder Dean Marney following a hamstring injury and striker Marvin Sordell.
The hosts dominated possession but made little impression on Burnley until Kieran Tripper produced a vital clearing header to deny Danny Welbeck in the 21st minute after Alex Oxlade-Chamberlain had raced away down the Burnley left.
From the resulting corner, Boyd – already booked for an earlier foul on Santi Cazorla - flicked the ball with a stray hand but the officials remained unmoved.
Arsenal found a way through down the right again in the 26th minute, but Cazorla smashed a shot wide from Oxlade-Chamberlain's pullback after good work from Alexis.
The visitors enjoyed a rare foray into the opposition penalty area five minutes later, only for Mathieu Flamini to make an important interception by prodding Scott Arfield's pass away from Danny Ings.
Alexis then twice went close, first cutting in from the left and forcing Heaton to push away his rasping drive before arching another 25-yard effort narrowly past the right-hand post.
Heaton proved Alexis' equal once more when the livewire forward raced into the box five minutes before the interval, as Burnley managed to reach the break unscathed.
Ings was unable to capitalise on a moment of confusion between Arsenal goalkeeper Wojciech Szczesny and Per Mertesacker in the 50th minute as Sean Dyche's men made a more assertive start to the second period.
The right flank remained fruitful for Arsenal but Alexis and Gibbs were unable to scramble the ball home from Oxlade-Chamberlain's latest cross.
Michael Duff produced some remarkable defensive heroics for the visitors as he flung himself to ground to deny Cazorla with the goal gaping after Chambers' cutback.
The pressure eventually told, though - almost inevitably from the Burnley left - as Alexis got ahead of Trippier to net a fine header from Chambers' cross.
The creator then became the goalscorer two minutes later, with Chambers on hand to steer home after a combination of Heaton and David Jones had denied Welbeck.
The Arsenal faithful warmly welcomed back Theo Walcott from his lengthy injury absence soon afterwards and the England man and fellow substitute Lukas Podolski drew magnificent saves from Heaton in the closing minutes as Burnley’s resistance faded, allowing Alexis to have the final word with a third goal.
