Right-back Clyne fired home from 30 yards out five minutes before half-time to ensure that Ronald Koeman's impressive start to his Southampton tenure continued with a third-round victory over a much-changed Arsenal side.
Alexis Sanchez had put Arsenal in front with a stunning free-kick, but Arsene Wenger's wait to win this competition for the first time goes on after the visitors mounted a fightback for a win which was no more than they deserved.
Dusan Tadic equalised with his first goal for the club from the penalty spot after Tomas Rosicky had brought down the impressive Sadio Mane and Clyne's thunderbolt put the south-coast club through.
Wenger made 10 changes to the side that beat Aston Villa in the Premier League on Saturday, with Abou Diaby making his first Arsenal start in 19 months after recovering from a knee injury, while goalkeeper David Ospina was handed a debut following his close-season move from Nice and Calum Chambers faced his former club.
Mane came in for his Southampton debut as one of five changes to the side that won at Swansea City at the weekend.
Steven Davis had a chance to mark his recall with a goal in the ninth minute, but Ospina reacted quickly to narrow the angle and thwart the midfielder after Tadic had picked him out with a lovely throughball.
It was the home side who were in front five minutes later, though, courtesy of a moment of class from Alexis.
The big-money signing from Barcelona lined up a free-kick 25 yards from goal and Fraser Forster could only watch on as the ball curled over the wall and into the net.
Southampton looked dangerous going forward and it came as no surprise when they pegged Arsenal bag after 20 minutes, when Tadic calmly slotted home a penalty after Rosicky had upended the lively Mane.
The equaliser was fully justified for the visitors and they were in front five minutes before half-time, as Clyne let fly with a venomous right-footed strike from 30 yards out which Ospina failed to keep out after a Tadic corner had been cleared into the defender's path.
Forster then got down well to keep out a Lukas Podolski strike at his near post to ensure Koeman's side led at the break.
Southampton continued to cause problems for the Arsenal defence and they could have extended their lead immediately after the restart, but Davis failed to get a clean contact on his shot after a lovely move.
Tadic was a constant menace to the home side and Ospina had to be alert to palm away the Serbian's left-footed free-kick midway through the second half.
Diaby was replaced by Santi Cazorla after 66 minutes as Wenger attempted to turn the tide and they were almost level soon afterwards, when Alexis fired straight at Forster from six yards out.
Alex Oxlade-Chamberlain also came on to face his former club, but Southampton stood firm.
Graziano Pelle ought to have sealed it four minutes from time when he headed Davis' corner over the crossbar, but Arsenal were unable to make the striker pay for that miss.
