Sami Hyypia put out a first-choice XI at the BayArena, but the 2. Bundesliga promotion chasers held firm, forcing extra time - in which Ruben Yttergard Jenssen struck to seal the shock of the round.
The visitors were on the back foot for much of the contest, but they were resolute as Leverkusen failed to conjure up anything impressive enough to break their lower-ranked opponents down.
As extra time loomed, Kaiserslautern found their threat on the counter and were awarded a penalty as Leverkusen debutant Andres Guardado brought Karim Matmour down, but Mohammadou Idrissou's weak effort was easily saved by Bernd Leno.
The Cameroonian had a hand in the winning goal, however - his cross landing at the feet of Yttergard Jenssen, who thrashed home.
Bayern displayed their dominance over German football once again with a 5-0 thrashing of out-of-sorts Hamburg at the Imtech Arena.
Mario Mandzukic grabbed a hat-trick, while Pep Guardiola will have been further boosted by the return of Bastian Schweinsteiger, who made a comeback to action after four months out with an ankle injury.
Mario Gotze gave Mandzukic the simple task of opening the scoring from close-range 21 minutes in, with Dante thundering home a Toni Kroos corner just four minutes later to put Bayern two up.
Kroos was again heavily involved in the visitors' third, dinking a neat ball over Hamburg's defence that Arjen Robben raced on to before finishing.
Mandzukic's second was again provided by Gotze - whose cross-cum-shot found the Croatian in space.
The striker's hat-trick - and the scoring in the tie - was wrapped up with 15 minutes to go as sloppy home defending gave him the chance to smash home.
As well as sending Bayern through, the result will crank the pressure up on Hamburg coach Bert van Marwijk, with club officials being forced to deny reports that he could be sacked as they sit second from bottom in the Bundesliga.
Wolfsburg made the next round with a 3-2 win at Hoffenheim, who were the architects of their own downfall.
Ricardo Rodriguez scored twice - both from the penalty spot - to sandwich Roberto Firmino's first-half equaliser.
Slack marking allowed Bas Dost to prod home on the hour, with Firmino's consolation in stoppage time counting for little.
Borussia Dortmund are the final side in the semi-final draw, having beaten Eintracht Frankfurt on Tuesday.
