Enyeama saved Zambia defender Thomas Nyirenda's attempt, then kept his composure to slot past opposite number Kennedy Mweene. The quarter-final finished goalless after extra time.
Nigeria will take on Ghana in Luanda on Thursday in a clash between two World Cup-bound teams while holders Egypt play Algeria in Benguela.
Nigeria were reduced to 10 men in extra time after Onyekachi Apam was sent off in the 106th minute for a second booking, but held on against a dogged Zambian team who belied their lowly world ranking.
Clear-cut chances were in short supply throughout.
Nigeria's John Obi Mikel forced Mweene to stretch down to his left and parry away his 20-metre shot on the half hour.
Zambia had a chance shortly before the break when Emmanuel Mbola rushed down the left flank and found the unmarked James Chamanga with a low cross, but the striker's shot went wide.
They came closer in the 65th when Hichani Himonde's header from a Chris Katongo corner was cleared off the line by Yusuf Mohammed.
Nigeria tried their luck on the break but Peter Odemwingie's runs were easily halted by the Zambia defence as the game headed into extra time.
Herve Renard's team, who advanced from a group featuring Cameroon, Gabon and Tunisia, failed to take advantage of Apam's sending off.
In the 108th minute, Noah Chivuta unleashed a 30-metre missile that was tipped over by Enyeama.
After that, Nigeria simply sit back and held on for the shootout.
'Obviously Messi is the best player in history and being compared to him means that you're doing things right, but I try to be myself': Barcelona star responds to comparisons with Argentine legend
'Darwin Nunez would be gold for Arsenal': Liverpool legend claims Gunners should have signed Uruguayan striker in January
