Gundogan's shot from 15 metres hit the post and bounced off the back of substitute goalkeeper Jasmin Fejzic and into the net in the 120th minute to break Fuerth's hearts.
Fejzic had only been on the pitch two minutes having been brought on ahead of what the hosts hoped would be penalties.
In the final Dortmund, who are chasing a domestic double and lead the Bundesliga title race by five points, will face Bayern Munich or Borussia Monchengladbach, who meet in the other semi on Wednesday.
"This is an absolute dream, especially after the way the game turned out today," said Dortmund coach Jurgen Klopp. "It was extremely difficult. Fuerth were amazing, very dangerous and then we go and score such a goal."
Second division leaders Fuerth, who had not conceded a goal in the competition until Tuesday, had started well and Bundesliga champions Dortmund had to wait 16 minutes before creating their first chance with a Sebastian Kehl header.
Dortmund dominated most of the game but failed to take their chances against their determined opponents.
Japan international Shinji Kagawa should have given the visitors the lead after 30 minutes but his header from point-blank range flew over the bar.
Kevin Grosskreutz also came close 10 minutes after the break but his bullet header was palmed away by keeper Max Gruen.
Fuerth coach Mike Bueskens, a former player and coach at Dortmund's bitter rivals Schalke 04, fired his players up from the sidelines and saw Canadian Olivier Occean threaten twice.
But he was left dejected when Gundogan scored seconds from time to send the visitors to their first cup final since 2008.
"Jasmin is outstanding at penalties in training and we decided to bring him on," Bueskens told reporters. "It was such a bitter way to go out. The keeper was unlucky, could do nothing with the way the ball bounced."
