Palace had won their home leg 1-0 thanks to an Anthony Gardner goal and it was the defender who levelled the score on aggregate with a headed own-goal after just seven minutes at the Cardiff City stadium.
The visitors were reduced to 10 men when captain Patrick McCarthy was sent off for a second yellow with 13 minutes remaining.
Extra-time failed to break the deadlock and the game went to penalties, despite Cardiff's Aron Gunnarsson hitting the bar in the final minute.
Palace missed their first two spot-kicks with Cardiff goalkeeper Tim Heaton saving from Jermaine Easter and Sean Scannell to become his team's hero as they booked a trip to Wembley, where they lost the 2008 FA Cup final to Portsmouth.
"It's a great night for Cardiff City," Heaton told the BBC. "I was a spectator for much of the game and when I got involved it was great to play a part in getting us to Wembley."
After levelling the score on aggregate, Cardiff twice went close to going ahead with Kenny Miller going agonisingly wide with one shot and hitting the post with another.
With their one-man advantage following McCarthy's dismissal after a challenge on Miller, Cardiff pushed for a winner but Palace held firm throughout the rest of normal time and extra-time to force the game into penalties.
"We've been at this stage before in this competition and the guys who took the penalties had nerves of steel," Cardiff manager Malky Mackay said.
"Tom Heaton has been our cup keeper and with some of the blame given to him after the goal in the first leg, maybe there was a bit of justice tonight."
Palace boss Dougie Freedman, whose team had been aiming to reach a first major final for more than 20 years, was proud of his side's efforts despite falling short.
"We've won a lot of hearts on the back of that performance," he said. "Cardiff are an outstanding team but we came at them. Football's not always about passing and movement, it's about hunger and desire.
"We showed some fantastic pace on the break. We fight for each other and on another night we'd have maybe taken one of our opportunities, but we deserved to take it to penalties."
Cardiff will learn who they will take on in next month's final on Wednesday when Premier League leaders Manchester City travel to seven-times winners Liverpool, who beat them 1-0 in the first leg.
