City, who will play Premier League strugglers Wigan Athletic in a May 11 final, dominated for two thirds of the game and led through Samir Nasri's 35th-minute strike and Sergio Aguero's header two minutes after the break.
Chelsea, in the midst of a fixture pile-up and playing their sixth game in 16 days, fought back after Demba Ba's acrobatic 65th minute volley and pressed strongly in the closing stages as Roberto Mancini's side held on.
"We deserved to win. We had some problems for 10 minutes in the second half. We had everything under control in the first half," City manager Roberto Mancini told a news conference.
"After we scored the second goal we probably played like the game was finished. Instead, the Demba Ba goal changed the game."
With Mancini having written off City's chances of retaining their Premier League title with Manchester United now 15 points clear, his side will start as heavy favourites to at least end their campaign with more silverware after they beat Chelsea in the Community Shield to open the season.
Wigan overcame Championship Millwall 2-0 at Wembley on Saturday in a match marred by fighting among the London club's supporters.
"Everyone can think we are favourites for this game but in 90 minutes anything can happen," Mancini warned.
"We know that Wigan play good football and if we want to beat them we have to prepare well."
Both sides will get an early dress rehearsal when they meet in the league at the Etihad stadium on Wednesday.
Chelsea, FA Cup winners in four of the last six seasons, were aggrieved to not be awarded a late penalty when Fernando Torres went down after wrestling with City captain Vincent Kompany.
Chelsea interim manager Rafael Benitez was adamant a spot-kick should have been given, and was less concerned by Aguero's ugly second-half challenge on David Luiz that appeared to be missed by referee Chris Foy.
Mancini said Aguero was a "good guy" while Benitez said he did not "like to point fingers at any player" after Aguero appeared to stamp on the Brazilian.
"The penalty not being given was a key moment in the game as Vincent Kompany could also have been sent off," Benitez said.
"It could have made a big difference. It was a clear penalty."
The Spaniard said tiredness was not a factor in Chelsea's defeat, with the Londoners having only two days full rest after their Europa League trip to Moscow where they sealed a semi-final place on Thursday by overcoming Rubin Kazan 5-4 on aggregate.
STRONG FINISH
"How do you explain how we finished so strongly?" added Benitez, who continued his squad rotation policy by leaving Frank Lampard and John Terry on the bench.
"You have to give credit to Manchester City. They are a very strong team... physically, technically. We talked at half-time about showing more character in the second half, and we did."
City set the tone for a dominant opening half when Petr Cech clawed away Aguero's early eff
