Manuel Pellegrini's men came from behind to triumph 3-1 over Sunderland at Wembley, banishing memories of last season's FA Cup final defeat to Wigan Athletic.
And Kompany revealed that disappointment had been an important factor in City's performances this season.
"Last year was probably... the fuel for our season so far," he said. "There's a lot of things you can do when you're beaten, but the main thing for us was to improve and work harder.
"And we're still doing so, so this is a fantastic trophy for us, but we'll get back this week and work hard again to make sure that we can keep our momentum."
City are still in the running for three other trophies, with the domestic treble looking a real possibility, but Kompany moved to cool expectations after City's first League Cup title since 1976.
"One thing at a time, let's bear in mind, for our club, it's a new trophy, a new experience and I want everybody to cherish it first of all," he added.
"It should give us the confidence and make us believe that of course we can still win trophies and of course we need to believe in the rest of the season."
