The relegation-battling Premier League outfit will be aiming for their first major trophy since the 1973 FA Cup when they take on City at Wembley on Sunday.
But Manuel Pellegrini's City – third and having scored 69 times in 26 league games – are hot favourites.
City were beaten 1-0 by Wigan in last year's FA Cup final and Poyet believes they will be eager to make amends.
"It's going to be extremely difficult. We are aware of Manchester City - the players they have, the opportunity, what happened to them last year in the FA Cup," the Uruguayan manager said.
"There are plenty of things that are there for them to go and use this opportunity to show the quality team they are.
"We are going to try our best. We are going to be much better than last weekend (in a 4-1 loss to Arsenal) I hope and we're going to try our best to make the Sunderland fans very proud."
Sunderland have won two of their past three meetings against City, including a 1-0 victory in November last year.
Goalkeeper Vito Mannone said it would be 'amazing' to win a trophy like the League Cup.
"It's great to be able to perform every week and give something back to the special fans," the former Arsenal shot-stopper said.
"They have been really supportive through the season with the difficulties that we had. I got my chance and I think I took it.
"Winning something always stays with you and everyone will remember it. It's something that you dream of as a kid to win a cup or win something, that's why we are footballers.
"We want to be winners and having the chance to win a cup, an important cup in England, would be amazing."
Defender Wes Brown, who enjoyed plenty of success at Manchester United, has given his team-mates simple advice.
"This week a few of the lads have asked me what it's like and I said 'you've just got to enjoy the day'," Brown said.
"You have to go out there and enjoy it and if you enjoy it you get the best out of yourself. If we can win the game it will be brilliant."
