Spurs suffered a third loss to West Ham this season after a poor first half left them with too much to do at Upton Park.
Younes Kaboul was dismissed for a foul on Stewart Downing before a Harry Kane own goal and Downing's free-kick ensured the hosts' top-flight status.
The visitors, meanwhile, suffered a dent to their UEFA Europa League hopes, but Sherwood dismissed the idea that his men were not up for the fight.
"The passion was there, they were up for it," he told BT Sport.
"It's difficult when you go down to 10 men. You have to give credit to West Ham but at 0-0 it was there for the taking.
"We showed our class and were more measured in our play.
"The second period, even with 10 men, we took it to them. We certainly had them on the back foot."
With the second goal, Sherwood stated his disappointment with the way the defensive wall capitulated to allow Downing's tame strike through.
Emmanuel Adebayor and Paulinho appeared to be the guilty parties and Sherwood acknowledged that it had been an error from his side.
"They were told. You have to take balls on the body. They know it's a mistake," Sherwood added.
"They accept that and it's given us a mountain to climb."
