Scott Sinclair grabbed the opening goal of Sherwood's tenure after 20 minutes, but they were unable to hold on as Mame Biram Diouf and Victor Moses – the latter with a stoppage-time penalty – secured a 2-1 win for the visitors.
An 11th Premier League game without a win sees Villa drop to 19th on goal difference after Burnley's 1-1 draw with leaders Chelsea, leaving Sherwood keen to get back on the training pitch as quickly as possible.
"I am bitterly disappointed for everyone involved with this football club," said the new Villa boss.
"It [the performance] wasn't great, it's a work in progress and there is a lot of work to be done.
"I was encouraged with some performances, I just think we need to work on it, continue to get on the ball and affect the football match.
"I didn't think we started fantastically well, we got the goal but weren't able to make it two. We switched off just before half-time from a throw and conceded, and it knocks the stuffing out of you.
"In the second half, I thought the longer the game went on it would just peter out into a draw. It looked like neither team were going to win it.
"So to go so late into the game and concede the penalty is obviously very, very disappointing."
Sherwood is, though, convinced that Villa can still beat the drop.
"I'm sure we've got the quality in the squad, it's just about putting the right jigsaw together to win games," he added.
"We've got to put this behind us now, we can't dwell on it. We need to move on.
"Games come thick and fast, so we can't dwell on it. The games are running out."
