West Ham were stunned by their League One visitors as the Yorkshire club prevailed on penalties after a 1-1 draw at Upton Park on Tuesday.
Allardyce's men led through Diafra Sakho before Winston Reid's own goal levelled matters, with Howard saving Enner Valencia's spot-kick to allow Michael Doyle to convert the winner in the shootout.
It offered Sheffield United a small taste of revenge after a Carlos Tevez-inspired West Ham controversially secured Premier League safety at their expense in 2007.
And Allardyce was in no doubt as to who had inspired the upset.
"They were fantastic saves by their goalkeeper because time after time we hit the target, and time after time he pulled off saves well beyond the league he is playing in," Allardyce told the club's official website.
"He has had one of those inspired nights and, of course, if he plays like that all season he won't be staying in League One.
"We found a keeper in his top form and that has cost us in the end because he's the one who saved the penalty as well.
"It's a disappointing night in the fact that we lost, but the football that we played and the people going out there and getting the game they needed under their belts gave an excellent account of themselves."
