The 22-year-old opened his account for the Championship club with a 14-minute second-half treble as the hosts put the relegation-threatened Premier League outfit to the sword.
Djamel Abdoun scored from the spot after 12 minutes to give Forest the lead, which was doubled midway through the second half by Paterson's first goal for the club
And the hosts' superiority was confirmed after 71 minutes as striker Paterson netted a second with a deflected effort, before claiming the match ball 11 minutes from time, with Andy Reid completing the scoring in stoppage time.
Allardyce's West Ham have just one win from their last nine matches in all competitions ahead of their League Cup semi-final first leg with Manchester City on Wednesday.
Forest made two alterations from the side that drew 1-1 with Reading in their last league encounter as midfielders Guy Moussi and Abdoun came into the starting XI, while Modibo Maiga featured up front for a much-changed West Ham team.
The home side made the better start to proceedings and took a deserved lead when Abdoun converted from the spot with a chipped penalty after George Moncur had felled Paterson inside the area.
West ham struggled to mount a response, although the visitors would have levelled if not for a fine save by Karl Darlow to keep out Ravel Morrison's curling free-kick.
The hosts were left aggrieved just before the half-hour mark as they were denied another penalty following Callum Driver's rash challenge on Greg Halford.
Forest finished the half on top and West Ham goalkeeper Adrian had to stand firm to deny Eric Lichaj from close range.
West Ham were much improved in the early stages of the second half, Sebastian Lletget and Morrison both going close as Allardyce's side pressed for an equaliser.
Still, the hosts remained a threat and midfielder Henri Lansbury perhaps should have done better when he headed Reid's free-kick narrowly over the crossbar.
Lansbury lashed over again shortly after, but his profligacy proved to be immaterial when Paterson found the bottom corner with a low strike after West Ham had failed to deal with a long ball forward after 65 minutes.
Forest refused to settle for two, though, with victory all but confirmed as Paterson's deflected effort found the net after West Ham had allowed Lansbury to waltz into the area.
Paterson cut inside to net his third with a smart finish beyond the reach of a despairing Adrian, and Reid rounded off the scoring with a close-range finish to cap a quick counter-attacking move.
