The diminutive striker combined well with Niall McGinn throughout proceedings but the pair's efforts were rewarded with 43 minutes on the clock when Paterson thumped the ball home.
That was enough to give Northern Ireland fans a first victory to celebrate since August 2011, when they ran out 4-0 winners over minnows Faroe Islands, while it also marked manager Michael O'Neill's first in charge.
For Russia, however, it was a huge opportunity missed to return to top of their group above group rivals Portugal.
With Jonny Evans and Kyle Lafferty both absent through injury or illness, and Chris Brunt suspended, Northern Ireland were forced to name goalkeeping coach Maik Taylor among the substitutes just to fill the bench.
Russia boss Fabio Capello, meanwhile, named only 30-year-old Aleksandr Kerzhakov up front as part of a line-up that included six in midfield.
Northern Ireland started brightly, with Aberdeen playmaker Niall McGinn firing in two early sighters on goal in the opening couple of minutes - the second of which looped just past Igor Akinfeev's left-hand post.
However, Russia fought back and at the other end West Brom's Gareth McAuley was forced to snuff out the threat from Alan Dzagoev as he raced in on goal with a well-timed challenge.
The hosts’ early attacking pressure in the opening quarter soon subsided, and eventually Russia's quality began to tell as they got a real foothold in the game.
Russian winger Vladimir Bystrov linked well down the right with Dzagoev, causing young left-back Shane Ferguson problems.
Then, in the 34th minute, the impressive Dzagoev created space for striker Kerzakhov, but the Zenit man took too long to get his shot away and McAuley cleared up.
As half-time neared, the hosts certainly did not seem to be missing defender Chris Baird at the back - the 31-year-old having been overlooked for the squad as he is currently without a club.
And despite Russia's attacking intent, it was Northern Ireland who finished the half ahead.
Derby County's Jamie Ward had already gone close when he sent a dipping 25-yard shot onto the post after being sent through by a quick free-kick from skipper Steve Davis.
And two minutes before the break Windsor Park was jumping.
McGinn was the architect when he whipped in cross from deep and Huddersfield Town's Martin Paterson leapt well above his man to head strongly into the corner - a remarkable feat considering that the 26-year-old was one of the smallest players on the pitch.
For the second period, Cappello rung the changes as he tried to get Russia back into the contest.
The Italian brought on Artem Dzyuba and Denis Cheryshev, although the latter was forced off with an injury after just six minutes.
But Northern Ireland still probed for a second goal, and they nearly managed it close to the hour mark only to be denied by the assistant referee’s flag.
Again it was Paterson with his head who found the net via a McAuley knockdown, but the forward was adjudged to be just offside. TV replays proved the call to be correct and showed that the striker had strayed fractionally off.
The aerial threat again posed Russia problems 15 minutes later when Ferguson pushed down the left flank, lifting another high ball into the box, and this time Akinfeev flapped at the delivery.
The ensuing scramble for the loose ball saw Paterson and Russian defender Vasiliy Berezutskiy booked.
The tenacity shown by Paterson was typical of the Irish, as they fought for everything going into the final 10 minutes.
And they were rarely tested in the closing stages as Windsor Park erupted at the whistle.
