The Colombian was straight in the action as he was fouled for Xherdan Shaqiri's second-minute penalty, though Roberto Mancini's men struggled in the first half at Stadio Atleti Azzurri d'Italia and Atalanta deservedly drew level through Maximiliano Moralez.
But Guarin then intervened with his first goal before the break, curling superbly into the bottom corner.
The hosts' cause was not helped when Yohan Benalouane talked himself into a second yellow card early in the second half and Guarin then hit home an even better strike just after the hour.
Rodrigo Palacio rounded off the scoring late on as Inter moved above city rivals Milan and into eighth place in the table.
Inter boss Mancini had to do without his top goalscorer Mauro Icardi who was ruled out with a stomach bug, with Lukas Podolski replacing him in the starting XI.
The visitors made a fast start and were awarded a penalty in the first minute when Guarin swivelled in the area and went down under pressure from Gianpaolo Bellini.
Shaqiri duly stepped up to score his first Serie A goal for the club, despite Atalanta keeper Marco Sportiello getting a hand to the ball.
It was the hosts who looked the likeliest to score thereafter, with Mauricio Pinilla first glancing a header just wide and then somehow blazing over from six yards shortly after Shaqiri's opener.
Atalanta got their deserved leveller 10 minutes later, though, as Pinilla flicked on a long ball to Moralez, who controlled expertly inside the area before firing a left-foot shot that found its way in off the post.
But Inter responded well to being pegged back, and Guarin restored their lead eight minutes before the break when he evaded two Atalanta players before curling in a superb low shot from outside the area.
Atalanta's chances of getting anything out of the game were hindered when Benalouane was booked for bringing down Palacio after 52 minutes, before then arguing with referee Luca Banti and earning himself a second caution.
Inter took full advantage through Guarin's second of the game. The Colombian was given time and space 25 yards out and picked his spot, bending superbly into the top corner after 62 minutes.
Shaqiri was denied a second soon after through a fine Sportiello save but the scoreline was made to look more emphatic 18 minutes from time when Guarin's throughball found Palacio and, as he was about to shoot, Guglielmo Stendardo's sliding challenge saw the ball hit the Inter man and find the net.
