The Highlands club took the lead after just six minutes as Richie Foran headed home from point-blank range after good work from Billy McKay and Aaron Doran.
Winless Kilmarnock levelled in bizarre fashion just six minutes into the second half, Graeme Shinnie's clearance cannoning off Barry Nicholson and into the net.
But parity was shortlived for the hosts, though, with McKay lobbing home the winner after 56 minutes to secure Inverness' fifth win of the campaign.
Defending champions Celtic remain three points behind Inverness with a game in hand after they cruised past troubled Hearts 3-1 at Tynecastle.
Kris Commons gave Neil Lennon's side the advantage from the penalty spot after 19 minutes, but Gary Locke's men responded early in the second half as Jason Holt beat Fraser Forster with a low drive.
However, the hosts, who were deducted 15 points at the start of the season after entering administration, could not hold on against the Glasgow outfit.
Anthony Stokes restored Celtic's lead, before recent signing Teemu Pukki wrapped things up with a goal on his debut for the club.
Motherwell continued their strong start to the season with a 1-0 triumph at St Mirren, John Sutton's strike seven minutes before the interval giving Stuart McCall's men all three points.
Meanwhile, Aberdeen made it two clean sheets in a row as they eased to a 3-0 success over Partick Thistle at Firhill Stadium
Calvin Zola, the striker signed from English League Two side Burton Albion, gave the visitors the lead before Peter Pawlett and Josh Magennis made sure of the win.
Elsewhere, Hibernian came from behind to beat St Johnstone 2-1, Paul Heffernan and James Collins turning things around for Pat Fenlon's men, who extended their unbeaten run to four matches after Stevie May had given the hosts the lead.
