Tuesday's comfortable 3-0 victory at neighbours Manchester United saw City close the gap on leaders Chelsea to three points, with Manuel Pellegrini's men boasting two games in hand.
However, Kompany insists the battle for the title is far from over and City must remain focused on the task in front of them.
Pellegrini's charges face Arsenal on Saturday and the Belgium defender expects another strong display to maintain their five-game unbeaten run in the league.
"It's just one little step (beating United)," Kompany told Sky Sports after the victory. "The other teams are still very good.
"I mean games have to be played. As long as we don't have the points, I don't consider ourselves there yet.
"With the game against Arsenal, obviously it's good in a way, because you don't have to make a big effort to get the lads focused again."
