Manchester United midfielder Fellaini scored twice in Saturday's 5-0 thumping of Cyprus, and made the difference again in Jerusalem - his ninth-minute strike putting Belgium top of Euro 2016 qualifying Group B.
Captain Kompany picked up a second yellow card 26 minutes from time to ensure the visitors endured a tense end to the game, but they held on to move above Wales on goal difference.
However, Kompany's dismissal means Belgium will be without their skipper when they travel to Cardiff for a key clash with Wales in June.
Belgium were on the front foot from the start and it came as little surprise to see Marc WIlmots' men break the deadlock.
There was little craft about the goal, as Kompany latched on to a sliced volley at the back post and saw his effort parried on to Fellaini, who bundled over the line.
Despite their control, Belgium struggled to craft clear-cut opportunities for the rest of the first period and Israel improved after the interval.
Thibaut Courtois was forced into a couple of fine saves by Eran Zahavi with the hour mark approaching, but Israel's biggest boost came a few minutes later, as Kompany picked up a second booking.
Courtois was again called into action to deny Ben Sahar shortly after Kompany's departure, but Belgium withstood late pressure to claim all three points.
