Marseille scored all three of their goals in the second half at the Stade Velodrome as they stayed on top of the table.
After a scoreless opening period, France striker Andre-Pierre Gignac netted a brace in the space of 13 minutes - making him Ligue 1's leading goalscorer with six.
Romain Alessandrini completed the scoring in injury time, coming off the bench to find the back of the net against his former club as Marseille stayed ahead of Bordeaux on goal difference after six matches.
Post-game, Rennes coach Philippe Montanier said his team "were not subjected to a lot of pressure", but Bielsa disagreed.
"Dominance in the game looked to us, the number of scoring opportunities was more or less equal," Bielsa told reporters.
"I disagree with the analysis of Montanier. We had more initiative, more of the ball, we finished our attacks better.
"The first period, we did not play well. In the second half we played better and we finished better attacks.
"I am happy because every time the team wins it gives me joy, but it plays only the first third of the season, I do not want to do anything but be moderate."
The win was particularly pleasing for Alessandrini, who swapped Rennes for Marseille during the close-season.
"When I scored I wanted to explode, but I thought 'I'll calm down, it's my ex-teammates," said the 25-year-old, who spent two years with Rennes.
"It makes me feel good, it's fun scoring against Rennes."
Marseille are in the midst of a four-game winning streak after losing to Montpellier last month, scoring 11 goals and keeping three clean sheets in the process.
However, Alessandrini is refusing to look too far ahead as Marseille aim to consolidate top spot over the coming weeks.
"A club with the standard of Marseille must play the top of the table. We must keep our principles of play that works and stick to it," he added.
"These victories are something strong, 4-0 against Nice, 3-0 today. But we will not get caught up in it."
