Elkin Soto gave the hosts a shock lead after 21 minutes, before Bastian Schweinsteiger restored parity with a curling free-kick three minutes later.
Battling Mainz looked set to cling on for a point, but Arjen Robben popped up in the 90th minute to snatch victory.
"A big compliment to Mainz," he said. "It was a very difficult match.
"Mainz have many fast players and are good on the counter-attack."
The victory sees Bayern head into the winter break with a record points tally of 45, and Guardiola is satisfied with his team's progress.
"If we analyse the whole season so far, I'm very, very pleased," he added. "No matter where we go, we always play with passion.
"We now need to stay calm and prepare for the second half of the season."
Robben shared his manager's view that Mainz were unfortunate to come away from the match empty-handed, adding: "I'm happy with the win, but I feel sorry for Mainz, who gave everything.
"They deserved a point, but that's football. Compliments to the team. We came back and won again."
