Klopp's side fought back 1-0 down to lead 2-1 but, after conceding an equaliser, needed two late goals to continue their recovery following a dismal first half to the season.
However, Klopp was worried by the relief his players showed after taking the lead for the first time and warned against it ahead of Friday's trip to Stuttgart.
"It is OK to be happy. The problem is that everyone is so relieved. That is a huge difference because you stay relaxed," explained the Dortmund boss.
"You saw it after we scored the second against Mainz. I saw three players who looked up to the sky. That is not a joke. It looked like they thought 'OK, we did it'.
"But we have not accomplished anything until the game is over. Of course I understand the reaction because the pressure is immense but in this relaxed state we lost our compactness and allowed Mainz back into the match."
The 2011 and 2012 Bundesliga champions have moved out of the relegation zone thanks to back-to-back wins but remain 15th after a disappointing campaign.
While his side are now on something of an upward curve, Klopp is eager not to look too far ahead.
"I do not want to think about the result of the season at this point, because I don't know where our journey leads us," he added.
"I am very optimistic. Of course, the last two matches were good for me. But I know that this is just one side of the coin."
