Goals from Pablo Armero, Teofilo Gutierrez and James Rodriguez ensured Colombia won their opening game of the competition in Group C.
The clash with Greece was also Colombia's first match at a World Cup since the 1998 edition of the tournament in France.
While Pekerman was pleased to get up and running with a victory, he warned against the dangers of getting carried away.
"It was certainly a wonderful debut for a side that didn't play a World Cup for (almost) 20 years," he said.
"Having said that, we haven't won anything yet, though I am very happy for this performance.
"We can prove how great Colombian football really is, but this was only the first game.
"Today we celebrate and tomorrow we'll start thinking about the next challenge against the Ivory Coast."
Colombia face Ivory Coast in Brasilia on Thursday before rounding off their group campaign with a match against Japan in Cuiaba on June 24.
