Goals from Pablo Armero, Teofilo Gutierrez and James Rodriguez ensured Colombia won their opening game of the competition in Group C.
The clash with Greece was also Colombia's first match at a World Cup since the 1998 edition of the tournament in France.
While Pekerman was pleased to get up and running with a victory, he warned against the dangers of getting carried away.
"It was certainly a wonderful debut for a side that didn't play a World Cup for (almost) 20 years," he said.
"Having said that, we haven't won anything yet, though I am very happy for this performance.
"We can prove how great Colombian football really is, but this was only the first game.
"Today we celebrate and tomorrow we'll start thinking about the next challenge against the Ivory Coast."
Colombia face Ivory Coast in Brasilia on Thursday before rounding off their group campaign with a match against Japan in Cuiaba on June 24.
‘It was without a doubt one of the most important matches that I played in my career. It wasn’t just Champions League football – it was a whole club at stake’: Ex-Chelsea star opens up on key game which changed the Blues' fortunes forever
'I was Arsenal captain and really enjoyed it, but when Man City come calling, you have to listen': England international reveals reason for swapping London with Manchester
