Goals from Pablo Armero, Teofilo Gutierrez and James Rodriguez ensured Colombia won their opening game of the competition in Group C.
The clash with Greece was also Colombia's first match at a World Cup since the 1998 edition of the tournament in France.
While Pekerman was pleased to get up and running with a victory, he warned against the dangers of getting carried away.
"It was certainly a wonderful debut for a side that didn't play a World Cup for (almost) 20 years," he said.
"Having said that, we haven't won anything yet, though I am very happy for this performance.
"We can prove how great Colombian football really is, but this was only the first game.
"Today we celebrate and tomorrow we'll start thinking about the next challenge against the Ivory Coast."
Colombia face Ivory Coast in Brasilia on Thursday before rounding off their group campaign with a match against Japan in Cuiaba on June 24.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
