Newcastle fell behind in the first minute of Saturday's 3-3 draw at St James' Park to Dwight Gayle's strike, and Palace were in front again in the 48th minute through Jason Puncheon's volley after Daryl Janmaat had levelled for Pardew's men.
Rolando Aarons' second-half introduction looked to have turned the game in the hosts' favour, as he netted a second equaliser for Newcastle before seeing his far-post cross cannon off the post to be turned home by Mike Willamson with two minutes left.
But Zaha - who returned to Palace on loan from Manchester United on Thursday - had the final say in the fifth minute of injury time to earn a point for new manager Neil Warnock.
That left Pardew to lament his side's concentration at the back, as their wait for a first Premier League win of the season goes on.
"We've conceded a goal after 30 seconds and we've conceded a goal straight after half-time, which is disappointing, but we've come back each time," Pardew said.
"We then take the lead and get ourselves in a position [to win]. I thought we got a little bit carried away when we took the lead and at this level you can't do that, you've got to make sure you see the victory out and we should have been better set before we conceded the free-kick for the [third] goal."
However, although he was disappointed with Newcastle's defensive performance, the 53-year-old felt they had not been helped by referee Mike Jones, who added seven minutes of injury time at the end of a game in which he felt Palace had set out to stall.
He added: "When a team comes here and waste time from the moment the game kicks off, on goal-kicks, and time and time again they took ages on everything - and then they're 3-2 down and he gives them seven minutes, I don't know it just didn't fit right with me.
"In the spirit of the game I thought that was wrong but we should have won and we didn't."
