The Liga champions responded well to Pablo Hernandez's outstanding opener on Saturday and goals from centre-back pairing Miranda and Diego Godin had them in front by the break.
But Miranda gave away a penalty eight minutes into the half, an opportunity converted by Nolito.
Atletico piled on the pressure in the latter stages, with Celta goalkeeper Sergio Alvarez making several saves, while substitute Alessio Cerci had an injury-time goal ruled out for offside.
Simeone's men had to settle for a point, though, and the coach said they must stop giving away silly penalties.
"The goals we conceded are different," Simeone said.
"The first one was something you hardly ever see and the second was from the penalty spot.
"It's true that two out of the three goals we have conceded in the last few matches were from the penalty spot.
"It's true we have to look for an improvement and it's clear that we need to stop conceding goals in this way."
Simeone felt the manner of his side's performance was pleasing and hailed their character to quickly respond from their early setback.
"I think it was one of the best matches we have played in recent weeks," he added.
"I liked the performance from my team, who responded very well to going behind.
"After Celta's first goal, our team reacted really well with two fine goals and we could even have scored the third with Antoine Griezmann.
"During the second half we suffered a bit and looked tired. [But still] we had so many chances. Their goalkeeper had a great match but at the end it is still a good point for us."
