The Cameroonian striker, from Malaga's youth team, came on as a second-half substitute and bundled the ball in off his thigh from close range in the 84th minute.
Olinga passed Athletic Bilbao forward Iker Muniain in the record books, Malaga said on their website. Muniain was 16 years and 289 days old when he netted in La Liga in October 2009.
Sevilla and Real Mallorca took temporary charge at the top of the standings with 2-1 home wins over Getafe and Espanyol respectively.
Celta had mounted a desperate late charge in front of their own fans and were unlucky not to take a point against Malaga when a shot that hit the bar bounced just short of the line.
The result is a boost for Malaga who, despite finishing fourth last season to earn a place in the Champions League playoffs for the first time, have suffered a troubled pre-season.
Delayed payments to players, the sales of Santi Cazorla and Salomon Rondon, the team's top scorer last season, and rumours that their Qatari owners were considering selling the club have overshadowed their preparations for the new campaign.
Malaga coach Manuel Pellegrini said a draw would have been a fair result, but preferred to switch his attentions to Wednesday's venture into Europe.
"The group had to endure some difficult moments and this will help strengthen minds and team spirit," the Chilean told a news conference.
"Now we have to think about the Champions League tie against Panathinaikos."
Federico Fazio headed Sevilla in front against Getafe and Spain striker Alvaro Negredo doubled the lead before the break from the penalty spot after Jose Antonio Reyes tumbled in the area.
Alexis pulled one back for the visitors with a header from a corner but Sevilla just about deserved the three points.
