The Brazilian hosts at the Estadio Cicero Pompeu de Toledo had been pegged back twice by the Colombian visitors, before Carlos' second goal in the 90th minute earned them a slight advantage for next week's second leg.
Earlier, Jadson struck on 14 minutes - only for Atletico to go into half-time level after Fernando Uribe equalised.
Carlos' first strike on 72 minutes gave the hosts the lead again, but Jefferson Duque fired another handy away goal for Atletico seven minutes later.
However, Carlos had the last say in proceedings - netting at the death to give Sao Paulo the all-important advantage heading to Colombia.
‘Even though it was late in the game, the Arsenal fans started singing my name. It felt like the moment I announced myself as a professional footballer’: Ex-Gunners star recalls scoring first senior goal as a teenager at Highbury
'I started to drive myself crazy': Former Manchester United midfielder explains why it all went wrong for him under Erik ten Hag
