Costa Rica weathered a first-half storm in Tampa before taking the lead in the 31st minute through captain Bryan Ruiz and Japan needed strikes from midfielders Yasuhito Endo and Shinji Kagawa after the break to triumph.
Yoichiro Kakitani then added a third for Japan in second-half stoppage time with an opportunistic strike at Raymond James Stadium.
Japan have scored eight goals in three games so far this year but only three have come from the four strikers coach Alberto Zaccheroni has named in his final 23-man squad for the FIFA World Cup.
Up against the Costa Ricans, Japan dominated possession but looked somewhat toothless up front until Shinji Okazaki - the Mainz striker, who scored twice against New Zealand in March - came on at half-time.
In Japan's second of three pre-tournament friendlies, Zaccheroni made four changes from his starting XI against Cyprus with Endo, Okazaki, Yuto Nagatomo and Kakitani missing out.
Maya Yoshida slotted into central defence with Yasuyuki Konno moving to right-back in Zaccheroni's 4-2-3-1 set-up, Toshihiro Aoyama moved into central midfield, Yoshito Okubo claimed the right wing spot and Yuya Osako led the line.
Costa Rica coach Jorge Luis Pinto made just one change from his team's last match against Paraguay in March, with Cristian Bolanos starting on the left in a 5-4-1 formation, while Marco Urena dropped to the bench.
A frantic start saw Ruiz and Okubo both miss promising opportunities in the first two minutes before Japan's goalkeeper Eiji Kawashima tipped Yeltsin Tejeda's shot past the post.
Japan began to take control as the half wore on with Osako heading over from close range and hitting the post, before Okubo forced Costa Rican gloveman Keylor Navas into a low save in the 25th minute.
But just after the half-hour mark, Pinto's men stung Japan on the counter-attack.
Junior Diaz got in behind Konno on Costa Rica's left flank and his driven cross was converted by Ruiz from inside the six-yard box.
The rest of the half followed a similar pattern as before the goal with Costa Rica struggling to get out of their defensive half, while Japan lacked cutting edge up front, meaning the score remained 1-0 at the break.
Zaccheroni made two changes at half-time with Okazaki and Endo coming on for Okubo and Aoyama, respectively.
The first change almost led to an immediate equaliser with Okazaki teeing up Kagawa in the 48th minute but Navas blocked the Manchester United midfielder's shot before Osako hit the rebound wide.
Okazaki was causing Diaz plenty of problems on Japan's right and blasted a shot straight at Navas in the 53rd minute before Joel Campbell forced Kawashima into a save soon after.
Japan equalised with 30 minutes left when Kagawa's fine chip over Costa Rica's defence found Honda.
Atsuto Uchida cleverly let Honda's cut-back roll between his legs and Endo finished from directly in front.
Zaccheroni's men broke through a second time in the 80th minute when Kagawa burst forward, played a fine one-two with Kakitani, and then slotted his shot past Navas.
Kakitani put his name forward for to lead the line when he slid in to score in the 92nd minute but Zaccheroni may need to try Okazaki as the central forward in Japan's final friendly against Zambia on Friday, as the man from Mainz has a record of 38 goals in 74 games for his country.
Costa Rica suffered their fourth defeat in five games, and Pinto will be expecting a better performance when they take on the Republic of Ireland in four days.
