The A-League side took the lead in the 24th minute through Nikita Rukavytsya, but Shoma Doi levelled on the hour mark for Kashima before Mu Kanazaki's 92nd-minute winner settled the game.
That defeat, coupled with Seoul's draw with Guangzhou Evergrande, leaves Western Sydney bottom of Group H with five points, one adrift of Seoul and Kashima, who meet in the final matchday next month.
Western Sydney will need a draw in that game and a victory over group winners Guangzhou in order to progress.
Al Hilal, last year's runners-up, are through from Group C following a 2-1 win over Lokomotiv Tashkent, with Al Sadd also qualifying for the last 16 alongside them courtesy of a 1-0 win against Foolad.
Nasaf remain very much in contention for a place in the knockout stages thanks to Artur Geworkyan's brace that gave them a 2-1 victory at Group D rivals Tractor Sazi.
Late goals from Ko Cha-won and Kaio gave Suwon Bluewings a 2-1 success away from home against Urawa Reds that sends them top of Group G courtesy of Beijing Guoan's 1-0 defeat to Brisbane Roar.
Suwon and Beijing are level on 10 points - three points ahead of Brisbane - and will qualify with a draw in their final group encounter in South Korea.
‘Milan didn’t collapse against Liverpool in Istanbul. We lost focus for six minutes – for the other 114, we played an extraordinary, beautiful game’: AC legend claims they were 'extraordinary' against Reds in 2005 Champions League final
‘Charlton could have qualified for Europe, but selling Scott Parker to Chelsea made the difference. The manager didn’t want him to go and there was a disagreement’: Former Addicks star highlights key reason why they failed to maintain 2003/04 form
