Former Arsenal player Steve Bould will replace him, manager Arsene Wenger said.
Rice's last match as Wenger's No.2 will be the Premier League season finale at West Bromwich Albion on Sunday when Arsenal will attempt to seal third place in the table.
The 63-year-old Rice, who played 49 times for Northern Ireland, first joined Arsenal as a youth player in 1964, was a member of their 1970/71 double-winning team and captained the side to FA Cup success in 1979.
He played 528 times for Arsenal over 14 seasons and after a four year spell at Watford from 1980-84, returned to Arsenal to coach the juniors and became Wenger's right-hand man when the Frenchman joined the club in 1996.
Wenger described Rice as an "Arsenal legend".
"He has committed almost his whole life to Arsenal Football Club, which shows huge loyalty and devotion," Wenger said.
"Pat has been amazing for me over the past 16 years.
"I will always be indebted to him for his expert insight into Arsenal and football as a whole. On the training pitches and on matchdays, Pat has always been a passionate, loyal and insightful colleague, who we will all miss."
Wenger told a news conference on Thursday that Bould, the current chief youth-team coach, would replace Rice. Bould, 49, played 288 league matches for Arsenal between 1988 and 1999 and won two League titles with the club.
He also had spells at his home-town club Stoke City and Sunderland and played twice for England.
‘Slot hasn’t come in and changed everything, making it all about himself – it’s been about the players he’s inherited, getting the best out of a system he likes’: Liverpool manager impressing former players with personality and tactical approach
‘We thought we’d signed him, but Chelsea blew us away. I think they signed him so we couldn’t have him, as much as because they really wanted him’: Liverpool actually failed to sign Mohamed Salah in 2014, three years before finally getting the Egyptian
