Croatia international Rakitic, whose contract ran out at the end of the season, joined Schalke in 2007 from Swiss club Basel, playing 131 games in all competitions and scoring 16 goals in total.
The German club did not give any details on the length of his new contract in Spain or the transfer fee.
Schalke have found a replacement for Rakitic, with 24-year-old Ghana international Annan set to join them from Norway's Rosenborg.
The Germans have yet to confirm the transfer but the Norwegian outfit said on their website Annan had passed his medical at Schalke.
"Anthony Annan is now officially ready for Schalke after passing the medical test," Rosenborg said on their website.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
