The 23-year-old is currently training with the national team ahead of the World Cup finals next month, and refused to confirm that he would definitely be a Villa player next season after Roberto Mancini's side saw a £20 million bid rejected for his servives.
"Do I expect to be at Villa next season? I don't know," Milner said. "I've worked as hard as I can and concentrated on club football while the season has been on.
"Now it has finished, I'm focusing on England and that's the only thing I am focusing on.
"The speculation about my own future is not distracting. There are always going to be rumours about every player in every club when the transfer window opens."
The former Leeds and Newcastle pass master netted 12 times in all competitions in 2009/10, leading to expected speculation tipping him to leave Martin O'Neill's side before the start of the new campaign.
But Milner insists his future - whether it be at Villa or elsewhere - will not be discussed until after this summer's global showpiece in South Africa.
"To be linked with any club is flattering but I'm just concentrating on England at this time," he said.
"The season finished at Villa a week and a bit ago and we all said we would discuss it [my future] after the World Cup.
"I've enjoyed my time at Aston Villa, it's a good club, we've had a lot of success this season, came close in two competitions, and we are moving forward."
By Owen Edwards
‘The pure simplicity of the way Slot has managed the squad is probably the biggest thing I could say about him. It’s not broken, so let’s get on with it’: Liverpool legend full of admiration for Jurgen Klopp's successor at Anfield
‘I trained at Spurs and thought they’d give me a chance. But I received a letter thanking me but saying they didn’t think I was good enough – I was gutted’: How Tottenham missed out on signing Wales legend John Toshack
