The Manchester City right-back decided against putting himself on standby for Roy Hodgson's Euro 2012 squad in favour of joining up with former boss Stuart Pearce and the one-off British selection.
However, with no England caps under his belt since a friendly with the Netherlands in February 2012, Richards has doubts over whether he will be able to force his way back into the national set-up under Hodgson.
"It was one of those. Do I go on standby or play in the Olympics?," he told The Sun. "I wanted to go and play under (Stuart) Pearce as I knew him from here at City.
"It was a decision I made; people will always think negatively about it.
"But, from my point of view, we'd won the league that year and the FA Cup the season before. Kyle Walker got injured and I was still being asked to go only on standby.
"Was I wrong for making the decision? Maybe. But I wanted to play in the Olympics. It was a once-in-a-lifetime chance and I enjoyed it. It was one of the best times of my life."
The 25-year-old still harbours dreams of lining up for England, but is aware he may miss out to emerging talent in his position.
"I still want to play for my country, everyone does," he added. "But what can I do now?
"Of course, there's still a lot of time. But time waits for no-one. We've got young players coming through now like Kyle and Nathaniel Clyne.
"So, yeah, time is still on my side. But sometimes with England, if you don't get in when you're young and stay in, they look to go younger. All I can do is play well.
"People ask me the question about getting back in the England squad and I think, 'I won the FA Cup and then the Premier League in 12 months and I still didn't get in'.
"So what can I do? I couldn't do any more than that."
Richards has 13 caps for England but only two of those have been gained since 2007.
