Galliani was involved in talks with Inter's board over a possible deal on Monday, with a loan move reported to be one of the options.
However, despite being without Daniele Bonera due to a fractured kneecap, Galliani confirmed Milan will target other defenders instead.
"Instead of getting closer with talks, we moved further away,” he told Sky Sport Italia.
"Therefore we are looking to other objectives instead. Have we abandoned the Silvestre option? Yes."
As well as Bonera, Milan also lost Robinho to injury in their International Champions Cup opener against Valencia on Saturday.
Galliani had previously confirmed that Robinho is expected back in time for the UEFA Champions League play-offs, which get under way on August 20 and 21.
‘Even though it was late in the game, the Arsenal fans started singing my name. It felt like the moment I announced myself as a professional footballer’: Ex-Gunners star recalls scoring first senior goal as a teenager at Highbury
'I started to drive myself crazy': Former Manchester United midfielder explains why it all went wrong for him under Erik ten Hag
