Galliani was involved in talks with Inter's board over a possible deal on Monday, with a loan move reported to be one of the options.
However, despite being without Daniele Bonera due to a fractured kneecap, Galliani confirmed Milan will target other defenders instead.
"Instead of getting closer with talks, we moved further away,” he told Sky Sport Italia.
"Therefore we are looking to other objectives instead. Have we abandoned the Silvestre option? Yes."
As well as Bonera, Milan also lost Robinho to injury in their International Champions Cup opener against Valencia on Saturday.
Galliani had previously confirmed that Robinho is expected back in time for the UEFA Champions League play-offs, which get under way on August 20 and 21.
Bizarre Jose Mourinho statement aims to pre-empt punishment after Fenerbahce boss clashes with Galatasaray counterpart
'Andy Robertson has a decision to make as a player, where you either roll your sleeves up and give someone great competition or you accept decisions and say that your time is done.' Alan Shearer on the conundrum facing Arne Slot's Liverpool
