Galliani was involved in talks with Inter's board over a possible deal on Monday, with a loan move reported to be one of the options.
However, despite being without Daniele Bonera due to a fractured kneecap, Galliani confirmed Milan will target other defenders instead.
"Instead of getting closer with talks, we moved further away,” he told Sky Sport Italia.
"Therefore we are looking to other objectives instead. Have we abandoned the Silvestre option? Yes."
As well as Bonera, Milan also lost Robinho to injury in their International Champions Cup opener against Valencia on Saturday.
Galliani had previously confirmed that Robinho is expected back in time for the UEFA Champions League play-offs, which get under way on August 20 and 21.
‘He instantly popped into our ratings as one of the best U21 midfielders in Europe. The impressive thing is Slot has made him into more of a no.6' Inside Ryan Gravenberch's transformation at Liverpool
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
