Long, who found the net 22 times in 87 games during a three-year spell at the Hawthorns, came back to haunt his former club on Saturday with the second goal in a 2-0 win for Hull.
The 27-year-old reportedly came close to joining Hull in August and went on to move to the KC Stadium in the next transfer window.
He has scored three goals since making the switch and Foster admits he is unsure as to why Long was allowed to leave.
"Sometimes I think why did we let him go to be honest with you," said Foster. "It's not even hindsight.
"He was one of our best players from the last couple of years and it's disappointing that he's no longer here.
"He's a fantastic player and we would love to have him now.
"If you had asked the players, we would have liked to have seen him stay. It's a business at the end of the day so it's out of our hands."
With Nicolas Anelka having also left West Brom, head coach Pepe Mel's primary attacking options are Victor Anichebe, Matej Vydra, Thievy Bifouma and Saido Berahino, who have 10 league goals between them this season.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
