The Welsh club fought back from 2-0 down to lead 3-2 at the Liberty Stadium but were denied three points courtesy of a 96th-minute Charlie Adam spot-kick.
Referee Robert Madley awarded the penalty after the ball appeared to strike Wayne Routledge on the arm in the area, although Williams was perplexed by the decision.
"It seemed like none of their players appealed for a penalty - maybe one of them was appealing for a corner," the defender told the South Wales Evening Post.
"It was a shock to everyone. I didn't see a handball from Wayne or Leon (Britton). I don't know.
"All I can say is that I didn't see a handball and I think both sides were surprised."
Adam's penalty ensured Swansea were denied a first victory in five matches and the chance to get back on track following defeat to rivals Cardiff City in their previous Premier League outing.
"It's a sickener because to get three points going into the mini-international break would have been good," Williams added.
"It (a win) is what we deserved and the table would have looked a bit better, but we can be pleased with what we did in the second half."
‘He instantly popped into our ratings as one of the best U21 midfielders in Europe. The impressive thing is Slot has made him into more of a no.6' Inside Ryan Gravenberch's transformation at Liverpool
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
