Algeria frustrated their opponents in the first half and took a surprise lead when Sofiane Feghouli was hauled down by Jan Vertonghen and picked got to his feet to convert the resulting penalty in the Group H opener at Estadio Mineirao in Belo Horizonte on Tuesday.
The African nation were dreaming of ending a run of six matches without a win at the World Cup, but Belgium fought back with goals from Marouane Fellaini and Dries Mertens in the final 20 minutes.
Halilhodzic rued his side's inability to keep Belgium at bay, but also believed Mertens' winner should not have been allowed to stand.
"The feat (of winning) was not far," he is quoted as saying in Le Buteur. "I am very disappointed, and rightly have regrets. We missed something big.
"We suffered in the second period.
"We left the Belgians excessive room on the first goal, that is naïve, (and) on the second goal there was a clear foul on Feghouli (by Kevin De Bruyne).
"This mistake would have been given if we had been Brazil. It's always like that with small teams like ours.
"But I cannot blame my players, they gave everything and we played as a team as I had requested. We (just) came across one of the best teams in Europe."
Defeat means Algeria have still not won a game in the finals since beating Chile in 1982.
'I’d probably pick Cole Palmer alongside Bellingham and Rice in there at the moment. For me, Declan Rice is good enough to play that central midfield role on his own' Thomas Tuchel advised to DROP Manchester City star by England legend
Manchester United staff threatened with sacking for leaked information... in a leaked email
