Boca have not won a league title in Argentina since 2011, suffering one of their least successful periods in recent history – and they are without a league win since November.
Belgrano recovered from conceding the opener at La Bombonera on Sunday to score the next three goals and while Luciano Acosta struck for Boca in second-half stoppage time, the visitors held on to win.
Cesar Pereyra was the key man for Belgrano, setting up Emiliano Rigoni for the visitors' first goal and then scoring himself in the 90th minute to make it 3-1.
Boca took the lead in just the seventh minute when defender Juan Forlin ran onto Juan Sanchez Mino's centring header to nod the ball past Belgrano goalkeeper Pablo Heredia.
The hosts stayed in front until half-time but two minutes after the break Rigoni equalised, running onto Pereyra's lofted pass, holding off two opponents and then finishing with aplomb.
A penalty from Luciano Lollo put Belgrano in front and the visitors looked to be cruising when Pereyra tapped home a rebound late on, only for Acosta to give Boca some hope with a stunning strike from the edge of the area.
Belgrano held on, however, and moved to four points from two games to sit fifth in the Argentine league, while Boca are 15th on one point.
Boca have failed to win in their past six Primera Division matches, stretching back to the latter stages of last season.
Estudiantes are top in Argentina with a perfect record from two games after their 1-0 win at All Boys, while Atletico Rafaela (four points) are second after thrashing Lanus 3-0.
Velez Sarsfield defeated Arsenal 3-1 to sit third, while Olimpo are fourth after a scoreless draw at Tigre.
In other results, River Plate drew 1-1 at Rosario Central, Gimnasia La Plata stunned Newell's Old Boys 3-0, Colon defeated Argentinos Juniors 1-0, San Lorenzo trumped Racing Club by the same scoreline and Godoy Cruz won 2-0 over Quilmes.
