Jurgen Klopp's men finally looked capable of replicating their fine UEFA Champions League form as they dominated their visitors throughout, but victory came via a stroke of luck as the home side wasted countless chances.
Former Monchengladbach star Marco Reus went close twice early on, with Yann Sommer nudging one effort on to the post.
But just when it appeared as though a combination of poor finishing and inspired goalkeeping from Sommer was going to thwart Dortmund, Kramer produced a moment that he will want to forget.
With 58 minutes played, the World Cup winner lobbed his own goalkeeper from near the halfway line as Dortmund made the breakthrough in the most fortuitous of circumstances.
Monchengladbach put Dortmund under some pressure towards the end, but the home side held on for a slender win, ending their opponents' unbeaten start to the season and lifting Klopp's men off the bottom of the table.
Dortmund showed few signs of being in such wretched form during the early exchanges, causing their visitors significant problems when going forward.
In fact, Reus almost opened the scoring with just over a minute played, as he burst into the left side of the area and sent a controlled effort just wide of the opposite post.
Dortmund went even closer to capitalising on their superiority in the ninth minute as Reus pounced on Henrikh Mkhitaryan's cut-back and saw his first-time effort tipped on to the upright by Sommer.
Monchengladbach were left chasing shadows at times as Dortmund worked the ball excellently in the final third, but they could not apply finish when presented with chances.
Lukasz Piszczek was particularly guilty when he spurned a glorious chance in the 29th minute, latching on to Mkhitaryan's defence-splitting pass before firing well wide when one-on-one with Sommer.
And the Armenian attacker followed suit eight minutes later, inexplicably blazing over the bar with just Sommer to beat after racing on to Sven Bender's inch-perfect throughball.
Dortmund started the second half in much the same manner as they generally dictated the tempo, but Monchengladbach remained resolute at the back.
That was until Kramer attempted a first-time back-pass from inside the centre-circle and succeeded only in looping the ball over a stranded Sommer to score a remarkable own goal.
Kramer buried his head in his hands and things almost got worse for the visitors two minutes later, Reus crashing a 20-yard drive off the crossbar.
Dortmund afforded Monchengladbach significantly more of the ball in the latter stages, but Lucien Favre's side could not break down Klopp's organised defensive unit and the hosts held on for just their third Bundesliga win of the season.
