Ginczek struck the decisive goal in the first minute of second-half stoppage-time at the Mercedes Benz Arena, capping off a dramatic finale, which included a red card to Stuttgart forward Martin Harnik and an equaliser from Bremen with four minutes left.
But as the clock ticked past 90 minutes in Stuttgart, the home side launched a counter-attack, which saw Geoffroy Serey Die burst through the middle, before sliding a pass to Ginczek and the striker finished between the legs of Bremen goalkeeper Raphael Wolf.
The home fans erupted in joy as Stuttgart moved ahead of last position in the Bundesliga standings for the first time since February 7.
Stuttgart (26 points) sit 17th in one of two automatic relegation positions, just ahead of Hamburg (25), with Paderborn (27) in 16th; a spot that entails playing a two-legged play-off with Bundesliga 2's third side in order to stay in the top division.
"We put in a huge performance," Ginczek told Stuttgart's website.
"Even with a man down and after conceding that late equaliser we still went all out for the win – and the winner was just pure emotion. We had that little bit of luck on our side in the end.
"We're still in a relegation place but we're determined to battle our way out of trouble. That's what we're all working towards and everyone has a part to play."
Stuttgart were reeled in twice before Ginczek finally gave them victory with Christian Gentner's drive on the quarter-hour mark cancelled out by a header from Davie Selke four minutes into the second half.
After Harnik wasted a number of gilt-edged chances to score around the hour mark, the Austrian teed up Ginczek in the 70th minute to make it 2-1, only for Jannik Vestergaard to equalise for Bremen from a corner 16 minutes later.
Coach Stevens hailed Stuttgart's "courage" to keep attacking, especially after a raft of misses and Harnik's second yellow card two minutes prior to Vestegaard's header.
"We should have done better with our chances in both halves and won it comfortably really, so we only have ourselves to blame in that regard," the 61-year-old coach said.
"The team showed a lot of courage and in the end they got the necessary stroke of luck. But we worked for it and that was a result of a concerted effort."
