Winger Mark Duffy made it a miserable debut for Chris Dunn when he caught the Yeovil goalkeeper out at his near post with a free-kick eight minutes from time on Friday.
Dean Furman had put Rovers in front with his first goal for the club, but was later sent off for a second bookable offence after Byron Webster had equalised just before half-time.
Paul Dickov's side held on for a win that moved them up to 18th in the table, while Yeovil remain rooted to the bottom.
Rovers made a bright start and they were rewarded when on-loan Manchester United striker Federico Macheda - making his second debut for the club - picked out Furman in the penalty area and the midfielder finished with his left foot after 10 minutes.
Yeovil responded and they were level three minutes before half-time when Shane Duffy headed into the path of Webster and the defender scored from close range.
Rovers' chances of securing three points looked to have diminished when Furman was shown a second yellow card for a late challenge on Ed Upson with just over 10 minutes remaining.
However, Mark Duffy beat Dunn with a free-kick from the left of the penalty area that curled inside the near post to win it for the home side eight minutes from time.
