Marseille, who have won the Cup a record 10 times but have not triumphed since 1989, struggled on a frozen pitch away to opponents playing in the fourth division.
The last 64 game was a 1-1 draw after normal and extra time and Marseille won the shoot-out 5-4, midfielder Benoit Cheyrou scoring the decisive penalty after an effort by Besancon striker Yohann Rangdet was blocked by Marseille keeper Steve Mandanda.
"My players did what they could in very difficult conditions," Marseille coach Eric Gerets told reporters after a game played in freezing temperatures. "That pitch was really, really hard."
The former European champions were 1-0 down with seven minutes left in normal time when Nigeria defender Taye Taiwo put them level from a penalty after Besancon defender Yannick Goyon handled the ball in the area.
Defender Ronald Zubar had put Besancon in front by kicking the ball into his own net under pressure from Rangdet on 67 minutes.
Besancon then resisted bravely under sustained pressure during extra time and did not show signs of nerves until the very end of the shoot-out.
Marseille, who were eliminated by fifth division Carquefou in the round of 16 last season, could face Olympique Lyon in the next round.
Following Sunday's draw for the round of 32, Lyon will entertain Marseille, providing the seven-times champions win their round of 64 tie against Concarneau, postponed twice this weekend because of a frozen pitch. No new date has been set.
Paris St Germain needed a second-half goal from Stephane Sessegnon to beat fourth division Montlucon 1-0 earlier on Sunday.
Benin midfielder Sessegnon tapped in from a corner on 52 minutes to make sure last season's losing finalists survived the first round involving top-flight clubs.
Monaco had earlier struggled to overcome fifth division Blagnac, needing an own goal from defender Frederic Marquet on 21 minutes to win 1-0.
