Vincenzo Montella's side moved ahead through Josip Ilicic's 20th-minute strike, but were pegged back at the Stadio Artemi Franchi on the hour mark by Guillermo Giacomazzi's leveller.
The Serie B outfit looked good value for extra time, but were denied by Compper's decider on 76 minutes.
Former Palermo midfielder Ilicic was one of five changes for Fiorentina, and he responded to his inclusion by smashing the opener past Simone Farelli.
The situation could have been more comfortable for the hosts who saw Joaquin and Alberto Aquilani squander good opportunities.
Their first-half wastefulness was punished after the braek when Giacomazzi flicked a header past Neto in the Fiorentina goal.
However, the home fans were celebrating the winner with 14 minutes left on the clock when Compper headed home from six yards to book Fiorentina's spot in the final four.
