Martinez's side looked finished in the third round replay at Upton Park, where Enner Valencia had opened the scoring before Aiden McGeady's red card.
But Kevin Mirallas levelled with a brilliant free-kick and Romelu Lukaku put the visitors ahead in extra time.
Carlton Cole drew West Ham level to 2-2 and they claimed a thrilling shoot-out 9-8, leaving Martinez distraught.
"It is heartbreaking because you have the hardest role when you go down to 10 men for such a long period," he said.
"I thought we were the better side. We created really, really good opportunities and probably we should have taken one or two before the one we scored.
"In terms of the performance and the character and the phases of good football that we showed, I couldn't be prouder."
Steven Naismith and Stewart Downing failed to convert in the shoot-out, with the latter missing the opportunity to win it for West Ham.
But Everton goalkeeper Joel Robles smashed his spot-kick into the crossbar before West Ham shot-stopper Adrian proved to be the hero when he converted the decisive penalty.
Martinez said his team deserved better.
"Sometimes in football you feel you are going to get what you deserve but clearly we were unfortunate in the penalty shootout today, when it was a lottery," he said.
"From the support that we had from our away end, to the desire of players coming on as subs and adapting to the flexibility of the tactics, I couldn't be prouder."
