Brown guided Hull to the Premier League in 2008 and victory over his former side would have seen Southend match their joint best FA Cup run, but the top-flight side's superiority showed at Roots Hall.
Goalscoring chances were relatively rare in the first half as Barry Corr and Yannick Sagbo both spurned their respective teams' best openings, before Michael Timlin struck the crossbar early in the second period for the League Two outfit.
The breakthrough finally came via Matty Fryatt with just over an hour gone as he produced a well-worked finish after a clever run to give Hull the lead.
Fryatt then added a second in stoppage time to make sure of their place in the fifth round.
Brown made two changes from the team that beat Millwall in the third round, reinstating Mark Phillips and top-scorer Kevan Hurst to his starting XI.
Hull had beaten Middlesbrough in the last round and among the three alterations from that side was Danny Graham, who was given the all-clear to play by parent club Sunderland
The tone of match was clear from early on as Hull attempted to string together intricate passing moves on the poor pitch, while Corr's early clattering of Maynor Figueroa suggested a physical encounter was in store.
The first chance of the game arrived in the 25th minute as Corr flicked on Anthony Straker’s long throw-in at the near post, but Steve Harper acrobatically tipped it over the crossbar.
Hull wasted a magnificent chance nine minutes later as George Boyd squared a pass across the edge of the area to Graham on the counter-attack, but he hit his tame effort straight at goalkeeper Daniel Bentley.
Southend almost got off to the perfect start in the second period as Timlin hit a fierce drive from 30 yards, but his effort came back off the crossbar to the relief of Harper.
Graham and David Meyler then forced Bentley in to good saves before the hour mark, but Hull did eventually take the lead in the 63rd minute.
Fryatt took the ball off Graham's foot on the edge of the area, but it proved to be a good decision as he skipped past one challenge, held off another and slotted into the bottom-right corner.
An away win rarely looked likely in the final stages, with Hull solid at the back and dangerous on the break.
And they ensured their passage by creating some breathing space in the 92nd minute, as Fryatt fired in from a narrow angle after good footwork in the area.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
