The 61-year-old returned to Priestfield for a second spell in charge in October 2013 to replace Martin Allen, originally on a temporary basis before taking the reins full-time a month later.
Taylor - who also managed Gillingham between 1999 and 2000 - steered the club a 17th-place finish in the 2013-14 campaign, avoiding relegation by six points.
However, Gillingham have continued to struggle this term and they sit outside the bottom four by just two points, having lost 10 of their 23 matches thus far.
A 3-1 loss at home to leaders Bristol City on Sunday proved the final straw for Taylor.
"Peter Taylor has been relieved of his duties with immediate effect," read a statement on the club's official website.
"The chairman has asked the coaching staff to assist Andy Hessenthaler with this weekend's game against Port Vale."
