The loss marked Hertha's 11th in the league this season and dropped the capital club to second bottom in the division with just 18 points.
Dutchman Luhukay was appointed in July 2012 and led them to a second promotion in three seasons in his maiden campaign in charge.
Hertha finished 11th last term but, after claiming just five league wins in 2014-15, they have decided to part company with Luhukay and assistants Markus Gellhaus and Rob Reekers.
A statement on Hertha's website read: "Hertha BSC would like to thank Jos Luhukay and his assistants Markus Gellhaus and Rob Reekers for their very successful work over the last years.
"Together we were able to get promoted to the first division of the Bundesliga and the year afterwards we managed to comfortably avoid relegation."
Youth team coach Pal Dardai will take charge of the team for Saturday's trip to Mainz, assisted by Rainer Widmayer.
Luhukay is the fifth coach from the top flight to leave his club this term, with Mirko Slomka, Jens Keller, Robin Dutt and Armin Veh suffering the same fate.
