ABSTRACT In this paper, a predator–prey model that takes into account the interactions between three species of fish, zooplankton, and phytoplankton has been developed. The logistic growth of phytoplankton has been considered in the absence of zooplankton. It is assumed that zooplankton eats phytoplankton. It is considered that the growth rate of zooplankton decreases due to the fear of fish. The release of toxins by phytoplankton and fish consumption are two further factors that are assumed to decrease the growth rate of zooplankton. The harvesting of fish has also been considered in this study. Local and global stability of the system around the equilibrium points has been investigated. The existence condition of oscillatory behaviour of the system has been studied through Hopf bifurcation. It is found that the increasing rate of minimum cost of fear may stabilize the system. It is obtained that the rate of increase of release of toxin substance by phytoplankton may be responsible for the stability of the system. It is also found that the system may become unstable for the increase of level of fear of zooplankton. Some numerical simulation results are presented to test the theoretical findings.