Aurantiochytrium strain BL10 is rich in highly unsaturated fatty acids (HUFAs), particularly docosahexaenoic acid (DHA). Previous research has shown that adding 2% of dry BL10 powder to increase the DHA content of feed for orange-spotted grouper (Epinephelus coioides) does not benefit growth but can significantly reduce size discrepancies between individuals and thereby potentially decrease the chance of cannibalism.To understand whether a 2% BL10 supplement can really reduce the cannibalism rates of E. coioides, whether increasing the BL10 dosage can reduce the cannibalism rates of E. coioides even further, and how the BL10 supplements affect cannibalistic behavior, we produced four isonitrogenous and isolipidic experimental diets dosed with different weight percentages (0, 2, 4, 6%) BL10 meal and fed it to E. coioides fingerlings (total length 3.8 ± 0.3 cm) for two weeks. During the feeding period, cannibalism in the experiment groups were observed. After the feeding period, we analyzed the brain fatty acid composition, brain 5-hydroxytryptamine (5-HT) levels, and serum cortisol levels in the experiment groups and used regression analysis to examine the relationships among these biochemical variants and cannibalism rates.The results indicate that 2–6% BL10 supplements, particularly 2% BL10 supplement, could significantly reduce the cannibalism rates of E. coioides. The BL10 supplements increased the proportions and contents of HUFAs in the brains of E. coioides, increased brain 5-HT levels, and reduced serum cortisol levels. The results of univariate linear regression indicated that the serum cortisol level exhibited a significant negative correlation with brain HUFA content, particularly arachidonic acid (ArA). Whereas, brain 5-HT was positively correlated with the ratio of n-3/n-6 fatty acids in the brain. Subsequent multivariate linear regression demonstrated that cannibalism is affected by serum cortisol, brain 5-HT levels and their interaction. Our results provide new insight into the physiological regulation mechanisms behind the cannibalistic behavior of teleost and provide a feasible solution at the same time.