Machine learning is a branch of artificial intelligence that allows computer systems to learn directly from examples, data, and experience. Statistical modeling is more about finding connections between variables and consequently the impact of these relationships, while also catering for prediction. It should be clear that these two methodologies are different in terms of their purpose, despite the fact that they use similar means to get there. The evaluation of the machine learning algorithm uses a set of tests to validate its accuracy. Although, for a statistical model, the analysis of regression parameters by confidence intervals, significance tests and other tests can be used to assess the legitimacy of the model. To demonstrate the applications and usefulness of this theory, an experimental study was conducted on zebrafish exposed to mycotoxin. Methods: Patulin (70 µg/L) and kojic acid (100 mg/L, 204 mg/L, and 284 mg/L) were administered by immersion to zebrafish once daily for a period of 7 days before the behavior testing. The following behavioral tests were performed: a novel tank test (NTT) (to assess the explorative behavior and anxiety); and a Y-maze test (which measures the spontaneous explorative behavior). Behavioral tests were performed on separate days. For the behavior tests, the statistical analysis was performed using ANOVA variation analysis (two-way ANOVA). All results are expressed as the mean ± standard error of the mean. The values of the general index F for which p < 0.05 were considered statistically significant. Results: Y-maze—patulin exposure led to an intensification of the locomotor activity and an increased traveled distance and number of arm entries. By increasing the spontaneous alternation between the aquarium’s arms, patulin has shown a stimulating effect on spatial memory. In the case of zebrafish exposed to 100 mg/L kojic acid, the traveled distance was shorter by 27% than the distance attained by those in the control group. The higher doses of kojic acid (204 mg/L and 284 mg/L) led to an increased locomotor activity, distance traveled, number of arm entries, and the spontaneous alternation. The increase in spontaneous alternation demonstrates that 204 mg/L and 284 mg/L kojic acid doses had a stimulating effect on spatial memory. Novel tank test—compared to the control group, the traveled distance of the patulin-exposed fish is slightly reduced. Compared to the control group, the traveled distance of the kojic acid-exposed fish is reduced, due to a shorter mobile time (by 25–27% in the case of fish exposed to 204 mg/L and 284 mg/L kojic acid). Patulin and kojic acid exhibit toxic effects on zebrafish liver, kidney, and myocardium and leads to severe alteration. We continued the analysis by trying some machine learning algorithms on the classification problems in the case of the two behavioral tests MAZE and NTT, after which we concluded that the results were better in the case of the NTT test relative to the MAZE test and that the use of decision tree algorithms leads to amazing results, knowing that their hierarchical structure allows them to learn signals from both classes. Conclusions: The groups exposed to patulin and kojic acid show histological changes in the liver, kidneys, and myocardial muscle tissue. The novel tank test, which assesses exploratory behavior, has been shown to be conclusive in the behavioral analysis of fish that have been given toxins, demonstrating that the intoxicated fish had a decreased explorative behavior and increased anxiety. We were able to detect a machine learning algorithm in the category of decision trees, which can be trained to classify the behavior of fish that were given a toxin in the category of those used in the experiment, only by analyzing the characteristic features of the NTT Behavior Test.