Xenopus laevis tadpoles exhibit an avoidance behavior when they encounter a moving visual stimulus. A visual avoidance event occurs when a moving object approaches the eye of a free-swimming animal at an approximately 90-degree angle and the animal turns in response to the encounter. Analysis of this behavior requires tracking both the free-swimming animal and the moving visual stimulus both prior to and after the encounter. Previous automated tracking software does not discriminate the moving animal from the moving stimulus, requiring time-consuming manual analysis. Here we present X-Tracker, an automated behavior tracking code that can detect and discriminate moving visual stimuli and free-swimming animals and score encounters and avoidance events. X-Tracker is as accurate as human analysis without the human time commitment. We also present software improvements to our previous visual stimulus presentation and image capture that optimize videos for automated analysis, and hardware improvements that increase the number of animal-stimulus encounters. X-Tracker is a high throughput, unbiased, and significant time-saving analysis system that will greatly facilitate visual avoidance behavior analysis of Xenopus laevis tadpoles, and potentially other free-swimming organisms. The tool is available at https://github.com/ClineLab/Tadpole-Behavior-Automation.