Animals from insects to humans perform visual escape behavior in response to looming stimuli, and these responses habituate if looms are presented repeatedly without consequence. While the basic visual processing and motor pathways involved in this behavior have been described, many of the nuances of predator perception and sensorimotor gating have not. Here, we have performed both behavioral analyses and brain-wide cellular-resolution calcium imaging in larval zebrafish while presenting them with visual loom stimuli or stimuli that selectively deliver either the movement or the dimming properties of full loom stimuli. Behaviorally, we find that, while responses to repeated loom stimuli habituate, no such habituation occurs when repeated movement stimuli (in the absence of luminance changes) are presented. Dim stimuli seldom elicit escape responses, and therefore cannot habituate. Neither repeated movement stimuli nor repeated dimming stimuli habituate the responses to subsequent full loom stimuli, suggesting that full looms are required for habituation. Our calcium imaging reveals that motion-sensitive neurons are abundant in the brain, that dim-sensitive neurons are present but more rare, and that neurons responsive to both stimuli (and to full loom stimuli) are concentrated in the tectum. Neurons selective to full loom stimuli (but not to movement or dimming) were not evident. Finally, we explored whether movement- or dim-sensitive neurons have characteristic response profiles during habituation to full looms. Such functional links between baseline responsiveness and habituation rate could suggest a specific role in the brain-wide habituation network, but no such relationships were found in our data. Overall, our results suggest that, while both movement- and dim-sensitive neurons contribute to predator escape behavior, neither plays a specific role in brain-wide visual habituation networks or in behavioral habituation.