Number sense, the ability to discriminate the quantity of objects, is crucial for survival. To understand how neurons work together and develop to mediate number sense, we used two-photon fluorescence light sheet microscopy to capture the activity of individual neurons throughout the brain of larval Danio rerio, while displaying a visual number stimulus to the animal. We identified number-selective neurons as early as 3 days post-fertilization and found a proportional increase of neurons tuned to larger quantities after 3 days. We used machine learning to predict the stimulus from the neuronal activity and observed that the prediction accuracy improves with age. We further tested ethanol's effect on number sense and found a decrease in number-selective neurons in the forebrain, suggesting cognitive impairment. These findings are a significant step towards understanding neural circuits devoted to discrete magnitudes and our methodology to track single-neuron activity across the whole brain is broadly applicable to other fields in neuroscience.