Pupillometry is widely used to measure arousal states. The primary functional role of the pupil, however, is to respond to the luminance of visual inputs. We previously demonstrated that cognitive effort-related arousal interacted multiplicatively with luminance, with the strongest pupillary effects of arousal occurring at low-to-mid luminances (< 37 cd/m2), implying a narrow range of conditions ideal for assessing cognitive arousal-driven pupillary differences. Does this generalize to other forms of arousal? To answer this, we assessed luminance-driven pupillary response functions while manipulating emotional arousal, using well-established visual and auditory stimulus sets. At the group level, emotional arousal interacted with the pupillary light response differently from cognitive arousal: the effects occurred primarily at much lower luminances (< 20 cd/m2). Analyses at the individual-participant level revealed qualitatively distinct patterns of modulation, with a sizable number of individuals displaying no arousal response to the visual or auditory stimuli, regardless of luminance. Together, our results suggest that effects of arousal on pupil size are not monolithic: different forms of arousal exert different patterns of effects. More practically, our findings suggest that lower luminances create better conditions for measuring pupil-linked arousal, and when selecting ambient luminance levels, consideration of the arousal manipulation and individual differences is critical.