Altered subjective visual sensitivity manifests as feelings of discomfort or overload elicited by intense and irritative visual stimuli. This can result in a host of visual aberrations including visual distortions, elementary visual hallucinations and visceral responses like dizziness and nausea, collectively referred to as "pattern glare." Current knowledge of the underlying neural mechanisms has focused on overall excitability of the visual cortex, but the individual contribution of excitatory and inhibitory systems has not yet been quantified. In this study, we focus on the role of glutamate and γ-aminobutyric acid (GABA) as potential mediators of individual differences in subjective visual sensitivity, measured by a computerized Pattern Glare Test-a series of monochromatic square-wave gratings with three different spatial frequencies, while controlling for psychological variables related to sensory sensitivity with multiple questionnaires. Resting neurotransmitter concentrations in primary visual cortex (V1) and right anterior insula were studied in 160 healthy participants using magnetic resonance spectroscopy. Data showed significant differences in the perception of visual distortions (VD) and comfort scores between men and women, with women generally reporting more VD, and therefore the modulatory effect of sex was considered in a further examination. A general linear model analysis showed a negative effect of occipital glutamate on a number of reported visual distortions, but also a significant role of several background psychological traits. When assessing comfort scores in women, an important intervening variable was the menstrual cycle. Our findings do not support that baseline neurotransmitter levels have a significant role in overreactivity to aversive stimuli in neurotypical population. However, we demonstrated that biological sex can have a significant impact on subjective responses. Based on this additional finding, we suggest that future studies investigate aversive visual stimuli while examining the role of biological sex.