The cognitive impairment of pituitary adenomas (PAs) has received increasing attention. Hyperprolactinemia and tumor mass effect are the potential causes. The aim of this study was to identify possible cognitive impairment and to further explore the correlation between these indices and prolactin (PRL) levels, based on the control of tumor size. Twenty-seven patients with prolactinomas (patient group) and twenty-six matched health control group (HC group) were enrolled in this study. All participants performed the flanker task while we continuously recorded electroencephalography data. On the behavioral performance level, patients showed a significantly slower reaction time (RT) in both flanker types. Concerning the event-related potentials level, patients elicited reduced P2 and enhanced N2 amplitudes compared with the HC group, suggesting an impairment of attentional processing (P2) and conflict monitoring (N2). Moreover, the patient group also induced lower P3 amplitudes relative to the HC group in both types, indicating that there were deficits in attentional resource allocation ability. We also found a significant correlation between the P3 amplitudes and incongruent condition RTs, as well as the subsequent PRL levels in the patient group. In conclusion, this is an innovative study that reveals the impaired cognition abilities in prolactinomas, and also proposes the possible cognitive toxicity of oversecreted PRL levels, which provides evidence for further research on the cognitive decline in PAs.