Background Chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS) is one of the most common diseases in urology, but its pathogenesis remains unclear. As a kind of chronic pain which the patients suffered for more than 3 months, we investigated the influence on patients' brain functional connectivity in resting state. Methods We recruited a cohort of 18 right-handed male patients with CP/CPPS and 21 healthy male right-handed age-matched controls. Their resting-state fMRI data and structural MRI data were preprocessed and processed by RESTPlus V1.22. To assess the integrity of the default mode network (DMN), we utilized the voxel-wised analysis that we set medial prefrontal cortex (mPFC) and posterior cingulate gyrus (PCC) as seed points to compare the global functional connectivity (FC) strength. Results Compared with healthy control, the FC strength between left mPFC and posterior DMN decreased in the group of CP/CPPS (P < 0.05, GFR correction, voxel P < 0.01, cluster P < 0.05), and the FC strength between the left anterior cerebellar lobe and posterior DMN increased (P < 0.05, GFR correction, voxel P < 0.01, cluster P < 0.05). In the patient group, there was a positive correlation between the increased FC strength and the score of the Hospital Anxiety and Depression Scale (HADS) anxiety subscale (r = 0.5509, P = 0.0178) in the left anterior cerebellar lobe, a negative correlation between the decreased FC strength and the score of the National Institutes of Health Chronic Prostatitis Symptom Index (r = −0.6281, P = 0.0053) in the area of left mPFC, and a negative correlation between the decreased FC strength and the score of HADS anxiety subscale (r = −0.5252, P = 0.0252). Conclusion Patients with CP/CPPS had alterations in brain function, which consisted of the default mode network's compromised integrity. These alterations might play a crucial role in the pathogenesis and development of CP/CPPS.