Prostate marker assays are widely used for detection of prostate cancer (PCa) but are associated with considerable sensitivity and specificity problems. Therefore, we investigated prostatic protein glycosylation profiles as a potential biomarker. We determined the urinary asparagine-linked glycan (N-glycan) profile of prostatic proteins of healthy volunteers (n = 25), patients with benign prostate hyperplasia (BPH; n = 62) and newly diagnosed PCa patients (n = 42) using DNA-sequencer-assisted fluorophore-assisted carbohydrate electrophoresis. Through squeezing of the prostate, a sufficient amount of prostatic proteins was obtained for direct structural analyses of N-glycan structures. N-glycans of PCa compared to BPH were characterized by a significant decrease in triantennary structures (p = 0.047) and overall fucosylation (p = 0.026). Prostate-specific antigen (PSA) and the urinary glycoprofile marker showed comparable overall receiver operating characteristic curve analysis as well as in the diagnostic gray zone with serum PSA values between 4 and 10 μg/L. However, when combining PSA and the urinary glycoprofile marker, the latter gave an additive diagnostic value to serum PSA (p ≤ 0.001). In conclusion, N-glycosylation profiling demonstrated differences between BPH and PCa. These changes could lead to the discovery of a new biomarker for PCa.