Pituitary neuroendocrine tumors (PitNETs) are one of the most common types of intracranial tumors. Currently, the cellular characteristics of normal pituitary and various other types of PitNETs are still not completely understood. We performed single-cell RNA sequencing (scRNA-seq) on 4 normal samples and 24 PitNET samples for comprehensive bioinformatics analysis. Findings regarding the function of PBK in the aggressive tumor cells were validated by siRNA knockdown, overexpression, and transwell experiments. We first constructed a reference cell atlas of the human pituitary. Subsequent scRNA-seq analysis of PitNET samples, representing major tumor subtypes, shed light on the intrinsic cellular heterogeneities of the tumor cells and tumor microenvironment (TME). We found that the expression of hormone-encoding genes defined the major variations of the PIT1-lineage tumor cell transcriptomic heterogeneities. A sub-population of TPIT-lineage tumor cells highly expressing GZMK suggested a novel subtype of corticotroph tumors. In immune cells, we found two clusters of tumor-associated macrophages, which were both highly enriched in PitNETs but with distinct functional characteristics. In PitNETs, the stress response pathway was significantly activated in T cells. While a majority of these tumors are benign, our study unveils a common existence of aggressive tumor cells in the studied samples, which highly express a set of malignant signature genes. The following functional experiments confirmed the oncogenic role of selected up-regulated genes. The over-expression of PBK could promote both tumor cell proliferation and migration, and it was also significantly associated with poor prognosis in PitNET patients. Our data and analysis manifested the basic cell types in the normal pituitary and inherent heterogeneity of PitNETs, identified several features of the tumor immune microenvironments, and found a novel epithelial cell sub-population with aggressive signatures across all the studied cases.