Most patients with prostate adenocarcinoma develop resistance to therapies targeting the androgen receptor (AR). Consequently, a portion of these patients develop AR-independent neuroendocrine prostate cancer (NEPC), a rapidly progressing cancer with limited therapies and poor survival outcomes. Current research to understand the progression to NEPC suggests a model of lineage plasticity whereby AR-dependent luminal-like tumors progress towards an AR-independent NEPC state. Genetic analysis of human NEPC identified frequent loss of RB1 and TP53, and the loss of both genes in experimental models mediates the transition to a neuroendocrine lineage. Transcriptomics studies have shown that lineage transcription factors ASCL1 and NEUROD1 are present in NEPC. In this study, we modeled the progression of prostate adenocarcinoma to NEPC by establishing prostate organoids and subsequently generating subcutaneous allograft tumors from genetically-engineered mouse models harboring Cre-induced loss of Rb1 and Trp53 with Myc overexpression (RPM). These tumors were heterogeneous and displayed adenocarcinoma, squamous, and neuroendocrine features. ASCL1 and NEUROD1 were expressed within neuroendocrine-defined regions, with ASCL1 being predominant. Genetic loss of Ascl1 in this model did not decrease tumor incidence, growth, or metastasis; however, there was a notable decrease in neuroendocrine identity and an increase in basal-like identity. This study provides an in vivo model to study progression to NEPC and establishes the requirement for ASCL1 in driving neuroendocrine differentiation in prostate cancer.