We determined the optimal timing for initiating androgen receptor signaling inhibitor (ARSI) therapy in patients with nonmetastatic castration-resistant prostate cancer (nmCRPC) and assessed its impact on oncological outcomes. This retrospective study included 145 nmCRPC patients who received enzalutamide, apalutamide or darolutamide at the Jikei University Hospital or its affiliated institutions between May 2014 and November 2022. Patients were stratified based on prostate-specific antigen (PSA) doubling time (PSADT) at CRPC diagnosis and PSA levels at ARSI initiation. Oncological outcomes, including progression-free survival (PFS), metastasis-free survival (MFS), cancer-specific survival and overall survival, were assessed using the Kaplan-Meier curve and Cox regression analysis. The median age of the patients was 73 (interquartile range [IQR]: 52-88) years, and the median follow-up duration was 36 (IQR: 2-104) months. The median PSA level at ARSI initiation was 5.4 (IQR: 2.2-48) ng/ml, and 44.8% of patients had a PSADT <3months. Multivariate analysis revealed that PSADT and PSA levels at ARSI initiation were independent MFS predictors. Patients with a PSADT ≤3months and a PSA level≥5.4ng/ml experienced significantly reduced PFS and MFS. Notably, ARSI initiation at a PSA level ≥5.4ng/ml was associated with worse outcomes, suggesting the potential benefit of earlier intervention. Patients with rapid PSADT are at increased risk of early disease progression, suggesting that immediate treatment may be warranted. In addition, initiating therapy at a PSA level <5.4ng/ml may be associated with improved patient outcomes in patients with low PSADT.