Background: Genistein (GEN) shows significant anticancer potential, particularly against prostate cancer. However, its clinical application is limited by poor water solubility, rapid metabolism and excretion, low bioavailability, and lack of targeted delivery to cancer cells, hindering its effectiveness as a chemopreventive or therapeutic agent. Objective: In this study, poly-ε-caprolactone (PCL) nanoparticles incorporating polyvinyl alcohol (PVA) as a stabilizer were engineered to encapsulate genistein (GEN) effectively. Utilizing a Quality by Design (QbD) methodology, the development and optimization of these nanoparticles were systematically approached. Methods: GEN-loaded PCL nanoparticles (NPs) were prepared using the Solvent Evaporation Technique, ideal for encapsulating hydrophobic drugs. A Plackett–Burman design (PBD) identified key factors, followed by a Box–Behnken design (BBD) to optimize nanoparticle quality. The NPs were evaluated for particle size, zeta potential (ZP), polydispersity index (PDI), morphology, encapsulation efficiency (EE), in vitro drug release, and cytotoxicity. Results: The optimized formulation containing PCL, PVA, and Volume of organic solvent as 43.7 mg, 6.2 mg, and 10.0 ml, respectively was chosen because it showed EE (%) of 94.0%, average particle size of 150 nm, PDI of 0.10, ZP of -28.0 and exhibited sustained release of GEN for around four days. The antiproliferative activities of GEN PCL NPs were confirmed by the MTT test in vitro on malignant prostate carcinoma cell lines (PC3). Flow cytometric analysis showed that the inhibition of cell proliferation of more potent GEN PCL NPs is comparable with the effects of free GEN. Conclusion: The findings indicate that genistein-loaded PCL nanoparticles have the potential to augment the anticancer efficacy of genistein, both in vitro and in vivo. This suggests their promise as a viable candidate for prostate cancer treatment.