Jackfruit (Artocarpus heterophyllus), a tropical fruit renowned for its diverse culinary uses, necessitates identifying the optimal growth stage to ensure superior flavor and texture. This research investigates employing deep learning techniques, particularly convolutional neural networks (CNNs), for accurately detecting jackfruit growth stages. Despite the challenge posed by the nuanced visual differences among fruits at various maturity stages, a meticulously curated dataset of labeled jackfruit images was developed in collaboration with experts, utilizing the BBCH scale. This dataset facilitated training and evaluation. A modified version of the Places 365 GoogLeNet CNN model was proposed for classifying four distinct growth stages of jackfruit, compared with a state-of-the-art CNN model. The trained models demonstrated varying levels of accuracy in classification. Furthermore, the proposed CNN model was trained and tested using original and augmented images, achieving an impressive overall validation accuracy of 90%. These results underscore the efficacy of deep learning in automating the detection of growth stages, offering promising implications for quality control and decision-making in jackfruit production and distribution.