Cerebral palsy (CP) is a neurological disorder caused by cerebral ischemia and hypoxia during fetal brain development. Early intervention in CP patients has a favorable impact on medications and therapy; however, challenges are associated with early age development, potential recovery, and neuroimaging sensitivity. Previously, the main focus was given on early diagnosis, brain injury characterization, and neurodevelopmental outcomes in infants; however, it is inevitable to study the brain developmental patterns in health controls (HC) and CP children. This study elaborates on the effects of aging on the brains of infants with HC and CP, age ranging from a few months and 17 years using deep learning (DL). The introduced DL-based models, namely SCT-DL and MCT-DL, trained on single contrast MRI (SC-MRI) and coupling of multi-contrast (MC-MRI) to learn vulnerable brain regions associated to brain development. The proposed models are equipped with fully-convolutional-based attention mechanism leveraging average pooling operation, directly passing salient features, parallel partial computing unit, specialized parameter sharing module, fusion, and spatial-channel attention with traversal placement to learn and associate brain development accurately. In addition, the study reports optimal SC-MRI and MC-MRI deeply associated with CP vulnerabilities and association to brain development. The SCT-DL outperformed the counterpart models with accumulative MAE=1.73 (C11C denotes HC and P denotes CP.=1.61 and P=1.63) underlying T1-w, where the MCT-DL is the improved version of SCT-DL and the prediction accuracy reached to MAE=1.08 (C=1.01 and P=1.12) over the coupling of T1-w⊕Sagittal. The trained models result in unmatching and distinct learning patterns for healthy and CP patients. Notably, the age-wise brain development-based results deduce a significance in age association at an early age (around two years) and poor at a later age. The study findings will assist neurodevelopmental processing and clinical practices in radiomics without concerning infants’ uncooperative movement or MRI artifacts.