PurposeFor timely treatment of extrahepatic metastasis and macrovascular invasion (aggressive progressive disease [PD]) in hepatocellular carcinoma, models aimed at stratifying the risks of subsequent aggressive PD should be constructed.Patients and MethodsAfter dividing 332 patients from five hospitals into training (n = 236) and validation (n = 96) datasets, non-invasive models, including clinical/semantic factors (ModelCS), deep learning radiomics (ModelD), and both (ModelCSD), were constructed to stratify patients according to the risk of aggressive PD. We examined the discrimination and calibration; similarly, we plotted a decision curve and devised a nomogram. Furthermore, we performed analyses of subgroups who received different treatments or those in different disease stages and compared time to aggressive PD and overall survival in the high- and low-risk subgroups.ResultsAmong the constructed models, ModelCSD, combining clinical/semantic factors and deep learning radiomics, outperformed ModelCS and ModelD (areas under the curve [AUCs] for the training dataset: 0.741, 0.815, and 0.856; validation dataset: 0.780, 0.836, and 0.862), with statistical difference per the net reclassification improvement, the integrated discrimination improvement, and/or the DeLong test in both datasets. Besides, ModelCSD had the best calibration and decision curves. The performance of ModelCSD was not affected by treatment types (AUC: resection = 0.839; transarterial chemoembolization = 0.895; p = 0.183) or disease stages (AUC: BCLC [Barcelona Clinic Liver Cancer] stage 0 and A = 0.827; BCLC stage AB &B = 0.861; p = 0.537). Moreover, the high-risk group had a significantly shorter median time to aggressive PD than the low-risk group (training dataset hazard ratio [HR] = 0.108, p < 0.001; validation dataset HR = 0.058, p < 0.001) and poorer overall survival (training dataset HR = 0.357, p < 0.001; validation dataset HR = 0.204, p < 0.001).ConclusionOur deep learning-based model successfully stratified the risks of aggressive PD. In the high-risk population, current guideline indicates that first-line treatments are insufficient to prevent extrahepatic metastasis and macrovascular invasion and ensure survival benefits, so more therapies may be explored for these patients.