Pure data-driven approaches may not completely meet the needs of practical clinical scenarios without a human-in-the-loop (HITL). Thus, we will test human involvement by incorporating expert knowledge (EK) as provided by radiation oncologists into personalized adaptive radiotherapy (pART) predictive models in Non-Small-Cell Lung Cancer (NSCLC) to increase tumor local control (LC) and avoid radiation-induced toxicities such as radiation pneumonitis with grade≥2 (RP2). We analyzed 108 NSCLC patients treated on prospective protocols. Each patient had 289 features from different biophysical resources including dosimetric information, single-nucleotide polymorphisms, miRNAs, clinical factors, pre- and during-treatment cytokines and PET radiomics data. 68 patients were used for training; the rest were reserved for independent testing (IT). Selected EK factors were “stage”, “GTV”, “age”, “chemotherapy”, “tumor gEUD” for LC prediction and “total lung volume”, “smoking status”, “lung gEUD”, “the change of TGF beta1”, “chemotherapy” for RP2 prediction. First, joint prediction of LC and RP2 based on EK were emulated by naïve Bayesian networks (BNs). Secondly, hierarchical BNs were developed by learning from a machine only approach to identify biophysical relationships among dosimetric, imaging, and biological variables, patients’ characteristics and outcomes. Then, a subjective BN approach was developed to build an HITL decision support system (DSS) by integrating EK with hierarchical BNs to improve situation awareness of pART before and during radiotherapy. Area under the free-response receiver operating characteristics (AU-FROC) was used with cross-validation (CV) to evaluate simultaneous prediction performance of LC/RP2 in a joint BN modeling framework. BNs were successfully formed for prediction of LC and/or RP2 before and during radiotherapy from three machine learning scenarios. Hierarchical relationships among those data types can be appreciated in the resulting BN graphs. The prediction performance of HITL outperformed those of EK or machine only approaches resulting in tighter confidence intervals (CIs) (Table 1). Our subjective BNs based HITL DSS improves radiation outcome prediction compared to current EK and machine only, and has the potential to be an important component of pART DSS. However, it may need further external independent validation via multi-institutional collaborations.Abstract 194; Table 1Performances of the naïve, hierarchical and subjective BNsScenarioCohortTreatment TimepointLC Only (AUC)RP2 Only (AUC)Joint LC and RP2AU-FROC95% CIHuman OnlyCVPre0.610.620.710.51-0.72During0.620.610.730.54-0.76ITPre0.540.530.63-During0.550.530.65-Machine OnlyCVPre0.810.820.800.70-0.86During0.850.870.850.75-0.91ITPre0.760.740.77-During0.790.750.79-Human-in-the-LoopCVPre0.830.820.830.75-0.90During0.860.860.870.79-0.93ITPre0.770.750.78-During0.780.760.79- Open table in a new tab