Radiotherapy (RT) is essential in head and neck cancer (HNC) treatments, but often causes significant toxicity. Different machine learning models have shown promise in predicting RT-induced toxicity, but none have yet integrated the fluctuating anatomical changes. By integrating daily cone-beam CTs (CBCT) allowing sequential anatomical views, our aim is to build a dynamic predictive model for three major HNC RT toxicities: reactive feeding tube placement, hospitalization and radionecrosis (RN). 292 HNC cases treated with curative RT between 2017 and 2019 at our institution were retrospectively analyzed for clinical and radiological data. VoxelMorph, a deep deformable registration model, integrated the daily anatomical deformations between each CBCT and the planning CT, then converted them to Jacobian determinant matrix (Jf). Resnet, a convolutional neural network with multiple layers was trained using a 5-fold cross validation to integrate both radiological and clinical data. Each toxicity was classified as a binary decision using the cross-entropy loss to account for a class imbalance. Its predictive performance was compared to the baseline model using only clinical data. The cohort included 78% men and 22% women, with a median age of 63 years (range 35-84). Primary cancer sites were 46% oropharynx, 19% larynx, 14% oral cavity, 7.5% nasopharynx, 5% hypopharynx, 4% unknown primary and 5% others; and stage ranged between Tx-4b N0 and 3b M0 (AJCC 8th Ed). Induction chemotherapy, concurrent chemotherapy, and adjuvant RT was used in 9%, 57% and 20% of patients, respectively. The incidence of feeding tube, hospitalization and RN was 19.9%, 7.2%, and 3.8%, respectively. Integrating Jf from the 10th RT CBCT showed better accuracy for each toxicity prediction: feeding tube (69.1% > 57.2%), hospitalization (75.3% > 63.1%) and RN (85.8% > 75.7%). Integrating both the raw CBCT and Jf improved hospitalization prediction (79.0% > 73.6%). Substituting Jf for the raw CBCT improved the prediction for RN (79.7% > 74.7%) and hospitalization (73.6% > 64.4%). For feeding tube, predictive performance of the Jf model trained against deformations showed a positive correlation between its performance and the RT received (r2 > 0.9) with increasing RT fractions, with a maximum accuracy of 83.1% at the 25th fraction. No such correlation was found for RN or hospitalization prediction. To our knowledge, this is the first study showing promising results to predict HNC RT toxicities using daily per-treatment CBCT. Next steps involve integrating both the radiomic and the dosimetric inputs to build a more powerful model. This could expand to predict therapeutic outcomes and, ultimately, could guide decisions in individualized RT.