Background and purposeAnatomical changes during radiotherapy pose a challenge to robustness of plans. Principal component analysis (PCA) is commonly used to model such changes. We propose a toolbox to evaluate how closely a given PCA model can represent actual deformations seen in the patient and highlight regions where the model struggles to capture these changes.Materials and methodsWe propose to calculate a residual error map from the difference between an actual displacement vector field (DVF) and the closest DVF that the PCA model can produce. This was done by taking the inner product of the DVF with the PCA components from the model. As a global measure of error, the 90th percentile of the residual errors ) across the whole scan was used. As proof of principle, we demonstrated this approach on both patient-specific cases and a population-based PCA in head and neck (H&N) cancer patients. These models were created using deformation data from deformable registrations between the planning computed tomography and cone-beam computed tomography (CBCTs), and were evaluated against DVFs from registrations of CBCTs not used to create the model.ResultsFor our example cases, the oropharyngeal and the nasal cavity regions showed the largest local residual error, indicating the PCA models struggle to predict deformations seen in these regions. ranged from 0.4 mm to 6.3 mm across the different models.ConclusionsA method to quantitatively evaluate how well PCA models represent observed anatomical changes was proposed. We demonstrated our approach on H&N PCA models, but it can be applied to other sites.