The Cleft Aesthetic Rating Scale (CARS) is a valid tool to assess the aesthetic outcome after unilateral cleft lip and palate (UCLP) repair. The goal of this study was to investigate whether the reliability of CARS increases by including fixed rates for atypical outcomes such as a smaller nostril, lower vermillion border or whistling deformity. 197 patients with UCLP who underwent cleft repair were included. Three cleft experts rated the post-operative photos using the original CARS, followed by a second assessment employing the modified CARS. Medisch Centrum Leeuwarden, Universitair Medisch Centrum Groningen, Radboud Universitair Medisch Centrum and Amsterdam University Medical Center. Photographs of 16- to 25-year-old patients with repaired UCLP. Two separate assessments using the original and modified CARS. The Intraclass Correlation Coefficient (ICC) was used to test inter-rater reliability of the original and modified version of the CARS. The reported ICC for the nose scored by means of the original and modified CARS were 0.68 (95% CI 0.62-0.74) and 0.66 (95% CI 0.59-0.72) respectively. The reported ICC for the lip assessed with the original and modified CARS were 0.53 (95% CI 0.33-0.67) and 0.57 (95% CI 0.34-0.72) respectively. No significant difference was found between the original and modified ICC of the nose and lip. The implementation of the fixed rates did not result in an increased reliability of the CARS. Therefore, we recommend the utilization of the original CARS.