ABSTRACT Objective Competency-based assessment on placement ideally involves evaluation of a trainee performing clinical skills using fit-for-purpose tools that are simple for clinical supervisors to use and informative to the trainee. The current study examined the potential utility of a new tool, the Psychology Competency Evaluation Tool – Summative (PsyCET-S), designed to assess the attainment of professional competencies in clinical psychology trainees. Method Twelve experts tested the inter-rater reliability of the vignette ordering, before the tool was used to assess performance of clinical psychology trainees on placement by both supervisors and trainees. Ninety-nine placement reviews of 45 trainees at different stages of training were analysed. Results Strong inter-rater reliability in the rank ordering of vignettes within each domain indicated their ability to capture competence development. Supervisor and trainee ratings showed a gradual increase in competence from early to later stages of training in all domains, with performance plateauing for the final placement. Evidence of both the leniency bias and halo effect were evident, with supervisors appearing more lenient than the trainees themselves. Conclusion The PsyCET-S encourages supervisors to evaluate the development of professional competencies in their trainees in a way that can optimise trainee learning and future practice as a Psychologist.