Self-assessment is an important aspect in the development of lifelong learning skills for medical students, crucial to maintaining a high level of competence in practice. Basic Life Support (BLS) is a skill that all health professionals must acquire and maintain competence in. This paper reports data from a study of 95 first-year graduate entry medical students at Flinders University in Australia, determining how well the students could assess their own BLS performance. These students were videotaped performing a short CPR scenario using a Resusci®Anne with SkillReporter™ (Laerdal, Norway). Using a six-point descriptive ratings scale, students graded themselves twice: once immediately after completing the task, and again after viewing a video of their performance. A single expert assessor viewed all the video recordings and, based on International Liaison Committee on Resuscitation (ILCOR) Guidelines, graded the students using the same scale. The hypothesis was that the intervention of viewing their performance on video would improve the correlation of their ratings with the expert assessor. The results showed that the students’ assessments did not agree with the expert assessor either before (weighted κ = 0.03) or after seeing the video (weighted κ = 0.002). Possible reasons, including student attitudes and lack of benchmarking, are discussed. Self-assessment skills of students warrant further attention.