Concept inventories are multiple choice exams designed with the intention to test core concepts on specific subjects and evaluate common misconceptions. These tests serve as a useful tool in the classroom to assess value added by the instructor's educational methods and to better understand how students learn. They can provide educators with a method to evaluate their current teaching strategies and to make modifications that enhance student learning and ultimately elevate the quality of medical physics education. The use of concept inventories in introductory college physics courses revealed important gaps in conceptual understanding of physics by undergraduate students and motivated a shift of physics teaching towards more effective methods, such as active learning techniques. The goal of this review is to introduce medical physicists to concept inventories as educational evaluation tools and discuss potential applications to medical physics education by development through multi-institutional collaboration.