Phishing is an increasing organizational threat that causes billions in losses and damage to productivity, trade secrets, and reputation each year. This work explores how organizations can use gamification techniques to improve phishing detection efforts by individuals to create a human firewall. We build on cognitive evaluation theory to begin a new area of inquiry in gamification of IT security. With three experiments in a mock work setting, we test leaderboard components of validation, attribution, incentives, and public presentation for improvements in experiential (e.g., motivation) and instrumental outcomes (e.g., hits and false positives) in phishing reporting. Our findings suggest public attribution with rewards and punishments best balance the competing necessities of accuracy with widespread reporting. Further, our results demonstrate leaderboards’ unique benefits to phishing reporting over and above other phishing mitigation techniques (training and warnings). However, we noted that unintended consequences in false alarms may arise from shifts in motivation resulting from public display of incentives.