The Balance Error Scoring System (BESS) test is a measure of postural stability that is a commonly used tool in concussion diagnosis and management. The data regarding both intra-rater reliability and inter-rater reliability of the BESS test, however, is limited. PURPOSE: We set out to determine the inter-rater and intra-rater reliability of the BESS test as scored by certified athletic trainers (ATs) in a pediatric sports medicine clinic and to determine if supplementary education could improve these reliability measures. METHODS: We recruited six volunteer athletes who had sustained concussions to be videotaped for BESS testing. These pediatric athletes were between the ages of 10 and 16 years, and all of them were patients in our concussion clinic. Twenty ATs scored the 6 videotaped BESS tests before and after educational training. This training reviewed best practices for BESS scoring including possible errors. Two months later, the ATs again scored the same videotaped BESS tests. At each time point, the BESS tests were presented in a random order. We used Spearman correlation coefficients (rs) to determine the intra-rater reliability for the ATs and used intraclass correlation coefficients (ICC) to calculate inter-rater reliability for the ATs in comparison with the gold standard BESS score determined by our two lead ATs who provide education on correct BESS test scoring. RESULTS: The intra-rater reliability was high (rs = 0.876; P < 0.0001) immediately after the educational intervention but decreased significantly (rs = 0.703; P < 0.0001) 2 months later. The inter-rater reliability was adequate at all times tested (ICC range 0.603-0.782; P = 0.0076). There was a very small but non-significant improvement in inter-rater reliability after education but this improvement disappeared 2 months post-intervention. CONCLUSIONS: The BESS test has inadequate reliability for clinical decision-making in the diagnosis and management of pediatric sport-related concussions.