The development of math skills is crucial for adequate functioning in academic and professional settings as well as in daily life. A factor that has been shown to negatively influence performance and acquisition of math skills is math anxiety. With the high prevalence of math anxiety in society and the long lasting effects on math performance, it is important to study the relation between math anxiety and math performance in young children. Since math anxiety is often more pronounced in women than in men, it is essential to take the effect of gender into account. While the effect of gender on the relation between math anxiety and math performance has been studied in adults and adolescents, less research has focused on children, especially children at young ages. To fill this gap, the current study examined how the relation between math anxiety and math performance differed between boys and girls in early elementary school years. Math anxiety and math performance was assessed in 124 second- and fourth-grade children (67 girls and 57 boys). Although boys and girls showed more or less equal levels of math anxiety and performed similarly at the arithmetic task, correlation analyses showed that only in girls, math anxiety significantly correlated with math performance. Analyses investigating if math anxiety moderated the effect of gender and grade on math performance revealed significant differences between boys and girls. Higher levels of math anxiety only significantly and negatively moderated math performance in girls, with the greatest effect observed in 2nd grade girls. These findings highlight the importance of taking gender differences into account when studying the effect of math anxiety. The results showed that math anxiety is already negatively linked to math performance in girls as early as second grade. The present findings emphasize the importance of the early identification and remediation of math anxiety in girls to prevent long lasting effects. Possible causes for the gender related differences will be discussed.