Mental abacus is mental arithmetic with the help of an imagined abacus. Children skilled in mental abacus have been shown to exhibit top-quality arithmetic abilities. The current study investigated whether children with high-level mental abacus ability could outperform untrained control children in non-symbolic number sense, which is considered to be fundamental for arithmetic development. One hundred and fifty children (75 children skilled in mental abacus and 75 controls) took part in this study. Children skilled in mental abacus completed a mental abacus level test. The two groups of children performed serial cognitive tasks, assessing non-symbolic number comparison, arithmetic, language, spatial processing, visual perception, attention, processing speed, working memory, and general intelligence. Results showed that children skilled in mental abacus had significantly better non-symbolic number sense than the other children after controlling for general intelligence. The significant difference in non-symbolic number sense remained after controlling for age, gender, all types of cognitive processing available, and arithmetic performance. A mediation model showed that non-symbolic number sense partially mediated the group difference in arithmetic development. These findings suggest that children skilled in mental abacus have enhanced non-symbolic number sense and raise the possibility that mental abacus training could directly improve children’s non-symbolic numerical skills.