ObjectivesTo investigate the in vivo diagnostic agreement between visual examination (VE) using the International Caries Detection and Assessment System (ICDAS) and an automated scanner system for detecting and classifying carious lesions in primary teeth. Methods5-year-old children (n = 216) underwent VE and intraoral scanning (TRIOS 4, 3Shape TRIOS A/S, Copenhagen, Denmark). Dental caries experience was recorded for each tooth surface using ICDAS. An automated, fluorescence-based caries scoring system was applied to eligible primary teeth occlusal surfaces on the 3D models using commercially available software. The automated system classified surfaces as sound, initial caries (ICDAS 01/02), or moderate-extensive caries (ICDAS ≥03). The diagnostic agreement was investigated using multi-level modelling and intraclass correlation coefficients. Analyses were repeated at both the initial threshold (ICDAS ≥01) and the moderate-extensive threshold (ICDAS ≥03). Results213 participants were included in the study, and 1525 primary molar occlusal surfaces were included in the analysis. The odds of detecting caries using the automated system were 46 % lower at the initial disease threshold (OR 0.54, 95 % CI 0.39–0.74) and 70 % lower at the moderate-extensive disease threshold (OR 0.30, 95 % CI 0.16–0.58) compared to VE. The intraclass correlation estimates at the initial and moderate-extensive thresholds were 0.90 (95 % CI 0.70–0.96) and 0.76 (95 % CI 0.22–0.94) respectively. ConclusionThe automated system is less likely to detect initial lesions and is more likely to underestimate lesion severity relative to visual examination using ICDAS. Clinical SignificanceClinically, using the automated tool to replace thorough visual inspection in primary teeth could result in missed opportunities to provide professional or self-care to arrest or reverse early disease. Additionally, it could misclassify moderate lesions as initial caries, potentially leading to complications associated with the delayed management of dental caries.