This study sought to disentangle several phenotypic correlates of pedophilic sexual interests, such as emotional congruence with children and lack of empathy. We utilized Implicit Association Tests and self-report questionnaires for emotional congruence with children and analyzed the psychometric properties of these measures. Further, we analyzed the associations between these measures and self-reported pedophilic sexual interests and empathy. The sample consisted of 110 adult community males (prevalence of sexual interest in children at least to some extent: 5.5%) with or without child-related jobs (43.6% vs. 56.4%) recruited online in the general population. Overall, we found equivocal parameters for reliability of the implicit and explicit measures of emotional congruence with children (α = 0.29 to 0.76). The self-reported emotional congruence with children was only weakly linked to the implicit positive evaluations of children (r = 0.170, p = 0.039), not linked to empathy except for distress in social interactions (r = 0.199, p = 0.019), and moderately linked to deviant sexual interests in children (r = 0.321 to 0.404, p < 0.001), especially in men working with children (R2 = 0.04, p = 0.027). Further studies are warranted to expand the sample to the offender population and explore the more complex network of constructs related to pedophilia.