Dogs exhibit remarkable phenotypic diversity, particularly in behavioral traits, making them an excellent model for studying the genetic basis of complex behaviors. Behavioral traits such as aggression and fear are highly heritable among different dog breeds, but their genetic basis is largely unknown. We used the genome-wide association study (GWAS) to identify candidate genes associated with nine behavioral traits including; stranger-directed aggression (SDA), owner-directed aggression (ODA), dog-directed aggression (DDA), stranger-directed fear (SDF), nonsocial fear (NF), dog-directed fear (DDF), touch sensitivity (TS), separation-related behavior (SRB) and attachment attention-seeking (AAS). The observed behavioral traits were collected from 38,714 to 40,460 individuals across 108 modern dog breeds. We performed a GWAS based on a latent trait extracted using the confirmatory factor analysis (CFA) method with nine observable behavioral traits and compared the results with those from the GWAS of the observed traits. Using both observed-trait and latent-trait GWAS, we identified 41 significant SNPs that were common between both GWAS methods, of which 26 were pleiotropic, as well as 10 SNPs unique to the latent-trait GWAS, and 5 SNPs unique to the observed-trait GWAS discovered. These SNPs were associated with 21 genes in latent-trait GWAS and 22 genes in the observed-trait GWAS, with 19 genes shared by both. According to previous studies, some of the genes from this study have been reported to be related to behavioral and neurological functions in dogs. In the human population, these identified genes play a role in either the formation of the nervous system or are linked to various mental health conditions. Taken together, our findings suggest that latent-trait GWAS for behavioral traits in dogs identifies significant latent genes that are neurologically prioritized.