The flavor differences in Oolong tea from different producing areas are caused by its complex differential compounds. In this study, representative samples of Oolong tea from four countries were collected, and their differential nonvolatile compounds were analyzed by a combination of widely targeted metabolomics, chemometrics, and quantitative taste evaluation. A total of 801 nonvolatile compounds were detected, which could be divided into 16 categories. We found that the difference in these compounds’ content among Oolong teas from three producing areas in China was the largest. There were 370 differential compounds related to the producing areas of Oolong tea, which were mainly distributed in 67 Kyoto Encyclopedia of Genes and Genomes (KEGG) metabolic pathways. In total, 81 differential nonvolatile compounds made important contributions to the taste differences in Oolong tea from different producing areas, among which the number of flavonoids was the largest. Finally, the characteristic compounds of Oolong tea in six producing areas were screened. This study comprehensively identifies the nonvolatile compounds of Oolong tea in different producing areas for the first time, which provides a basis for the analysis of flavor characteristics, quality directional control, and the identification and protection of geographical landmark agricultural products of Oolong tea from different producing areas.