Japanese apricot (Prunus mume) is remarkably valuable for its high ornamental and economic importance due to its distinctive features. Low temperature is a serious environmental constraint for this species, restricting its cultivation and dispersal in the north of China. To address this issue, breeding requires an understanding of the molecular mechanisms underlying responses to cold stress. We examined the leaf physiological and transcriptome profile by RNA sequencing in ‘Bungo’ scion cultivar grafted onto Prunus mume (cold-sensitive) and Prunus armeniaca (cold-tolerant) rootstocks at 4 °C for 0, 6, and 24 h. Our results revealed that the increased MDA concentration in the leaves of P. mume cultivar (cold-sensitive) suggests that cold stress might cause oxidative damage and increased sensitivity. Moreover, the cold-tolerant cultivar (P. armeniaca) considerably enhances the enzyme activities (i.e., SOD, POD, and CAT), as well as osmo-protectants (soluble sugars and proline) compared with sensitive cultivar, which helps plants to withstand oxidative damage caused by cold stress. Additionally, differentially expressed genes were shown to be enriched in plant hormone signal transduction, ribosome, MAPK signaling, and circadian rhythm pathway. After 24 h of cold stress, genes related to PYL4, histidine kinase 1, SAUR36, bHLH130, bHLH123, TIFY 6B-like, WRKY 40, WRKY 57, and 60S acidic ribosomal protein P1 were differentially expressed, implying that these DEGs involved in multiple pathways are involved in cold tolerance in Japanese apricot. This study improved our current understanding of the mechanism of cold tolerance in Japanese apricot, and the findings could be utilized for other related fruit species.