Restrictive cardiomyopathy (RCM) is a rare disease characterized by increased ventricular stiffness and preserved ventricular contraction. Various sarcomere gene variants are known to cause RCM; however, more than a half of patients do not harbor such pathogenic variants. We recently demonstrated that cardiac fibroblasts (CFs) play important roles in inhibiting the diastolic function of cardiomyocytes via humoral factors and direct cell–cell contact regardless of sarcomere gene mutations. However, the mechanical properties of CFs that are crucial for intercellular communication and the cardiomyocyte microenvironment remain less understood. In this study, we evaluated the rheological properties of CFs derived from pediatric patients with RCM and healthy control CFs via atomic force microscopy. Then, we estimated the cellular modulus scale factor related to the cell stiffness, fluidity, and Newtonian viscosity of single cells based on the single power-law rheology model and analyzed the comprehensive gene expression profiles via RNA-sequencing. RCM-derived CFs showed significantly higher stiffness and viscosity and lower fluidity compared to healthy control CFs. Furthermore, RNA-sequencing revealed that the signaling pathways associated with cytoskeleton elements were affected in RCM CFs; specifically, cytoskeletal actin-associated genes (ACTN1, ACTA2, and PALLD) were highly expressed in RCM CFs, whereas several tubulin genes (TUBB3, TUBB, TUBA1C, and TUBA1B) were down-regulated. These results implies that the signaling pathways associated with cytoskeletal elements alter the rheological properties of RCM CFs, particularly those related to CF–cardiomyocyte interactions, thereby leading to diastolic cardiac dysfunction in RCM.