The formation and maintenance of tissues is regulated by various signals triggered by biological, chemical, and physical factors. Data increasingly confirm that matrix or tissue elasticity plays an influential role in regulating numerous cell functions. The aim of the present study was to better understand the regulation of cellular differentiation by mechanical cues. We studied the influence of matrix stiffness on the osteodifferentiation of two cell lineages characterized by different responses: mesenchymal stromal/stem cells isolated from the Wharton's jelly of the umbilical cord (UC-MSCs) with strong stiffness-dependent responses; and bone-derived cells (BDCs), which are insensitive to changes in matrix rigidity. The study also aimed to delineate how matrix stiffness affects intracellular signaling through focal adhesion kinase (FAK) activity—one of the key components in integrin-mediated signaling pathways. The effect of substrate stiffness on the expression of α2, α5, and β1 integrin was studied using real time PCR and Western blot using cells cultured in an osteogenic medium on tunable polyacrylamide gels coated with type I collagen, with elasticities corresponding to Young's moduli of 1.46 kPa and 26.12 kPa. FAK activity was monitored using ELISA assays. We demonstrate for the first time the changes in the expression of α2, α5, and β1 integrin subunits in perinatal stem cells and in adult osteoblast precursor cells during in vitro osteogenic differentiation on surfaces characterized by different stiffness. We found that matrix rigidity significantly affects the osteogenic differentiation of UC-MSCs through α2 integrin-mediated mechanotransduction events, though not through the α5 integrin subunit. In BDCs, there were no significant changes in the expression levels of the tested protein associated with varying stiffness. Our results provide evidence that matrix rigidity affects the osteogenic differentiation of UC-MSCs via mechanotransduction events mediated by α2 integrin subunits.