Previously, a novel technique was proposed to quantify the relationship between the muscle stiffness and its nonfatigue contraction intensity. The method extended the measured range of isometric contraction to 100% maximum voluntary contraction (MVC) using an ultrasonic shear wave measurement setup. Yet, it has not been revealed how this relationship could be affected by factors like age or sex. To clarify these questions, vastus intermedius (VI) stiffness of 40 healthy subjects was assessed under 11 step levels of isometric contraction. The subjects were divided into four groups: young males, young females, elderly males, and elderly females (n = 10 for each). In a relaxed state, no significant difference was observed between the male and female subjects (p = 0.156) nor between the young and elderly subjects (p = 0.221). However, when performing isometric contraction, the VI stiffness of males was found to be significantly higher than that of females at the same level (p < 0.001), and that of the young was higher than the elderly (p < 0.001). Meanwhile, for two knee joint angles used, the stiffness measured at a 90° knee joint angle was always significantly larger than that measured at 60° (p < 0.001). Recognizing the active muscle stiffness of VI contributes to body stability, and these results may provide insight into the age and sex bias in musculoskeletal studies, such as those on fall risks.