In varying environments, it is beneficial for organisms to utilize available cues to infer the conditions they may encounter and express potentially favourable traits. However, external cues can be unreliable or too costly to use. We consider an alternative strategy where organisms exploit internal sources of information. Even without sensing environmental cues, their internal states may become correlated with the environment as a result of selection, which then form a memory that helps predict future conditions. To demonstrate the adaptive value of such internal cues in varying environments, we revisit the classic example of seed dormancy in annual plants. Previous studies have considered the germination fraction of seeds and its dependence on environmental cues. In contrast, we consider a model of germination fraction that depends on the seed age, which is an internal state that can serve as a memory. We show that, if the environmental variation has temporal structure, then age-dependent germination fractions will allow the population to have an increased long-term growth rate. The more the organisms can remember through their internal states, the higher the growth rate a population can potentially achieve. Our results suggest experimental ways to infer internal memory and its benefit for adaptation in varying environments.