This study aims for multistate systems (MSSs) with aging multistate components (MSCs) to construct a time-replacement policy and thereby determine the optimal time to replace the entire system. The nonhomogeneous continuous time Markov models (NHCTMMs) quantify the transition intensities among the degradation states of each component. The dynamic system state probabilities are therefore assessed using the established NHCTMMs. Solving NHCTMMs is rather complicated compared to homogeneous continuous time Markov models (HCTMMs) in determining reliability related performance indexes. Often, traditional mathematics cannot acquire accurate explicit expressions, in particular, for multiple components that are involved in designed system configuration. To overcome this difficulty, this study uses Markov reward models and the bound approximation approach to assess rewards of MSSs with MSCs, including such things as total maintenance costs and the benefits of the system staying in acceptable working states. Accordingly, we established a long-run expected benefit (LREB) per unit time, representing overall MSS performance through a lifetime, to determine the optimal time to replace the entire system, at which time the LREB values are maximized. Finally, a simulated case illustrates the practicability of the proposed approach.