The advent of ever more complex systems in extensive areas of industries hampers efficient and accurate analysis of reliability and effective reliability-based decision making. Such difficulties may arise from the intricate formulation of system failure events, statistical dependence between component failure events, and the convoluted quantification of underlying probabilities of basic events. So-called k-out-of-N systems, which survive or succeed when at least k components are available among the total of N components, give rise to a high level of complexity. This type of systems are commonly introduced to secure a proper level of redundancy in operating engineering systems, but the intricate definition of the system events may elude the system reliability analysis. It is noted that such k-out-of-N systems are often tested and corrected over a certain period of time before their official usage or release in order to assure the target reliability of the system. For the purpose of reliability prognosis based on the data collected from the test period, reliability growth models (RGMs) have been widely used in software and hardware engineering. However, RGMs have been applied mostly to individual components, not at the system level. Furthermore, in complex systems such as k-out-of-N system, it is challenging to relate the reliability growth of components with that of the system. To address this need, in this paper, the matrix-based system reliability (MSR) method is extended to k-out-of-N systems by modifying the formulations of event and probability vectors. The proposed methods can incorporate statistical dependence between component failures for both homogeneous and non-homogeneous k-out-of-N systems, and can compute measures related to parameter sensitivity and relative importance of components. The reliability growths of components represented by RGMs are incorporated into the proposed system reliability method, so that the trend of system reliability growth can be effortlessly evaluated and predicted. Two numerical examples are introduced in this paper to demonstrate the proposed method and its applications: (1) hypothetical systems each consisting of series, parallel and k-out-of-N subsystems, and (2) a simplified high speed train system modeled by multiple k-out-of-N subsystems. Two types of RGMs, i.e. non-homogeneous Poisson process (NHPP) and Duane models are employed in these examples.