This paper proposes a novel smart component model replacement (SCMR) approach that is not only efficient for large-scale nonlinear structural analysis, but also accurate in the refined simulation of components of interest. The method creates a FE model with a relatively coarse mesh to simulate the entire structure. During analysis, when a component is identified to predefined criteria defined by the users, its model will be replaced by another more refined “substructure” model that can simulate the damage and failure more accurately. A few “replacement” strategies are presented for smoothly transiting between different models: (1) A gradually transiting strategy is presented to avoid the possible jumps of responses due to the model differences to guarantee numerical stability; (2) The historical boundary conditions are applied to the substructure model to address the historical dependence of responses; (3) The boundary connection between main and substructure is considered by using a flexible client/server technology, to simplify the replacement. The SCMR approach is verified by three application examples. The results demonstrate that the proposed modeling approach can efficiently perform large-scale nonlinear structural analysis, while capturing the failure mechanism in RC structures well.