For seismic resilience-based design (RBD), a selection of recorded time histories for dynamic structural analysis is usually required. In order to make individual structures and communities regain their target functions as promptly as possible, uncertainty of the structural response estimates is in great need of reduction. The ground motion (GM) selection based on a single target response spectrum, such as acceleration or displacement response spectrum, would bias structural response estimates leading significant uncertainty, even though response spectrum variance is taken into account. In addition, resilience of an individual structure is not governed by its own performance, but depends severely on the performance of other systems in the same community. Thus, evaluation of resilience of a community using records matching target spectrum at whole periods would be reasonable because the fundamental periods of systems in the community may be varied. This paper presents a GM selection approach based on a probabilistic framework to find an optimal set of records to match multiple target spectra, including acceleration and displacement response spectra. Two major steps are included in that framework. Generation of multiple sub-spectra from target displacement response spectrum for selecting sets of GMs was proposed as the first step. Likewise, the process as genetic algorithm (GA), evolvement of individuals previously generated, is the second step, rather than using crossover and mutation techniques. A novel technique improving the match between acceleration response spectra of samples and targets is proposed as the second evolvement step. It is proved computationally efficient for the proposed algorithm by comparing with two developed GM selection algorithms. Finally, the proposed algorithm is applied to select GM records according to seismic codes for analysis of four archetype reinforced concrete (RC) frames aiming to evaluate the influence of GM selection considering two design response spectra on structural responses. The implications of design response spectra especially the displacement response spectrum and GM selection algorithm are summarized.