For the purpose of reliability assessment of a structure subject to stochastic excitations, the probability of the occurrence of at least one failure event over a time interval, i.e. the first-passage probability, often needs to be evaluated. In this paper, a new method is proposed to incorporate constraints on the first-passage probability into reliability-based optimization of structural design or topology. For efficient evaluations of first-passage probability during the optimization, the failure event is described as a series system event consisting of instantaneous failure events defined at discrete time points. The probability of the series system event is then computed by use of a system reliability analysis method termed as the sequential compounding method. The adjoint sensitivity formulation is derived for calculating the parameter sensitivity of the first-passage probability to facilitate the use of efficient gradient-based optimization algorithms. The proposed method is successfully demonstrated by numerical examples of a space truss and building structures subjected to stochastic earthquake ground motions.