Seismic inversion, such as velocity and impedance, is an ill-posed problem. To solve this problem, swarm intelligence (SI) algorithms have been increasingly applied as the global optimization approach, such as differential evolution (DE) and particle swarm optimization (PSO). Based on the well logs, the sparse probability distribution (PD) of the reflectivity distribution is spatial stationarity. Therefore, we proposed a general SI scheme with constrained by a priori sparse distribution of the reflectivity, which helps to provide more accurate potential solutions for the seismic inversion. In the proposed scheme, as two key operations, the creating of probability density function library and probability transformation are inserted into standard SI algorithms. In particular, two targeted DE-PD and PSO-PD algorithms are implemented. Numerical example of Marmousi2 model and field example of gas hydrates show that the DE-PD and PSO-PD estimate better inversion solutions than the results of the original DE and PSO. In particular, the DE-PD is the best performer both in terms of mean error and fitness value of velocity and impendence inversion. Overall, the proposed SI with sparse distribution scheme is feasible and effective for seismic inversion.