In the implementation of time-domain boundary element method for elasto-dynamic problems, there are two types of singularities: the wave front singularity arising when the product of wave velocity and time is equal to the distance between the source point and the field point, and the spatial singularity arising when the source point coincides with the field point. In this paper, the singularity of the first type in the integrand is eliminated by an analytical integration over time, Cauchy principal value and Hadamard finite part integral. Four types of singularities with different orders appear in the integrand after analytical time integration. In order to accurately calculate the integral, in which the integrand is piecewise continuous, the integral domain is subdivided into several patches based on the relation between the product of wave velocity and time and the distance. In singular patches, the integrands are separated into a regular part and a singular part. The regular part can be computed by traditional numerical integration method such as Gaussian integration, while the singular part can be analytically integrated. Using the proposed method, the spatial singular integrals can be calculated directly. Numerical examples using various kinds of elements are presented to verify the proposed method.