Aims. In this paper we discuss the cluster and field star formation in the central part of the Small Magellanic Cloud. The main goal is to study the correlation between young objects and their interstellar environment. Methods. The ages of about 164 associations and 311 clusters younger than 1 Gyr are determined using isochrone fitting. The spatial distribution of the clusters is compared with the HI maps, with the HI velocity dispersion field, with the location of the CO clouds and with the distribution of young field stars. Results. The cluster age distribution supports the idea that clusters formed in the last 1 Gyr of the SMC history in a roughly continuous way with periods of enhancements. The two super-shells 37A and 304A detected in the HI distribution are clearly visible in the age distribution of the clusters: an enhancement in the cluster formation rate has taken place from the epoch of the shell formation. A tight correlation between young clusters and the HI intensity is found. The degree of correlation is decreasing with the age of the clusters. Clusters older than 300 Myr are located away from the HI peaks. Clusters and associations younger than 10 Myr are related to the CO clouds in the SW region of the SMC disk. A positive correlation between the location of the young clusters and the velocity dispersion field of the atomic gas is derived only for the shell 304A, suggesting that the cloud-cloud collision is probably not the most important mechanism of cluster formation. Evidence of gravitational triggered episode due to the most recent close interaction between SMC and LMC is found both in the cluster and field star distribution.