The coronavirus disease 2019 (COVID-19) is an emerging and rapidly evolving profound pandemic, which causes severe acute respiratory syndrome and results in significant case fatality around the world including Bangladesh. We conducted this study to assess how COVID-19 cases clustered across districts in Bangladesh and whether the pattern and duration of clusters changed following the country's containment strategy using Geographic information system (GIS) software. We calculated the epidemiological measures including incidence, case fatality rate (CFR) and spatiotemporal pattern of COVID-19. We used inverse distance weighting (IDW), Geographically weighted regression (GWR), Moran's I and Getis-Ord Gi* statistics for prediction, spatial autocorrelation and hotspot identification. We used retrospective space-time scan statistic to analyse clusters of COVID-19 cases. COVID-19 has a CFR of 1.4%. Over 50% of cases were reported among young adults (21-40years age). The incidence varies from 0.03 - 0.95 at the end of March to 15.59-308.62 per 100,000, at the end of July. Global Moran's Index indicates a robust spatial autocorrelation of COVID-19 cases. Local Moran's I analysis stated a distinct High-High (HH) clustering of COVID-19 cases among Dhaka, Gazipur and Narayanganj districts. Twelve statistically significant high rated clusters were identified by space-time scan statistics using a discrete Poisson model. IDW predicted the cases at the undetermined area, and GWR showed a strong relationship between population density and case frequency, which was further established with Moran's I (0.734; p≤0.01). Dhaka and its surrounding six districts were identified as the significant hotspot whereas Chattogram was an extended infected area, indicating the gradual spread of the virus to peripheral districts. This study provides novel insights into the geostatistical analysis of COVID-19 clusters and hotspots that might assist the policy planner to predict the spatiotemporal transmission dynamics and formulate imperative control strategies of SARS-CoV-2 in Bangladesh. The geospatial modeling tools can be used to prevent and control future epidemics and pandemics.