Before-after analysis methods in traffic safety often aggregate traffic crashes into crash frequencies using relatively long aggregation time periods, such as a year. The implicit assumption is that the treatment effect is temporally stable over the aggregation period. However, certain “treatments”, such as the COVID-19 pandemic, may result in fast-evolving changes to road safety. By aggregating individual crashes, it is difficult to investigate the temporal characteristics of crashes and capture the potential temporal instability in treatment effect at detailed temporal levels, such as within a year. Therefore, this study exploits the disaggregated nature of crashes and proposes a survival analysis with random parameter (SARP) before-after analysis approach that can flexibly accommodate the temporal instability in treatment effect at various temporal levels. To validate and test the proposed approach, a statistical simulation study and an empirical case study that investigates the safety impact of COVID-19 lockdown in Manhattan, New York, are conducted. The statistical simulation study shows that the SARP method can unbiasedly estimate different patterns of temporally instable treatment effect at various temporal levels. The estimated monthly crash modification factors from the case study display an increasing trend after the largest decrease in the first month after the lockdown, which implies that traffic safety conditions are gradually returning to normal and provides evidence of temporal instability in treatment effect. The proposed SARP approach is promising to investigate the evolving safety impact of emerging technologies in transportation, such as the deployment of connected and autonomous vehicles.