AbstractAimIn an age of increased anthropogenic changes, it is crucially important to understand how and why ecological communities change. Community assembly is governed by colonization and extinction processes, and the simplest model describing it is dynamic equilibrium (DE), which assumes that communities are shaped solely by stochastic colonization and extinction events. Deviations from this model can point to the role of species life histories, niches or human stressors acting on communities. Despite its potential to serve as a null model for community dynamics, there is currently no accepted methodology for identifying and measuring such deviations.InnovationHere we propose a new, easily applicable methodology for testing and quantifying deviations of empirically observed time series of community dynamics from the predictions and assumptions of the classical DE model, in which assemblages of independent species undergo stochastic colonization–extinction dynamics with constant rates. The methodology consists of a novel randomization‐based null model to generate synthetic time series (PARIS) and a set of statistics that quantify and test how different facets of community dynamics deviate from DE. These statistics are designed to test the assumptions of the theory, namely species independence and constancy of colonization and extinction rates, and the predictions of the theory regarding the magnitude of changes in species richness and composition.Main conclusionsTested against simulated data and a case study, the proposed methodology is shown to have good statistical properties: acceptable type I error rates, good statistical power and robustness against errors in the data. We discuss alternative methods and present guidelines for practical use of the methodology, hoping it will enhance the applicability of DE as a reference for studying changes in ecological communities.