Commercial wind turbine blades are progressively becoming longer and more flexible; in order to achieve load reduction, the use of shape modifying devices is currently under research. While such modifications facilitate cost reduction, they also render the blade susceptible to the unstable aeroelastic phenomenon of flutter. To be able to detect the onset of flutter, and to modify the load control algorithm accordingly, it is desirable to perform online identification of system dynamics. In this paper, a recursive subspace identification algorithm is augmented with a nuclear norm-based cost function for the rapid identification of changes in the dominant system behavior. The time-consuming singular value thresholding step involved in the identification is replaced by a fast randomized algorithm. The method developed is used to identify the changes in the dynamics of an experimental wind turbine equipped with shape-modifying actuators, and operated under controlled conditions in a wind tunnel. The proposed identification method shows high sensitivity to changes in system dynamics, and is shown capable of stably and rapidly identifying the onset of aeroelastic flutter.