The global increase in the penetration of renewable energy is pushing electrical power systems into uncharted territory, especially in terms of transient and dynamic stability. In particular, the greater penetration of wind generation in European power networks is, at times, displacing a significant capacity of conventional synchronous generation with fixed-speed induction generation and now more commonly, doubly fed induction generators. The impact of such changes in the generation mix requires careful monitoring to assess the impact on transient and dynamic stability. This study presents a measurement-based method for the early detection of power system oscillations, with consideration of mode damping, in order to raise alarms and develop strategies to actively improve power system dynamic stability and security. A method is developed based on wavelet-based support vector data description (SVDD) to detect oscillation modes in wind farm output power, which may excite dynamic instabilities in the wider system. The wavelet transform is used as a filter to identify oscillations in frequency bands, whereas the SVDD method is used to extract dominant features from different scales and generate an assessment boundary according to the extracted features. Poorly damped oscillations of a large magnitude, or that are resonant, can be alarmed to the system operator, to reduce the risk of system instability. The proposed method is exemplified using measured data from a chosen wind farm site.