Musical Onset Research Articles

Predicting emotions in music using the onset curve

The relationship between parameters extracted from the musical stimuli and emotional response has been traditionally approached using several physical measures extracted from time or frequency domains. From time-domain measures, the musical onset is defined as the moment in that any musical instrument or human voice issues a musical note. The onsets’ sequence in the performance of a specific musical score creates what is known as the onset curve (OC). The influence of the structure of OC on the emotional judgment of people is not known. To this end, we have applied principal component analysis on a complete set of variables extracted from the OC to capture their statistical structure. We have found a trifactorial structure related to activation and valence dimensions of emotional judgment. The structure has been cross-validated using different participants and stimuli. In this way, we propose the factorial scores of the OC as a reliable and relevant piece of information to predict the emotional judgment of music.

Musical note onset detection based on a spectral sparsity measure

If music is the language of the universe, musical note onsets may be the syllables for this language. Not only do note onsets define the temporal pattern of a musical piece, but their time-frequency characteristics also contain rich information about the identity of the musical instrument producing the notes. Note onset detection (NOD) is the basic component for many music information retrieval tasks and has attracted significant interest in audio signal processing research. In this paper, we propose an NOD method based on a novel feature coined as Normalized Identification of Note Onset based on Spectral Sparsity (NINOS2). The NINOS2 feature can be thought of as a spectral sparsity measure, aiming to exploit the difference in spectral sparsity between the different parts of a musical note. This spectral structure is revealed when focusing on low-magnitude spectral components that are traditionally filtered out when computing note onset features. We present an extensive set of NOD simulation results covering a wide range of instruments, playing styles, and mixing options. The proposed algorithm consistently outperforms the baseline Logarithmic Spectral Flux (LSF) feature for the most difficult group of instruments which are the sustained-strings instruments. It also shows better performance for challenging scenarios including polyphonic music and vibrato performances.

Individual differences in human frequency-following response predict pitch labeling ability

The frequency-following response (FFR) provides a measure of phase-locked auditory encoding in humans and has been used to study subcortical processing in the auditory system. While effects of experience on the FFR have been reported, few studies have examined whether individual differences in early sensory encoding have measurable effects on human performance. Absolute pitch (AP), the rare ability to label musical notes without reference notes, provides an excellent model system for testing how early neural encoding supports specialized auditory skills. Results show that the FFR predicts pitch labelling performance better than traditional measures related to AP (age of music onset, tonal language experience, pitch adjustment and just-noticeable-difference scores). Moreover, the stimulus type used to elicit the FFR (tones or speech) impacts predictive performance in a manner that is consistent with prior research. Additionally, the FFR predicts labelling performance for piano tones better than unfamiliar sine tones. Taken together, the FFR reliably distinguishes individuals based on their explicit pitch labeling abilities, which highlights the complex dynamics between sensory processing and cognition.

Rhythmic variability, language, and style: A replication and extension of nPVI findings with the RISM dataset

The normalised pairwise variability index (nPVI), has been frequently used to measure the rhythmic variance between musical onsets. For example, researchers have proposed connections between the nationalities of composers and the nPVI values of their music and native language. One particular issue, however, lies in the notion of intended nationality – composers frequently wrote in national styles other than their own. This study employs the RISM-World dataset to both replicate previous findings relating the rhythmic variance of melodies to the nationalities of composers, and to examine intended nationality in the music of Mozart and Handel, namely their operas and oratorios.

Musical Onset Detection Using Constrained Linear Reconstruction

This letter presents a multi-frame extension of the well-known spectral flux method for unsupervised musical onset detection. Instead of comparing only the spectral content of two frames, the proposed method takes into account a wider temporal context to evaluate the dissimilarity between a given frame and its previous frames. More specifically, the dissimilarity is measured by using the previous frames to obtain a linear reconstruction of the given frame, and then calculating the rectified, ${l_2}$ -norm reconstruction error. Evaluation on a dataset comprising 2,169 onset events of 12 instruments shows that this simple idea works fairly well. When a non-negativity constraint is imposed in the linear reconstruction, the proposed method can outperform the state-of-the-art unsupervised method SuperFlux by 2.9% in F-score. Moreover, the proposed method is particularly effective for instruments with soft onsets, such as violin, cello, and ney. The proposed method is efficient, easy to implement, and is applicable to scenarios of online onset detection.

Note onset detection based on sparse decomposition

Music onset detection is significant and essential for obtaining the high-level music features such as rhythm, beat, music paragraph and structure. The traditional methods for onset detection which...

Gamelan Music Onset Detection based on Spectral Features

This research detects onsets of percussive instruments by examining the performance on the sound signals of gamelan instruments as one of traditional music instruments in Indonesia. Onset plays important role in determining musical rythmic structure, like beat, tempo, measure, and is highly required in many applications of music information retrieval. Four onset detection methods that employ spectral features, such as magnitude, phase, and the combination of both are compared in this paper. They are phase slope (PS), weighted phase deviation (WPD), spectral flux (SF), and rectified complex domain (RCD). Features are extracted by representing the sound signals into time-frequency domain using overlapped Short-time Fourier Transform (STFT) and by varying the window length. Onset detection functions are processed through peak-picking using dynamic threshold. The results showed that by using suitable window length and parameter setting of dynamic threshold, F-measure which is greater than 0.80 can be obtained for certain methods.

Gamelan Music Onset Detection based on Spectral Features

Gamelan Music Onset Detection based on Spectral Features

Real-time detection of musical onsets with linear prediction and sinusoidal modeling

Real-time musical note onset detection plays a vital role in many audio analysis processes, such as score following, beat detection and various sound synthesis by analysis methods. This article provides a review of some of the most commonly used techniques for real-time onset detection. We suggest ways to improve these techniques by incorporating linear prediction as well as presenting a novel algorithm for real-time onset detection using sinusoidal modelling. We provide comprehensive results for both the detection accuracy and the computational performance of all of the described techniques, evaluated using Modal, our new open source library for musical onset detection, which comes with a free database of samples with hand-labelled note onsets.

Adaptive Linear Prediction Filtering in DWT Domain for Real-Time Musical Onset Detection

Onset detection is a typical digital signal processing task in acoustic signal analysis, with many applications as in the musical field. Many techniques have been proposed so far, which are typically reliable in terms of performances but often not suitable to real-time computing, for example, they require knowledge of the whole piece to perform optimally, or they are too computationally intensive for most embedded processors. Up to the authors' knowledge, the real-time implementation problem for musical onset detection has been scarcely addressed within the literature, which has motivated them to propose a scalable and computationally efficient algorithm with good detection capabilities. Comparison with other techniques and porting to a real-time embedded processor are discussed as well: provided experimental results seem to confirm the effectiveness of the approach.

Music Onset Detection Based on Resonator Time Frequency Image

This paper describes a new method for music onset detection. The novelty of the approach consists mainly of two elements: the time-frequency processing and the detection stages. The resonator time frequency image (RTFI) is the basic time-frequency analysis tool. The time-frequency processing part is in charge of transforming the RTFI energy spectrum into more natural energy- change and pitch-change cues that are then used as input elements for the detection of music onsets by detection tools. Two detection algorithms have been developed: an energy-based algorithm and a pitch-based one. The energy-based detection algorithm exploits energy-change cues and performs particularly well for the detection of hard onsets. The pitch-based algorithm successfully exploits stable pitch cues for the onset detection in polyphonic music, and achieves much better performances than the energy-based algorithm when applied to the detection of soft onsets. Results for both the energy-based and pitch-based detection algorithms have been obtained on a large music dataset.

Melody Extraction and Musical Onset Detection via Probabilistic Models of Framewise STFT Peak Data

We propose a probabilistic method for the joint segmentation and melody extraction for musical audio signals which arise from a monophonic score. The method operates on framewise short-time Fourier transform (STFT) peaks, enabling a computationally efficient inference of note onset, duration, and pitch attributes while retaining sufficient information for pitch determination and spectral change detection. The system explicitly models note events in terms of transient and steady-state regions as well as possible gaps between note events. In this way, the system readily distinguishes abrupt spectral changes associated with musical onsets from other abrupt change events. Additionally, the method may incorporate melodic context by modeling note-to-note dependences. The method is successfully applied to a variety of piano and violin recordings containing reverberation, effective polyphony due to legato playing style, expressive pitch variations, and background voices. While the method does not provide a sample-accurate segmentation, it facilitates the latter in subsequent processing by isolating musical onsets to frame neighborhoods and identifying possible pitch content before and after the true onset sample location

Instrumental and vocal music effects on EEG and EKG in neonates of depressed and non-depressed mothers

Neonates (M age = 16 days) born to depressed and non-depressed mothers were randomly assigned to hear an audiotaped lullaby of instrumental music with vocals or without vocals. Neonatal EEG and EKG were recorded for 2 min (baseline) of silence and for 2 min of one or the other music presentation. Neonates of non-depressed mothers showed greater relative right frontal EEG asymmetry to both types of music, suggesting a withdrawal response. Neonates of depressed mothers on the other hand showed greater relative left frontal EEG asymmetry to the instrumental without vocal segment, suggesting an approach response, and greater relative right frontal EEG asymmetry to the instrumental with vocal segment, suggesting a withdrawal response. Heart rate decelerations occurred following the music onset for both groups of infants, however, compared to infants of non-depressed mothers, infants of depressed mothers showed a delayed heart rate deceleration, suggesting slower processing and/or delayed attention. These findings suggest that neonates of depressed and non-depressed mothers show different EKG and EEG responses to instrumental music with versus without vocals.

On the Use of Phase and Energy for Musical Onset Detection in the Complex Domain

We present a study on the combined use of energy and phase information for the detection of onsets in musical signals. The resulting method improves upon both energy-based and phase-based approaches. The detection function, generated from the analysis of the signal in the complex frequency domain is sharp at the position of onsets and smooth everywhere else. Results on a database of recordings show high detection rates for low rates of errors. The approach is more robust than its predecessors both theoretically and practically.