Musical Instrument Recognition Research Articles

Automatic Ontology Generation for Musical Instruments Based on Audio Analysis

In this paper we present a novel hybrid system that involves a formal method of automatic ontology generation for web-based audio signal processing applications. An ontology is seen as a knowledge management structure that represents domain knowledge in a machine interpretable format. It describes concepts and relationships within a particular domain, in our case, the domain of musical instruments. However, the different tasks of ontology engineering including manual annotation, hierarchical structuring and organization of data can be laborious and challenging. For these reasons, we investigate how the process of creating ontologies can be made less dependent on human supervision by exploring concept analysis techniques in a Semantic Web environment. In this study, various musical instruments, from wind to string families, are classified using timbre features extracted from audio. To obtain models of the analysed instrument recordings, we use K-means clustering to determine an optimised codebook of Line Spectral Frequencies (LSFs), or Mel-frequency Cepstral Coefficients (MFCCs). Two classification techniques based on Multi-Layer Perceptron (MLP) neural network and Support Vector Machines (SVM) were tested. Then, Formal Concept Analysis (FCA) is used to automatically build the hierarchical structure of musical instrument ontologies. Finally, the generated ontologies are expressed using the Ontology Web Language (OWL). System performance was evaluated under natural recording conditions using databases of isolated notes and melodic phrases. Analysis of Variance (ANOVA) were conducted with the feature and classifier attributes as independent variables and the musical instrument recognition F-measure as dependent variable. Based on these statistical analyses, a detailed comparison between musical instrument recognition models is made to investigate their effects on the automatic ontology generation system. The proposed system is general and also applicable to other research fields that are related to ontologies and the Semantic Web.

Instrument timbre and trait attribution.

An instrument's timbre is one of the determining factors in the decision to play it. Previous studies have shown gender stereotypes of certain instruments, and personality differences among musicians playing specific instruments. The present study was conducted with 2 aims: first, to examine whether nonmusicians hold the same stereotypes of instrumentalists as musicians; second, to examine whether hearing a melody in different instrumental timbres influences trait attribution to the assumed performers. To examine the first question, 26 nonmusicians and 22 musicians (control groups) rated flautists, trumpet players, and violinists on a list of 10 personality traits. Nonmusicians rated trumpet players as less introverted and sensitive than flautists and violinists. Musicians rated trumpet players as tougher than flautists and violinists, and violinists were rated as more egocentric than trumpet players or flautists. In the second phase, 80 participants (experimental groups) rated fictional musicians after hearing the same melody played in the 3 timbres, presented in different orders. Results show that trumpet timbre led to significantly different attributions from flute or violin. Trumpet players were generally rated more extraverted, friendly, tough, and assertive, and less introverted, anxious, sensitive, and shy. Finally, comparisons between control and experimental groups showed significant differences in ratings. Results demonstrate the strong effect of hearing instruments' timbre on trait attribution. However, these effects were different for the different instruments. Whereas nonmusicians do not hold the same stereotypes of instrumentalists as musicians, hearing a melody in the 3 different timbres led to ratings in the direction of gender stereotypes.Keywords: timbre, personality, musical instrument, nonmusiciansTimbre is the attribute of sound that allows the perceptual distinction between different instruments playing the same pitch at the same loudness level. Studies on timbre perception have mostly concentrated on the physical parameters of sound, such as the number and intensity of overtones, or attack and decay, for the recognition of musical instruments (Handel & Erickson, 2004; Iverson, 1995; Iverson & Krumhansl, 1993; Lakatos, 2000; Schel- lenberg, Iverson, & McKinnon, 1999). The present study sought to examine another effect of timbre: its association with personality traits. Previous studies have shown that voice timbre influences likability ratings and is related to personality traits (Spasova, 2011; Weiss & Burkhardt, 2010). The aim of the study was to examine a similar effect with musical instruments. Would the same melody, played in different instrument timbres, influence the attribution of personality traits to the performing musicians in nonmusicians?Sensitivity to timbre is evident from infancy, and children as young as 2 months are able to distinguish between various instru- ments (McDonald & Simons, 1989; Trehub, Endman, & Thorpe, 1990). As children grow older, the sound and feel of different instruments become an important aspect in determining their at- traction to specific musical instruments, and to some degree pre- dicts future involvement with the same instrument (Abeles, 2009; Conway, 2000; Cutietta & Mercadoocasio, 2003; Delzell & Lep- pla, 1992; Fortney, Boyle, & DeCarbo, 1993; Gordon, 1991).The connotations evoked by different instruments' sound qual- ities lead to the endowing of personality traits to the instrument, or the person playing it (Kemp, 1996). The association between an instrument's sound and gender is one important aspect of this phenomenon. Certain instruments, such as drums, bass, tuba, sax- ophone, trumpet, or guitar, are considered more masculine, while others, such as harp, flute, violin, and piano, are seen as more feminine (Abeles, 2009; Cramer, Million, & Perrault, 2002; Gris- wold & Chroback, 1981; S. …

Musical Instrument Recognition in Polyphonic Audio Using Missing Feature Approach

A method is described for musical instrument recognition in polyphonic audio signals where several sound sources are active at the same time. The proposed method is based on local spectral features and missing-feature techniques. A novel mask estimation algorithm is described that identifies spectral regions that contain reliable information for each sound source, and bounded marginalization is then used to treat the feature vector elements that are determined to be unreliable. The mask estimation technique is based on the assumption that the spectral envelopes of musical sounds tend to be slowly-varying as a function of log-frequency and unreliable spectral components can therefore be detected as positive deviations from an estimated smooth spectral envelope. A computationally efficient algorithm is proposed for marginalizing the mask in the classification process. In simulations, the proposed method clearly outperforms reference methods for mixture signals. The proposed mask estimation technique leads to a recognition accuracy that is approximately half-way between a trivial all-one mask (all features are assumed reliable) and an ideal “oracle” mask.

Auditory Training: Evidence for Neural Plasticity in Older Adults.

Improvements in digital amplification, cochlear implants, and other innovations have extended the potential for improving hearing function; yet, there remains a need for further hearing improvement in challenging listening situations, such as when trying to understand speech in noise or when listening to music. Here, we review evidence from animal and human models of plasticity in the brain's ability to process speech and other meaningful stimuli. We considered studies targeting populations of younger through older adults, emphasizing studies that have employed randomized controlled designs and have made connections between neural and behavioral changes. Overall results indicate that the brain remains malleable through older adulthood, provided that treatment algorithms have been modified to allow for changes in learning with age. Improvements in speech-in-noise perception and cognition function accompany neural changes in auditory processing. The training-related improvements noted across studies support the need to consider auditory training strategies in the management of individuals who express concerns about hearing in difficult listening situations. Given evidence from studies engaging the brain's reward centers, future research should consider how these centers can be naturally activated during training.

A Study : Analysis of Music Features for Musical Instrument Recognition and Music Similarity Search

Lots of work has been done on speech and speaker recognition. Many technologies were developed for the analysis of speech waveforms. Musical instrument recognition is an important aspect of music information retrieval system. In this paper we analyzed features for musical instruments recognition and a brief study on music similarity. Music similarity search is done by using mel-frequency cepstral coefficients (MFCC), linear predictive coefficients (LPC) and the classifier used is K- Nearest Neighbor with Dynamic Time Warping.

The Effects of Training on Recognition of Musical Instruments by Adults with Cochlear Implants

This study examines the efficiency and effectiveness of three types of training on recognition of musical instruments by adults with cochlear implants (CI). Seventy-one adults with CIs were randomly assigned to one of three training conditions: feedback on response accuracy, feedback-plus (response accuracy plus correct answer), and direct instruction. Each participant completed three training sessions per week over a five-week time period in which they listened to recorded excerpts of eight different musical instruments. Results showed significant pre-to-posttest improvement in music instrument recognition accuracy for all three training conditions (22.9-25.7%, p< 0.0001). Time when tested (week), bilateral CI use, and age were significant predictors of performance. Participants who wore bilateral implants scored significantly higher than participants with unilateral implants at all three time points; hearing aid use was not a significant predictor. These results may have practical implications for numerous types of auditory rehabilitation for persons who use CIs.

Music in Our Ears: The Biological Bases of Musical Timbre Perception

Timbre is the attribute of sound that allows humans and other animals to distinguish among different sound sources. Studies based on psychophysical judgments of musical timbre, ecological analyses of sound's physical characteristics as well as machine learning approaches have all suggested that timbre is a multifaceted attribute that invokes both spectral and temporal sound features. Here, we explored the neural underpinnings of musical timbre. We used a neuro-computational framework based on spectro-temporal receptive fields, recorded from over a thousand neurons in the mammalian primary auditory cortex as well as from simulated cortical neurons, augmented with a nonlinear classifier. The model was able to perform robust instrument classification irrespective of pitch and playing style, with an accuracy of 98.7%. Using the same front end, the model was also able to reproduce perceptual distance judgments between timbres as perceived by human listeners. The study demonstrates that joint spectro-temporal features, such as those observed in the mammalian primary auditory cortex, are critical to provide the rich-enough representation necessary to account for perceptual judgments of timbre by human listeners, as well as recognition of musical instruments.

Musical Instrument Recognition using Zero Crossing Rate and Short-time Energy

Traditionally, musical instrument recognition is mainly based on frequency domain analysis (sinusoidal analysis, cepstral coefficients) and shape analysis to extract a set of various features. Instruments are usually classified using k-NN classifiers, HMM, Kohonen SOM and Neural Networks. Recognition of musical instruments in multi-instrumental, polyphonic music is a difficult challenge which is yet far from being solved. Successful instrument recognition techniques in solos (monophonic or polyphonic recordings of single instruments) can help to deal with this task. We introduce an instrument recognition process in solo recordings of a set of instruments (flute, guitar and harmonium), which yields a high recognition rate. A large solo database is used in order to encompass the different sound possibilities of each instrument and evaluate the generalization abilities of the classification process. The basic characteristics are computed in 1sec interval and result shows that the estimation of zero crossing rate and short time energy reflects more effectively the difference in musical instruments.

Polyphonic Pitch Estimation and Instrument Identification by Joint Modeling of Sustained and Attack Sounds

Polyphonic pitch estimation and musical instrument identification are some of the most challenging tasks in the field of music information retrieval (MIR). While existing approaches have focused on the modeling of harmonic partials, we design a joint Gaussian mixture model of the harmonic partials and the inharmonic attack of each note. This model encodes the power of each partial over time as well as the spectral envelope of the attack part. We derive an expectation-maximization (EM) algorithm to estimate the pitch and the parameters of the notes. We then extract timbre features both from the harmonic and the attack part via principal component analysis (PCA) over the estimated model parameters. Musical instrument recognition for each estimated note is finally carried out with a support vector machine (SVM) classifier. Experiments conducted on mixtures of isolated notes as well as real-world polyphonic music show higher accuracy over state-of-the-art approaches based on the modeling of harmonic partials only.

Analysis of Recognition of a Musical Instrument in Sound Mixes Using Support Vector Machines

Experiments with recognition of the dominating musical instrument in sound mixes are interesting from the point of view of music information retrieval, but this task can be very difficult if the mi...

UCC 음원분류를 위한 연주악기 분류에 대한 연구

사용자가 직접 연주하여 제작한 콘텐츠에서 많이 사용되는 악기는 기타, 피아노, 그리고 바이올린 이다. 이중 기타와 피아노가 만들어 내는 오디오 신호의 특성이 비슷하여 구분하기가 어렵다. 하지만 시간에 따른 신호의 에너지 변화가 피크(Peak)들을 중심으로 서로 다른 패턴을 보이는 것으로 분석되었다. 누적 히스토그램을 이용하여 피크 존재 가능성의 확률적 분포를 구한 후, 피크를 중심으로 그 주변의 주파수 대역 별에너지 변화 패턴을 통계적 방법으로 모델링하여 실험한 결과 피아노와 기타의 구분 성공률이 최고 14% 정도의 향상을 보였다. A guitar, a piano, and a violin are popular musical instruments for User Created Contents(UCC). However the patterns of audio signal generated by a guitar and a piano are too similar to differentiate. The difference between two musical instruments can be found by analyzing the frequency variation per each band near signal peaks. The distribution of probability on the existence of signal peaks based on Cumulative Histogram were applied to musical instrument recognition. Experiments with statistical models of the frequency variation per each band near signal peaks showed the 14% improvement of musical instrument recognition.

The cognitive organization of music knowledge: a clinical analysis

Despite much recent interest in the clinical neuroscience of music processing, the cognitive organization of music as a domain of non-verbal knowledge has been little studied. Here we addressed this issue systematically in two expert musicians with clinical diagnoses of semantic dementia and Alzheimer’s disease, in comparison with a control group of healthy expert musicians. In a series of neuropsychological experiments, we investigated associative knowledge of musical compositions (musical objects), musical emotions, musical instruments (musical sources) and music notation (musical symbols). These aspects of music knowledge were assessed in relation to musical perceptual abilities and extra-musical neuropsychological functions. The patient with semantic dementia showed relatively preserved recognition of musical compositions and musical symbols despite severely impaired recognition of musical emotions and musical instruments from sound. In contrast, the patient with Alzheimer’s disease showed impaired recognition of compositions, with somewhat better recognition of composer and musical era, and impaired comprehension of musical symbols, but normal recognition of musical emotions and musical instruments from sound. The findings suggest that music knowledge is fractionated, and superordinate musical knowledge is relatively more robust than knowledge of particular music. We propose that music constitutes a distinct domain of non-verbal knowledge but shares certain cognitive organizational features with other brain knowledge systems. Within the domain of music knowledge, dissociable cognitive mechanisms process knowledge derived from physical sources and the knowledge of abstract musical entities.

Incorporating scale information with cepstral features: Experiments on musical instrument recognition

We present two sets of novel features that combine multiscale representations of signals with the compact timbral description of Mel-frequency cepstral coefficients (MFCCs). We define one set of features, OverCs, from overcomplete transforms at multiple scales. We define the second set of features, SparCs, from a signal model found by sparse approximation. We compare the descriptiveness of our features against that of MFCCs by performing two simple tasks: pairwise musical instrument discrimination, and musical instrument classification. Our tests show that both OverCs and SparCs improve the characterization of the global timbre and local stationarity of an audio signal than do mean MFCCs with respect to these tasks.

Progressive associative phonagnosia: A neuropsychological analysis

There are few detailed studies of impaired voice recognition, or phonagnosia. Here we describe two patients with progressive phonagnosia in the context of frontotemporal lobar degeneration. Patient QR presented with behavioural decline and increasing difficulty recognising familiar voices, while patient KL presented with progressive prosopagnosia. In a series of neuropsychological experiments we assessed the ability of QR and KL to recognise and judge the familiarity of voices, faces and proper names, to recognise vocal emotions, to perceive and discriminate voices, and to recognise environmental sounds and musical instruments. The patients were assessed in relation to a group of healthy age-matched control subjects. QR exhibited severe impairments of voice identification and familiarity judgments with relatively preserved recognition of difficulty-matched faces and environmental sounds; recognition of musical instruments was impaired, though better than recognition of voices. In contrast, patient KL exhibited severe impairments of both voice and face recognition, with relatively preserved recognition of musical instruments and environmental sounds. Both patients demonstrated preserved ability to analyse perceptual properties of voices and to recognise vocal emotions. The voice processing deficit in both patients could be characterised as associative phonagnosia: in the case of QR, this was relatively selective for voices, while in the case of KL, there was evidence for a multimodal impairment of person knowledge. The findings have implications for current cognitive models of voice recognition.

Purging Musical Instrument Sample Databases Using Automatic Musical Instrument Recognition Methods

Compilation of musical instrument sample databases requires careful elimination of badly recorded samples and validation of sample classification into correct categories. This paper introduces algorithms for automatic removal of instrument samples using automatic musical instrument recognition and outlier detection techniques. Best evaluation results on a methodically contaminated sound database are achieved using the introduced MCIQR method, which removes 70.1% bad samples with 0.9% false-alarm rate and 90.4% with 8.8% false-alarm rate.

Temporal Integration for Audio Classification With Application to Musical Instrument Classification

Nowadays, it appears essential to design automatic indexing tools which provide meaningful and efficient means to describe the musical audio content. There is in fact a growing interest for music information retrieval (MIR) applications amongst which the most popular are related to music similarity retrieval, artist identification, musical genre or instrument recognition. Current MIR-related classification systems usually do not take into account the mid-term temporal properties of the signal (over several frames) and lie on the assumption that the observations of the features in different frames are statistically independent. The aim of this paper is to demonstrate the usefulness of the information carried by the evolution of these characteristics over time. To that purpose, we propose a number of methods for early and late temporal integration and provide an in-depth experimental study on their interest for the task of musical instrument recognition on solo musical phrases. In particular, the impact of the time horizon over which the temporal integration is performed will be assessed both for fixed and variable frame length analysis. Also, a number of proposed alignment kernels will be used for late temporal integration. For all experiments, the results are compared to a state of the art musical instrument recognition system.

Effects of Training on Recognition of Musical Instruments Presented through Cochlear Implant Simulations

The simulation of the CI (cochlear implant) signal presents a degraded representation of each musical instrument, which makes recognition difficult. To examine the efficiency and effectiveness of three types of training on recognition of musical instruments as presented through simulations of the sounds transmitted through a CI. Participants were randomly assigned to one of three training conditions: repeated exposure, feedback, and direct instruction. Sixty-six adults with normal hearing. Each participant completed three training sessions per week, over a five-week time period, in which they listened to the CI simulations of eight different musical instruments. Analyses on percent of instruments identified correctly showed statistically significant differences between recognition accuracy of the three training conditions (p < .01). those assigned to the direct instruction group showed the greatest improvement over the five-week training period as well as sustained improvement after training. The feedback group achieved the next highest level of recognition accuracy. The repeated exposure group showed modest improvement during the first three-week time period, but no subsequent improvements. These results indicate that different types of training are differentially effective with regard to improving recognition of musical instruments presented through a degraded signal, which has practical implications for the auditory rehabilitation of persons who use cochlear implants.

Sound Isolation by Harmonic Peak Partition For Music Instrument Recognition

Identification of music instruments in polyphonic sounds is difficult and challenging, especially where heterogeneous harmonic partials are overlapping with each other. This has stimulated the research on sound separation for content-based automatic music information retrieval. Numerous successful approaches on musical data feature extraction and selection have been proposed for instrument recognition in monophonic sounds. Unfortunately, none of those algorithms can be successfully applied to polyphonic sounds. Based on recent research in sound classification of monophonic sounds and studies in speech recognition, Moving Picture Experts Group (MPEG) standardized a set of features of the digital audio content data for the purpose of interpretation of the informationmeaning. Most of themare in a formof largematrix or vector of large size, which are not suitable for traditional data mining algorithms; while other features in smaller size are not sufficient for instrument recognition in polyphonic sounds. Therefore, these acoustical features themselves alone cannot be successfully applied to classification of polyphonic sounds. However, these features contain critical information, which implies music instruments' signatures. We have proposed a novel music information retrieval system with MPEG-7-based descriptors and we built classifiers which can retrieve the important time-frequency timbre information and isolate sound sources in polyphonic musical objects, where two instruments are playing at the same time, by energy clustering between heterogeneous harmonic peaks.

Sinusoidal wavelet transforms for pitch estimation of music signals

Pitch is related to fundamental frequency in an audio signal. It is a perceptual measure and depends on the auditory system. Discrete Fourier transforms which are most commonly used provide same resolution through the entire range of frequencies and give the output as an average over the entire bin. Resolution of human auditory system varies logarithmically which maps just as the multiresolution analysis based on wavelet transforms. Sinusoidal wavelet transforms are proposed here to detect two continuously occurring signals to a good resolution both in frequency and time. The accuracy can be increased to a desired value by increasing the number of scaling factors used. The technique can be used for note identification and musical instrument recognition through pattern classification.

Instrogram: Probabilistic Representation of Instrument Existence for Polyphonic Music

This paper presents a new technique for recognizing musical instruments in polyphonic music. Since conventional musical instrument recognition in polyphonic music is performed notewise, i.e., for each note, accurate estimation of the onset time and fundamental frequency (F0) of each note is required. However, these estimations are generally not easy in polyphonic music, and thus estimation errors severely deteriorated the recognition performance. Without these estimations, our technique calculates the temporal trajectory of instrument existence probabilities for every possible F0. The instrument existence probability is defined as the product of a nonspecific instrument existence probability calculated using the PreFEst and a conditional instrument existence probability calculated using hidden Markov models. The instrument existence probability is visualized as a spectrogram-like graphical representation called the instrogram and is applied to MPEG-7 annotation and instrumentation-similaritybased music information retrieval. Experimental results from both synthesized music and real performance recordings have shown that instrograms achieved MPEG-7 annotation (instrument identification) with a precision rate of 87.5% for synthesized music and 69.4% for real performances on average and that the instrumentation similarity measure reflected the actual instrumentation better than an MFCC-based measure.