Musical Instrument Recognition Research Articles

Research on the application of vocal music teaching methods and techniques in colleges and universities based on proportional integral differentiation algorithm

Abstract With the continuous development of modern information technology, the combination of intelligent audio processing technology and vocal music teaching has gradually become a research hot spot. In this paper, we first build a vocal music teaching system based on music emotion and instrument recognition, optimize the support vector machine using the PSO algorithm, construct the music emotion recognition and instrument recognition method based on SVM, and control and optimize the vocal music teaching system through multi-objective proportional integral differentiation algorithm. Through the comparison experiments of different models of music emotion recognition and musical instrument recognition, the performance of music emotion recognition and musical instrument recognition of this paper’s model is explored. Then, the application effect analysis of the vocal music teaching system is carried out. The results show that the SVM model optimized by PSO has a more satisfactory effect on music emotion recognition, with a recognition accuracy 16.67% higher than the comparison model and an average adaptability of 70%–90%. In addition, this model has a higher instrument recognition rate of 18.17% and 7.45% compared to the other two models. After using the vocal teaching system, 63.04% of the students thought that it could promote learning, 47.83% of the student’s classroom interest increased, and more than 70% of the students were more satisfied with its functions. In this paper, the vocal teaching system can be applied to college vocal teaching to promote the improvement of the vocal teaching effect.

Attention-Based Deep Spiking Neural Networks for Temporal Credit Assignment Problems.

The temporal credit assignment (TCA) problem, which aims to detect predictive features hidden in distracting background streams, remains a core challenge in biological and machine learning. Aggregate-label (AL) learning is proposed by researchers to resolve this problem by matching spikes with delayed feedback. However, the existing AL learning algorithms only consider the information of a single timestep, which is inconsistent with the real situation. Meanwhile, there is no quantitative evaluation method for TCA problems. To address these limitations, we propose a novel attention-based TCA (ATCA) algorithm and a minimum editing distance (MED)-based quantitative evaluation method. Specifically, we define a loss function based on the attention mechanism to deal with the information contained within the spike clusters and use MED to evaluate the similarity between the spike train and the target clue flow. Experimental results on musical instrument recognition (MedleyDB), speech recognition (TIDIGITS), and gesture recognition (DVS128-Gesture) show that the ATCA algorithm can reach the state-of-the-art (SOTA) level compared with other AL learning algorithms.

Pencirian Isyarat Data Muzik yang Terhasil dari Alat Muzik Tradisional Menggunakan Ciri Fraktal

Musical instruments are usually distinguished by their sound produced through human perception which may lead to misinterpretation due to auditory perception bias and other disturbances. Therefore, recognition using music signals is carried out to help characterize signals from different musical instruments. Fractal analysis is a mathematical tool used to study complex and irregular patterns in various systems. In this study, fractal analysis was used to study and analyze musical notes signal data from different instruments. The fractal analysis method used is the box counting method. The traditional musical instruments involved in this study are the seruling, cak lempong, kompang, and gambus. Matlab software was used to analyze the musical data signal. First, the fractal dimension of the music data signal in the form of time domain is calculated. Then, the mean and standard deviation of the fractal dimension values were determined to recognize different musical instruments. Additionally, different image resolutions and box sizes are also used to calculate the fractal dimension of musical instrument from the time domain data. The error of best fit line, E will also be calculated to ensure the reliability of box counting dimensions using leastsquares regression method. The results show that the value of the fractal dimension for all 18 data is between 1.2636 to 1.7543. In terms of musical instrument recognition, the recognition of seruling and kompang using mean fractal dimension is successful. However, this method is less suitable for identification for cak lempong and gambus due to the high standard deviation. By using different image resolutions and box sizes, the accuracy of fractal dimension values will be affected. Increasing these parameters increases the accuracy of the results. The results of the study show that an image with a resolution of 1024 x 1024 pixels and a scaling factor of 9 is suitable to analyze the fractal characteristics of musical instrument data signals by using the box counting method.

APLIKASI PENGENALAN ANATOMI SISTEM PENCERNAAN PADA TUBUH MANUSIA MENGGUNAKAN TEKNOLOGI AUGMENTED REALITY

Natural Sciences (IPA) is a science related to human endeavors in understanding and finding out about the universe through precise observations and using procedures that can be explained by reasoning so as to get a clear conclusion. There are 5 main branches in Natural Sciences, namely Astronomy, Biology, Chemistry, Earth Sciences, and Physics. Science learning should be able to provide direct experience so as to increase the ability to understand and apply the concepts that have been learned. Anatomy is a branch of biology that deals with the structure of living things, in the anatomy of the human body, there are organs that support the formation of the body as a whole. The organs of the body such as bones, joints, skeletal muscles, brain, digestive system, and circulatory system and other organs. The digestive system is a process of mechanically and chemically breaking food into a form that is easier for the body to absorb. interactively in real time. The existence of technology in the form of Augmented Reality will facilitate the process of introducing information and displaying Anatomy Learning objects. This research aims to introduce anatomy to a digestive system in the human body using Augmented Reality technology which can facilitate students in the learning process. The software development method used to design and implement traditional musical instrument recognition applications is the Rational Unified Process method. Application testing is carried out using black-box testing and user testing. User testing was carried out by distributing questionnaires at SMP Negeri 2 Pedamaran Timur. Testing user satisfaction is calculated using a Likert scale and the results of questionnaire questions which can have a final score of 87% which is included in the strongly agree category

APLIKASI PENGENALAN ALAT MUSIK TRADISIONAL SUMATERA SELATAN MENGGUNAKAN TEKNOLOGI AUGMENTED REALITY

South Sumatra has several traditional musical instruments in the form of Burdah OKU, Gong, Kolintang, Flute, Gambus, Trumpet, Weaving, Genggong, Tambourine, Kenong Basemah. In this research, an application for recognizing traditional South Sumatran musical instruments has been designed and implemented by utilizing Augmented Reality technology. Augmented Reality is a combination of real objects and virtual objects that run interactively in real time. With the existence of technology in the form of Augmented Reality, it will facilitate the process of recognizing information and displaying objects of traditional musical instruments in South Sumatra. This research aims to help reintroduce residents to increase their knowledge of traditional musical instruments in South Sumatra. The software development method used to design and implement traditional musical instrument recognition applications is the Rational Unified Process method. Application testing is carried out using black-box testing and user testing. User testing was carried out by distributing questionnaires at Madrasah Ibtidaiyah Marfu'ah to grade 5 students. User satisfaction testing was calculated using a Likert scale and the results of the questionnaire questions obtained had a final score of 91% which was included in the strongly agree category

AAM: a dataset of Artificial Audio Multitracks for diverse music information retrieval tasks

We present a new dataset of 3000 artificial music tracks with rich annotations based on real instrument samples and generated by algorithmic composition with respect to music theory. Our collection provides ground truth onset information and has several advantages compared to many available datasets. It can be used to compare and optimize algorithms for various music information retrieval tasks like music segmentation, instrument recognition, source separation, onset detection, key and chord recognition, or tempo estimation. As the audio is perfectly aligned to original MIDIs, all annotations (onsets, pitches, instruments, keys, tempos, chords, beats, and segment boundaries) are absolutely precise. Because of that, specific scenarios can be addressed, for instance, detection of segment boundaries with instrument and key change only, or onset detection only in tracks with drums and slow tempo. This allows for the exhaustive evaluation and identification of individual weak points of algorithms. In contrast to datasets with commercial music, all audio tracks are freely available, allowing for extraction of own audio features. All music pieces are stored as single instrument audio tracks and a mix track, so that different augmentations and DSP effects can be applied to extend training sets and create individual mixes, e.g., for deep neural networks. In three case studies, we show how different algorithms and neural network models can be analyzed and compared for music segmentation, instrument recognition, and onset detection. In future, the dataset can be easily extended under consideration of specific demands to the composition process.

A Comprehensive Review on Audio based Musical Instrument Recognition: Human-Machine Interaction Towards Industry 4.0

Over the last two decades, the application of machine technology has shifted from industrial to residential use. Further, advances in hardware and software sectors have led machine technology to its utmost application, the human-machine interaction, a multimodal communication. Multimodal communication refers to the integration of various modalities of information like speech, image, music, gesture, and facial expressions. Music is the non-verbal type of communication that humans often use to express their minds. Thus, Music Information Retrieval (MIR) has become a booming field of research and has gained a lot of interest from the academic community, music industry, and vast multimedia users. The problem in MIR is accessing and retrieving a specific type of music as demanded from the extensive music data. The most inherent problem in MIR is music classification. The essential MIR tasks are artist identification, genre classification, mood classification, music annotation, and instrument recognition. Among these, instrument recognition is a vital sub-task in MIR for various reasons, including retrieval of music information, sound source separation, and automatic music transcription. In recent past years, many researchers have reported different machine learning techniques for musical instrument recognition and proved some of them to be good ones. This article provides a systematic, comprehensive review of the advanced machine learning techniques used for musical instrument recognition. We have stressed on different audio feature descriptors of common choices of classifier learning used for musical instrument recognition. This review article emphasizes on the recent developments in music classification techniques and discusses a few associated future research problems.

Testing the role of primary musical instrument on context effects in music perception

Musicians display numerous perceptual benefits versus nonmusicians, such as better pitch and melody perception (the “musician advantage”). Recently, Shorey et al. (2021 ASA) investigated whether this musician advantage extended to spectral contrast effects (SCEs; categorization shifts produced by acoustic properties of surrounding sounds) in musical instrument recognition. Musicians and nonmusicians listened to a context sound (filtered string quartet passage highlighting frequencies of the horn or saxophone), then categorized a target sound (tone from a six-step series varying from horn to saxophone). Although musicians displayed superior pitch discrimination, their SCEs did not differ from those of nonmusicians. Importantly, separate research has reported that a musician’s instrument of training heavily influences musical perception, potentially improving frequency discrimination and rhythm perception/production. However, in the Shorey et al. study, musicians were recruited without respect to their primary instrument. This follow-up study uses the same methodology as Shorey et al. but recruits only musicians who play horn or saxophone (the instruments used as target sounds) as their primary instrument. It is predicted that horn and saxophone players will display larger SCEs than nonmusicians due to their intimate familiarity with the instrument timbres. Preliminary data are trending in the predicted direction; full results will be discussed.

Neural Models for Target-Based Computer-Assisted Musical Orchestration: A Preliminary Study

In this paper we will perform a preliminary exploration on how neural networks can be used for the task of target-based computer-assisted musical orchestration. We will show how it is possible to model this  musical problem as a classification task and we will propose two deep learning models. We will show, first, how they perform as classifiers for musical instrument recognition by comparing them with specific baselines. We will then show how they perform, both qualitatively and quantitatively, in the task of computer-assisted orchestration by comparing them with state-of-the-art systems. Finally, we will highlight benefits and problems of neural approaches for assisted orchestration and we will propose possible future steps. This paper is an extended version of the paper "A Study on Neural Models for Target-Based Computer-Assisted Musical Orchestration" published in the proceedings of The 2020 Joint Conference on AI Music Creativity. 

Construction of Vocal Timbre Evaluation System Based on Classification Algorithm

With the continuous development of communication technology, computer technology, and network technology, a large amount of information such as images, videos, and audios has grown exponentially, and people have started to be exposed to massive multimedia contents, which can easily and quickly access the increasingly rich music resources, so new technologies are urgently needed for their effective management, and automatic classification of audio signals has become the focus of engineering and academic attention. Currently, music retrieval can be achieved by selecting song titles and singer names, but as people’s living standards continue to improve, the spiritual realm is also enriched. People want to be able to select music with different types of emotional expressions with their emotions. It mainly includes the basic principles of audio classification, the analysis and extraction of music emotion features, and the selection of the best classifier. Two classification algorithms, hybrid Gaussian model and AdaBoost, are used to classify music emotions, and the two classifiers are combined. In this paper, we propose the Discrete Harmonic Transform (DHT), a sparse transform based on harmonic frequencies. This paper derives and proves the formula of Discrete Harmonic Transform and further analyzes the harmonic structure of musical tone signal and the accuracy of harmonic structure. Since the timbre of musical instruments depends on the harmonic structure, and similar instruments have similar harmonic structures, the discrete harmonic transform coefficients can be defined as objective indicators corresponding to the timbre of musical instruments, and thus the concept of timbre expression spectrum is proposed, and a specific construction algorithm is given in this paper. In the application of musical instrument recognition, the 53-dimensional combined features of LPCC, MFCC, and timbre expression spectrum are selected, and a nonlinear support vector machine is used as the classifier. The classification recognition rate is improved by reducing the number of feature dimensions.

Construction of Intelligent Recognition and Learning Education Platform of National Music Genre Under Deep Learning.

In order to study the application of the deep learning (DL) method in music genre recognition, this study introduces the music feature extraction method and the deep belief network (DBN) in DL and proposes the parameter extraction feature and the recognition classification method of an ethnic music genre based on the DBN with five kinds of ethnic musical instruments as the experimental objects. A national musical instrument recognition and classification network structure based on the DBN is proposed. On this basis, a music library classification retrieval learning platform has been established and tested. The results show that, when the DBN only contains one hidden layer and the number of neural nodes in the hidden layer is 117, the basic convergence accuracy is approximately 98%. The first hidden layer has the greatest impact on the prediction results. When the input sample feature size is one-third of the number of nodes in the first hidden layer, the network performance is basically convergent. The DBN is the best way for softmax to identify and classify national musical instruments, and the accuracy rate is 99.2%. Therefore, the proposed DL algorithm performs better in identifying music genres.

Development of Augmented Reality Traditional Musical Education Applications

Technology that is developing very rapidly today is no exception in the world of education, the convenience offered by technology cannot be denied by shifting the way children learn who were born and raised in the digital era. In the field of education, augmented reality technology has been widely implemented, such as applications that use augmented reality book aids. This study aims to create an alternative application as a learning medium for the introduction of traditional musical instruments with a multimedia development method by an elementary school educational institution. In this study, the augmented reality method used is Image Based Tracking, which uses images as markers. The tool that helps in making AR is the Vuforia SDK which is used as a storage area marker. By doing this research, the results were obtained in the form of an Augmented Reality Traditional Musical Instrument Recognition application that runs on the Android platform and respondent testing using the User Acceptance Testing (UAT) method with a value of 86%.

Audio recognition of Chinese traditional instruments based on machine learning

This paper is part of a special issue on Music Technology. We study the type recognition of traditional Chinese musical instrument audio in the common way. Using MEL spectrum characteristics as input, we train an 8-layer convolutional neural network, and finally achieve 99.3% accuracy. After that, this paper mainly studies the performance skill recognition of Chinese traditional musical instruments. Firstly, for a single instrument, the features were extracted by using the pre-trained ResNet model, and then the SVM algorithm was used to classify all the instruments with an accuracy of 99%. Then, in order to improve the generalization of the model, the paper proposes the performance skill recognition of the same kind of instruments. In this way, the regularity of the same playing technique of different instruments can be utilized. Finally, the recognition accuracy of the four kinds of instruments is as follows: 95.7% for blowing instruments, 82.2% for plucked-string instruments, 88.3% for strings instruments, and 97.5% for percussion instruments. We open source the audio database of traditional Chinese musical instruments and the Python source code of the whole experiment for further research.

Correlation Analysis between Cultural Differences and Normal Music Perception Based on Embedded Voice Multisensor

In order to effectively improve the sense of difference brought by the extracorporeal machine to users and minimize the related derived problems, the implementation based on embedded multisensor has become a major breakthrough in the research of cochlear implant. To explore the impact of different cultural differences on timbre perception, effectively evaluate the correlation between cultural differences and music perception teaching based on embedded multisensor normal hearing, evaluate the discrimination ability of embedded multisensor normal hearing to music timbre, and analyse the correlation between cultural differences and timbre perception, it provides a basis for the evaluation of music perception of normal hearing people with embedded multisensor and the design and development of evaluation tool. In this paper, adults with normal hearing in different cultures matched with music experience are selected to test their recognition ability of different musical instruments and the number of musical instruments by using music evaluation software, and the recognition accuracy of the two tests is recorded. The results show that the accuracy of musical instrument recognition in the mother tongue group is 15% higher than that in the foreign language group, and the average recognition rates of oboe, trumpet, and xylophone in the foreign language group are lower than those in the mother tongue group, the recognition rate of oboe and trumpet in wind instruments was low in both groups, and the recognition rate of oboe and trumpet in foreign language group was high.

Musical Instrument Recognition with a Convolutional Neural Network and Staged Training

Musical instrument recognition is an important task within the broad field of Music Information Retrieval. It helps to build recommendation systems, compute similarity between musical compositions and enable automatic search in music collections with regard to the instrument. The task has two variants differed by the difficulty level. The simpler one is classification based on the sound of a single instrument, while the more difficult challenge is to recognize the predominant instrument in polyphonic recordings. In this paper, we used a convolutional neural network to solve both of these problems. As the analysis of monotimbral recordings is relatively easy, we used the knowledge acquired during solving it to train a neural network to tackle the more complex predominant instrument recognition problem. Within this staged training scenario, we also examined the impact of introducing some intermediate stages during the training sessions. The results showed that such a training approach has a potential to improve the accuracy of classification.

Use of Speaker Recognition Approaches for Learning and Evaluating Embedding Representations of Musical Instrument Sounds

Constructing an embedding space for musical instrument sounds that can meaningfully represent new and unseen instruments is important for downstream music generation tasks such as multi-instrument synthesis and timbre transfer. The framework of Automatic Speaker Verification (ASV) provides us with architectures and evaluation methodologies for verifying the identities of unseen speakers, and these can be repurposed for the task of learning and evaluating a musical instrument sound embedding space that can support unseen instruments. Borrowing from state-of-the-art ASV techniques, we construct a musical instrument recognition model that uses a SincNet front-end, a ResNet architecture, and an angular softmax objective function. Experiments on the NSynth and RWC datasets show our model’s effectiveness in terms of equal error rate (EER) for unseen instruments, and ablation studies show the importance of data augmentation and the angular softmax objective. Experiments also show the benefit of using a CQT-based filterbank for initializing SincNet over a Mel filterbank initialization. Further complementary analysis of the learned embedding space is conducted with t-SNE visualizations and probing classification tasks, which show that including instrument family labels as a multi-task learning target can help to regularize the embedding space and incorporate useful structure, and that meaningful information such as playing style, which was not included during training, is contained in the embeddings of unseen instruments.

Android-Based Musical Instrument Recognition Application For Vocational High School Level

With the old way of learning and methods, Vocational High School students will be bored, especially in the field of Arts and Culture, which on average are considered difficult and not liked by most students. Therefore, it is necessary to make other learning media based on Android so that learning becomes more interesting and enjoyable. In addition, with this learning media, it is hoped that it can help children's creativity and thinking power, because it is equipped with several important components that can hone children's imaginative power such as pictures, videos in the material provided. Therefore, the authors design and create an application about interactive learning media that includes these components. This application will be aimed at the Vocational High School level with the title "Android-Based Musical Instrument Recognition Application for Vocational High School Level". Therefore, research was conducted at the Vocational High School level. Users want attractive images of each musical instrument such as guitar, bass, drums, keyboard, have video links, there are questions in the application, a menu in the form of icons, and attractive colors. After designing, manufacturing, and testing this application, several conclusions can be drawn. First, with this application, Vocational High School students can easily learn musical instruments in different ways. In addition, with this learning method, students become enthusiastic and learning becomes fun in studying cultural arts lessons in the field of musical instruments, especially musical instrument techniques and chords of musical instruments. Second, this application is able to hone the cognitive abilities of students because in this application there are practice questions, the questions are integrated with the material provided, so that students can easily practice their abilities through the practice questions available in this application. Third, and lastly, according to the author, this application is lacking in terms of animation in the application that was made late. Hopefully this application can be a learning material for students who want to take a Thesis with a title related to learning musical instruments. The author really expects criticism and suggestions because this design system still has many shortcomings

An Automatic Instrument Recognition Approach Based on Deep Convolutional Neural Network

Background: This paper presents an automatic instrument recognition method highlighting the deep learning aspect of instrument identification in order to advance the automatic process of video monitoring remotely equipment of substation. Methods: This work utilizes the Scale Invariant Feature Transform approach (SIFT) and the Gaussian difference model for instrument positioning while proposing a design scheme of the instrument identification system. Results: The experimental outcomes obtained proves that the proposed system is capable of automatically recognizing a modest graphical interface and study independently while improving the effectiveness of the appliance, thereby, realizing the purpose of spontaneous self-check. The proposed approach is applicable for musical instrument recognition, and it provides 92% of the accuracy rate, 87.5% precision value and recall rate of 91.2%. Conclusion: The comparative analysis with other state-of-the-art methods justifies that the proposed deep learning-based music recognition method outperforms the other existing approaches in terms of accuracy, thereby providing a practicable music instrument recognition solution.

Piano Education of Children Using Musical Instrument Recognition and Deep Learning Technologies Under the Educational Psychology.

The objective of the study was to enhance quality education in the traditional pre-school piano education. Deep Learning (DL) technology is applied to piano education of children to improve their interest in learning music. Firstly, the problems of the traditional piano education of children were analyzed with the teaching patterns discussed under educational psychology, and a targeted music education plan was established. Secondly, musical instrument recognition technology was introduced, and the musical instrument recognition model was implemented based on DL. Thirdly, the proposed model was applied to the piano education of children to guide the music learning of students and improve their interest in piano learning. The feature recognition and acquisition of the proposed model were improved. Finally, the different teaching patterns were comparatively analyzed through the Questionnaire Survey (QS). The experimental results showed that the instrument recognition accuracy of Hybrid Neural Network (HNN) is 97.2%, and with the increase of iterations, the recognition error rate of the model decreases and stabilizes. Therefore, the proposed HNN based on DL for musical instrument recognition can accurately identify musical features. The QS results showed that the introduction of musical instrument recognition technology in the piano education of children can improve their interest in piano learning. Therefore, the establishment of the piano education patterns based on the piano education model can improve the effectiveness of teaching piano to students. This research provides a reference for the intelligentization of children's piano education.

Emotion recognition of musical instruments based on convolution long short time memory depth neural network

In this paper, a method of emotion recognition for musical instruments based on convolution long short time memory depth neural network is proposed, and an emotion recognition music database composed of four musical instruments is established, including keyboard instruments, wind instruments, string instruments and percussion instruments. The emotional types of these four instruments are divided into happiness, anger, sadness and fear. Through the establishment of CLDNN model based musical instrument emotion recognition architecture, MFCCs, CNN and CFS are used for feature extraction and training. The experimental structure shows that the best classification effect is to use long-term memory (LSTM) and deep neural network (DNN) to extract and combine the emotional feature sets of musical instruments, which has the highest accuracy. Considering the dynamic changes of musical features in musical instruments, the modeling method is used to predict the emotional changes of musical instruments.