Music Score Recognition Research Articles

Application of artificial intelligence based on pattern recognition in music entertainment environment and automatic music recognition

The development of digital technology has given music dissemination a technological advantage of high productivity and efficiency. With the continuous improvement of digital technology and the diversification of communication forms, information visualization design pays more attention to the audience’s experience and feelings. This article analyzes the application of pattern recognition based artificial intelligence in music entertainment environments and automatic recognition of music. Digital media can achieve bidirectional information transmission through user participation and feedback, and can provide customers with more entertainment music experiences through entertainment games and other means. This article focuses on the image enhancement processing, note recognition, and automatic recognition conversion of music scores in complex contexts, and develops an automatic music score recognition system to digitize music score information. Through online dissemination, the public can fully experience the beauty of music in the process of communication and interaction, and at the same time, music can be brought into the entertainment environment.

Exploring Countermeasures for the Problem of Music Score Recognition in Piano Teaching for the Visually Impaired

Students with visual impairment encounter many difficulties in the learning process of piano, the most prominent of which is the problem of reading music score. Due to objective constraints, visually impaired students have difficulty in quickly accessing information in music scores, which makes them slow, inefficient and prone to errors. At this point in time, piano teaching should change the traditional "score reading - sight-reading" training method, give full play to the strengths of the other senses of visually impaired students, reduce the impact of visual impairment, develop a suitable way of reading music for visually impaired students. In this paper, we study the common problem of reading music for visually impaired piano students, analyze the special situation of visually impaired piano students, sort out the causes and solutions of the problem of reading music in visually impaired piano teaching, and discuss the strategies to solve the problem of reading music in visually impaired piano teaching on the basis of actual teaching research. It is hoped that this study will provide a reference for piano educators who face such problems, and help visually impaired piano students to get rid of their difficulties in reading music, so that they can learn piano successfully.

Music Score Recognition Method Based on Deep Learning.

In recent years, the recommendation application of artificial intelligence and deep music has gradually become a research hotspot. As a complex machine learning algorithm, deep learning can extract features with value laws through training samples. The rise of deep learning network will promote the development of artificial intelligence and also provide a new idea for music score recognition. In this paper, the improved deep learning algorithm is applied to the research of music score recognition. Based on the traditional neural network, the attention weight value improved convolutional neural network (CNN) and high execution efficiency deep belief network (DBN) are introduced to realize the feature extraction and intelligent recognition of music score. Taking the feature vector set extracted by CNN-DBN as input set, a feature learning algorithm based on CNN&DBN was established to extract music score. Experiments show that the proposed model in a variety of different types of polyphony music recognition showed more accurate recognition and good performance; the recognition rate of the improved algorithm applied to the soundtrack identification is as high as 98.4%, which is significantly better than those of other classic algorithms, proving that CNN&DBN can achieve better effect in music information retrieval. It provides data support for constructing knowledge graph in music field and indicates that deep learning has great research value in music retrieval field.

Music Score Recognition and Composition Application Based on Deep Learning

Optical score recognition is a critical technology for retrieving music information, and note recognition is a critical component of score recognition. This article evaluates and discusses the current state of research on important technologies for score recognition. To address the issues of low note recognition accuracy and intricate steps in the present music score image, a deep learning-based music score recognition model is proposed. The model employs a deep network, accepts the entire score image as input, and outputs the note's time value and pitch directly. Experiments on music scores demonstrate that the method described in this study has a high note identification accuracy of 0.95 for time values and 0.97 for pitch, which is suitable for composition.

Improved Feature Pyramid Convolutional Neural Network for Effective Recognition of Music Scores

Music written by composers and performed by multidimensional instruments is an art form that reflects real-life emotions. Historically, people disseminated music primarily through sheet music recording and oral transmission. Among them, recording music in sheet music form was a great musical invention. It became the carrier of music communication and inheritance, as well as a record of humanity's magnificent music culture. The advent of digital technology solves the problem of difficult musical score storage and distribution. However, there are many drawbacks to using data in image format, and extracting music score information in editable form from image data is currently a challenge. An improved convolutional neural network for musical score recognition is proposed in this paper. Because the traditional convolutional neural network SEGNET misclassifies some pixels, this paper employs the feature pyramid structure. Use additional branch paths to fuse shallow image details, shallow texture features that are beneficial to small objects, and high-level features of global information, enrich the multi-scale semantic information of the model, and alleviate the problem of the lack of multiscale semantic information in the model. Poor recognition performance is caused by semantic information. By comparing the recognition effects of other models, the experimental results show that the proposed musical score recognition model has a higher recognition accuracy and a stronger generalization performance. The improved generalization performance allows the musical score recognition method to be applied to more types of musical score recognition scenarios, and such a recognition model has more practical value.

Fuzzy Theory-based Air Valve Control for Auto-Score-Recognition Soprano Recorder Machines

In the past research, there are many disadvantages to score recognition and flute performance. In view of this, we will improve the above disadvantages in this article. First, for the music score recognition, a y-axis projection method is used to detect the staff position and eliminate it to replace the erosion and expansion in morphology. This feature can be used to distinguish the notes, which have a specific writing style on the staff. For the soprano recorder playing, in the past we used finger-shaped electric arms to press the blow hole to cause that the speed of the score cannot be kept up. To improve this drawback, the motor is changed to a solenoid valve to facilitate the pneumatic cylinder to smoothly press the blow hole. In addition, since the difference in pitch of the soprano recorder requires different air pressure, we increase one valve to three valves. Moreover, the range is divided into bass, midrange, and treble. Not only that, fuzzy theory-based air valve control is applied to auto-score-recognition soprano recorder machines to greatly improve the sound distortion caused by the original single air valve. Experiments prove that the fuzzy theory-based air valve control combined with sheet music recognition techniques can fully realize the functions of autoplaying soprano recorder machines.

Music Signal Recognition Based on the Mathematical and Physical Equation Inversion Method

Digitization and analysis processing technology of music signals is the core of digital music technology. The paper studies the music signal feature recognition technology based on the mathematical equation inversion method, which is aimed at designing a method that can help music learners in music learning and music composition. The paper firstly studies the modeling of music signal and its analysis and processing algorithm, combining the four elements of music sound, analyzing and extracting the characteristic parameters of notes, and establishing the mathematical model of single note signal and music score signal. The single note recognition algorithm is studied to extract the Mel frequency cepstrum coefficient of the signal and improve the DTW algorithm to achieve single note recognition. Based on the implementation of the single note algorithm, we combine the note temporal segmentation method based on the energy-entropy ratio to segment the music score into single note sequences to realize the music score recognition. The paper then goes on to study the music synthesis algorithm and perform simulations. The benchmark model demonstrates the positive correlation of pitch features on recognition through comparative experiments and explores the number of harmonics that should be attended to when recognizing different instruments. The attention network-based classification model draws on the properties of human auditory attention to improve the recognition scores of the main playing instruments and the overall recognition accuracy of all instruments. The two-stage classification model is divided into a first-stage classification model and a second-stage classification model, and the second-stage classification model consists of three residual networks, which are trained separately to specifically identify strings, winds, and percussions. This method has the highest recognition score and overall accuracy.

State-of-the-Art Model for Music Object Recognition with Deep Learning

Optical music recognition (OMR) is an area in music information retrieval. Music object detection is a key part of the OMR pipeline. Notes are used to record pitch and duration and have semantic information. Therefore, note recognition is the core and key aspect of music score recognition. This paper proposes an end-to-end detection model based on a deep convolutional neural network and feature fusion. This model is able to directly process the entire image and then output the symbol categories and the pitch and duration of notes. We show a state-of-the-art recognition model for general music symbols which can get 0.92 duration accurary and 0.96 pitch accuracy .

Research on the theory and practice of multimedia music image photography in music research

Note recognition is the core and key of music score recognition. In this paper, according to the diversity and polymorphism of notes, a structure-based recognition scheme is determined, and its recognition process is divided into two stages: primitive extraction and structure analysis. In the aspect of note primitive extraction, the rough extraction method based on vertical run-length coding and the fine detection method based on horizontal run-length coding are proposed, which overcome the shortcomings of the existing methods such as poor adaptability of complex notes and incomplete extraction results. A solid head extraction method is designed, which first divides and then detects the features. This method uses the prior knowledge of notes and the existing spectral lines and trunk recognition. In the aspect of note structure analysis, a new method of note structure analysis based on action field is proposed. In this method, the concept of action field in physics is introduced into the relationship expression of note primitives, and the unity of knowledge, generality and accuracy is realized. On this basis, six note substructures are defined, and a note structure analysis model of key structure priority location is established, which realizes the reconstruction of the note base metadata to the note object. This model reflects the thinking habit of highlighting key features and from the whole to the details in manual spectrum recognition. It not only reduces the complexity of analysis, but also has a strong ability of primitive redundancy error removal. The test results show that the overall recognition performance has reached the level of the current excellent commercial music score recognition system, and it has obvious advantages in note recognition, adaptability in different data environments and execution speed.

From Optical Music Recognition to Handwritten Music Recognition: A baseline

Optical Music Recognition (OMR) is the branch of document image analysis that aims to convert images of musical scores into a computer-readable format. Despite decades of research, the recognition of handwritten music scores, concretely the Western notation, is still an open problem, and the few existing works only focus on a specific stage of OMR. In this work, we propose a full Handwritten Music Recognition (HMR) system based on Convolutional Recurrent Neural Networks, data augmentation and transfer learning, that can serve as a baseline for the research community.

Recognition of Music Scores with Non-Linear Distortions in Mobile Devices

Optical music recognition (OMR), when the input music score is captured by a handheld or a mobile phone camera, suffers from severe degradation in the image quality and distortions caused by non-planar document curvature and perspective projection. Hence the binarization of the input often fails to preserve the details of the original music score, leading to a poor performance in recognition of music symbols. This paper addresses the issue of staff line detection, which is the most important step in OMR, in the presence of nonlinear distortions and describes how to cope with severe degradations in recognition of music symbols. First, a RANSAC-based detection of curved staff lines is presented and staves are segmented into sub-areas for the rectification with bi-quadratic transformation. Then, run length coding is used to recognize music symbols such as stem, note head, flag, and beam. The proposed system is implemented on smart phones, and it shows promising results with music score images captured in the mobile environment.

An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition.

Image-based sequence recognition has been a long-standing research topic in computer vision. In this paper, we investigate the problem of scene text recognition, which is among the most important and challenging tasks in image-based sequence recognition. A novel neural network architecture, which integrates feature extraction, sequence modeling and transcription into a unified framework, is proposed. Compared with previous systems for scene text recognition, the proposed architecture possesses four distinctive properties: (1) It is end-to-end trainable, in contrast to most of the existing algorithms whose components are separately trained and tuned. (2) It naturally handles sequences in arbitrary lengths, involving no character segmentation or horizontal scale normalization. (3) It is not confined to any predefined lexicon and achieves remarkable performances in both lexicon-free and lexicon-based scene text recognition tasks. (4) It generates an effective yet much smaller model, which is more practical for real-world application scenarios. The experiments on standard benchmarks, including the IIIT-5K, Street View Text and ICDAR datasets, demonstrate the superiority of the proposed algorithm over the prior arts. Moreover, the proposed algorithm performs well in the task of image-based music score recognition, which evidently verifies the generality of it.

Staff removal using image operator learning

Staff removal is an image processing task that aims to facilitate further analysis of music score images. Even when restricted to images in specific domains such as music score recognition, solving image processing problems usually requires the design of customized algorithms. To cope with image variabilities and the growing amount of data, machine learning based techniques emerge as a natural approach to be employed in image processing problems. In this sense, image operator learning methods are concerned with estimating, from sample pairs of input-output images of a transformation, a local function that characterizes the image transformation. These methods require the definition of some parameters, including the local information to be considered in the processing which is defined by a window. In this work we show how to apply the image operator learning technique to the staff line removal problem. We present an algorithm for window determination and show that it captures visual information relevant for staff removal. We also present a reference window set to be used in cases where the training set is not sufficiently large. Experimental results obtained with respect to synthetic and handwritten music scores under varying image conditions show that the learned image operators are comparable with especially designed state-of-the-art heuristic algorithms.

Design of Music Learning Assistant Based on Audio Music and Music Score Recognition

Mastering a musical instrument for an unskilled beginning learner is not an easy task. It requires playing every note correctly and maintaining the tempo accurately. Any music comes in two forms, a music score and it rendition into an audio music. The proposed method of assisting beginning music players in both aspects employs two popular pattern recognition methods for audio-visual analysis; they are support vector machine (SVM) for music score recognition and hidden Markov model (HMM) for audio music performance tracking. With proper synchronization of the two results, the proposed music learning assistant system can give useful feedback to self-training beginners.

Music Score Recognition Based on a Collaborative Model

Recognition musical symbols are very important in music score system and they depend on these methods of researchers. Most of existing approaches for OMR (optical music recognition) removes staff lines before symbols are detected, therefore the symbols can get damaged easily. Another method recognizes symbols without staff line removal but all of them have a low accuracy rate and high processing time for recognizing symbols. In this paper, none staff removal and staff removal are suggested and these new methods are proposed to improve appreciation result of symbols. A lot of symbols are detected before deleted staff line as vertical lines, note head, pitch, beam, tail and then these staff lines are removed to identify other symbols using connected component. The proposed method is applied to the Samsung smart phone which embeds a high resolution camera. Experimental results show that the recognition rate is higher than existing methods and the computation time is reduced significantly.

A Lightweight and Effective Music Score Recognition on Mobile Phones

Recognition systems for scanned or printed music scores that have been implemented on personal computers have received attention from numerous scientists and have achieved significant results over many years. A modern trend with music scores being captured and played directly on mobile devices has become more interesting to researchers. The limitation of resources and the effects of illumination, distortion, and inclination on input images are still challenges to these recognition systems. In this paper, we introduce a novel approach for recognizing music scores captured by mobile cameras. To reduce the complexity, as well as the computational time of the system, we grouped all of the symbols extracted from music scores into ten main classes. We then applied each major class to SVM to classify the musical symbols separately. The experimental results showed that our proposed method could be applied to real time applications and that its performance is competitive with other methods.

악보인식 전처리를 위한 강건한 오선 두께와 간격 추정 방법

이 논문에서는 모바일 기기상에서 카메라기반 악보인식을 위한 오선 두께와 오선 간격을 추정하는 전처리 기술을 제안한다. 캡쳐된 영상은 조명이나, 흐려짐, 저해상도 등의 많은 왜곡으로 인해 인식에 어려움이 있다. 특히 복잡한 배경을 가지고 있는 악보 영상인식의 경우 더욱 그렇다. 악보 기호 인식에서 오선 두께와 오선 간격은 인식에 큰 영향을 끼친다. 이들 정보는 이진화에도 사용되는데, 복잡한 배경을 가지고 있는 경우 일반적인 이진 영상은 오선 두께와 간격을 추정하는데 만족스럽지 못하다. 따라서 우리는 에지영상에서 런-길이 엔코딩 기술을 이용해 오선 두께와 간격 추정하는 강건한 알고리즘을 제안한다. 제안된 방법은 2단계로 구성되어 있다. 첫 번째 단계는 소벨 연산자에 의해 영역별로 에지 영상을 기반으로 오선 두께와 간격을 추정한다. 각 에지 영상의 열은 런-길이 엔코딩 알고리즘에 의해 기술된다. 두 번째 단계는 안정한 경로 알고리즘을 이용한 오선 검출과 오선 위치를 추적하는 적응적 LTH알고리즘을 이용한 오선 제거이다. 실험결과 복잡한 영상의 경우에도 강건함과 높은 인식률을 보였다. In this paper, we propose a robust pre-processing module for camera-based Optical Music Score Recognition (OMR) on mobile device. The captured images likely suffer for recognition from many distortions such as illumination, blur, low resolution, etc. Especially, the complex background music sheets recognition are difficult. Through any symbol recognition system, the staff line height and staff line space are used many times and have a big impact on recognition module. A robust and accurate staff line height and staff line space are essential. Some staff line height and staff line space are proposed for binary image. But in case of complex background music sheet image, the binarization results from common binarization algorithm are not satisfactory. It can cause incorrect staff line height and staff line space estimation. We propose a robust staff line height and staff line space estimation by using run-length encoding technique on edge image. Proposed method is composed of two steps, first step, we conducted the staff line height and staff line space estimation based on edge image using by Sobel operator on image blocks. Each column of edge image is encoded by run-length encoding algorithm Second step, we detect the staff line using by Stable Path algorithm and removal the staff line using by adaptive Line Track Height algorithm which is to track the staff lines positions. The result has shown that robust and accurate estimation is possible even in complex background cases.

SOM과 개선된 ART-1을 이용한 악보 인식

본 논문에서는 SOM과 개선된 ART-1을 이용하여 악보를 인식하는 방법을 제안한다. 악보 인식을 위해 스캔된 악보 이미지를 호프 변환, Otsu's 이진화를 원본 이미지에 적용하고, 히스토그램 분석을 통해 구분된 작은악절에서 오선을 제거하여 악보의 음표 성분을 추출할 수 있는 이미지 전처리 단계를 수행한다. 오선이 제거된 작은악절은 레이블링을 이용하여 음표 성분을 분리한다. 추출된 음표들은 SOM 알고리즘을 적용하여 일정한 크기로 정규화하고, 정규화된 음표 정보들을 개선된 ART-I 알고리즘을 적용하여 학습과 인식한다. 제안된 방법을 적용하여 음표 인식 실험을 한 결과, 제안된 방법이 음표 인식에 효율적임을 확인하였다. In this paper, we propose a Musical Score Recognition with SOM and Enhanced ART-1 Algorithm. First, we apply Hough transform and Otsu's binarization to the original BMP format image and extract notes from separated passages by histogram analysis with removing staff lines. Then extracted musical notes are normalized to same size by SOM algorithm and ART-1 algorithm plays the learning and recognition role. The experiment verifies that the proposed method is quite effective on printed musical score recognition.

VECTORIZATION AND LINE DETECTION FOR AUTOMATIC IMAGE RECOGNITION

We propose an algorithm for creating line graphs from binary images. The algorithm consists of a vectorizer followed by a line detector that can handle a large variety of binary images and is tolerant to noise. The proposed algorithm can accurately extract higher-level geometry from the images lending itself well to automatic image recognition tasks. Our algorithm revisits the technique of image polygonization proposing a very robust variant based on subpixel resolution and the construction of directed paths along the center of the border pixels where each pixel can correspond to multiple nodes along one path. The algorithm has been used in the areas of chemical structure and musical score recognition and is available for testing at www.docnition.com. Extensive testing of the algorithm against commercial and noncommercial methods has been conducted with favorable results.

An Overview of Optical Music Recognition in China

OMR (Optical Music Recognition) is a technology for digital musical score image processing and recognition by computer, which has broad applications in the digital music library, contemporary music education, music theory, music automatic classification, music and audio sync dissemination and etc. This paper first has a brief description of OMR research and focuses on describing the research of Chinese OMR literature, it represents the research status and results in China, then the paper pointes out that the target of OMR research in China must tend to Chinese traditional musical score image processing and pattern recognition.