Genre Classification Task Research Articles

An improved random forest model for music genre classification algorithm based on sparrow search algorithm

In this study, a random forest model optimised based on the sparrow search algorithm is explored for the music genre classification problem. By introducing the sparrow search algorithm, the model is able to search the feature space more comprehensively during the training process, which improves the learning ability of complex data relationships. After 15 iterations of optimisation, the fitness value of the model is improved from 0.2002 to 0.197, showing better convergence. The performance on the training set shows that the model predicts 2,543 music genres with an accuracy of 99.59% and only 10 incorrect predictions, which verifies its excellent performance on the training data. While on the test set, the model's prediction accuracy for 623 music genres is 79.78%, which is slightly lower than the result of the training set, but still reflects good generalisation ability. The results of this paper imply that the improved random forest model based on the sparrow search algorithm has potential application prospects in the music genre classification task. By optimising the parameter settings and feature selection, we not only improve the prediction accuracy and generalisation ability of the model, but also provide new ideas for solving other complex data classification problems. In the future, we can further expand the sample size, try different feature combinations, and conduct in-depth research by combining more domain knowledge in order to further improve the model performance and promote the development of related fields.

Exploration of Speech and Music Information for Movie Genre Classification

Movie genre prediction from trailers is mostly attempted in a multi-modal manner. However, the characteristics of movie trailer audio indicate that this modality alone might be highly effective in genre prediction. Movie trailer audio predominantly consists of speech and music signals in isolation or overlapping conditions. This work hypothesizes that the genre labels of movie trailers might relate to the composition of their audio component. In this regard, speech-music confidence sequences for the trailer audio are used as a feature. In addition, two other features previously proposed for discriminating speech-music are also adopted in the current task. This work proposes a time and channel Attention Convolutional Neural Network (ACNN) classifier for the genre classification task. The convolutional layers in ACNN learn the spatial relationships in the input features. The time and channel attention layers learn to focus on crucial timesteps and CNN kernel outputs, respectively. The Moviescope dataset is used to perform the experiments, and two audio-based baseline methods are employed to benchmark this work. The proposed feature set with the ACNN classifier improves the genre classification performance over the baselines. Moreover, decent generalization performance is obtained for genre prediction of movies with different cultural influences (EmoGDB).

Comparison Among AlexNet, GoogLeNet and ResNet-18 in Autiomatic Music Genre Classification

This paper compared the performance among three famous convolutional neural networks in classifying music genres. Being carried out on a prominent dataset named Free Music Archive, the experiments show that ResNet-18 performs much better than AlexNet and GoogLeNet in classifying the music genres in a relatively small dataset. Meanwhile, the classification accuracy of each model for each music genre was also recorded. It indicates that different models could be expert in identifying distinct genres. Several genres, including blues, hip-hop and international, were not closely related to the change of models. In general, ResNet-18 reached the highest average classification accuracy at approximate 80%, while AlexNet did best in finding hip-hop music and GoogLeNet had relatively less difference in recognition rates for every genre. Those findings can serve as a reference in future music genre classification tasks and personalized music recommendation based on big data.

Audio Scanning Network: Bridging Time and Frequency Domains for Audio Classification

With the rapid growth of audio data, there's a pressing need for automatic audio classification. As a type of time-series data, audio exhibits waveform fluctuations in both the time and frequency domains that evolve over time, with similar instances sharing consistent patterns. This study introduces the Audio Scanning Network (ASNet), designed to leverage abundant information for achieving stable and effective audio classification. ASNet captures real-time changes in audio waveforms across both time and frequency domains through reservoir computing, supported by Reservoir Kernel Canonical Correlation Analysis (RKCCA) to explore correlations between time-domain and frequency-domain waveform fluctuations. This innovative approach empowers ASNet to comprehensively capture the changes and inherent correlations within the audio waveform, and without the need for time-consuming iterative training. Instead of converting audio into spectrograms, ASNet directly utilizes audio feature sequences to uncover associations between time and frequency fluctuations. Experiments on environmental sound and music genre classification tasks demonstrate ASNet's comparable performance to state-of-the-art methods.

A Multi-Branch Residual Network Based on Depth Correlation Features for the Classification of Chinese Ink Paintings

Despite significant strides in digital classification of Chinese ink paintings, existing methods predominantly rely on low-level features, insufficient for capturing the nuanced artistic styles of such works. This study introduces a novel multi-branch residual network that leverages depth correlation features to enhance the classification of Chinese ink paintings. We innovatively combine global style features, extracted using the Gram matrix, with local brushstroke features obtained via the Holistically-Nested Edge Detection(HED) method. This dual-feature approach addresses the limitations of previous studies by incorporating high-level stylistic nuances alongside low-level details, resulting in a more robust classification system. Quantitative results demonstrate a marked improvement in classification accuracy, with our network outperforming existing state-of-the-art models by significant margins in both artist and genre classification tasks. This advancement not only underscores the efficacy of integrating diverse feature sets but also paves the way for more sensitive and accurate management of digital art repositories.

Parts-of-Speech (PoS) Analysis and Classification of Various Text Genres

Abstract Natural language processing (NLP) has made significant leaps over the past two decades due to the advancements in machine learning algorithms. Text classification is pivotal today due to a wide range of digital documents. Multiple feature classes have been proposed for classification by numerous researchers. Genre classification tasks form the basis for advanced techniques such as native language identification, readability assessment, author identification etc. These tasks are based on the linguistic composition and complexity of the text. Rather than extracting hundreds of variables, a simple premise of text classification using only the text feature of parts-of-speech (PoS) is presented here. A new dataset gathered from Project Gutenberg is highlighted in this study. PoS analysis of each text in the created dataset was carried out. Further grouping of these texts into fictional and non-fictional texts was carried out to measure their classification accuracy using the artificial neural networks (ANN) classifier. The results indicate an overall classification accuracy of 98 and 35 % for the genre and sub-genre classification, respectively. The results of the present study highlight the importance of PoS not only as an important feature for text processing but also as a sole text feature classifier for text classification.

MusicNeXt: Addressing category bias in fused music using musical features and genre-sensitive adjustment layer

Convolutional neural networks (CNNs) have been successfully applied to music genre classification tasks. With the development of diverse music, genre fusion has become common. Fused music exhibits multiple similar musical features such as rhythm, timbre, and structure, which typically arise from the temporal information in the spectrum. However, traditional CNNs cannot effectively capture temporal information, leading to difficulties in distinguishing fused music. To address this issue, this study proposes a CNN model called MusicNeXt for music genre classification. Its goal is to enhance the feature extraction method to increase focus on musical features, and increase the distinctiveness between different genres, thereby reducing classification result bias. Specifically, we construct the feature extraction module which can fully utilize temporal information, thereby enhancing its focus on music features. It exhibits an improved understanding of the complexity of fused music. Additionally, we introduce a genre-sensitive adjustment layer that strengthens the learning of differences between different genres through within-class angle constraints. This leads to increased distinctiveness between genres and provides interpretability for the classification results. Experimental results demonstrate that our proposed MusicNeXt model outperforms baseline networks and other state-of-the-art methods in music genre classification tasks, without generating category bias in the classification results.

Deep learning for video game genre classification

In this paper, we propose a new multi-modal deep learning framework with a visual modality and a textual modality for video game genre classification. The proposed framework consists of three parts: two deeep networks for textual data and imaginary data, a feature concatenation algorithm, and then a softmax classifier. Video game covers and textual descriptions are usually the very first impression to its consumers and they often convey important information about the video games. Video game genre classification based on its cover and textual description would be utterly beneficial to many modern identification, collocation, and retrieval systems. At the same time, it is also an extremely challenging task due to the following reasons: First, there exists a wide variety of video game genres, many of which are not concretely defined. Second, video game covers vary in many different ways such as colors, styles, textual information, etc, even for games of the same genre. Third, cover designs and textual descriptions may vary due to many external factors such as country, culture, target reader populations, etc. With the growing competitiveness in the video game industry, the cover designers and typographers push the cover designs to its limit in the hope of attracting sales. The computer-based automatic video game genre classification systems become a particularly exciting research topic in recent years. The contribution of this paper is four-fold. First, we compiles a large dataset consisting of 50,000 video games from 21 genres made of cover images, description text, and title text and the genre information. Second, image-based and text-based, state-of-the-art models are evaluated thoroughly for the task of genre classification for video games. Third, we developed an efficient and scalable multi-modal framework based on both images and texts. Fourth, a thorough analysis of the experimental results is given and future works to improve the performance is suggested. The results show that the multi-modal framework outperforms the current state-of-the-art image-based or text-based models. Several challenges are outlined for this task. More efforts and resources are needed for this classification task in order to reach a satisfactory level.

Panel-Page-Aware Comic Genre Understanding.

Using a sequence of discrete still images to tell a story or introduce a process has become a tradition in the field of digital visual media. With the surge in these media and the requirements in downstream tasks, acquiring their main topics or genres in a very short time is urgently needed. As a representative form of the media, comic enjoys a huge boom as it has gone digital. However, different from natural images, comic images are divided by panels, and the images are not visually consistent from page to page. Therefore, existing works tailored for natural images perform poorly in analyzing comics. Considering the identification of comic genres is tied to the overall story plotting, a long-term understanding that makes full use of the semantic interactions between multi-level comic fragments needs to be fully exploited. In this paper, we propose [Formula: see text]Comic, a Panel-Page-aware Comic genre classification model, which takes page sequences of comics as the input and produces class-wise probabilities. [Formula: see text]Comic utilizes detected panel boxes to extract panel representations and deploys self-attention to construct panel-page understanding, assisted with interdependent classifiers to model label correlation. We develop the first comic dataset for the task of comic genre classification with multi-genre labels. Our approach is proved by experiments to outperform state-of-the-art methods on related tasks. We also validate the extensibility of our network to perform in the multi-modal scenario. Finally, we show the practicability of our approach by giving effective genre prediction results for whole comic books.

Multi-Modal Song Mood Detection with Deep Learning.

The production and consumption of music in the contemporary era results in big data generation and creates new needs for automated and more effective management of these data. Automated music mood detection constitutes an active task in the field of MIR (Music Information Retrieval). The first approach to correlating music and mood was made in 1990 by Gordon Burner who researched the way that musical emotion affects marketing. In 2016, Lidy and Schiner trained a CNN for the task of genre and mood classification based on audio. In 2018, Delbouys et al. developed a multi-modal Deep Learning system combining CNN and LSTM architectures and concluded that multi-modal approaches overcome single channel models. This work will examine and compare single channel and multi-modal approaches for the task of music mood detection applying Deep Learning architectures. Our first approach tries to utilize the audio signal and the lyrics of a musical track separately, while the second approach applies a uniform multi-modal analysis to classify the given data into mood classes. The available data we will use to train and evaluate our models comes from the MoodyLyrics dataset, which includes 2000 song titles with labels from four mood classes, {happy, angry, sad, relaxed}. The result of this work leads to a uniform prediction of the mood that represents a music track and has usage in many applications.

Distance Metric Learnt Kernel-Based Music Classification Using Timbral Descriptors

Automatic music genre classification based on distance metric learning (DML) is proposed in this paper. Three types of timbral descriptors, namely, mel-frequency cepstral coefficient (MFCC) features, modified group delay features (MODGDF) and low-level timbral feature sets are combined at the feature level. We experimented with k nearest neighbor (kNN) and support vector machine (SVM)-based classifiers for standard and DML kernels (DMLK) using GTZAN and Folk music dataset. Standard kernel-based kNN and SVM-based classifiers report classification accuracy (in%) of 79.03 and 90.16, respectively, on GTZAN dataset and 86.60 and 92.26, respectively, for Folk music dataset, with the best performing RBF kernel. A further improvement was observed when DML kernels were used in place of standard kernels in the kernel kNN and SVM-based classifiers with an accuracy of 84.46%, 92.74% (GTZAN), 90.00 and 96.23 (Folk music dataset) for DMLK-kNN and DMLK-SVM, respectively. The results demonstrate the potential of DML kernels in music genre classification task.

DBTMPE: Deep Bidirectional Transformers-Based Masked Predictive Encoder Approach for Music Genre Classification

Music is a type of time-series data. As the size of the data increases, it is a challenge to build robust music genre classification systems from massive amounts of music data. Robust systems require large amounts of labeled music data, which necessitates time- and labor-intensive data-labeling efforts and expert knowledge. This paper proposes a musical instrument digital interface (MIDI) preprocessing method, Pitch to Vector (Pitch2vec), and a deep bidirectional transformers-based masked predictive encoder (MPE) method for music genre classification. The MIDI files are considered as input. MIDI files are converted to the vector sequence by Pitch2vec before being input into the MPE. By unsupervised learning, the MPE based on deep bidirectional transformers is designed to extract bidirectional representations automatically, which are musicological insight. In contrast to other deep-learning models, such as recurrent neural network (RNN)-based models, the MPE method enables parallelization over time-steps, leading to faster training. To evaluate the performance of the proposed method, experiments were conducted on the Lakh MIDI music dataset. During MPE training, approximately 400,000 MIDI segments were utilized for the MPE, for which the recovery accuracy rate reached 97%. In the music genre classification task, the accuracy rate and other indicators of the proposed method were more than 94%. The experimental results indicate that the proposed method improves classification performance compared with state-of-the-art models.

Poster-Based Multiple Movie Genre Classification Using Inter-Channel Features

As the scale of the film industry grows, the demand for well-established movie databases is also growing. The genre of a movie supplies information on its overall content and has multiple values. Therefore, it should be well classified utilizing the characteristics of movies, without omissions in the database. In this study, we extract the optimal information and characteristics from movie posters to aid the classification of movies into genres and propose the use of a Gram layer in a convolutional neural network (CNN). The Gram layer first extracts style features by applying the Gram matrix to produce a feature map of a poster image. Using this as a style weight, the existing feature map is merged with style information to perform the genre classification task. The proposed Gram layer performed multi-genre classification tasks with higher efficiency than a residual neural network (ResNet), which is the current CNN model used for such tasks. We compared the activation map with the Squeeze-and-Excitation network, which gives weight to the image, and we confirmed that the introduction of the Gram layer actually focuses on the style of the movie poster. To classify the movie genres, we reconstructed the poster dataset into 12 multi-genres that emphasized the characteristics of each poster.

An Analysis of the Effect of Data Augmentation Methods: Experiments for a Musical Genre Classification Task

Supervised machine learning relies on the accessibility of large datasets of annotated data. This is essential since small datasets generally lead to overfitting when training high-dimensional machine-learning models. Since the manual annotation of such large datasets is a long, tedious and expensive process, another possibility is to artificially increase the size of the dataset. This is known as data augmentation. In this paper we provide an in-depth analysis of two data augmentation methods: sound transformations and sound segmentation. The first transforms a music track to a set of new music tracks by applying processes such as pitch-shifting, time-stretching or filtering. The second one splits a long sound signal into a set of shorter time segments. We study the effect of these two techniques (and the parameters of those) for a genre classification task using public datasets. The main contribution of this work is to detail by experimentation the benefit of these methods, used alone or together, during training and/or testing. We also demonstrate their use in improving the robustness of potentially unknown sound degradations. By analyzing these results, good practice recommendations are provided.

Deep attention based music genre classification

As an important component of music information retrieval, music genre classification attracts great attentions these years. Benefitting from the outstanding performance of deep neural networks in computer vision, some researchers apply CNN on music genre classification tasks with audio spectrograms as input instead, which has similarities with RGB images. These methods are based on a latent assumption that spectrums with different temporal steps have equal importance. However, it goes against the theory of processing bottleneck in psychology as well as our observation from audio spectrograms. By considering the differences of spectrums, we propose a new model incorporating with attention mechanism based on Bidirectional Recurrent Neural Network. Furthermore, two attention-based models (serial attention and parallelized attention) are implemented in this paper. Comparing with serial attention, parallelized attention is more flexible and gets better results in our experiments. Especially, the CNN-based parallelized attention models with taking STFT spectrograms as input outperform the previous work.

Fine-art painting classification via two-channel dual path networks

Fine-art painting expresses the state of mind and social culture of mankind. Automatic fine-art painting classification is an important task to assist the analysis of fine-art paintings. In this paper, we propose a novel two-channel dual path networks for the task of style, artist and genre classification on fine-art painting image. It includes the RGB and the brush stroke information channels. Besides the RGB information channel is used to represent the color information in fine-art painting images, the brush stroke information channel is used to extract brush stroke information from fine-art painting images. And the four-directional gray-level co-occurrence matrix is used in deep learning to detect the brush stroke information, which has never been considered in the task of fine-art painting classification. Experiments on two datasets demonstrate that the four-directional gray-level co-occurrence matrix is effective in feature representation of fine-art painting images. And the proposed model achieves best classification accuracy and good generalization performance when compared with other methods.

Using Generic Summarization to Improve Music Information Retrieval Tasks

In order to satisfy processing time constraints, many MIR tasks process only a segment of the whole music signal. This practice may lead to decreasing performance, since the most important information for the tasks may not be in those processed segments. In this paper, we leverage generic summarization algorithms, previously applied to text and speech summarization, to summarize items in music datasets. These algorithms build summaries, that are both concise and diverse, by selecting appropriate segments from the input signal which makes them good candidates to summarize music as well. We evaluate the summarization process on binary and multiclass music genre classification tasks, by comparing the performance obtained using summarized datasets against the performances obtained using continuous segments (which is the traditional method used for addressing the previously mentioned time constraints) and full songs of the same original dataset. We show that GRASSHOPPER, LexRank, LSA, MMR, and a Support Sets-based Centrality model improve classification performance when compared to selected 30-second baselines. We also show that summarized datasets lead to a classification performance whose difference is not statistically significant from using full songs. Furthermore, we make an argument stating the advantages of sharing summarized datasets for future MIR research.

Music Genre and Emotion Recognition Using Gaussian Processes

Gaussian Processes (GPs) are Bayesian nonparametric models that are becoming more and more popular for their superior capabilities to capture highly nonlinear data relationships in various tasks, such as dimensionality reduction, time series analysis, novelty detection, as well as classical regression and classification tasks. In this paper, we investigate the feasibility and applicability of GP models for music genre classification and music emotion estimation. These are two of the main tasks in the music information retrieval (MIR) field. So far, the support vector machine (SVM) has been the dominant model used in MIR systems. Like SVM, GP models are based on kernel functions and Gram matrices; but, in contrast, they produce truly probabilistic outputs with an explicit degree of prediction uncertainty. In addition, there exist algorithms for GP hyperparameter learning-something the SVM framework lacks. In this paper, we built two systems, one for music genre classification and another for music emotion estimation using both SVM and GP models, and compared their performances on two databases of similar size. In all cases, the music audio signal was processed in the same way, and the effects of different feature extraction methods and their various combinations were also investigated. The evaluation experiments clearly showed that in both music genre classification and music emotion estimation tasks the GP performed consistently better than the SVM. The GP achieved a 13.6% relative genre classification error reduction and up to an 11% absolute increase of the coefficient of determination in the emotion estimation task.

High level feature extraction for the self-taught learning algorithm

Availability of large amounts of raw unlabeled data has sparked the recent surge in semi-supervised learning research. In most works, however, it is assumed that labeled and unlabeled data come from the same distribution. This restriction is removed in the self-taught learning algorithm where unlabeled data can be different, but nevertheless have similar structure. First, a representation is learned from the unlabeled samples by decomposing their data matrix into two matrices called bases and activations matrix respectively. This procedure is justified by the assumption that each sample is a linear combination of the columns in the bases matrix which can be viewed as high level features representing the knowledge learned from the unlabeled data in an unsupervised way. Next, activations of the labeled data are obtained using the bases which are kept fixed. Finally, a classifier is built using these activations instead of the original labeled data. In this work, we investigated the performance of three popular methods for matrix decomposition: Principal Component Analysis (PCA), Non-negative Matrix Factorization (NMF) and Sparse Coding (SC) as unsupervised high level feature extractors for the self-taught learning algorithm. We implemented this algorithm for the music genre classification task using two different databases: one as unlabeled data pool and the other as data for supervised classifier training. Music pieces come from 10 and 6 genres for each database respectively, while only one genre is common for the both of them. Results from wide variety of experimental settings show that the self-taught learning method improves the classification rate when the amount of labeled data is small and, more interestingly, that consistent improvement can be achieved for a wide range of unlabeled data sizes. The best performance among the matrix decomposition approaches was shown by the Sparse Coding method.

‘I love Rock ‘n’ Roll’—Music genre preference modulates brain responses to music

The present study examined the effect of participants’ music genre preference on the neural processes underlying evaluative and cognitive judgements of music using the event-related potential technique. To this aim, two participant groups differing in their preference for Latin American and Heavy Metal music performed a liking judgement and a genre classification task on a variety of excerpts of either music genre. A late positive potential (LPP) was elicited in all conditions between 600 and 900ms after stimulus onset. During the genre classification task, an early negativity was elicited by the preferred compared to the non-preferred music at around 230–370ms whereas the non-preferred genre was characterized by a larger LPP. The findings suggest that evaluative and cognitive judgements of music are accompanied by affective responses and that the valence of music may spontaneously modulate early processes of music categorization even when no overt liking judgement is required.