Spectrogram Features Research Articles

Multimodal and Multitask Learning with Additive Angular Penalty Focus Loss for Speech Emotion Recognition

Speech emotion recognition has lots of applications such as human-computer interaction and health management. The current methods are challenged with the problems of fuzzy decision boundary and imbalance between difficult and easy samples in the training data. This paper first proposes an additive angle penalty focus loss function (APFL), which strictly refines the fuzzy decision boundary by introducing angle penalty factors to improve the compactness within the class and enlarge the distance between classes. It also assigns the larger loss to difficult samples to make the model pay more attention to them, as they are easily misclassified. Simultaneously, due to the lack of training samples, the framework of multimodal and multitask learning with APFL is further proposed, which extracts spectrogram features by deep neural network, text features by the pretrained language model, and audio features by the pretrained sound model. It uses the gender recognition as an auxiliary task. The experimental results verify the effectiveness of the proposed loss function and framework.

MM-EMOR: Multi-Modal Emotion Recognition of Social Media Using Concatenated Deep Learning Networks

Emotion recognition is crucial in artificial intelligence, particularly in the domain of human–computer interaction. The ability to accurately discern and interpret emotions plays a critical role in helping machines to effectively decipher users’ underlying intentions, allowing for a more streamlined interaction process that invariably translates into an elevated user experience. The recent increase in social media usage, as well as the availability of an immense amount of unstructured data, has resulted in a significant demand for the deployment of automated emotion recognition systems. Artificial intelligence (AI) techniques have emerged as a powerful solution to this pressing concern in this context. In particular, the incorporation of multimodal AI-driven approaches for emotion recognition has proven beneficial in capturing the intricate interplay of diverse human expression cues that manifest across multiple modalities. The current study aims to develop an effective multimodal emotion recognition system known as MM-EMOR in order to improve the efficacy of emotion recognition efforts focused on audio and text modalities. The use of Mel spectrogram features, Chromagram features, and the Mobilenet Convolutional Neural Network (CNN) for processing audio data are central to the operation of this system, while an attention-based Roberta model caters to the text modality. The methodology of this study is based on an exhaustive evaluation of this approach across three different datasets. Notably, the empirical findings show that MM-EMOR outperforms competing models across the same datasets. This performance boost is noticeable, with accuracy gains of an impressive 7% on one dataset and a substantial 8% on another. Most significantly, the observed increase in accuracy for the final dataset was an astounding 18%.

Sound recognition method for white feather broilers based on spectrogram features and the fusion classification model

This paper presents a sound recognition method for white feather broilers using spectrogram features and a fusion classification model, with the goal of achieving accurate classification of white feather broilers sound signals and providing a reliable basis for monitoring their health. In the training part, after five steps of sound signal acquisition, pre-processing, feature extraction, feature optimization, and model training, a fusion classification model with strong reliability is constructed for practical application scenarios. In the testing part, the method is applied to a real farming scenario of white feather broilers, and the stability of the multi-classification models and the reliability of the fusion classification model are verified. The fusion classification model comprises Random Forest, K-nearest neighbor, and RBF-based SVM. Results from multiple tests showed that the highest classification accuracies achieved by the three multi-classification models were 100%, 86.67%, and 93.33%, respectively. The average prediction accuracy of the fusion classification model on multiple audio signals was 98.57%, the results effectively demonstrate the feasibility and practicality of the proposed method.

Experimental Investigation of Acoustic Features to Optimize Intelligibility in Cochlear Implants.

Although cochlear implants work well for people with hearing impairment in quiet conditions, it is well-known that they are not as effective in noisy environments. Noise reduction algorithms based on machine learning allied with appropriate speech features can be used to address this problem. The purpose of this study is to investigate the importance of acoustic features in such algorithms. Acoustic features are extracted from speech and noise mixtures and used in conjunction with the ideal binary mask to train a deep neural network to estimate masks for speech synthesis to produce enhanced speech. The intelligibility of this speech is objectively measured using metrics such as Short-time Objective Intelligibility (STOI), Hit Rate minus False Alarm Rate (HIT-FA) and Normalized Covariance Measure (NCM) for both simulated normal-hearing and hearing-impaired scenarios. A wide range of existing features is experimentally evaluated, including features that have not been traditionally applied in this application. The results demonstrate that frequency domain features perform best. In particular, Gammatone features performed best for normal hearing over a range of signal-to-noise ratios and noise types (STOI = 0.7826). Mel spectrogram features exhibited the best overall performance for hearing impairment (NCM = 0.7314). There is a stronger correlation between STOI and NCM than HIT-FA and NCM, suggesting that the former is a better predictor of intelligibility for hearing-impaired listeners. The results of this study may be useful in the design of adaptive intelligibility enhancement systems for cochlear implants based on both the noise level and the nature of the noise (stationary or non-stationary).

Multi-Featured Sea Ice Classification with SAR Image Based on Convolutional Neural Network

Sea ice is a significant factor in influencing environmental change on Earth. Monitoring sea ice is of major importance, and one of the main objectives of this monitoring is sea ice classification. Currently, synthetic aperture radar (SAR) data are primarily used for sea ice classification, with a single polarization band or simple combinations of polarization bands being common choices. While much of the current research has focused on optimizing network structures to achieve high classification accuracy, which requires substantial training resources, we aim to extract more information from the SAR data for classification. Therefore we propose a multi-featured SAR sea ice classification method that combines polarization features calculated by polarization decomposition and spectrogram features calculated by joint time-frequency analysis (JTFA). We built a convolutional neural network (CNN) structure for learning the multi-features of sea ice, which combines spatial features and physical properties, including polarization and spectrogram features of sea ice. In this paper, we utilized ALOS PALSAR SLC data with HH, HV, VH, and VV, four types of polarization for the multi-featured sea ice classification method. We divided the sea ice into new ice (NI), first-year ice (FI), old ice (OI), deformed ice (DI), and open water (OW). Then, the accuracy calculation by confusion matrix and comparative analysis were carried out. Our experimental results demonstrate that the multi-feature method proposed in this paper can achieve high accuracy with a smaller data volume and computational effort. In the four scenes selected for validation, the overall accuracy could reach 95%, 91%, 96%, and 95%, respectively, which represents a significant improvement compared to the single-feature sea ice classification method.

Sound-Based Real-Time Scrap Falling State Monitoring in the Shearing Process of Wide Heavy Plate Mill

The scrap falling state in the shearing process of a Wide Heavy Plate Mill depends on manual monitoring and judgment. The steel plate can only move forward after the scrap falls, otherwise, the equipment will be damaged. An experienced operator can use the sound they hear to estimate the falling state. To automatically recognize sound events, a scheme of sound data acquisition processing and recognition for monitoring the scrap state of the shearing process is proposed in this paper. Firstly, a pickup is installed at the monitoring point of the crop shear to collect the sound. Afterward, the sound data is processed and finally fed into the model for recognition. Since many sound segments in the field are not related to key actions, an endpoint detection technique based on a short-term energy threshold and the variable window length strategy is applied in the data processing section to obtain the sound segments to be identified. The Log-Mel spectrogram feature of the sound segment is then extracted. After a data filling to ensure a uniform data shape, the feature map is fed into a lightweight deep convolution neural network called MobileNetV2 to quickly identify sound events. For the shearing process events, the model recognition accuracy can reach 95% of the F1 score in a realistic noise environment and we can identify the scrap falling 0.3s after the action starts. Finally, the model was applied to the UrbanSound8K dataset and achieved 95% recognition accuracy, validating the performance of the method.

An Automatic Classification System for Environmental Sound in Smart Cities.

With the continuous promotion of "smart cities" worldwide, the approach to be used in combining smart cities with modern advanced technologies (Internet of Things, cloud computing, artificial intelligence) has become a hot topic. However, due to the non-stationary nature of environmental sound and the interference of urban noise, it is challenging to fully extract features from the model with a single input and achieve ideal classification results, even with deep learning methods. To improve the recognition accuracy of ESC (environmental sound classification), we propose a dual-branch residual network (dual-resnet) based on feature fusion. Furthermore, in terms of data pre-processing, a loop-padding method is proposed to patch shorter data, enabling it to obtain more useful information. At the same time, in order to prevent the occurrence of overfitting, we use the time-frequency data enhancement method to expand the dataset. After uniform pre-processing of all the original audio, the dual-branch residual network automatically extracts the frequency domain features of the log-Mel spectrogram and log-spectrogram. Then, the two different audio features are fused to make the representation of the audio features more comprehensive. The experimental results show that compared with other models, the classification accuracy of the UrbanSound8k dataset has been improved to different degrees.

Crack identification of Yunnan walnuts based on acoustic vibration and feature fusion

AbstractCracks in walnuts during processing and storage can adversely affect their quality and cause economic losses. To achieve efficient identification of cracked walnuts, this study proposed a method of walnut crack identification based on acoustic vibration and feature fusion. First, the sound signals of intact and cracked walnuts were collected using an acoustic signal acquisition system, and 44 time‐domain features, 13 frequency‐domain features, and 768 Mel spectrogram features (the number of pixel frequencies corresponding to the gray‐scale values of R, G, and B channels) of the sound signals were extracted. Then, the classification models of support vector machines (SVM), least squares support vector machines (LSSVM), and extreme learning machines (ELM) were established based on single class features data and fusion of different feature groups data respectively. The results indicated that the LSSVM model with the fusion of the three feature sets was optimal, with an accuracy of 85% in the testing set. Next, three feature selection methods were employed to reduce the dimensionality of the best fused feature data. Subsequently, the LSSVM classification model was established based on the feature selection data. Finally, arithmetic optimization algorithm (AOA), particle swarm optimization (PSO), and gray wolf optimization (GWO) were introduced to optimize the parameters c and of the classification model. The results indicated that the best classification model was VISSA‐IRIV‐GWO‐LSSVM, with 95% accuracy in the testing set. This study provides theoretical support for the research and development of online detection equipment for walnut crack in Yunnan Yangbi.Practical applicationsCracks in walnut during harvesting, transportation and peeling may lead to economic losses and food safety problems. Aiming at the difficulty and low accuracy of crack identification in Yunnan walnut, this paper proposed a method for crack detection based on acoustic vibration identification combine with feature fusion. The features of time domain, frequency domain and Mel spectrogram were extracted from the effective sound signal, and the features were fused. The influence of three feature selection methods, three models and three optimization algorithms on walnut sound signal recognition was analyzed and compared. The results indicated that the best classification model was VISSA‐IRIV‐GWO‐LSSVM, and the method proposed in this study provides theoretical support for the research and development of online detection equipment for walnut crack in Yunnan Yangbi.

Analysis of Application and Creation Skills of Story-Based MV Micro Video and Big Multimedia Data in Music Communication

MV is an art type that uses TV pictures to supplement information and content that cannot be covered by music. With the rapid development of micro video technology and network technology, micro video has been rapidly popularized. Combining MV and micro video with mobile devices and platforms, it breaks the limitations of traditional movies, TV, and animation and becomes a new highlight of cultural communication in the new era. This paper starts with the selection of MV songs, story script, content design, MV shooting and editing skills, and introduces the specific creation methods of story MV in combination with local culture and proposes a music genre classification algorithm model DCNN-SSA based on spectral space domain feature concern. DCNN-SSA model effectively marks genre features of different music Mel spectrograms in the spatial domain and changes the network structure, thus improving the feature extraction effect while ensuring the validity of the model, thus improving the accuracy of music genre classification.

Deep Learning Approaches for Automatic Drum Transcription

Drum transcription is the task of transcribing audio or music into drum notation. Drum notation is helpful to help drummers as instruction in playing drums and could also be useful for students to learn about drum music theories. Unfortunately, transcribing music is not an easy task. A good transcription can usually be obtained only by an experienced musician. On the other side, musical notation is beneficial not only for professionals but also for amateurs. This study develops an Automatic Drum Transcription (ADT) application using the segment and classify method with Deep Learning as the classification method. The segment and classify method is divided into two steps. First, the segmentation step achieved a score of 76.14% in macro F1 after doing a grid search to tune the parameters. Second, the spectrogram feature is extracted on the detected onsets as the input for the classification models. The models are evaluated using the multi-objective optimization (MOO) of macro F1 score and time consumption for prediction. The result shows that the LSTM model outperformed the other models with MOO scores of 77.42%, 86.97%, and 82.87% on MDB Drums, IDMT-SMT Drums, and combined datasets, respectively. The model is then used in the ADT application. The application is built using the FastAPI framework, which delivers the transcription result as a drum tab.

Feeding intensity assessment of aquaculture fish using Mel Spectrogram and deep learning algorithms

Accurately and objectively analyzing fish feeding intensity is essential to guiding feeding and production techniques. Fish feeding intensity in recirculating aquaculture systems (RAS) can be used to indicate a fish's appetite. However, the low efficiency and lack of objectivity of manual observation are problems with current fish feeding intensity assessment processes. Applying acoustic techniques to aquaculture issues is an insufficiently explored area that requires new investigations, particularly into methods that explore temporal information in acoustic spectrograms. With Oplegnathus punctatus as the experimental species, we proposed a fish feeding intensity detection method based on the Mel Spectrogram and MobileNetV3-SBSC lightweight networks. First, the Oplegnathus punctatus feeding sound dataset, which has a total of 3 types—"strong," "medium," and "none," was built. Next, Mel Spectrogram feature maps were extracted using steps including preprocessing, Fast Fourier Transform (FFT), Mel filter bank (MFB), etc. Finally, the MobileNetV3-SBSC lightweight network was used to detect and recognize the obtained feature maps. Experimental results indicated that the proposed MobileNetV3-SBSC model, as compared to the MobileNetV3-S model, improved test accuracy by 4.6% and decreased test loss by 67.4% with only a 0.84% increase in the number of parameters and a 3.08% increase in the model size. Additionally, we have verified that the accuracy of the test set was 59.6%, 53.3%, 83.3%, 85.3%, and 85.9% for groups of 5, 15, 40, 70, and 100 fish, respectively. This study demonstrated that the proposed method is not applicable to a small number of fish, which means that when the numbers of fish are small, changes in the feeding of individual fish would have a significant effect on the whole feeding feature map, leading to negligible changes in feeding features. However, in view of the commonly high aquaculture density, the proposed method can be used to automatically and objectively examine fish feeding, which could provide a theoretical basis and methodological support for further feeding decisions.

A SVM-based method for identifying fracture modes of rock using WVD spectrogram features of AE signals

In order to achieve the highly efficient and accurate identification of fracture modes including tension or shear fractures during rock failure, an intelligent identification method based on Wigner-Ville distribution (WVD) spectrogram features of acoustic emission (AE) signals was proposed. This method was mainly constructed by the following steps: Firstly, AE hits corre-sponding to tension and shear fractures were obtained through conducting the Brazilian disc test (tension fracture) and direct shear test (shear fracture) of limestone. Secondly, the WVD spectro-grams of these tensile-type and shear-type AE hits were respectively extracted and then trans-formed into the image features of relatively low-dimension as the sample set based on the gray-level cooccurrence matrix (GLCM) and histogram of oriented gradient (HOG). Finally, on the basis of the processed and classified sample set of the WVD spectrogram features, an identifica-tion model of rock fracture modes was established by a support vector machine (SVM) learning algorithm. To verify this method, the fracture modes of limestone subjected to biaxial compres-sion were identified by the method. The results showed that the method not only can greatly re-veal the fracture modes change from tension-dominated to shear-dominated fractures, but also has advantages over the RA-AF value method, such as applicability, accuracy and practicality.

Hybrid data augmentation and deep attention-based dilated convolutional-recurrent neural networks for speech emotion recognition

Recently, speech emotion recognition (SER) has become an active research area in speech processing, particularly with the advent of deep learning (DL). Numerous DL-based methods have been proposed for SER. However, most of the existing DL-based models are complex and require a large amounts of data to achieve a good performance. In this study, a new framework of deep attention-based dilated convolutional-recurrent neural networks coupled with a hybrid data augmentation method was proposed for addressing SER tasks. The hybrid data augmentation method constitutes an upsampling technique for generating more speech data samples based on the traditional and generative adversarial network approaches. By leveraging both convolutional and recurrent neural networks in a dilated form along with an attention mechanism, the proposed DL framework can extract high-level representations from three-dimensional log Mel spectrogram features. Dilated convolutional neural networks acquire larger receptive fields, whereas dilated recurrent neural networks overcome complex dependencies as well as the vanishing and exploding gradient issues. Furthermore, the loss functions are reconfigured by combining the SoftMax loss and the center-based losses to classify various emotional states. The proposed framework was implemented using the Python programming language and the TensorFlow deep learning library. To validate the proposed framework, the EmoDB and ERC benchmark datasets, which are imbalanced and/or small datasets, were employed. The experimental results indicate that the proposed framework outperforms other related state-of-the-art methods, yielding the highest unweighted recall rates of 88.03 ± 1.39 (%) and 66.56 ± 0.67 (%) for the EmoDB and ERC datasets, respectively.

Speech Recognition of Accented Mandarin Based on Improved Conformer.

The convolution module in Conformer is capable of providing translationally invariant convolution in time and space. This is often used in Mandarin recognition tasks to address the diversity of speech signals by treating the time-frequency maps of speech signals as images. However, convolutional networks are more effective in local feature modeling, while dialect recognition tasks require the extraction of a long sequence of contextual information features; therefore, the SE-Conformer-TCN is proposed in this paper. By embedding the squeeze-excitation block into the Conformer, the interdependence between the features of channels can be explicitly modeled to enhance the model's ability to select interrelated channels, thus increasing the weight of effective speech spectrogram features and decreasing the weight of ineffective or less effective feature maps. The multi-head self-attention and temporal convolutional network is built in parallel, in which the dilated causal convolutions module can cover the input time series by increasing the expansion factor and convolutional kernel to capture the location information implied between the sequences and enhance the model's access to location information. Experiments on four public datasets demonstrate that the proposed model has a higher performance for the recognition of Mandarin with an accent, and the sentence error rate is reduced by 2.1% compared to the Conformer, with only 4.9% character error rate.

A Genetic Algorithm Approach to Automate Architecture Design for Acoustic Scene Classification

Convolutional neural networks (CNNs) have been widely used with remarkable success in the acoustic scene classification (ASC) task. However, the performance of these CNNs highly relies on their architectures, requiring a lot of effort and expertise to design CNNs suitable for the investigated problem. In this work, we propose an efficient genetic algorithm (GA) that aims to find optimized CNN architectures for the ASC task. The proposed algorithm uses frequency-dimension splitting of the input spectrograms in order to explore the architecture search space in sub-CNN models in addition to classical single-path CNNs. Specifically, this algorithm aims to find the best number of sub-CNNs in addition to their architectures to better capture the distinct features of the input spectrograms. The proposed GA is specifically designed for sound classification to suit the ASC task than many other GAs that optimize conventional single-path CNN architectures. Experimental results on three benchmark datasets demonstrate the effectiveness of the proposed method. Specifically, the proposed algorithm has achieved around 17.8%, 16%, and 17.2%, relative improvement in accuracy with respect to the baseline systems on the development datasets of DCASE2018-Task1A, DCASE2019-Task1A, and DCASE2020-Task1A, respectively.

Study on fault diagnosis of ultra-low-speed rolling bearings based on full vector sound spectrogram

By exploring the mapping relationship between the multi-directional data and fault characteristics of bearings, a time-frequency analysis method for considering the multi-directional acoustic emission (AE) data of bearings is proposed. Firstly, using the full vector spectrum (FVS) theory, the full vector sound spectrogram of the dual-channel AE signal of a bearing is extracted to enhance the representation of the fault state using time-frequency characteristics. Then, the obtained full vector sound spectrogram is transformed into a specific size as the input feature map and a convolutional neural network (CNN) classifier model is established. Next, the Softmax classifier is used to classify the bearing faults in order to realise the intelligent fault diagnosis of an ultra-low-speed rolling bearing. The comparison of the different models shows that the average recognition accuracy using the full vector sound spectrogram CNN model can reach 95.61%, which is better than the other three methods. The feature extraction using the full vector sound spectrogram feature analysis method has a high degree of recognition for bearing faults in an ultra-low-speed state and can provide high accuracy and stability under noisy conditions.

Deep Learning-Based Speech Enhancement of an Extrinsic Fabry-Perot Interferometric Fiber Acoustic Sensor System.

To achieve high-quality voice communication technology without noise interference in flammable, explosive and strong electromagnetic environments, the speech enhancement technology of a fiber-optic external Fabry-Perot interferometric (EFPI) acoustic sensor based on deep learning is studied in this paper. The combination of a complex-valued convolutional neural network and a long short-term memory (CV-CNN-LSTM) model is proposed for speech enhancement in the EFPI acoustic sensing system. Moreover, the 3 × 3 coupler algorithm is used to demodulate voice signals. Then, the short-time Fourier transform (STFT) spectrogram features of voice signals are divided into a training set and a test set. The training set is input into the established CV-CNN-LSTM model for model training, and the test set is input into the trained model for testing. The experimental findings reveal that the proposed CV-CNN-LSTM model demonstrates exceptional speech enhancement performance, boasting an average Perceptual Evaluation of Speech Quality (PESQ) score of 3.148. In comparison to the CV-CNN and CV-LSTM models, this innovative model achieves a remarkable PESQ score improvement of 9.7% and 11.4%, respectively. Furthermore, the average Short-Time Objective Intelligibility (STOI) score witnesses significant enhancements of 4.04 and 2.83 when contrasted with the CV-CNN and CV-LSTM models, respectively.

Automatic diagnosis of multi-task in essential tremor: Dynamic handwriting analysis using multi-modal fusion neural network

Essential tremor (ET) is one of the most common movement disorders, and patients with ET have more than a fourfold increased risk of developing Parkinson’s disease (PD). Currently, the handwriting assessment remains the gold standard for ET diagnosis and quantification of tremor severity. However, the traditional evaluation of tremors is subjective and partial, primarily limited by the experience of the neurologist and the progression of the disease course. With advanced sensing and computer technology development, digital pad-based electronic handwriting analysis can provide clinical information on pen tip kinematics. This study proposes a novel spatial–temporal–spectral fusion neural network (STSNet) for multi-task fine-grained assessment of tremor severity. The model can efficiently fuse complementary information from static handwriting images and dynamic multi-sensory fusion signals. Specifically, we design a static spatial branch for learning the global spatial features of the electronic handwriting image and a dynamic temporal branch to learn the spectrogram features in the time–frequency domain. The spectrograms are obtained by the transformation algorithm of the kinematic signals of the nib sensors to represent the spectral energy associated with tremor. In addition, we also validate the utility of the attention mechanism, transfer learning strategy, and the introduction of demographic features-based prior knowledge by ablation studies. STSNet obtains an average accuracy of 97.33–97.39%​ (five categories) on our collected database containing 147 ET patients. In addition, we test the robustness and effectiveness of the proposed method in three publicly available databases of PD diagnostics. These results show that STSNet outperforms other advanced works, which can automatically provide objective and accurate scores for ET symptom assessment or PD diagnosing.

A hybrid meta-heuristic ensemble based classification technique speech emotion recognition

This research intends to propose a new unique and automated speech emotion reorganization model by going through. Initially, the obtained raw speech data is filtered using Butterworth filter and then the signals are segmented into frames. In addition, using the Convolutional Neural Network (CNN), the Spectrogram feature is recovered from the frames. After that, the extracted acoustic and Spectrogram feature is amalgamated to make a hybrid feature vector. Furthermore, the Enhanced-ReliefF (E-ReliefF) is utilized to choose the most important features from the hybrid feature vector. The speech emotion recognition phase is modelled with an ensemble-of-classifiers. The proposed EC consists of a Recurrent Neural Network (RNN), DBN, and an Artificial Neural Network (ANN). The ANN and DBN are trained with the hybrid feature vector. The results of DBN and ANN are fed into an optimized RNN, which will provide the final outcome corresponding to the emotions expressed in the input speech. Furthermore, a RNN weight is adjusted using a hybrid optimization technique to improve speech emotion categorization precision. Arithmetic Exploration updated Wildbeast Model (AEUWB) is a newly presented hybrid optimization model that combines two classic optimization models, namely Arithmetic Optimization Algorithm (AOA) and Wildebeest herd optimization (WHO). Moreover, a comparative analysis validates the projected model's (AEUWB+EC) effectiveness. Accordingly, the accuracy of the presented method is 12.3%, 9.2%, 7.2%, 15.45, 17.5% and 13.4% superior than the conventional models like LA+EC, SSO+EC, SSO+EC, WHO+EC, AOA+EC, respectively, at the 90th LP.

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks.

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.