Gramian Angular Field Research Articles

Identification of apple watercore based on ConvNeXt and Vis/NIR spectra

This paper proposes a method for discriminating between normal apples and watercore apples using visible/near-infrared spectroscopy technology, which combines Gramian Angular Fields (GAF) encoding technology and the ConvNeXt deep learning network. The existing feature extraction methods for visible/near-infrared spectroscopy data do not perform deeper information mining on the extracted features, which results in the quality of the established model being entirely determined by the extracted features. Additionally, the process of building a visible/near-infrared spectroscopy data classification model is complex and time-consuming, and the accuracy of the established model is not high. To address these issues, the experimental visible/near-infrared spectroscopy data of apples was first transformed into two-dimensional images using Gramian Angular Summation Fields (GASF) and Gramian Angular Difference Fields (GADF) with sizes of 64, 128, 256, and 512. These images were then input into the ConvNeXt network, and the performance of different encoding methods and sizes was compared. The results showed that, under the conditions provided in this paper, the GADF encoding method with a size of 256 achieved the highest classification accuracy of 98.48%. Next, ResNet, EfficientNet, and RegNet deep learning networks were selected to classify the encoded images under the same conditions. The results above indicate that the apple variety discrimination method based on GAF encoding technology and ConvNeXt network combined with visible/near-infrared spectroscopy technology can achieve deep information mining of visible/near-infrared spectroscopy data and provide a relatively simple method for establishing qualitative classification models of visible/near-infrared spectroscopy. This method has a relatively excellent discrimination effect between normal apples and watercore apples.

Research on Ploughing to Shearing Transition in Micro Milling of Titanium Alloy Using Gramian Angular Field and Machine Learning Methods

Research on Ploughing to Shearing Transition in Micro Milling of Titanium Alloy Using Gramian Angular Field and Machine Learning Methods

RS-Net: Hyperspectral Image Land Cover Classification Based on Spectral Imager Combined with Random Forest Algorithm

Recursive neural networks and transformers have recently become dominant in hyperspectral (HS) image classification due to their ability to capture long-range dependencies in spectral sequences. Despite the success of these sequential architectures, mainstream deep learning methods primarily handle two-dimensional structured data. However, challenges such as the curse of dimensionality, spectral variability, and confounding factors in hyperspectral remote sensing images limit their effectiveness, especially in remote sensing applications. To address this issue, this paper proposes a novel land cover classification algorithm that integrates random forests with a spectral transformer network structure (RS-Net). Firstly, this paper presents a combination of the Gramian Angular Field (GASF) and Gramian Angular Difference Field (GADF) algorithms, which effectively maps the multidimensional time series constructed for each pixel onto two-dimensional image features, enabling precise extraction and recognition in the backend network algorithms and improving the classification accuracy of land cover types. Secondly, to capture the relationships between features at different scales, this paper proposes a SpectralFormer network architecture using the Context and Structure Encoding (CASE) module to effectively learn dependencies between channels. This architecture enhances important features and suppresses unimportant ones, thereby addressing the semantic gap and improving the recognition capability of land cover features. Finally, the final prediction results are determined by a voting mechanism from the Random Forest algorithm, which synthesizes predictions from multiple decision trees to enhance classification stability and accuracy. To better compare the performance of RS-Net, this paper conducted extensive experiments on three benchmark HS datasets obtained from satellite and airborne imagers, comparing various classic neural network models. Surprisingly, the RS-Net algorithm achieves high performance and efficiency, offering a new and effective tool for land cover classification.

Acoustic emission-based leakage detection for gas safety valves: Leveraging a multi-domain encoding learning algorithm

Safety valves in gas distribution systems are crucial for preventing excessive pressure buildups. Leakage in these valves compromises system stability, causing equipment damage and environmental pollution. Acoustic emission signal analysis from the valve body is essential for detecting leakage, but these signals are often weak and easily disrupted by noise, complicating detection. This paper proposes a multi-domain encoding learning algorithm to address these challenges. Unlike conventional single-domain methods, it combines two encoding images, the Gramian angular difference field and the Markov transition field, to enhance the comprehensive expression of leakage features. A lightweight convolutional neural network reduces computing resource dependency, and a convolutional block attention mechanism improves feature identification. Experimental results demonstrate that our method detects gas leakage with an accuracy of at least 96.77%. It maintains superior performance even under intense noise interference, offering a promising solution for detecting weak gas leakages in safety valves amidst high background noise.

On image transformation for partial discharge source identification in vehicle cable terminals of high‐speed trains

AbstractPartial discharge (PD) detection of cable terminals is crucial for the safe operation of the traction power system in trains. However, similar PD signals in complex train‐operating environments cause difficulty to recognise the insulation defects. Therefore, a PD signal image transformation recognition method is proposed for PD detection of cable terminal defects to identify defects in cable terminals with similar PD characteristics accurately. In the proposed method, the raw PD signals are firstly transformed to images via the Gramian angular field (GAF) representation. This can reveal the discriminative characteristics embedded in the original PD signals and subsequently facilitate differentiating the PD sources, which exhibit similar characteristic in the time domain. The obtained GAF representation of PD signals (named as PD GAF images) is extracted from local and global features to train an efficient MobileVIT model, which is then utilised to identify similar types of PD sources in cable terminals. The results show that the proposed method achieves 97.5% recognition accuracy in the field experiment, which is superior to other methods.

SPRout-DBN: a cross domain bearing fault diagnosis method based on spatial pyramid pooling residual network-DBN

Abstract Effectively leveraging the spatial features of time series signals to improve the accuracy of bearing fault classification in neural networks presents a significant challenge. To address this issue of different operating conditions, a novel model termed spatial pyramid pooling residual network-deep belief network (SPRout-DBN) is proposed. First and foremost, the Gramian angular difference fields (GADF) are utilized to encode original vibration signals of bearings. Secondly, two-dimensional images transformed by GADF from original signals are input to a novel designed residual network with spatial pyramid pooling to extract fixed-size temporal fusion feature vectors. Finally, a deep belief network is employed for classification and cross-domain learning, enabling the identification of fault samples under varying operating conditions. The proposed method is validated by two sets of datasets from Case Western Reserve University and Jiangnan University, achieving accuracies of 99.81% and 99.0% under identical operating conditions, and 99.41% and 98.43% under different operating conditions with 40 samples. Comparative analysis indicates that the proposed SPRout-DBN remains more robust and effective compared with other methods such as K-nearest neighbors, support vector machines, LeNet-5, ResNet-18, domain adaptation networks, and domain-adversarial neural networks in diverse operating environments.

Radio Frequency Signal-Based Drone Classification with Frequency Domain Gramian Angular Field and Convolutional Neural Network

Over the past few years, drones have been utilized in a wide range of applications. However, the illegal operation of drones may pose a series of security risks to sensitive areas such as airports and military bases. Hence, it is vital to develop an effective method of identifying drones to address the above issues. Existing drone classification methods based on radio frequency (RF) signals have low accuracy or a high computational cost. In this paper, we propose a novel RF signal image representation scheme that incorporates a convolutional neural network (CNN), named the frequency domain Gramian Angular Field with a CNN (FDGAF-CNN), to perform drone classification. Specifically, we first compute the time–frequency spectrum of raw RF signals based on short-time Fourier transform (STFT). Then, the 1D frequency spectrum series is encoded as 2D images using a modified GAF transform. Moreover, to further improve the recognition performance, the images obtained from different channels are fused to serve as the input of a CNN classifier. Finally, numerous experiments were conducted on the two available open-source DroneRF and DroneRFa datasets. The experimental results show that the proposed FDGAF-CNN can achieve a relatively high classification accuracy of 98.72% and 98.67% on the above two datasets, respectively, confirming the effectiveness and generalization ability of the proposed method.

Combined ResNet Attention Multi-Head Net (CRAMNet): A Novel Approach to Fault Diagnosis of Rolling Bearings Using Acoustic Radiation Signals and Advanced Deep Learning Techniques

The fault diagnosis of rolling bearing acoustic radiation signals holds significant importance in industrial equipment maintenance. It effectively prevents equipment failures and downtime, ensuring the smooth operation of the production process. Compared with traditional vibration signals, acoustic radiation signals have the advantage of non-contact measurement. They can diagnose faults in special conditions where sensors cannot be installed and provide more comprehensive equipment status information. Therefore, to extract the fault characteristic information of rolling bearings from complex acoustic signals, this paper proposes an advanced deep learning model combining Gramian Angular Field (GAF), ResNet1D, ResNet2D, and multi-head attention mechanism, named CRAMNet (Combined ResNet Attention Multi-Head Net), to diagnose the faults of rolling bearing acoustic radiation signals. Firstly, this method includes converting one-dimensional signals into GAF images and performing data standardization and segmentation. Then, the method utilizes ResNet1D to extract features from one-dimensional signals and ResNet2D to extract features from GAF images. Further, it combines the multi-head attention mechanism to enhance feature representation and capture dependencies between different channels. Finally, this paper compares the proposed method with several traditional models (including CNN, LSTM, DenseNet, and CNN-Transformers). Experimental results show that the proposed method performs outstandingly in terms of accuracy and robustness. The combination of residual networks and multi-head attention mechanism in the model significantly enhances its ability to accurately diagnose rolling bearing faults, proving the superiority of the algorithm.

Predicting Adherence to Computer-Based Cognitive Training Programs Among Older Adults: Study of Domain Adaptation and Deep Learning.

Cognitive impairment and dementia pose a significant challenge to the aging population, impacting the well-being, quality of life, and autonomy of affected individuals. As the population ages, this will place enormous strain on health care and economic systems. While computerized cognitive training programs have demonstrated some promise in addressing cognitive decline, adherence to these interventions can be challenging. The objective of this study is to improve the accuracy of predicting adherence lapses to ultimately develop tailored adherence support systems to promote engagement with cognitive training among older adults. Data from 2 previously conducted cognitive training intervention studies were used to forecast adherence levels among older participants. Deep convolutional neural networks were used to leverage their feature learning capabilities and predict adherence patterns based on past behavior. Domain adaptation (DA) was used to address the challenge of limited training data for each participant, by using data from other participants with similar playing patterns. Time series data were converted into image format using Gramian angular fields, to facilitate clustering of participants during DA. To the best of our knowledge, this is the first effort to use DA techniques to predict older adults' daily adherence to cognitive training programs. Our results demonstrated the promise and potential of deep neural networks and DA for predicting adherence lapses. In all 3 studies, using 2 independent datasets, DA consistently produced the best accuracy values. Our findings highlight that deep learning and DA techniques can aid in the development of adherence support systems for computerized cognitive training, as well as for other interventions aimed at improving health, cognition, and well-being. These techniques can improve engagement and maximize the benefits of such interventions, ultimately enhancing the quality of life of individuals at risk for cognitive impairments. This research informs the development of more effective interventions, benefiting individuals and society by improving conditions associated with aging.

Optimizing battery deployment: Aging trajectory prediction enabling homogenous performance grouping

As battery deployments in electric vehicles and energy storage systems grow, ensuring homogeneous performance across units is crucial. We propose a multi-derivative imaging fusion (MDIF) model, employing advanced imaging and machine learning to predict battery aging trajectories from minimal initial data, thus facilitating effective performance grouping before deployment. Utilizing a derivative strategy and Gramian Angular Difference Field for dimensional enhancement, the MDIF model uncovers subtle predictive features from discharge curve data after only ten cycles. The architecture includes a parallel convolutional neural network with lateral connections to enhance feature integration and extraction. Tested on a self-developed dataset, the model achieves an average root-mean-square error of 0.047 Ah and an average mean absolute percentage error of 1.60%, demonstrating high precision and reliability. Its robustness is further validated through transfer learning on two publicly available datasets, adapting with minimal retraining. This approach significantly reduces the testing cycles required, lowering both time and costs associated with battery testing. By enabling precise battery behavior predictions with limited data, the MDIF model optimizes battery utilization and deployment strategies, enhancing system efficiency and sustainability.

Research on Fault Diagnosis of Rolling Bearing Based on Gramian Angular Field and Lightweight Model.

Due to the limitations of deep learning models in processing one-dimensional signal feature extraction, and high model complexity leading to low training accuracy and large consumption of computing resources, this paper innovatively proposes a rolling bearing fault diagnosis method based on Gramian Angular Field (GAF) and enhanced lightweight residual network. Firstly, the one-dimensional signal is transformed into a two-dimensional GAF image, fully preserving the signal's temporal dependency. Secondly, to address the parameter redundancy and high computational complexity of the ResNet-18 model, its residual blocks are improved. The second convolutional layer in the downsampling residual blocks is removed, traditional convolutional layers are replaced with depthwise separable convolutions, and the lightweight Efficient Channel Attention (ECA) module is embedded after each residual block. This further enhances the model's ability to capture key features while maintaining low computational cost, resulting in a lightweight model referred to as E-ResNet13. Finally, the generated GAF feature maps are fed into the E-ResNet13 model for training, and through a global average pooling layer, they are mapped to a fully connected layer for classifying the faults of rolling bearings. Verifying the superiority of the proposed GAF-E-ResNet13 model, experimental results show that the GAF image encoding method achieves higher fault recognition accuracy compared to other encoding methods. Compared with other intelligent diagnosis methods, the E-ResNet13 model demonstrates strong diagnostic performance and generalization capability under both a single condition and complex varying conditions, fully proving the innovation and practicality of this method.

Research on prediction of yellow flesh peach firmness using a novel acoustic real-time detection device and Vis/NIR technology

Firmness is a critical indicator for predicting fruit ripeness, optimal harvest date, and shelf life. In this study, a novel fruit acoustic real-time detection prototype device and a conventional visible near-infrared (Vis/NIR) spectroscopy real-time detection device were used to collect acoustic and spectral signals from yellow flesh peaches to jointly predict their firmness. The acoustic and optical signals were generated into one- and two-dimensional feature data by complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), continuous wavelet transform (CWT) and Gramian angular field (GAF) data processing methods. Based on these data, a variety of yellow flesh peach firmness prediction models were constructed in this study, including partial least square (PLS), support vector regression (SVR), Swin Transformer (SwinT), and SwinT-PLS/SVR. The experimental results showed that the SwinT-PLS model based on the fusion of competitive adaptive re-weighted sampling (CARS)-acoustic image features and CARS-Vis/NIR spectral features showed the best prediction performance (R2P = 0.951, the RMSEP = 0.443 N/mm, RPDP = 4.339), and the prediction performance is significantly higher than that of the prediction model based on single acoustic and Vis/NIR spectral data. The method proposed can fast, non-destructively, accurately predict fruit firmness and has excellent prospects for commercial real-time fruit sorting applications.

Integration of multiscale fusion of residual neural network with 2-D gramian angular fields for lower limb movement recognition based on multi-channel sEMG signals

The human lower limb movements recognition (LLMR) plays a pivotal role in active lower limb exoskeleton robots. Employing surface electromyography (sEMG) signals for LLMR allows for the convenient, rapid and stable capture of signal variations, facilitating efficient identification of lower limb motion patterns. However, current sEMG-based LLMR methods face challenges such as incomplete feature extraction, limited contextual information and restricted feature extraction scales during feature extraction. This paper proposed a LLMR method based on Gramian Angular Fields (GAF) and multiscale fusion of Residual Neural Network (MS-ResNet). The denoised sEMG time series was transformed into Gramian Angular Difference Field (GADF) matrix based on GAF. The MS-ResNet model, incorporating ResNet and multiscale feature fusion concepts, was proposed to comprehensively capture global and local information through different-scale feature extraction and fusion, so as to improve recognition performance. sEMG signals from 11 muscles of the preferred leg of 15 healthy subjects were recorded during six common lower limb movements. Experimental analysis investigated the impact of the convolutional kernel size (k × k) in Stream 2 of MS-ResNet and the number of muscles involved on recognition performance. The study revealed that selecting k as 13, coupled with 11 muscles, yielded optimal model performance with the average cross-individual recognition accuracy reaching 97.62 %, demonstrating the model’s efficiency in LLMR. This method could provide a viable solution for developing more efficient and reliable LLMR systems, applicable to lower limb exoskeleton robots and intelligent prosthetics.

A gear fault diagnosis method based on reactive power and semi-supervised learning

Abstract In gearbox gear fault diagnosis based on motor current signals, the gear fault characteristic frequency component is often overshadowed by the fundamental frequency component of the current. In addition, the complex working conditions during actual production and use make it difficult to collect gear operation monitoring data containing labeled feature information. To address the above problems, a semi-supervised learning method based on reactive power signals is proposed for gear fault diagnosis of gearboxes. First, the method utilizes the Hilbert transform to process the current signal of the drive motor in the mechanical system, from which the reactive power is constructed. Then, the reactive power signal is analyzed by spectral analysis as a basis for gear fault diagnosis. Subsequently, the GAF-CNN-MTDL(Gramian angular field—convolutional neural network-mean teacher deep learning) fault diagnosis model is proposed to convert the reactive power signal into a two-dimensional image by using the GAF, and the semi-supervised training method of the average teacher is applied to input the fault dataset into the gear fault diagnosis model which is based on the CNN as the main backbone after the fault dataset has been divided into the labeled and the unlabeled dataset in accordance with a certain ratio. Finally, the gear fault dataset is used for method validation. The experimental outcomes demonstrate the method’s proficiency in effectively emphasizing the fault feature information pertaining to the gear part, and the introduced GAF-CNN-MTDL fault diagnosis model enables the utilization of a minimal number of labeled samples to achieve highly accurate gear fault diagnosis.

Predicting odor from vibrational spectra: a data-driven approach

This study investigates olfaction, a complex and not well-understood sensory modality. The chemical mechanism behind smell can be described by so far proposed two theories: vibrational and docking theories. The vibrational theory has been gaining acceptance lately but needs more extensive validation. To fill this gap for the first time, we, with the help of data-driven classification, clustering, and Explainable AI techniques, systematically analyze a large dataset of vibrational spectra (VS) of 3018 molecules obtained from the atomistic simulation. The study utlizes image representations of VS using Gramian Angular Fields and Markov Transition Fields, allowing computer vision techniques to be applied for better feature extraction and improved odor classification. Furthermore, we fuse the PCA-reduced fingerprint features with image features, which show additional improvement in classification results. We use two clustering methods, agglomerative hierarchical (AHC) and k-means, on dimensionality reduced (UMAP, MDS, t-SNE, and PCA) VS and image features, which shed further insight into the connections between molecular structure, VS, and odor. Additionally, we contrast our method with an earlier work that employed traditional machine learning on fingerprint features for the same dataset, and demonstrate that even with a representative subset of 3018 molecules, our deep learning model outperforms previous results. This comprehensive and systematic analysis highlights the potential of deep learning in furthering the field of olfactory research while confirming the vibrational theory of olfaction.

Comparative Analysis of Machine Learning and Deep Learning Algorithms for Assessing Agricultural Product Quality Using NIRS.

The success of near-infrared spectroscopy (NIRS) analysis hinges on the precision and robustness of the calibration model. Shallow learning (SL) algorithms like partial least squares discriminant analysis (PLS-DA) often fall short in capturing the interrelationships between adjacent spectral variables, and the analysis results are easily affected by spectral noise, which dramatically limits the breadth and depth of applications of NIRS. Deep learning (DL) methods, with their capacity to discern intricate features from limited samples, have been progressively integrated into NIRS. In this paper, two discriminant analysis problems, including wheat kernels and Yali pears as examples, and several representative calibration models were used to research the robustness and effectiveness of the model. Additionally, this article proposed a near-infrared calibration model, which was based on the Gramian angular difference field method and coordinate attention convolutional neural networks (G-CACNNs). The research results show that, compared with SL, spectral preprocessing has a smaller impact on the analysis accuracy of consensus learning (CL) and DL, and the latter has the highest analysis accuracy in the modeling results using the original spectrum. The accuracy of G-CACNNs in two discrimination tasks was 98.48% and 99.39%. Finally, this research compared the performance of various models under noise to evaluate the robustness and noise resistance of the proposed method.

Classification and identification of extreme wind events by CNNs based on Shapelets and improved GASF-GADF

In this manuscript, we propose an automatic classification and recognition method for extreme wind events based on Convolutional Neural Networks (CNNs) and combining the Shapelet Transform (ST) algorithm with the improved Gramian Angular Summation Field - Gramian Angular Difference Field (GASF-GADF) 2D images construction format. First, a CNN model suitable for wind speed time series 2D images classification and recognition among five mainstream CNNs (ResNet-50, ShuffleNet0.5 × , DenseNet-121, EfficientNet-B2, and EfficientNetV2-S) is preferred based on the basic Gramian Angular Field (GAF) method; then, the improved GASF-GADF images construction format is proposed, and the optimal CNN is used to compare the classification and recognition results based on other three image conversion methods: Markov Transition Field (MTF), GASF, GADF. Last, it is proposed to utilize the ST algorithm to extract the feature subsequence Shapelets of wind speed time series to further improve the classification and recognition effect on extreme wind events. The effectiveness and applicability of the proposed method were verified through three extreme wind event datasets in this paper.The results show that the combination of Shapelets and the improved GASF-GADF images transformation method proposed in this paper can effectively enhance the classification and recognition of extreme wind events by CNNs. Among them, EfficientNetV2-S combined with the method proposed in this paper achieves 99.50%, 99.50% and 97.50% recognition Accuracy for thunderstorm, gust front and typhoon, respectively. Meanwhile, it still has good robustness for extreme wind events disturbed by noise.

Planetary gearbox fault classification based on tooth root strain and GAF pseudo images

Traditional signal processing methods based on acceleration signals can determine whether a fault has occurred in a planetary gearbox. However, acceleration signals are severely affected by interference, causing difficulties in fault identification. This study proposes a gear fault classification method based on root strain and pseudo images. Firstly, fiber optic sensors are employed to directly acquire strain data from the ring gear root. Next, the strain signals are preprocessed using resampling and a time-domain synchronous averaging algorithm. The processed signals are encoded into two-dimensional images using Gramian Angular Fields (GAF). Then, CN-EfficientNet with contrast learning is proposed to analyze and extract deeper fault features from the image texture features. In the classification experiments for different types of faults, the accuracy reached 96.84%. The results indicate that the method can effectively accomplish the task of fault classification in planetary gearboxes. Comparative experiments with other common classification models further indicate the superior performance of the proposed learning model. Visualization based on Grad-CAM provides interpretability for the fault recognition network’s results and reveals the underlying mechanism for its excellent classification performance.

Fractional Diversity Entropy: A Vibration Signal Measure to Assist a Diffusion Model in the Fault Diagnosis of Automotive Machines

Real-world vibration signal acquisition of automotive machines often results in imbalanced sample sets due to restricted test conditions, adversely impacting fault diagnostic accuracy. To address this problem, we propose fractional diversity entropy (FrDivEn) and incorporate it into the classifier-guided diffusion model (CGDM) to synthesize high-quality samples. Additionally, we present a corresponding imbalanced fault diagnostic method. This method first converts vibration data to Gramian angular field (GAF) image samples through GAF transformation. Then, FrDivEn is mapped to the gradient scale of CGDM to trade off the diversity and fidelity of synthetic samples. These synthetic samples are mixed with real samples to obtain a balanced sample set, which is fed to the fine-tuned pretrained ConvNeXt for fault diagnosis. Various sample synthesizers and fault classifiers were combined to conduct imbalanced fault diagnosis experiments across bearing, gearbox, and rotor datasets. The results indicate that for the three datasets, the diagnostic accuracies of the proposed CGDM using FrDivEn at an imbalance ratio of 40:1 are 91.22%, 87.90%, and 98.89%, respectively, which are 7.32%, 11.59%, and 3.48% higher than that of the Wasserstein generative adversarial network (WGAN), respectively. The experimental results across the three datasets validated the validity and generalizability of the proposed diagnostic method.

A Novel Method Based on Stepwise Variational Modal Decomposition and Gramian Angular Difference Field for Bearing Health Monitoring

A Novel Method Based on Stepwise Variational Modal Decomposition and Gramian Angular Difference Field for Bearing Health Monitoring