Machine Learning-based Methods Research Articles

Phishing Website Detection

Abstract: Phishing is an online threat where an attacker impersonates an authentic and trustworthy organization to obtain sensitive information from a victim. One example of such is trolling, which has long been considered a problem. However, recent advances in phishing detection, such as machine learning-based methods, have assisted in combatting these attacks. Therefore, this paper develops and compares four models for investigating the efficiency of using machine learning to detect phishing domains. It also compares the most accurate model of the four with existing solutions in the literature. The work carried out in this study is an update in the previous systematic literature surveys with more focus on the latest trends in phishing detection techniques. This study enhances readers' understanding of different types of phishing website detection techniques, the data sets used, and the comparative performance of algorithms used. Our findings show that the model based on the K means clustering is the most accurate of the other four techniques and outperforms other solutions in the literature.

Intrusion Detection System Using Probabilistic Adaptive Learning

Abstract: Machine learning and deep learning techniques are widely used to evaluate intrusion detection systems (IDS) capable of rapidly and automatically recognizing and classifying cyber-attacks on networks and hosts. However, when destructive attacks are becoming more extensive, more challenges develop, needing a comprehensive response. Numerous intrusion detection datasets are publicly accessible for further analysis by the cybersecurity research community. The Intrusion Detection System (IDS) is an effective tool utilized in cybersecurity systems to detect and identify intrusion attacks. With the increasing volume of data generation, the possibility of various forms of intrusion attacks also increases. Feature selection is crucial and often necessary enhance performance. The structure of the dataset can impact the efficiency of the machine learning model. Furthermore, data imbalance can pose a problem, but sampling approaches can help mitigate it. With technological advancements, machine learning-based methods have emerged as the cornerstone of modern intrusion detection, enabling more precise identification of abnormal behaviors and potential intrusions by learning the patterns of normal network traffic. In response to these challenges, this paper introduces an innovative intrusion detection model that amalgamates the Probabilistic Adaptiv eLearning Network architecture.

A comprehensive study of machine learning-based methods to predict epileptic seizures

People with epilepsy have many difficulties as a result of this complicated brain condition, which is typified by frequent convulsions. Symptoms of these seizures include bizarre behaviors, odd sensations, and in extreme cases, loss of awareness. These seizures appear as episodes of aberrant electrical impulses in the central nervous system. Successful epilepsy management depends on early seizure detection and identification, which allows for appropriate intervention to minimize risks and improve patient outcomes. Two major reasons have contributed to the extraordinary advancements in the area of epilepsy investigations in the last few years: the explosive development of machine learning techniques and the decreasing cost of non-invasive electroencephalography (EEG) apparatus. The availability of low-cost EEG equipment has made it easier to gather information on brain activity, which has opened up new avenues for monitoring and analyzing episodes of epileptic seizures away from conventional medical settings. The abundance of data and the advancement of machine learning methods have created new opportunities for the early identification and forecasting of seizures. Machine learning algorithms can predict seizures based on EEG data, providing patients with epilepsy with more control and informed decision-making. This paper offers a current review of current methods for treating epileptic seizures. The feature extraction techniques and classification algorithms receive particular focus. The most popular EEG datasets and their accessibility are listed. The approaches that are examined range from those that use more established machine learning techniques, such as naive Bayes models, Support Vector Machines (SVM), and Linear Discriminant Analysis (LDA), to those that take advantage of more recent deep learning techniques, like (Long-Short Term Memory, or LSTM), and deep Convolutional-Neural-Networks (CNN).

Negligible effects of read trimming on the accuracy of germline short variant calling in the human genome

Background Next generation sequencing (NGS) has become a standard tool in the molecular diagnostics of Mendelian disease, and the precision of such diagnostics is greatly affected by the accuracy of variant calling from sequencing data. Recently, we have comprehensively evaluated the performance of multiple variant calling pipelines. However, no systematic analysis of the effects of read trimming on variant discovery with modern variant calling software has yet been performed. Methods In this work, we systematically evaluated the effects of adapters on the performance of 8 variant calling and filtering methods using 14 standard reference Genome-in-a-Bottle (GIAB) samples. Variant calls were compared to the ground truth variant sets, and the effect of adapter trimming with different tools was assessed using major performance metrics (precision, recall, and F1 score). Results We show that adapter trimming has no effect on the accuracy of the best-performing variant callers (e.g., DeepVariant) on whole-genome sequencing (WGS) data. For whole-exome sequencing (WES) datasets subtle improvement of accuracy was observed in some of the samples. In high-coverage WES data (~200x mean coverage), adapter removal allowed for discovery of 2-4 additional true positive variants in only two out of seven datasets tested. Moreover, this effect was not dependent on the median insert size and proportion of adapter sequences in reads. Surprisingly, the effect of trimming on variant calling was reversed when moderate coverage (~80-100x) WES data was used. Finally, we show that some of the recently developed machine learning-based variant callers demonstrate greater dependence on the presence of adapters in reads. Conclusions Taken together, our results indicate that adapter removal is unnecessary when calling germline variants, but suggest that preprocessing methods should be carefully chosen when developing and using machine learning-based variant analysis methods.

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

Medium-term forecasting of global horizontal solar radiation in Brazil using machine learning-based methods

The generation of electric energy through renewable sources, such as solar photovoltaic (PV) systems, has emerged as one solution to the climate change crisis. To avoid fluctuations in energy generation on the electricity grid, reliable and accurate forecasts are required. Statistical downscaling methodologies often rely on data from General Circulation Model (GCM) to provide information on the local level. In contrast to traditional downscaling approaches, the present study focuses on a data-driven approach using the Global Data Assimilation System (GDAS) database in three different ways to feed the machine learning (ML) models, based on boosting and bagging, to provide medium-term Global Horizontal Irradiance (GHI) forecasts, i.e. up to seven days ahead. In addition, feature selection and feature augmentation techniques are also applied. The results showed that Extra Trees method outperformed the other tested in the present study, with a RMSE of 99.6 and 137.6 W/m2, and correlation 0.31 and 0.52 for June and October, respectively. Thus far, this work marks the initial suggestion of incorporating GDAS analysis data into a data-driven forecasting approach employing bagging and boosting models. This innovative approach demonstrates the potential for integrating global-scale data assimilation systems with local measurements using ML techniques. The outcomes of this research contribute to advancing the field of traditional downscaling methods and provides more robust and specialized forecasting.

Application of machine learning for antibiotic resistance in water and wastewater: A systematic review

Antibiotic resistance (AR) is considered one of the greatest global threats in the current century, which can only be overcome if all interconnected areas of humans, animals and the environment are taken into account as part of the One Health concept proposed by the World Health Organization (WHO). Water and wastewater are among the most important environmental media of AR sources, where the phenomena are generally non-linear. Therefore, the aim of this study was to investigate the application of machine learning-based methods (MLMs) to solve AR-induced problems in water and wastewater. For this purpose, most relevant databases were searched in the period between 1987 and 2023 to systematically analyze and categorize the applications. Accordingly, the results showed that out of 12 applications, 11 (91.6%) were for shallow learning and 1 (8.3%) for deep learning. In shallow learning category, n = 6, 50% of the applications were regression and n = 4, 33.3% were classification, mainly using artificial neural networks, decision trees and Bayesian methods for the following objectives: Predicting the survival of antibiotic-resistant bacteria (ARB), determining the order of influencing parameters on AR-based scores, and identifying the major sources of antibiotic resistance genes (ARGs). In addition, only one study (8.3%) was found for clustering and no study for association. Surprisingly, deep learning had been used in only one study (8.3%) to predict ARGs sequences. Therefore, working on the knowledge gaps of AR, especially using clustering, association and deep learning methods, would be a promising option to analyze more aspects of the related problems. However, there is still a long way to go to consider and apply MLMs as unique approaches to study different aspects of AR in water and wastewater.

Raman spectroscopy for esophageal tumor diagnosis and delineation using machine learning and the portable Raman spectrometer

Esophageal cancer is one of the leading causes of cancer-related deaths worldwide. The identification of residual tumor tissues in the surgical margin of esophageal cancer is essential for the treatment and prognosis of cancer patients. But the current diagnostic methods, either pathological frozen section or paraffin section examination, are laborious, time-consuming, and inconvenient. Raman spectroscopy is a label-free and non-invasive analytical technique that provides molecular information with high specificity. Here, we report the use of a portable Raman system and machine learning algorithms to achieve accurate diagnosis of esophageal tumor tissue in surgically resected specimens. We tested five machine learning-based classification methods, including k-Nearest Neighbors, Adaptive Boosting, Random Forest, Principal Component Analysis-Linear Discriminant Analysis, and Support Vector Machine (SVM). Among them, SVM shows the highest accuracy (88.61 %) in classifying the esophageal tumor and normal tissues. The portable Raman system demonstrates robust measurements with an acceptable focal plane shift of up to 3 mm, which enables large-area Raman mapping on resected tissues. Based on this, we finally achieve successful Raman visualization of tumor boundaries on surgical margin specimens, and the Raman measurement time is less than 5 min. This work provides a robust, convenient, accurate, and cost-effective tool for the diagnosis of esophageal cancer tumors, advancing toward Raman-based clinical intraoperative applications.

Far-Field Super-Resolution Optical Microscopy for Nanostructures in a Reflective Substrate

The resolution of an optical microscope is determined by the overall point spread function of the system. When examining structures significantly smaller than the wavelength of light, the contribution of the background or surrounding environment can profoundly affect the point spread function. This research delves into the impact of reflective planar substrate structures on the system’s resolution. We establish a comprehensive forward imaging model for a reflection-type confocal laser scanning optical microscope, incorporating vector field manipulation to image densely packed nanoparticle clusters. Both theoretical and experimental findings indicate that the substrate causes an interference effect between the background field and the scattered field from the nanoparticles, markedly enhancing the overall spatial resolution. The integration of vector field manipulation with an interferometric scattering approach results in superior spatial resolution for imaging isolated particles and densely distributed nanoscale particle clusters even with deep subwavelength gaps as small as 20 nm between them. However, the method still struggles to resolve nanoparticles positioned directly next to each other without any gap, necessitating further work to enhance the resolving ability. This may involve techniques like deconvolution or machine learning-based post-processing methods.

Image-driven machine learning for automatic characterization of grain size and distribution in smart vanadium dioxide thin films

This study focuses on characterizing grain size and distribution in vanadium dioxide (VO2) thin films using Atomic Force Microscope (AFM) images. Traditional segmentation methods face challenges in accurately detecting grains, and this paper addresses these limitations by employing machine learning-based classification approaches, defining three classes: grains, joints or dividing lines, and the background. Various classification methods, including supervised, unsupervised, and thresholding-based approaches, were investigated, demonstrating the superiority of machine learning-based methods over traditional segmentation techniques. Deep Learning (DL) models, such as Convolutional Neural Networks (CNN), Deep Belief Networks (DBN), and Deep Boltzmann Machines (DBM), achieved the best results in accuracy and precision. The DBN algorithm, in particular, outperformed other methods with a 93.66% accuracy in grain detection. This could be attributed to its probabilistic modeling and effective handling of challenging pixels, making it a powerful tool for accurate grain size and distribution analysis.

Generalized zero-shot learning for classifying unseen wafer map patterns

A data-driven approach for classifying defect patterns in wafer maps is crucial for automating the quality assurance process during semiconductor fabrication. Existing works have proposed machine learning-based methods to accomplish this important problem. However, many of these approaches assume model training from a supervised learning perspective, in which a substantial number of annotated wafer maps, each with its actual defect pattern, need to be available. Due to the cost and time required to generate such an annotated dataset, recently a few works started to explore the problem of identifying previously unseen defect patterns. However, the methods proposed in those works are able to only detect the presence of unseen defect patterns without identifying what those defect patterns are. To address this issue, this paper introduces a generalized zero-shot learning (GZSL)-based framework capable of classifying and identifying both seen and unseen defect patterns. It consists of a visual feature extraction phase based on semi-supervised contrastive learning (CL), a clustering phase using distance-based initialization, and an alignment phase based on the Wasserstein-Distance-based Visual Structure Constraint method. Our framework further reduces the need for expensive annotated datasets previously required to train supervised learning-based models, thereby improving the practicality of the data-driven approach for wafer map pattern classification. Furthermore, our framework introduces an effective way of incorporating additional semantic information in wafer map pattern classification that traditionally relied on training models solely from wafer maps. To the best of the authors’ knowledge, our work is the first work to adopt GZSL in wafer map pattern classification.

XGBoost Regression Analysis of Dielectric Properties of Epoxy Resin with Inorganic Hybrid Nanofillers

Our research described in this paper investigated the influence of varying weight percentages of a hybrid nano filler composed of titanium dioxide (TiO2) and silicon dioxide (SiO2) in an epoxy polymer composite. The dielectric properties were analyzed across a broadband frequency range. Additionally, X-ray diffraction (XRD) was employed to examine the structural changes and crystalline phases within the composite materials. The Influence of hybrid inorganic nano-fillers in the epoxy composite on the dielectric properties are discussed thoroughly in this report. To improve material design and prediction modeling, our research utilized machine learning algorithms, such as the XGBoost regressor. The aim was to assess the effectiveness of regression techniques in evaluating the material properties. By employing performance metrics, like R2 score and RMSE, the tests could achieve accuracies ranging from 30% to 60%. Simulation results demonstrated that machine learning-based methods can significantly expedite the forecasting of the dielectric properties, i.e. dielectric constant and dielectric loss at intermediate frequencies, thereby saving both time and energy.

Identification of cloud-to-ground lightning and intra-cloud lightning based on their radiated electric field signatures using different types of neural networks and machine learning classifiers

The scope of this paper is to test the efficiency of three different neural networks in the area of cloud-to-ground/intra-cloud lightning classification using a not very large data set of electric field data. Effective neural network learning verified for a small lightning event dataset but with whole 2-seconds particular record length is an important feature of fast and efficient procedures implemented into lightning location systems among which discrimination between cloud-to-ground and intra-cloud lightning is one of the most important task. Measurement data obtained at the Lightning Observatory in Rzeszow were used for the analysis. About 100 extremely low/medium frequency bandwidth electric field of cloud-to ground lightning registrations and the same number of intra-cloud lightning events were taken into account. During the study 81% of those registrations were dedicated for learning of neural networks while the remaining 19% were put into the testing set. All data were preselected manually with consideration of the electric field variation and information from the lightning location system database. Lightning events were selected uniformly with the well-pronounced presence of their basic components and reported distance to the registration station which ensured a wide variation of expected lightning electric field signatures. The multilayer perceptron neural network, the radial basis function neural network and the convolutional neural network were tested and compared in different configurations while distinguishing lightning. The results were compared with other conventional classification approaches, such as traditional machine learning methods as well as one a little more contemporary architecture, the long short-term memory neural network. The multilayer perceptron neural network achieved the best detection accuracy overall. Presented research is an extension of lightning identification studies available so far mainly based on the convolutional neural network by the results achieved using the multilayer perceptron neural network, the radial basis function neural network, the long short-term memory neural network and several machine learning-based classification methods.

Defect Detection of Cell Phone Screen Using a Faster Regional Convolutional Neural Network with Multi-head Attention Mechanism

Since the glass outer screen of a cell phone is the main sensory part of the human eye when using a cell phone, the advantages and disadvantages of the cell phone screen directly affect people's sense of use. Therefore, the defect detection requirements for cell phone screens are high and need to meet the needs of high-volume factory inspection. Most of the traditional defect detection methods use visual methods, the detection results are overly dependent on the subjectivity and experience of workers, the efficiency of this method is low, and the accuracy is poor. Currently, machine learning-based detection methods are applied in numerous industries. In this paper, a faster Regional Convolutional Neural Network (R-CNN) with multi-head attention mechanism for defect detection of cell phone screen is proposed. To enhance the network's capability in extracting feature information, a four-head attention mechanism is added to the last convolutional layer of the ResNet50 network. An improved Region of Interest (ROI) Align is proposed to replace the original ROI Pooling to reduce the localization error of cell phone screen defects. Replace the original Rectified Linear Unit (ReLU) activation function with the Copy Exponential Linear Unit (CELU) activation function to expedite the convergence capability of the network. Finally, by comparing with other classical model training, the evaluation results indicate that the proposed method achieved an average accuracy of 95.71%, which is a 5.34% improvement compared to the original faster R-CNN network.

Estimation of instantaneous peak flows in Canadian rivers: an evaluation of conventional, nonlinear regression, and machine learning methods.

Instantaneous peak flows (IPFs) are often required to derive design values for sizing various hydraulic structures, such as culverts, bridges, and small dams/levees, in addition to informing several water resources management-related activities. Compared to mean daily flows (MDFs), which represent averaged flows over a period of 24 h, information on IPFs is often missing or unavailable in instrumental records. In this study, conventional methods for estimating IPFs from MDFs are evaluated and new methods based on the nonlinear regression framework and machine learning architectures are proposed and evaluated using streamflow records from all Canadian hydrometric stations with natural and regulated flow regimes. Based on a robust model selection criterion, it was found that multiple methods are suitable for estimating IPFs from MDFs, which precludes the idea of a single universal method. The performance of machine learning-based methods was also found reasonable compared to conventional and regression-based methods. To build on the strengths of individual methods, the fusion modeling concept from the machine learning area was invoked to synthesize outputs of multiple methods. The study findings are expected to be useful to the climate change adaptation community, which currently heavily relies on MDFs simulated by hydrologic models.

Galaxy merger challenge: A comparison study between machine learning-based detection methods

Various galaxy merger detection methods have been applied to diverse datasets. However, it is difficult to understand how they compare. Our aim is to benchmark the relative performance of merger detection methods based on machine learning (ML). We explore six leading ML methods using three main datasets. The first dataset consists of mock observations from the IllustrisTNG simulations, which acts as the training data and allows us to quantify the performance metrics of the detection methods. The second dataset consists of mock observations from the Horizon-AGN simulations, introduced to evaluate the performance of classifiers trained on different, but comparable data to those employed for training. The third dataset is composed of real observations from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) survey. We also compare mergers and non-mergers detected by the different methods with a subset of HSC-SSP visually identified galaxies. For the simplest binary classification task (i.e. mergers vs. non-mergers), all six methods perform reasonably well in the domain of the training data. At the lowest redshift explored $0.1<z<0.3$, precision and recall generally range between sim 70<!PCT!> and 80<!PCT!>, both of which decrease with increasing $z$ as expected (by sim 5<!PCT!> for precision and sim 10<!PCT!> for recall at the highest $z$ explored $0.76<z<1.0$). When transferred to a different domain, the precision of all classifiers is only slightly reduced, but the recall is significantly worse (by sim 20-40<!PCT!> depending on the method). Zoobot offers the best overall performance in terms of precision and F1 score. When applied to real HSC observations, different methods agree well with visual labels of clear mergers, but can differ by more than an order of magnitude in predicting the overall fraction of major mergers. For the more challenging multi-class classification task to distinguish between pre-mergers, ongoing-mergers, and post-mergers, none of the methods in their current set-ups offer good performance, which could be partly due to the limitations in resolution and the depth of the data. In particular, ongoing-mergers and post-mergers are much more difficult to classify than pre-mergers. With the advent of better quality data (e.g. from JWST and Euclid), it is of great importance to improve our ability to detect mergers and distinguish between merger stages.

Relationships between Resident Activities and Physical Space in Shrinking Cities in China—The Case of Chaoyang City

Shrinking cities suffer from a decreased level of resident activities. As a result, areas with low levels of resident activities may become breeding grounds for social issues. To ease and prevent social issues, it is important to deploy physical space optimisation strategies to effectively guide the distribution of resident activities in shrinking cities. To support the development of such spatial strategies, this paper introduces machine learning-based methods for analysing the nuanced non-linear relationship between resident activities and physical space in shrinking cities. Utilising dual-scale grids, this study calculates multi-source spatial elements, which are subsequently integrated with resident activity data to construct a gradient boosting decision tree model. It then analyses the weight of different spatial elements’ impacts on resident activities and their nonlinear relationships. The model proposed in this study demonstrates good precision in construing the relationship between resident activities and physical space. Based on the research findings, strategies for different types of spatial development in shrinking cities are drawn out. This paper advocates for the application of this analytical approach before conducting spatial planning in shrinking cities to maximise the effectiveness of spatial development in guiding resident activities.

A review of machine learning-based methods for predicting drug-target interactions.

The prediction of drug-target interactions (DTI) is a crucial preliminary stage in drug discovery and development, given the substantial risk of failure and the prolonged validation period associated with in vitro and in vivo experiments. In the contemporary landscape, various machine learning-based methods have emerged as indispensable tools for DTI prediction. This paper begins by placing emphasis on the data representation employed by these methods, delineating five representations for drugs and four for proteins. The methods are then categorized into traditional machine learning-based approaches and deep learning-based ones, with a discussion of representative approaches in each category and the introduction of a novel taxonomy for deep neural network models in DTI prediction. Additionally, we present a synthesis of commonly used datasets and evaluation metrics to facilitate practical implementation. In conclusion, we address current challenges and outline potential future directions in this research field.

Machine learning-based virtual screening of multi-target anti-obesity compounds from medicinal and edible plants: A combined in silico and in vitro study

In response to the limited effectiveness of existing weight loss food products, we sought to apply machine learning-based virtual screening methods to identify potential anti-obesity functional compounds from medicinal and edible plants and validate their in vitro activities. Firstly, we construct and evaluate the machine learning (ML) screening models using Multilayer Perceptron (MLP) and Random Forest (RF) algorithms. The receiver operating characteristic (ROC) curve demonstrates the high accuracy of MLP and RF models in screening for obese-related targets PL (pancreatic lipase) and AMPK (Adenosine 5′-monophosphate activated protein kinase). Subsequently, the tested ML models are employed to screen the constructed database, and Gypenoside LXVI (GYP) and alisol-b-23-acetate (ALI) are identified as compounds exhibiting favorable activity against both targets. The hit compounds are tested for their impact on lipase activity and lipid accumulation. The test results show that GYP and ALI have favorable inhibitory effects on pancreatic lipase (PL), with IC50 of 359.7 and 433.8 μg/mL. Furthermore, both GYP and ALI significantly reduced cellular lipid accumulation by 72.89% and 79.01% with the concentration increase to 40 μg/mL. The molecular docking results indicate that GYP and ALI can interact with several amino acid residues on the two target proteins, thereby affecting the activity of the target proteins. In conclusion, GYP and ALI can prevent and alleviate obesity by inhibiting PL activity and regulating AMPK signaling factors. We innovatively applied virtual screening based on ML to discover functional factors in food for anti-obesity purposes. This novel computational screening technique holds significant potential in the development of dietary supplements to combat obesity.

Interpretable intelligent fault diagnosis strategy for fixed-wing UAV elevator fault diagnosis based on improved cross entropy loss

The current popular machine learning-based fault diagnosis methods make it difficult to explain the diagnostic results, leading to low user trust in such diagnostic techniques. In this regard, this paper explores the study of the interpretability of intelligent fault diagnosis algorithms using the elevator of a fixed-wing unmanned aerial vehicle (UAV) as a diagnostic object. The Transformer model combines excellent modeling capability and efficient sequence data processing, is chosen to mine fault signal features to guarantee accurate diagnosis. Among the proposed interpretable fault diagnosis models, the local interpretable model-agnostic explanations (LIME) model is used to provide explicit interpretability for the decision-making process of the diagnosis model. In addition, a loss function called RDCE (reinforced diagnostic cross-entropy) Loss is designed to minimize the negative impact of different sample sizes for different fault types on the diagnostic performance. This loss function is designed to weigh the various types of faults to speed up the convergence of the model and improve the diagnostic accuracy. By comparing the proposed diagnostic strategy with other commonly used machine learning models, including long short term memory and recurrent neural network (RNN), the average diagnostic accuracy of the proposed diagnostic strategy is 99.97%, significantly better than that of the comparison algorithms. At the same time, this paper provides an in-depth interpretable analysis of the diagnostic process of the Transformer. The diagnostic process of the Transformer model gives the reasons for the diagnostic results from the point of view of the kind of features processed by the model. Based on this, the diagnostic model is simplified. After streamlining the number of features from 40 to 24 according to their importance, the diagnostic accuracy of the model is improved by 0.26%, and the diagnostic efficiency is improved. In addition, the proposed diagnostic strategy also shows significant advantages in terms of noise robustness.