Self-attention Research Articles

Joint-Partition Group Attention for skeleton-based action recognition

Skeleton-based action recognition aims to recognize human actions from the coordinates of human joints. By encoding coordinates as joint tokens, previous methods have successfully utilized the self-attention (SA) mechanism to capture the relationship of each pair of joints. However, the attention map generated from the joint Query and joint Key in SA only captures joint-to-joint correlations at a single granularity, which is obviously insufficient for human actions that express semantics in terms of body parts. In this paper, we argue that SA should have a more comprehensive mechanism to capture correlations in joint-to-joint and joint-to-partition patterns for a higher semantic representation of skeleton-based actions. Therefore, we propose Joint-Partition Group Attention (JPGA) to simultaneously capture correlations between joints and body parts of different granularity sizes. Specifically, JPGA integrates the joint tokens according to the joint’s human body partition attributes and produces different body parts tokens (partition-tokens) with different granularities. Then the attention map of JPGA is computed from joint-token and partition-token of different granularity sizes to represent the relationship between joints and body parts. To adaptively partition human body parts at different granularities, we apply the reparameterization trick to adaptively learn the multi-granularity partitioning matrix. Based on JPGA, we construct our Joint-Partition Former (JPFormer), and conduct extensive experiments on NTU-RGB+D, NTU-RRB+D 120, and Northwestern UCLA datasets and achieve state-of-the-art results, which highlights the effectiveness of our design and practice.

An attention-based multi-scale temporal convolutional network for remaining useful life prediction

Remaining Useful Life (RUL) prediction is one of the key technologies in Prognostics and Health Management (PHM). Accurate RUL prediction is crucial for equipment reliability, and equipment sensor data contains information at different scales that can be used for RUL prediction. Current mainstream methods usually extract features from only a single scale, neglecting the information from other scales. Additionally, the attention mechanisms used in these methods do not incorporate multi-dimensional information, which can lead to a loss in prediction performance. To address this issue, this paper proposes an end-to-end framework which utilizes Multi-Scale Temporal Convolutional Network (MSTCN) to extract multi-scale features and introduces Self-Attention (SA) and Global Fusion Attention (GFA) mechanisms to assist MSTCN's work. SA is located at the head of MSTCN, emphasizing the features of those steps that are particularly important, thereby improving the extraction efficiency of MSTCN. GFA is an original design of attention mechanism located at the end of MSTCN that intelligently integrates multi-dimensional attention information, suppressing redundant information in the multi-scale features. To validate the effectiveness of the proposed method, we conduct experiments on the C-MAPSS dataset released by NASA. The results show that our method outperforms other state-of-the-art prediction methods in terms of RUL prediction performance.

Research on the Fiber-to-the-Room Network Traffic Prediction Method Based on Crested Porcupine Optimizer Optimization

In order to solve the problem of traffic burst due to the increase in access points and user movement in an FTTR network, as well as to meet the demand for a high-performance network, it is necessary to rationally allocate network resources, and accurate traffic prediction is very important for dynamic bandwidth allocation in such a network. Therefore, this paper introduces a novel traffic prediction model, named CPO-BiTCN-BiLSTM-SA, which integrates the Crested Porcupine Optimizer (CPO), bidirectional temporal convolution (BiTCN), and bidirectional long short-term memory (BiLSTM) networks. BiTCN extends the traditional TCN by incorporating bidirectional data information, while BiLSTM enhances the network’s capability to learn from long sequences. Moreover, self-attention (SA) mechanisms are utilized to emphasize the crucial segments in the data. Subsequently, the BiTCN-BiLSTM-SA model is optimized by CPO to obtain the best network hyperparameters, and model training prediction is performed to achieve multi-step predictions based on single-step prediction. To evaluate the model’s generalization ability, two distinct datasets are employed for traffic prediction. Experimental findings demonstrate that the proposed model surpasses existing models in terms of the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). In comparison with the traditional XGBoost model, the proposed model has an average reduction of 29.50%, 25.43%, and 25.00% in RMSE, MAE, and MAPE, respectively, with a 6.70% improvement in R2.

Automatic detection of epilepsy from EEGs using a temporal convolutional network with a self-attention layer

BackgroundOver 60% of epilepsy patients globally are children, whose early diagnosis and treatment are critical for their development and can substantially reduce the disease’s burden on both families and society. Numerous algorithms for automated epilepsy detection from EEGs have been proposed. Yet, the occurrence of epileptic seizures during an EEG exam cannot always be guaranteed in clinical practice. Models that exclusively use seizure EEGs for detection risk artificially enhanced performance metrics. Therefore, there is a pressing need for a universally applicable model that can perform automatic epilepsy detection in a variety of complex real-world scenarios.MethodTo address this problem, we have devised a novel technique employing a temporal convolutional neural network with self-attention (TCN-SA). Our model comprises two primary components: a TCN for extracting time-variant features from EEG signals, followed by a self-attention (SA) layer that assigns importance to these features. By focusing on key features, our model achieves heightened classification accuracy for epilepsy detection.ResultsThe efficacy of our model was validated on a pediatric epilepsy dataset we collected and on the Bonn dataset, attaining accuracies of 95.50% on our dataset, and 97.37% (A v. E), and 93.50% (B vs E), respectively. When compared with other deep learning architectures (temporal convolutional neural network, self-attention network, and standardized convolutional neural network) using the same datasets, our TCN-SA model demonstrated superior performance in the automated detection of epilepsy.ConclusionThe proven effectiveness of the TCN-SA approach substantiates its potential as a valuable tool for the automated detection of epilepsy, offering significant benefits in diverse and complex real-world clinical settings.

Prior-guided adaptive generative adversarial network method for various types of borehole image inpainting

During geophysical exploration, inpainting defective logging images caused by mismatches between logging tools and borehole sizes can affect fracture and hole identification, petrographic analysis, and stratigraphic studies. However, existing methods do not describe sufficient stratigraphic continuity. In addition, they ignore the completeness of characterization in terms of fractures, gravel structures, and fine-grained textures in the logging images. To address these issues, we develop a deep-learning method for inpainting stratigraphic features. First, to enhance the continuity of image inpainting, we build a generative adversarial network and train it on numerous natural images to extract relevant features that guide the recovery of continuity characteristics. Second, to ensure that complete structural and textural features are found in geologic formations, we introduce a feature-extraction-fusion module with a cooccurrence mechanism consisting of channel attention (CA) and self-attention (SA). CA improves texture effects by adaptively adjusting control parameters based on highly correlated prior features from electrical logging images. SA captures long-range contextual associations across preinpainted gaps to improve completeness in fractures and gravel structure representation. Our method has been tested on various borehole images demonstrating its reliability and robustness.

Assessment of land use and land cover change detection and prediction using deep learning techniques for the southwestern coastal region, Goa, India.

Understanding the connections between human activities and the natural environment depends heavily on information about land use and land cover (LULC) in the form of accurate LULC maps. Environmental monitoring using deep learning (DL) is rapidly growing to preserve a sustainable environment in the long term. For establishing effective policies, regulations, and implementation, DL can be a valuable tool for assessing environmental conditions and natural resources that will positively impact the ecosystem. This paper presents the assessment of land use and land cover change detection (LULCCD) and prediction using DL techniques for the southwestern coastal region, Goa, also known as the tourist destination of India. It consists of three components: (i) change detection (CD), (ii) quantification of LULC changes, and (iii) prediction. A new CD assessment framework, Spatio-Temporal Encoder-Decoder Self Attention Network (STEDSAN), is proposed for the LULCCD process. A dual branch encoder-decoder network is constructed using strided convolution with downsampling for the encoder and transpose convolution with upsampling for the decoder to assess the bitemporal images spatially. The self-attention (SA) mechanism captures the complex global spatial-temporal (ST) interactions between individual pixels over space-time to produce more distinct features. Each branch accepts the LULC map of 2 years as one of its inputs to determine binary and multiclass changes among the bitemporal images. The STEDSAN model determines the patterns, trends, and conversion from one LULC type to another for the assessment period from 2005 to 2018. The binary change maps were also compared with the existing state of the art (SOTA) CD methods, with STEDSAN having an overall accuracy of 94.93%. The prediction was made using an recurrent neural network (RNN) known as long short term memory network (LSTM) for the year 2025. Experiments were conducted to determine area-wise changes in several LULC classes, such as built-up (BU), crops (kharif crop (KC), rabi crop (RC), zaid crop (ZC), double/triple (D/T C)), current fallow (CF), plantation (PL), forests (evergreen forest (EF), deciduous forest (DF), degraded/scurb forest (D/SF) ), littoral swamp (LS), grassland (GL), wasteland (WL), waterbodies max (Wmx), and waterbodies min (Wmn). As per the analysis, over the period of 13 years, there has been a net increase in the amount of BU (1.25%), RC (1.17%), and D/TC( 2.42%) and a net decrease in DF (3.29%) and WL(1.44%) being the most dominant classes being changed. These findings will offer a thorough description of identifying trends in coastal areas that may incorporate methodological hints for future studies. This study will also promote handling the spatial and temporal complexity of remotely sensed data employed in categorizing the coastal LULC of a heterogeneous landscape.

CSL-SFNet for Cooperative Spectrum Sensing in Cognitive Satellite Network with GEO and LEO Satellites

In a cognitive satellite network (CSN) with GEO and LEO satellites, there is a large propagation losses between the sensing satellite and the ground station. The results of spectrum sensing from a single satellite may be inaccurate, which will create serious interference in the primary satellite system. Cooperative spectrum sensing (CSS) has become the key technology for solving the above problems in recent years. However, most of the current CSS techniques are model-driven. They are difficult to model and implement in CSNs since their detection performance is strongly dependent on an assumed statistical model. Thus, we propose a novel CSS scheme, which uses convolutional neural networks (CNNs), self-attention (SA) modules, long short-term memory networks (LSTMs), and soft fusion networks, called CSL-SFNet. This scheme combines the advantages of CNNs, SA modules, and LSTMs to extract the features of the input signals from the spatial and temporal domains. Additionally, the CSL-SFNet makes use of a novel soft fusion technique that improves detection performance while also considerably reducing communication overhead. The simulation results demonstrate that the proposed algorithm can achieve a detection probability of 90% when the signal-to-noise ratio is −20 dB; it has a shorter running time and always outperforms the other CSS algorithms.

ScCompressSA: dual-channel self-attention based deep autoencoder model for single-cell clustering by compressing gene-gene interactions.

Single-cell clustering has played an important role in exploring the molecular mechanisms about cell differentiation and human diseases. Due to highly-stochastic transcriptomics data, accurate detection of cell types is still challenged, especially for RNA-sequencing data from human beings. In this case, deep neural networks have been increasingly employed to mine cell type specific patterns and haveoutperformed statistic approachesin cell clustering. Using cross-correlation to capture gene-gene interactions, this study proposes the scCompressSA method to integrate topological patterns from scRNA-seq data, with support of self-attention (SA) based coefficient compression (CC) block. This SA-based CC block is able to extract and employ static gene-gene interactions from scRNA-seq data. This proposed scCompressSA method has enhanced clustering accuracy in multiple benchmark scRNA-seq datasets by integrating topological and temporalfeatures. Static gene-gene interactions have been extracted as temporal features to boost clustering performance insingle-cell clustering For the scCompressSA method, dual-channel SA based CC block is able to integrate topological features and has exhibited extraordinary detection accuracy compared with previous clustering approaches that onlyemploy temporal patterns.

Segmentation of bone surface from ultrasound using a lightweight network UBS-Net

Objective. Ultrasound-assisted orthopaedic navigation held promise due to its non-ionizing feature, portability, low cost, and real-time performance. To facilitate the applications, it was critical to have accurate and real-time bone surface segmentation. Nevertheless, the imaging artifacts and low signal-to-noise ratios in the tomographical B-mode ultrasound (B-US) images created substantial challenges in bone surface detection. In this study, we presented an end-to-end lightweight US bone segmentation network (UBS-Net) for bone surface detection. Approach. We presented an end-to-end lightweight UBS-Net for bone surface detection, using the U-Net structure as the base framework and a level set loss function for improved sensitivity to bone surface detectability. A dual attention (DA) mechanism was introduced at the end of the encoder, which considered both position and channel information to obtain the correlation between the position and channel dimensions of the feature map, where axial attention (AA) replaced the traditional self-attention (SA) mechanism in the position attention module for better computational efficiency. The position attention and channel attention (CA) were combined with a two-class fusion module for the DA map. The decoding module finally completed the bone surface detection. Main Results. As a result, a frame rate of 21 frames per second (fps) in detection were achieved. It outperformed the state-of-the-art method with higher segmentation accuracy (Dice similarity coefficient: 88.76% versus 87.22%) when applied the retrospective ultrasound (US) data from 11 volunteers. Significance. The proposed UBS-Net for bone surface detection in ultrasound achieved outstanding accuracy and real-time performance. The new method out-performed the state-of-the-art methods. It had potential in US-guided orthopaedic surgery applications.

Learning Scene Representations for Human-assistive Displays Using Self-attention Networks

Video-see-through (VST) augmented reality (AR) is widely used to present novel augmentative visual experiences by processing video frames for viewers. Among VST AR systems, assistive vision displays aim to compensate for low vision or blindness, presenting enhanced visual information to support activities of daily living for the vision impaired/deprived. Despite progress, current assistive displays suffer from a visual information bottleneck, limiting their functional outcomes compared to healthy vision. This motivates the exploration of methods to selectively enhance and augment salient visual information. Traditionally, vision processing pipelines for assistive displays rely on hand-crafted, single-modality filters, lacking adaptability to time-varying and environment-dependent needs. This article proposes the use of Deep Reinforcement Learning (DRL) and Self-attention (SA) networks as a means to learn vision processing pipelines for assistive displays. SA networks selectively attend to task-relevant features, offering a more parameter—and compute-efficient approach to RL-based task learning. We assess the feasibility of using SA networks in a simulation-trained model to generate relevant representations of real-world states for navigation with prosthetic vision displays. We explore two prosthetic vision applications, vision-to-auditory encoding, and retinal prostheses, using simulated phosphene visualisations. This article introduces SA-px, a general-purpose vision processing pipeline using self-attention networks, and SA-phos, a display-specific formulation targeting low-resolution assistive displays. We present novel scene visualisations derived from SA image patches importance rankings to support mobility with prosthetic vision devices. To the best of our knowledge, this is the first application of self-attention networks to the task of learning vision processing pipelines for prosthetic vision or assistive displays.

SlowR50-SA: A Self-Attention Enhanced Dynamic Facial Expression Recognition Model for Tactile Internet Applications

Emotion recognition from facial expressions is a challenging task due to the subtle and nuanced nature of facial expressions. Within the framework of Tactile Internet (TI), the integration of this technology has the capacity to completely transform real-time user interactions, by delivering customized emotional input. The influence of this technology is far-reaching, as it may be used in immersive virtual reality interactions and remote tele-care applications to identify emotional states in patients. In this paper, a novel emotion recognition algorithm is presented that integrates a Self-Attention (SA) module into the SlowR50 backbone (SlowR50-SA). The experiments on the DFEW and FERV39K datasets demonstrate that the proposed model achieves good performance in terms of both Unweighted Average Recall (UAR) and Weighted Average Recall (WAR) metrics, achieving a UAR (WAR) of 57.09% (69.87%) on the DFEW dataset, and UAR (WAR) of 39.48% (49.34%) on the FERV39K dataset. Notably, SlowR50-SA operates with only eight frames of input at low temporal resolution, highlighting its efficiency. Furthermore, the algorithm has the potential to be integrated into Tactile Internet applications, where it can be used to enhance the user experience by providing real-time emotion feedback. SlowR50-SA can also be used to enhance virtual reality experiences by providing personalized haptic feedback based on the user’s emotional state. It can also be used in remote tele-care applications to detect signs of stress, anxiety, or depression in patients.

Multi-scale window transformer for cervical cytopathology image recognition

Cervical cancer is a major global health issue, particularly in developing countries where access to healthcare is limited. Early detection of pre-cancerous lesions is crucial for successful treatment and reducing mortality rates. However, traditional screening and diagnostic processes require cytopathology doctors to manually interpret a huge number of cells, which is time-consuming, costly, and prone to human experiences. In this paper, we propose a Multi-scale Window Transformer (MWT) for cervical cytopathology image recognition. We design multi-scale window multi-head self-attention (MW-MSA) to simultaneously integrate cell features of different scales. Small window self-attention is used to extract local cell detail features, and large window self-attention aims to integrate features from smaller-scale window attention to achieve window-to-window information interaction. Our design enables long-range feature integration but avoids whole image self-attention (SA) in ViT or twice local window SA in Swin Transformer. We find convolutional feed-forward networks (CFFN) are more efficient than original MLP-based FFN for representing cytopathology images. Our overall model adopts a pyramid architecture. We establish two multi-center cervical cell classification datasets of two-category 192,123 images and four-category 174,138 images. Extensive experiments demonstrate that our MWT outperforms state-of-the-art general classification networks and specialized classifiers for cytopathology images in the internal and external test sets. The results on large-scale datasets prove the effectiveness and generalization of our proposed model. Our work provides a reliable cytopathology image recognition method and helps establish computer-aided screening for cervical cancer. Our code is available at https://github.com/nmyz669/MWT, and our web service tool can be accessed at https://huggingface.co/spaces/nmyz/MWTdemo.

DAS-VSP coupled noise suppression based on U-Net network

Abstract The emerging distributed fiber-optic acoustic sensing (DAS) technology has broad prospects for application in vertical seismic profiles (VSP). However, the acquired DAS-VSP data often suffers from coupled noise that seriously affects data quality. Traditional methods for suppressing coupled noise are usually time-consuming and not suitable for the large-scale denoising of DAS-VSP data. To address this, a coupled noise suppression method based on the U-Net network is proposed, and a self-attention (SA) block is introduced to enhance the denoising ability of the network. Transfer learning is employed to achieve coupled noise suppression from synthetic data to field data. Denoising results demonstrate that the network can effectively suppress coupled noise in DAS-VSP data while preserving signal energy to a certain extent, exhibiting strong generalization capability. Upon completion of network training, denoising results can be obtained within seconds, making it more convenient and efficient compared to traditional methods.

Designing a set of criteria for evaluating artificial neural networks trained with physics-based data to replicate molecular dynamics and other particle method trajectories

This article presents an in-depth analysis and evaluation of artificial neural networks (ANNs) when applied to replicate trajectories in molecular dynamics (MD) simulations or other particle methods. This study focuses on several architectures—feedforward neural networks (FNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), time convolutions (TCs), self-attention (SA), graph neural networks (GNNs), neural ordinary differential equation (ODENets), and an example of physics-informed machine learning (PIML) model—assessing their effectiveness and limitations in understanding and replicating the underlying physics of particle systems. Through this analysis, this paper introduces a comprehensive set of criteria designed to evaluate the capability of ANNs in this context. These criteria include the minimization of losses, the permutability of particle indices, the ability to predict trajectories recursively, the conservation of particles, the model’s handling of boundary conditions, and its scalability. Each network type is systematically examined to determine its strengths and weaknesses in adhering to these criteria. While, predictably, none of the networks fully meets all criteria, this study extends beyond the simple conclusion that only by integrating physics-based models into ANNs is it possible to fully replicate complex particle trajectories. Instead, it probes and delineates the extent to which various neural networks can “understand” and interpret aspects of the underlying physics, with each criterion targeting a distinct aspect of this understanding.

Spatio-temporal prediction of groundwater vulnerability based on CNN-LSTM model with self-attention mechanism: A case study in Hetao Plain, northern China

Located in northern China, the Hetao Plain is an important agro-economic zone and population centre. The deterioration of local groundwater quality has had a serious impact on human health and economic development. Nowadays, the groundwater vulnerability assessment (GVA) has become an essential task to identify the current status and development trend of groundwater quality. In this study, the Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) models are integrated to realize the spatio-temporal prediction of regional groundwater vulnerability by introducing the Self-attention mechanism. The study firstly builds the CNN-LSTM model with self-attention (SA) mechanism and evaluates the prediction accuracy of the model for groundwater vulnerability compared to other common machine learning models such as Support Vector Machine (SVM), Random Forest (RF), and Extreme Gradient Boosting (XGBoost). The results indicate that the CNN-LSTM model outperforms these models, demonstrating its significance in groundwater vulnerability assessment. It can be posited that the predictions indicate an increased risk of groundwater vulnerability in the study area over the coming years. This increase can be attributed to the synergistic impact of global climate anomalies and intensified local human activities. Moreover, the overall groundwater vulnerability risk in the entire region has increased, evident from both the notably high value and standard deviation. This suggests that the spatial variability of groundwater vulnerability in the area is expected to expand in the future due to the sustained progression of climate change and human activities. The model can be optimized for diverse applications across regional environmental assessment, pollution prediction, and risk statistics. This study holds particular significance for ecological protection and groundwater resource management.

Exploiting the Social-Like Prior in Transformer for Visual Reasoning

Benefiting from instrumental global dependency modeling of self-attention (SA), transformer-based approaches have become the pivotal choices for numerous downstream visual reasoning tasks, such as visual question answering (VQA) and referring expression comprehension (REC). However, some studies have recently suggested that SA tends to suffer from rank collapse thereby inevitably leads to representation degradation as the transformer layer goes deeper. Inspired by social network theory, we attempt to make an analogy between social behavior and regional information interaction in SA, and harness two crucial notions of structural hole and degree centrality in social network to explore the possible optimization towards SA learning, which naturally deduces two plug-and-play social-like modules. Based on structural hole, the former module allows to make information interaction in SA more structured, which effectively avoids redundant information aggregation and global feature homogenization for better rank remedy, followed by latter module to comprehensively characterize and refine the representation discrimination via considering degree centrality of regions and transitivity of relations. Without bells and whistles, our model outperforms a bunch of baselines by a noticeable margin when considering our social-like prior on five benchmarks in VQA and REC tasks, and a series of explanatory results are showcased to sufficiently reveal the social-like behaviors in SA.

Target Focused Shallow Transformer Framework for Efficient Visual Tracking

Template learning transformer trackers have achieved significant performance improvement recently due to the longdependency learning using the self-attention (SA) mechanism. However, the typical SA mechanisms in transformers adopt a less discriminative design approach which is inadequate for focusing on the most important target information during tracking. Therefore, existing trackers are easily distracted by background information and have constraints in handling tracking challenges. The focus of our research is to develop a target-focused discriminative shallow transformer tracking framework that can learn to distinguish the target from the background and enable accurate tracking with fast speed. Extensive experiments will be performed on several popular benchmarks, including OTB100, UAV123, GOT10k, LaSOT, and TrackingNet, to demonstrate the effectiveness of the proposed framework.

CSAP-UNet: Convolution and self-attention paralleling network for medical image segmentation with edge enhancement

Convolution operation is performed within a local window of the input image. Therefore, convolutional neural network (CNN) is skilled in obtaining local information. Meanwhile, the self-attention (SA) mechanism extracts features by calculating the correlation between tokens from all positions in the image, which has advantage in obtaining global information. Therefore, the two modules can complement each other to improve feature extraction ability. An effective fusion method is a problem worthy of further study. In this paper, we propose a CNN and SA paralleling network CSAP-UNet with U-Net as backbone. The encoder consists of two parallel branches of CNN and Transformer to extract the feature from the input image, which takes into account both the global dependencies and the local information. Because medical images come from certain frequency bands within the spectrum, their color channels are not as uniform as natural images. Meanwhile, medical segmentation pays more attention to lesion regions in the image. Attention fusion module (AFM) integrates channel attention and spatial attention in series to fuse the output features of the two branches. The medical image segmentation task is essentially to locate the boundary of the object in the image. The boundary enhancement module (BEM) is designed in the shallow layer of the proposed network to focus more specifically on pixel-level edge details. Experimental results on three public datasets validate that CSAP-UNet outperforms state-of-the-art networks, particularly on the ISIC 2017 dataset. The cross-dataset evaluation on Kvasir and CVC-ClinicDB shows that CSAP-UNet has strong generalization ability. Ablation experiments also indicate the effectiveness of the designed modules. The code for training and test is available at https://github.com/zhouzhou1201/CSAP-UNet.git.

Surrogate model-based multi-objective optimization algorithm for train-ballast turnout system: Balancing safety and vibration damping in parameter design

In rail transportation structural parameter design, multi-objective optimal algorithms (MOOAs) have advanced significantly in safety, comfort, and aerodynamics domain. However, the vital factor of environment vibration remains neglected, impeding depot cover technology advancements. To address this challenge, with a focus on both safety and vibration damping. Firstly, the mechanistic models are constructed to establish a surrogate modeling database. Subsequently, a surrogate model is proposed, integrating stacked denoising autoencoders (SdAEs), self-attention (SA), and the gated recurrent unit (GRU). In addition, A novel MOOA, Dynamic Population-Multi-Objective Harris Hawks Optimization-Crowding Distance-Elitist Learning Strategy (D-MOHHO-CD-ELS), is proposed and its efficacy is validated against five existing MOOAs using the CTP1∼CTP7 test functions. Moreover, the train-ballast No.12 turnout system is considered as an engineering case. Importantly, the accuracy of both the mechanistic model and surrogate model is thoroughly validated. Among them, the performance of the data preprocessing and prediction algorithms is substantiated through a comparison of the proposed SdAEs-SA-GRU model against simulation results and eight other algorithms. Furthermore, the surrogate-based D-MOHHO-CD-ELS method, applied to tackle a multi-objective optimization problem with nine optimization parameters and four optimization objectives, demonstrates an efficiency improvement of 1 to 2 orders of magnitude compared to traditional methods. Consequently, following optimization, the wheel load reduction rate, derailment coefficient, capsizing coefficient and vibration damping effect are improved by 15.5%, 11.1%, 5.5% and 24.8% respectively, which reveals the effectiveness of enhancements in both the safety and damping efffect. the study offers a novel perspective for parameter design of TBT.

An intelligent fault detection and diagnosis model for refrigeration systems with a comprehensive feature selection method

Feature selection and model establishment are two essential steps for fault detection and diagnosis (FDD) of refrigeration systems. A robust and powerful FDD model combined with a suitable feature selection method can exhibit excellent performance in FDD tasks for refrigeration systems. In this study, a novel FDD method that integrates a comprehensive feature selection method and a deep learning-based intelligent FDD model is proposed. Including three steps, the comprehensive feature selection method combines filter methods and wrapper methods. It can optimize the features and the model jointly by using the multi-objective optimization algorithm to achieve a better performance. In addition, a novel FDD model that combines one-dimensional convolutional neural network (1D-CNN) and self-attention (SA) mechanism is proposed based on the deep learning technology. To evaluate the proposed method, experiments are performed on a miniature refrigeration system under 4 situations with multiple working conditions, forming a dataset for the FDD study. The proposed three-step feature selection method is utilized to obtain the best feature subset. The 1D-CNN and SA FDD model is constructed and the model is jointly optimized with the features. Several comparisons are carried out to demonstrate the effectiveness and superiority of the proposed feature selection method and the FDD model. The results demonstrate that the presented integrated optimization achieved a test accuracy of around 99.66 %, surpassing other popular FDD models including MLP, CNN, and LSTM.