Improve Prediction Performance Research Articles

A data recovery method for extra-long-span railway bridge health monitoring based on TVFEMD and CNN-GRU

Structure health monitoring systems play a crucial role in understanding the condition of structures. However, owing to various uncertain factors, sensor data may be missing, which poses a great challenge to capture the real-time dynamic characteristics of the bridge. To address this problem, this paper proposes a model that integrates time varying filtering based empirical mode decomposition (TVFEMD), convolutional neural network (CNN), and gated recurrent unit (GRU) to forecast and fill in the missing data. The proposed model initially employs TVFEMD to decompose the signal into several intrinsic mode functions (IMFs) of different frequency bands. Subsequently, CNN is utilized to extract data features for each IMF, followed by prediction through GRU. The model linearly combines the prediction results obtained from each IMF to obtain the actual prediction result. Simulation data and measured data from the large railway bridge are utilized in this research to validate the model’s efficacy. The analysis results demonstrate a significant improvement in prediction performance compared to traditional models, showcasing strong generalization ability and robustness. In conclusion, the model proposed in this paper has a high utilization value in health monitoring data recovery.

Learning semi-supervised enrichment of longitudinal imaging-genetic data for improved prediction of cognitive decline

BackgroundAlzheimer’s Disease (AD) is a progressive memory disorder that causes irreversible cognitive decline. Given that there is currently no cure, it is critical to detect AD in its early stage during the disease progression. Recently, many statistical learning methods have been presented to identify cognitive decline with temporal data, but few of these methods integrate heterogeneous phenotype and genetic information together to improve the accuracy of prediction. In addition, many of these models are often unable to handle incomplete temporal data; this often manifests itself in the removal of records to ensure consistency in the number of records across participants.ResultsTo address these issues, in this work we propose a novel approach to integrate the genetic data and the longitudinal phenotype data to learn a fixed-length “enriched” biomarker representation derived from the temporal heterogeneous neuroimaging records. Armed with this enriched representation, as a fixed-length vector per participant, conventional machine learning models can be used to predict clinical outcomes associated with AD.ConclusionThe proposed method shows improved prediction performance when applied to data derived from Alzheimer’s Disease Neruoimaging Initiative cohort. In addition, our approach can be easily interpreted to allow for the identification and validation of biomarkers associated with cognitive decline.

Periodic Transformer Encoder for Multi-Horizon Travel Time Prediction

In the domain of Intelligent Transportation Systems (ITS), ensuring reliable travel time predictions is crucial for enhancing the efficiency of transportation management systems and supporting long-term planning. Recent advancements in deep learning have demonstrated the ability to effectively leverage large datasets for accurate travel time predictions. These innovations are particularly vital as they address both short-term and long-term travel demands, which are essential for effective traffic management and scheduled routing planning. Despite advances in deep learning applications for traffic analysis, the dynamic nature of traffic patterns frequently challenges the forecasting capabilities of existing models, especially when forecasting both immediate and future traffic conditions across various time horizons. Additionally, the area of long-term travel time forecasting still remains not fully explored in current research due to these complexities. In response to these challenges, this study introduces the Periodic Transformer Encoder (PTE). PTE is a Transformer-based model designed to enhance traffic time predictions by effectively capturing temporal dependencies across various horizons. Utilizing attention mechanisms, PTE learns from long-range periodic traffic data for handling both short-term and long-term fluctuations. Furthermore, PTE employs a streamlined encoder-only architecture that eliminates the need for a traditional decoder, thus significantly simplifying the model’s structure and reducing its computational demands. This architecture enhances both the training efficiency and the performance of direct travel time predictions. With these enhancements, PTE effectively tackles the challenges presented by dynamic traffic patterns, significantly improving prediction performance across multiple time horizons. Comprehensive evaluations on an extensive real-world traffic dataset demonstrate PTE’s superior performance in predicting travel times over multiple horizons compared to existing methods. PTE is notably effective in adapting to high-variability road segments and peak traffic hours. These results prove PTE’s effectiveness and robustness across diverse traffic environments, indicating its significant contribution to advancing traffic prediction capabilities within ITS.

MPHGCL-DDI: Meta-Path-Based Heterogeneous Graph Contrastive Learning for Drug-Drug Interaction Prediction

The combinatorial therapy with multiple drugs may lead to unexpected drug-drug interactions (DDIs) and result in adverse reactions to patients. Predicting DDI events can mitigate the potential risks of combinatorial therapy and enhance drug safety. In recent years, deep models based on heterogeneous graph representation learning have attracted widespread interest in DDI event prediction and have yielded satisfactory results, but there is still room for improvement in prediction performance. In this study, we proposed a meta-path-based heterogeneous graph contrastive learning model, MPHGCL-DDI, for DDI event prediction. The model constructs two contrastive views based on meta-paths: an average graph view and an augmented graph view. The former represents that there are connections between drugs, while the latter reveals how the drugs connect with each other. We defined three levels of data augmentation schemes in the augmented graph view and adopted a combination of three losses in the model training phase: multi-relation prediction loss, unsupervised contrastive loss and supervised contrastive loss. Furthermore, the model incorporates indirect drug information, protein–protein interactions (PPIs), to reveal latent relations of drugs. We evaluated MPHGCL-DDI on three different tasks of two datasets. Experimental results demonstrate that MPHGCL-DDI surpasses several state-of-the-art methods in performance.

A retrospective view on non-linear methods in chemometrics, and future directions

This perspective article reviews how the chemometrics community approached non-linear methods in the early years. In addition to the basic chemometric methods, some methods that fall under the term “machine learning” are also mentioned. Thereafter, types of non-linearity are briefly presented, followed by discussions on important aspects of modeling related to non-linear data. Lastly, a simulated data set with non-linear properties is analyzed for quantitative prediction and batch monitoring. The conclusion is that the latent variable methods to a large extent handle non-linearities by adding more linear combinations of the original variables. Nevertheless, with strong non-linearities between the X and Y space, non-linear methods such as Support Vector Machines might improve prediction performance at the cost of interpretability into both the sample and variable space. Applying multiple local models can improve performance compared to a single global model, of both linear and non-linear nature. When non-linear methods are applied, the need for conservative model validation is even more important. Another approach is pre-processing of the data which can make the data more linear before the actual modeling and prediction phase.

Dynamic Perception-Based Vehicle Trajectory Prediction Using a Memory-Enhanced Spatio-Temporal Graph Network

In the realm of intelligent transportation systems, accurately predicting vehicle trajectories is paramount for enhancing road safety and optimizing traffic flow management. Addressing the impacts of complex traffic environments and efficiently modeling the diverse behaviors of vehicles are the key challenges at present. To achieve precise prediction of vehicle trajectories, it is essential to fully consider the dynamic changes in traffic conditions and the long-term dependencies of time-series data. In response to these challenges, we propose the Memory-Enhanced Spatio-Temporal Graph Network (MESTGN), an innovative model that integrates a Spatio-Temporal Graph Convolutional Network (STGCN) with an attention-enhanced Long Short-Term Memory (LSTM)-based sequence to sequence (Seq2Seq) encoder–decoder structure. MESTGN utilizes STGCN to capture the complex spatial dependencies between vehicles and reflects the interactions within the traffic network through road traffic data and network topology, which significantly influences trajectory prediction. Additionally, the model focuses on historical vehicle trajectory data points using an attention-weighted mechanism under a traditional LSTM prediction architecture, calculating the importance of critical trajectory points. Finally, our experiments conducted on the urban traffic dataset ApolloSpace validate the effectiveness of our proposed model. We demonstrate that MESTGN shows a significant performance improvement in vehicle trajectory prediction compared with existing mainstream models, thereby confirming its increased prediction accuracy.

Integrated analysis of blood DNA methylation, genetic variants, circulating proteins, microRNAs, and kidney failure in type 1 diabetes.

Variation in DNA methylation (DNAmet) in white blood cells and other cells/tissues has been implicated in the etiology of progressive diabetic kidney disease (DKD). However, the specific mechanisms linking DNAmet variation in blood cells with risk of kidney failure (KF) and utility of measuring blood cell DNAmet in personalized medicine are not clear. We measured blood cell DNAmet in 277 individuals with type 1 diabetes and DKD using Illumina EPIC arrays; 51% of the cohort developed KF during 7 to 20 years of follow-up. Our epigenome-wide analysis identified DNAmet at 17 CpGs (5'-cytosine-phosphate-guanine-3' loci) associated with risk of KF independent of major clinical risk factors. DNAmet at these KF-associated CpGs remained stable over a median period of 4.7 years. Furthermore, DNAmet variations at seven KF-associated CpGs were strongly associated with multiple genetic variants at seven genomic regions, suggesting a strong genetic influence on DNAmet. The effects of DNAmet variations at the KF-associated CpGs on risk of KF were partially mediated by multiple KF-associated circulating proteins and KF-associated circulating miRNAs. A prediction model for risk of KF was developed by adding blood cell DNAmet at eight selected KF-associated CpGs to the clinical model. This updated model significantly improved prediction performance (c-statistic = 0.93) versus the clinical model (c-statistic = 0.85) at P = 6.62 × 10-14. In conclusion, our multiomics study provides insights into mechanisms through which variation of DNAmet may affect KF development and shows that blood cell DNAmet at certain CpGs can improve risk prediction for KF in T1D.

Topic and knowledge-enhanced modeling for edge-enabled IoT user identity linkage across social networks

The Internet of Things (IoT) devices spawn growing diverse social platforms and online data at the network edge, propelling the development of cross-platform applications. To integrate cross-platform data, user identity linkage is envisioned as a promising technique by detecting whether different accounts from multiple social networks belong to the same identity. The profile and social relationship information of IoT users may be inconsistent, which deteriorates the reliability of the effectiveness of identity linkage. To this end, we propose a topic and knowledge-enhanced model for edge-enabled IoT user identity linkage across social networks, named TKM, which conducts feature representation of user generated contents from both post-level and account-level for identity linkage. Specifically, a topic-enhanced method is designed to extract features at the post-level. Meanwhile, we develop an external knowledge-based Siamese neural network for user-generated content alignment at the account-level. Finally, we show the superiority of TKM over existing methods on two real-world datasets. The results demonstrate the improvement in prediction and retrieval performance achieved by utilizing both post-level and account-level representation for identity linkage across social networks.

Developing machine learning models to predict multi-class functional outcomes and death three months after stroke in Sweden.

Globally, stroke is the third-leading cause of mortality and disability combined, and one of the costliest diseases in society. More accurate predictions of stroke outcomes can guide healthcare organizations in allocating appropriate resources to improve care and reduce both the economic and social burden of the disease. We aim to develop and evaluate the performance and explainability of three supervised machine learning models and the traditional multinomial logistic regression (mLR) in predicting functional dependence and death three months after stroke, using routinely-collected data. This prognostic study included adult patients, registered in the Swedish Stroke Registry (Riksstroke) from 2015 to 2020. Riksstroke contains information on stroke care and outcomes among patients treated in hospitals in Sweden. Prognostic factors (features) included demographic characteristics, pre-stroke functional status, cardiovascular risk factors, medications, acute care, stroke type, and severity. The outcome was measured using the modified Rankin Scale at three months after stroke (a scale of 0-2 indicates independent, 3-5 dependent, and 6 dead). Outcome prediction models included support vector machines, artificial neural networks (ANN), eXtreme Gradient Boosting (XGBoost), and mLR. The models were trained and evaluated on 75% and 25% of the dataset, respectively. Model predictions were explained using SHAP values. The study included 102,135 patients (85.8% ischemic stroke, 53.3% male, mean age 75.8 years, and median NIHSS of 3). All models demonstrated similar overall accuracy (69%-70%). The ANN and XGBoost models performed significantly better than the mLR in classifying dependence with F1-scores of 0.603 (95% CI; 0.594-0.611) and 0.577 (95% CI; 0.568-0.586), versus 0.544 (95% CI; 0.545-0.563) for the mLR model. The factors that contributed most to the predictions were expectedly similar in the models, based on clinical knowledge. Our ANN and XGBoost models showed a modest improvement in prediction performance and explainability compared to mLR using routinely-collected data. Their improved ability to predict functional dependence may be of particular importance for the planning and organization of acute stroke care and rehabilitation.

Prediction of energy consumption for manufacturing small and medium-sized enterprises (SMEs) considering industry characteristics

There has been a growing demand for energy consumption statistics in the manufacturing industry to establish national energy and greenhouse gas policies. Despite its importance, the Korean government faces significant challenges in collecting energy data at a facility level in a precise and timely manner. To address the lack of timely data, this paper employs machine-learning models to predict the annual total energy consumptions of each manufacturing facility. We first designed four prediction models that take into account the characteristics and energy consumption behaviors of industry sub-sectors. As input variables, these prediction models mainly included electricity consumption, employee size, energy types, gas consumption and other accessible data. Finally, we conducted numerical experiments on approximately 100,000 facilities and evaluated the prediction performance of various machine-learning algorithms such as linear regression, decision tree regression, random forest regression, gradient boost regression, and extreme gradient boosting regression. The numerical experiments provided insights into which model and algorithm offer the best prediction performance for each industry sub-sector. In addition, we identified the important variables for predicting total energy consumption, revealing that not only electricity but also various other energy sources and variables representing industry-specific characteristics play a crucial role in improving prediction performance.

Precise prediction of CO2 separation performance of metal–organic framework mixed matrix membranes based on feature selection and machine learning

Nowadays, the price of fossil fuels keeps setting new records, escalating continuing concerns about global warming from CO2 production from fuel combustion. As a promising membrane separation technique dealing with carbon capture, metal–organic framework (MOF) mixed matrix membranes (MMMs) have been extensively studied. Herein, a genetic algorithm (GA) optimized artificial neural network (ANN) was developed to form prediction model of MOF MMMs performances towards CO2/N2 separation. The MOF properties, polymer properties, and the operating conditions were used as the characteristic variables. To overcome the limitation, molecular descriptors were incorporated to reflect the physicochemical properties of polymers and target encoding was applied to digitalize the MOF and polymer types. In addition, recursive feature elimination algorithm was used to filter the optimal feature subset and Shapley additive explanations was utilized to analyze the feature importance. The results demonstrated that the model has a dramatically improved prediction performance than other machine learning methods.

Traffic Flow Prediction by an Ensemble Framework with Data Denoising and Functional Principal Components Analysis

Accurate traffic flow data are important for congestion identification and real-time traffic control. Raw traffic flow data may be disturbed by different noises during the acquisition process, which leads to the degradation of model prediction performance. To address this problems, this paper proposes a data denoising method based on the improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) combined with wavelet threshold denoising (WTD) to suppress potential outliers in the data, and then introduces functional principal component analysis to accomplish the traffic forecasting task. We use the proposed framework for real traffic flow prediction and validate the prediction performance of the proposed framework using root mean square error and mean absolute error. In the prediction performance comparison, five denoising methods are considered: empirical mode decomposition (EMD), ensemble empirical mode decomposition, complete ensemble empirical mode decomposition with adaptive noise, ICEEMDAN, and WTD. The prediction results show that the model combined with the denoising method outperforms the model without the denoising method. ICEEMDAN combined with WTD’s denoising method improves prediction performance more than other methods. Furthermore, the WTD method with dmey type achieves higher accuracy than other types.

Attention-Oriented CNN Method for Type 2 Diabetes Prediction

Diabetes is caused by insulin deficiency or impaired biological action, and long-term hyperglycemia leads to a variety of tissue damage and dysfunction. Therefore, the early prediction of diabetes and timely intervention and treatment are crucial. This paper proposes a robust framework for the prediction and diagnosis of type 2 diabetes (T2DM) to aid in diabetes applications in clinical diagnosis. The data-preprocessing stage includes steps such as outlier removal, missing value filling, data standardization, and assigning class weights to ensure the quality and consistency of the data, thereby improving the performance and stability of the model. This experiment used the National Health and Nutrition Examination Survey (NHANES) dataset and the publicly available PIMA Indian dataset (PID). For T2DM classification, we designed a convolutional neural network (CNN) and proposed a novel attention-oriented convolutional neural network (SECNN) through the channel attention mechanism. To optimize the hyperparameters of the model, we used grid search and K-fold cross-validation methods. In addition, we also comparatively analyzed various machine learning (ML) models such as support vector machine (SVM), logistic regression (LR), decision tree (DT), random forest (RF), and artificial neural network (ANN). Finally, we evaluated the performance of the model using performance evaluation metrics such as precision, recall, F1-Score, accuracy, and AUC. Experimental results show that the SECNN model has an accuracy of 94.12% on the NHANES dataset and an accuracy of 89.47% on the PIMA Indian dataset. SECNN models and CNN models show significant improvements in diabetes prediction performance compared to traditional ML models. The comparative analysis of the SECNN model and the CNN model has significantly improved performance, further verifying the advantages of introducing the channel attention mechanism. The robust diabetes prediction framework proposed in this article establishes an effective foundation for diabetes diagnosis and prediction, and has a positive impact on the development of health management and medical industries.

Molecular property prediction based on graph structure learning.

Molecular property prediction (MPP) is a fundamental but challenging task in the computer-aided drug discovery process. More and more recent works employ different graph-based models for MPP, which have achieved considerable progress in improving prediction performance. However, current models often ignore relationships between molecules, which could be also helpful for MPP. For this sake, in this paper we propose a graph structure learning (GSL) based MPP approach, called GSL-MPP. Specifically, we first apply graph neural network (GNN) over molecular graphs to extract molecular representations. Then, with molecular fingerprints, we construct a molecular similarity graph (MSG). Following that, we conduct graph structure learning on the MSG, i.e., molecule-level graph structure learning, to get the final molecular embeddings, which are the results of fusing both GNN encoded molecular representations and the relationships among molecules. That is, combining both intra-molecule and inter-molecule information. Finally, we use these molecular embeddings to perform MPP. Extensive experiments on ten various benchmark datasets show that our method could achieve state- of-the-art performance in most cases, especially on classification tasks. Further visualization studies also demonstrate the good molecular representations of our method. Source code is available at https://github.com/zby961104/GSL-MPP.

Enhancing geospatial prediction models with feature engineering from road networks: a graph-driven approach

Traditional geospatial predictive models for property valuation have naturally relied on coordinates as well as ‘hedonic’ (internal and external) features. In particular, location-centric methods such as Geographically Weighted Regression (GWR) and Kriging have focused on intrinsic target characteristics together with distances between individual targets. However, especially in the context of heavily urbanized areas, these approaches might overlook crucial aspects arising from the underlying topological structure that presents itself in such areas. Concretely, in this work, we focus on the structure arising from the road network connecting properties. We introduce a novel though straightforward technique for feature engineering based on graphs constructed on a road network. We then extract relevant features from these and utilize those as inputs for predictive models and assess their performance benefits when used together with a variety of both well-known geospatial models and state-of-art machine learning models. To this end, we present an exhaustive experiment using four different real-life data sets across various regions and exhibiting sizes outperforming many comparative works in the field. Our findings reveal that our feature engineering approach offers significant improvements in predictive performance. Finally, we apply Shapley values as an interpretability technique to confirm the reliability and effectiveness of our approach.

Sampling-based novel heterogeneous multi-layer stacking ensemble method for telecom customer churn prediction

In recent times, customer churn has become one of the most significant issues in business-oriented sectors with telecommunication being no exception. Maintaining current customers is particularly valuable due to the high degree of rivalry among telecommunication companies and the costs of acquiring new ones. The early prediction of churned customers may help telecommunication companies to identify the causes of churn and design industrial tactics to address or mitigate the churn problem. Controlling customer churn by developing efficient and reliable customer churn prediction (CCP) solutions is essential to achieving this objective. Findings from existing CCP studies have shown that numerous methods, such as rule-based and machine-learning (ML) mechanisms, have been devised to solve the CCP problem. Nonetheless, the problems of adaptability and the resilience of rule-based CCP solutions are its major weaknesses, and the skewed pattern of churn datasets (class imbalance) is detrimental to the prediction performances of conventional ML models in CCP. Hence, this research developed a robust heterogeneous multi-layer stacking ensemble method (HMSE) for effective CCP. Specifically, in the HMSE method, the prediction prowess of five ML classifiers (Random Forest (RF), Bayesian network (BN), Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and Repeated Incremental Pruning to Produce Error Reduction (RIPPER)) with distinct computational characteristics are ensembled based on stacking and the resulting model is further enhanced using a forest penalizing attribute (FPA) model. The synthetic minority oversampling technique (SMOTE) is integrated with the proposed HMSE to balance the skewed class label present in the original experimental datasets. Extensive tests were carried out to determine the effectiveness of the proposed HMSE and S-HMSE on standard telecom CCP datasets. Observed findings from the experimental results showed that HMSE and S-HMSE can effectively predict churners even with the class imbalance (skewed datasets) problem. In addition, comparison studies demonstrated that the suggested S-HMSE offered improved prediction performance and optimum solutions for CCP in the telecom sector in comparison with baseline classifiers, homogeneous ensemble methods, and current CCP approaches.

Just-in-Time crash prediction for mobile apps

Just-In-Time (JIT) defect prediction aims to identify defects early, at commit time. Hence, developers can take precautions to avoid defects when the code changes are still fresh in their minds. However, the utility of JIT defect prediction has not been investigated in relation to crashes of mobile apps. We therefore conducted a multi-case study employing both quantitative and qualitative analysis. In the quantitative analysis, we used machine learning techniques for prediction. We collected 113 reliability-related metrics for about 30,000 commits from 14 Android apps and selected 14 important metrics for prediction. We found that both standard JIT metrics and static analysis warnings are important for JIT prediction of mobile app crashes. We further optimized prediction performance, comparing seven state-of-the-art defect prediction techniques with hyperparameter optimization. Our results showed that Random Forest is the best performing model with an AUC-ROC of 0.83. In our qualitative analysis, we manually analysed a sample of 642 commits and identified different types of changes that are common in crash-inducing commits. We explored whether different aspects of changes can be used as metrics in JIT models to improve prediction performance. We found these metrics improve the prediction performance significantly. Hence, we suggest considering static analysis warnings and Android-specific metrics to adapt standard JIT defect prediction models for a mobile context to predict crashes. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.

Predicting the Intensity of Tropical Cyclones over the Western North Pacific Using a Dual-Branch Spatiotemporal Attention Convolutional Network

Abstract This paper proposes a spatiotemporal attention convolutional network (STAC-Pred) that leverages deep learning techniques to model the spatiotemporal features of tropical cyclones (TCs) and enable real-time prediction of their intensity. The proposed model employs dual branches to concurrently extract and integrate features from intensity heatmaps and satellite cloud imagery. Additionally, a residual attention (RA) module is integrated into the three-channel cloud imagery convolution process to automatically respond to high wind speed regions. TC’s longitude, latitude, and radius of winds are injected into the multi-timepoint prediction model to assist in the prediction task. Furthermore, a rolling mechanism (RM) is employed to smooth the fluctuation of losses, achieving accurate prediction of TC intensity. We use several TC records to evaluate and validate the universality and effectiveness of the model. The results indicate that STAC-Pred achieves satisfactory performance. Specifically, the STAC-Pred model improves prediction performance by 47.69% and 28.26% compared to the baseline (official institutions) at 3- and 6-h intervals, respectively. Significance Statement Tropical cyclones are one of the most deadly and damaging natural disasters in coastal areas worldwide. Early prediction can significantly reduce casualties and property losses. This study innovatively conducts dimensionality augmentation on one-dimensional intensity numerical sequences and proposes a new network model for rolling forecast of their future intensity. The proposed prototype model (not yet incorporating any atmospheric conditions) shows promising results for 3- and 6-h advance forecasts, providing valuable guidance for forecasters regarding real-time operational predictions of short-term tropical cyclone intensity.

LMCrot: an enhanced protein crotonylation site predictor by leveraging an interpretable window-level embedding from a transformer-based protein language model.

Recent advancements in natural language processing have highlighted the effectiveness of global contextualized representations from protein language models (pLMs) in numerous downstream tasks. Nonetheless, strategies to encode the site-of-interest leveraging pLMs for per-residue prediction tasks, such as crotonylation (Kcr) prediction, remain largely uncharted. Herein, we adopt a range of approaches for utilizing pLMs by experimenting with different input sequence types (full-length protein sequence versus window sequence), assessing the implications of utilizing per-residue embedding of the site-of-interest as well as embeddings of window residues centered around it. Building upon these insights, we developed a novel residual ConvBiLSTM network designed to process window-level embeddings of the site-of-interest generated by the ProtT5-XL-UniRef50 pLM using full-length sequences as input. This model, termed T5ResConvBiLSTM, surpasses existing state-of-the-art Kcr predictors in performance across three diverse datasets. To validate our approach of utilizing full sequence-based window-level embeddings, we also delved into the interpretability of ProtT5-derived embedding tensors in two ways: firstly, by scrutinizing the attention weights obtained from the transformer's encoder block; and secondly, by computing SHAP values for these tensors, providing a model-agnostic interpretation of the prediction results. Additionally, we enhance the latent representation of ProtT5 by incorporating two additional local representations, one derived from amino acid properties and the other from supervised embedding layer, through an intermediate fusion stacked generalization approach, using an n-mer window sequence (or, peptide/fragment). The resultant stacked model, dubbed LMCrot, exhibits a more pronounced improvement in predictive performance across the tested datasets. LMCrot is publicly available at https://github.com/KCLabMTU/LMCrot.

Improved training framework in a neural network model for disruption prediction and its application on EXL-50

A neural network model with a classical annotation method has been used on the EXL-50 tokamak to predict impending disruption. However, the results revealed issues of overfitting and overconfidence in predictions caused by inaccurate labeling. To mitigate these issues, an improved training framework has been proposed. In this approach, soft labels from previous training serve as teachers to supervise the further learning process; this has lead to a significant improvement in predictive model performance. Notably, this enhancement is primarily attributed to the coupling effect of the soft labels and correction mechanism. This improved training framework introduces an instance-specific label smoothing method, which reflects a more nuanced model assessment on the likelihood of a disruption. It presents a possible solution to effectively address the challenges associated with accurate labeling across different machines.