Traditional Prediction Methods Research Articles

Exploring Pathogen Presence Prediction in Pastured Poultry Farms through Transformer-Based Models and Attention Mechanism Explainability.

In this study, we explore how transformer models, which are known for their attention mechanisms, can improve pathogen prediction in pastured poultry farming. By combining farm management practices with microbiome data, our model outperforms traditional prediction methods in terms of the F1 score-an evaluation metric for model performance-thus fulfilling an essential need in predictive microbiology. Additionally, the emphasis is on making our model's predictions explainable. We introduce a novel approach for identifying feature importance using the model's attention matrix and the PageRank algorithm, offering insights that enhance our comprehension of established techniques such as DeepLIFT. Our results showcase the efficacy of transformer models in pathogen prediction for food safety and mark a noteworthy contribution to the progress of explainable AI within the biomedical sciences. This study sheds light on the impact of effective farm management practices and highlights the importance of technological advancements in ensuring food safety.

Research on internal leakage detection of the ball valves based on stacking ensemble learning

Abstract Natural gas is an important clean energy source that is mainly transported through pipelines. The ball valve is a crucial piece of control equipment for the pipeline transportation system for natural gas, and the failure of internal leakage of the ball valve will seriously affect the natural gas transmission and increase the risk of sudden safety accidents. In response to the problems of the limitations of a single machine learning model in the traditional ball valve internal leakage rate prediction methods and failure to qualitatively analyze unilateral and bilateral internal leakage recognition of ball valve, a study of ball valve internal leakage detection based on Stacking ensemble learning is proposed. A total of 15 time and frequency domain feature parameters were obtained by feature extraction of 125 and 96 sets of raw acoustic emission signals from the ball valve; the parameters of a single machine learning model were adjusted by Bayesian optimization grid search. An internal leakage rate prediction model and an internal leakage recognition model are constructed, and the proposed model is compared and analyzed with a single model through a field ball valve internal leakage test. The results indicate that the Stacking ensemble learning model outperforms each single machine learning model in terms of SMAPE (17.2583), RMSE (1.1009), and MAE (0.9375) for internal leakage rate prediction. The Stacking ensemble learning model outperformed the single machine learning model in terms of accuracy (1), recall (1), precision (1), FAR(0), and F1-score (1) for internal leakage recognition. Stacking ensemble learning significantly enhances the model's ability to detect internal ball valve leaks.

Carbon emissions trading price forecasts by multi-perspective fusion

<p>The precise prediction of carbon emissions trading prices is the foundation for the stable and sustainable development of the carbon financial market. In recent years, influenced by a combination of factors such as the pandemic, trading regulations, and policies, carbon prices have exhibited strong random volatility and clear non-stationary characteristics. Traditional single-perspective prediction methods based on conventional statistical models are increasingly inadequate due to the homogenization of features and are struggling to adapt to China's regional carbon emissions trading market. Therefore, this paper proposes a multi-perspective fusion-based prediction method tailored to the Chinese market. It leverages carbon emissions trading information from key cities as relevant features to predict the price changes in individual cities. Inspired by the development of artificial intelligence, this paper implements various time series models based on deep neural networks. The effectiveness of the multi-perspective approach is validated through multiple metrics. It provides scientific decision-making tools for domestic carbon emissions trading investors, making a significant contribution to strengthening carbon market risk management and promoting the establishment and rational development of a unified carbon market in China.</p>

Integrating deep learning, satellite image processing, and spatial-temporal analysis for urban flood prediction

Urban flooding is escalating worldwide due to the increasing impervious surfaces from urban developments and frequency of extreme rainfall events by climate change. Traditional flood extent prediction and mapping methods based on physical-based hydrological principles often face limitations due to model complexity and computational burden. In response to these challenges, there has been a notable shift toward satellite image processing and Artificial Intelligence (AI) based approaches, such as Deep Learning (DL) models, including architectures like Convolutional Neural Networks (CNN). The objective of this research is to predict near real-time (NRT) flood extents within urban areas. This research integrated CNN (U-Net) with Sentinel-1 satellite imagery, Digital Elevation Model (DEM), hydrologic soil group (HSG), imperviousness, and rainfall data to create a flood extent prediction model. To detect flooded areas, a binary raster map was created using calibrated backscatter values derived from the VV (vertical transmit and vertical receive) polarization mode of Sentinel-1 imagery, which was highlighted as having a significant impact on backscatter behavior and prediction results. Application of the model was demonstrated in urban areas of Miami-Dade County, Florida. The results demonstrated the capability of the model to provide rapid and accurate flood extent predictions at a spatial resolution of 10 m, with an overall accuracy of 97.05 %, F-1 Score of 92.49 %, and AUC of 93 % in the study area. The U-Net model’s flood predictions were compared with historical floodplain data and then using GIS overlay analysis, resulting in a Ground Truth Index of 84.05 % that shows the accuracy of the model in identifying flooded areas. The research incorporated crucial flood-influencing data (including rainfall) to the flood extent prediction models and expanded the focus models beyond major rainfall events only to encompass a wider range of flood events. The presented NRT flood extent mapping model has a broad range of applicability, including, but not limited to, the continuous monitoring of flood events and their potential impacts on civil infrastructure assets (e.g., construction, operation, and maintenance of roads and bridges), early warning systems for timely evacuation and preparedness measures, and insurance risk assessment.

Ship pitch prediction method based on LSTMC and multi-head attention

When ships sail in the sea, ship motion is unstable and stochasticity due to the ever-changing weather and marine conditions. Aiming to address the issue of low prediction accuracy in traditional prediction methods, short term predictions of ship pitch at 1 step, 3 steps, 6 steps, and 12 steps in advance are conducted, and the impact of input-output ratios on the prediction model is studied. The seven types of data of the marine environment and ship motion are taken as model inputs, with pitch as the model output. When predicting 1 step in advance, a prediction model based on LSTM and multi-head attention (LMA) is proposed. This model firstly uses LSTM layer to extract the temporal feature of the input sequence, and then uses the multi-head attention layer to assign larger weights to the key information to enhance prediction accuracy. When predicting 3 steps, 6 steps, and 12 steps in advance, as the input-output ratio increases, the length of the input sequence keeps growing. The model needs to sufficiently capture the local spatial characteristic and long-range dependency between multi-variable inputs, a deep prediction model based on Conv2d and LMA model is proposed. The model firstly uses two-dimensional convolution layer to extract local spatial feature of long sequences, and then through the attention layer, assign larger weights to temporal information and spatial information that are conducive to continuous prediction in order to enhance prediction accuracy. Research on the input-output ratio indicates that, as the number of predicted steps increases, the appropriate ratio continuously decreases, but the length of the input sequence keeps increasing. Finally, four evaluation metrics are used to evaluate the proposed algorithms on real ship data, and the prediction stability and accuracy of the LMA model and the LCMA model are also verified.

Effect Evaluation of Staged Fracturing and Productivity Prediction of Horizontal Wells in Tight Reservoirs

In this paper, the effect evaluation and production prediction of staged fracturing for horizontal wells in tight reservoirs are studied. Firstly, the basic characteristics and value of horizontal wells in tight reservoirs are introduced, their geological characteristics, flow mechanism and permeability model are analyzed and the application of grey theory in effect analysis is discussed. Considering the problems of staged fracturing effect evaluation and the production prediction of horizontal wells in tight reservoirs, a BP neural network model based on deep learning is proposed. Due to the interference of multiple physical parameters and the complex functional relationship in the development of tight reservoir fracturing, the traditional prediction method has low accuracy and it is difficult to establish an accurate mapping relationship. In this paper, a BP neural network is used to simulate multivariable nonlinear mapping by modifying the model, and its advantages in solving the coupling relationship of complex functions are brought into play. A neural network model with fracturing parameters as input and oil and gas production as output is designed. Through the training and testing of data sets, the accuracy and applicability of the proposed model for effect evaluation and yield prediction are verified. The research results show that the model can fit the complex mapping relationship between fracturing information and production and provide an effective evaluation and prediction tool for the development of the staged fracturing of horizontal wells in tight reservoirs.

Spatio-temporal communication network traffic prediction method based on graph neural network

The function of network traffic prediction plays an important role in many network operations such as security, path planning and congestion control etc. Most traditional traffic prediction methods only consider temporal correlation but ignore spatial correlation, which may result in limited accuracy. In this paper, we propose an effective traffic prediction method based on the graph multi-head attention convolution neural network model, termed as FlowDiviner, which combines graph convolutional network (GCN) and multi-head attention mechanism in its encoder-decoder architecture. Specifically, GCN is used to extract spatial correlation from complex network topologies and multi-head attention mechanism is used to capture dynamic temporal correlations based on monitored traffic behaviors. Meanwhile, a middle attention module is introduced between encoder and decoder to model the relationship between historical and future timesteps of traffic, thus it can alleviate the error accumulation and improve accuracy. The experiments based on both real-life dataset as well as synthetically generated traffic traces show that FlowDiviner can effectively obtain temporal and spatial correlation from the network historical traffic data, and the test results of all metrics are significantly improved from the baseline schemes.

Research on the Short-Term Prediction of Offshore Wind Power Based on Unit Classification

The traditional power prediction methods cannot fully take into account the differences and similarities between units. In the face of the complex and changeable sea climate, the strong coupling effect of atmospheric circulation, ocean current movement, and wave fluctuation, the characteristics of wind processes under different incoming currents and different weather are very different, and the spatio-temporal correlation law of offshore wind processes is highly complex, which leads to traditional power prediction not being able to accurately predict the short-term power of offshore wind farms. Therefore, aiming at the characteristics and complexity of offshore wind power, this paper proposes an innovative short-term power prediction method for offshore wind farms based on a Gaussian mixture model (GMM). This method considers the correlation between units according to the characteristics of the measured data of units, and it divides units with high correlation into a category. The Bayesian information criterion (BIC) and contour coefficient method (SC) were used to obtain the optimal number of groups. The average intra-group correlation coefficient (AICC) was used to evaluate the reliability of measurements for the same quantized feature to select the representative units for each classification. Practical examples show that the short-term power prediction accuracy of the model after unit classification is 2.12% and 1.1% higher than that without group processing, and the mean square error and average absolute error of the short-term power prediction accuracy are reduced, respectively, which provides a basis for the optimization of prediction accuracy and economic operation of offshore wind farms.

Design of a UAV Trajectory Prediction System Based on Multi-Flight Modes

With the burgeoning impact of artificial intelligence on the traditional UAV industry, the pursuit of autonomous UAV flight has emerged as a focal point of contemporary research. Addressing the imperative for advancing critical technologies in autonomous flight, this paper delves into the realm of UAV flight state recognition and trajectory prediction. Presenting an innovative approach focused on improving the precision of unmanned aerial vehicle (UAV) path forecasting via the identification of flight states, this study demonstrates its efficacy through the implementation of two prediction models. Firstly, UAV flight data acquisition was realized in this paper by the use of multi-sensors. Finally, two models for UAV trajectory prediction were designed based on machine learning methods and classical mathematical prediction methods, respectively, and the results before and after flight pattern recognition are compared. The experimental results show that the prediction error of the UAV trajectory prediction method based on multiple flight modes is smaller than the traditional trajectory prediction method in different flight stages.

An Adaptive Similar Scenario Matching Method for Predicting Aircraft Taxiing Time

Accurate prediction of taxiing time is important in ensuring efficient and safe operations on the airport surface. It helps improve ground operation efficiency, reduce fuel waste, and improve carbon emissions at the airport. In actual operations, taxiing time is influenced by various factors, including a large number of categorical features. However, few previous studies have focused on selecting such features. Additionally, traditional taxiing time prediction methods are often black-box models that only provide a single prediction result; they fail to provide effective practical references for controllers. Therefore, this paper analyses the features that affect taxiing time from different data types and forms a taxi feature set consisting of nine key features. We also propose a taxiing time prediction method based on adaptive scenario matching rules. This process classifies the scenarios into multiple typical historical scenario sets and adaptively matches the current target scenario to a typical scenario set based on quantified rules. Then, based on the matching results, a pre-trained model obtained from the corresponding scenario set is used to predict the taxiing time of an aircraft in the target scenario, aiming to mitigate the impact of data heterogeneity on prediction results. Experimental results show that compared to baseline methods, the mean absolute error and root mean square error of the proposed method decreased by 4.8% and 12.6%, respectively. This method significantly reduces the fluctuations in results caused by sample heterogeneity and enhances controllers’ acceptance of prediction results from the model. It can be used to further improve auxiliary decision making systems and enhance the precise control capabilities of airport surface operations.

Multimodal Shot Prediction Based on Spatial-Temporal Interaction between Players in Soccer Videos

Sports data analysis has significantly advanced and become an indispensable technology for planning strategy and enhancing competitiveness. In soccer, shot prediction has been realized on the basis of historical match situations, and its results contribute to the evaluation of plays and team tactics. However, traditional event prediction methods required tracking data acquired with expensive instrumentation and event stream data annotated by experts, and the benefits were limited to only some professional athletes. To tackle this problem, we propose a novel shot prediction method using soccer videos. Our method constructs a graph considering player relationships with audio and visual features as graph nodes. Specifically, by introducing players’ importance into the graph edge based on their field positions and team information, our method enables the utilization of knowledge that reflects the detailed match situation. Next, we extract latent features considering spatial–temporal interactions from the graph and predict event occurrences with uncertainty based on the probabilistic deep learning method. In comparison with several baseline methods and ablation studies using professional soccer match data, our method was confirmed to be effective as it demonstrated the highest average precision of 0.948, surpassing other methods.

Anomaly Detection Using Machine Learning in Hydrochemical Data From Hot Springs: Implications for Earthquake Prediction

AbstractThis study explores the potential of machine learning algorithms for earthquake prediction, utilizing fluid chemical anomaly data from hot springs. Six hot springs, located within an active fault zone along the southeastern coast of China, were carefully chosen as hydrochemical monitoring sites for an extended period of two and a half years. Using this data, a prediction model integrating six algorithms was developed to forecast M ≥ 5 earthquakes in Taiwan. The model's performance was validated against recorded earthquake events, and the factors influencing its predictive capability were analyzed. Our comprehensive analysis conclusively demonstrates the superiority of machine learning algorithms over traditional statistical methods for earthquake prediction. Additionally, including sampling time in the data sets significantly improves the model's predictive performance. However, it is important to note that the model's predictive performance varies across different hot spring and indicators type, highlighting the importance of identifying optimal indicators for specific scenarios. The model parameters, including the anomaly detection rate (P) and earthquake response time threshold (M), significantly impact the model's predictive capabilities. Therefore, adjustments are needed to optimize the model's performance for practical use. Despite limitations such as the inability to differentiate pre‐earthquake anomalies from post‐earthquake anomalies and pinpoint the precise location of earthquakes, this study successfully showcases the potential of machine learning algorithms in earthquake prediction, paving the way for further research and improved prediction methods.

Multi-channel deterioration trend prediction of pumping storage unit based on temporal graph convolution network

Abstract The safe and stable operation of pumped storage units is related to the integrity of the power grid, and it is necessary to study the prediction of unit deterioration trends. Traditional degradation trend prediction methods are mostly for single channel, without considering the interaction between different channels of the shaft system. Therefore, a multi-channel degradation trend prediction model for pumped storage units shaft system is proposed. Firstly, the healthy model is obtained by constructing the mapping relationship between the working parameters and the swing of the shaft system through the fully connected network. Secondly, the deterioration trend sequence of each channel is calculated based on the historical monitoring data of each channel of the unit shaft system and the corresponding healthy model. Finally, the correlation coefficient analysis method is used to calculate the correlation between each channel, and the temporality of the degradation sequence of each channel and the correlation between channels are learned through the spatio-temporal graph convolutional network to realize the prediction of multi-channel degradation trends. The proposed model is verified by using the data of a pumped storage power station. The results show that the proposed model can accurately predict the multi-channel deterioration trend.

Development of a Cost Prediction Model for Design Changes: Case of Korean Apartment Housing Projects

Apartment buildings are significantly popular among South Korean construction companies. However, design changes present a common yet challenging aspect, often leading to cost overruns. Traditional cost prediction methods, which primarily rely on numerical data, have a gap in fully capitalizing on the rich insights that textual descriptions of design changes offer. Addressing this gap, this research employs machine learning (ML) and natural language processing (NLP) techniques, analyzing a dataset of 35,194 instances of design changes from 517 projects by a major public real estate developer. The proposed models demonstrate acceptable performance, with R-square values ranging from 0.930 to 0.985, underscoring the potential of integrating structured and unstructured data for enhanced predictive analytics in construction project management. The predictor using Extreme Gradient Boosting (XGB) shows better predictive ability (R2 = 0.930; MAE = 16.05; RMSE = 75.09) compared to the traditional Multilinear Regression (MLR) model (R2 = 0.585; MAE = 43.85; RMSE = 101.41). For whole project cost changes predictions, the proposed models exhibit good predictive ability, both including price fluctuations (R2 = 0.985; MAE = 605.1; RMSE = 1009.5) and excluding price fluctuations (R2 = 0.982; MAE = 302.1; RMSE = 548.5). Additionally, a stacked model combining CatBoost and Support Vector Machine (SVM) algorithms was developed, showcasing the effective prediction of cost changes, with or without price fluctuations.

FusPB-ESM2: Fusion model of ProtBERT and ESM-2 for cell-penetrating peptide prediction

Cell-penetrating peptides have attracted much attention for their ability to break through cell membrane barriers, which can improve drug bioavailability, reduce side effects, and promote the development of gene therapy. Traditional wet-lab prediction methods are time-consuming and costly, and computational methods provide a short-time and low-cost alternative. Still, the accuracy and reliability need to be further improved. To solve this problem, this study proposes a feature fusion-based prediction model, where the protein pre-trained language models ProtBERT and ESM-2 are used as feature extractors, and the extracted features from both are fused to obtain a more comprehensive and effective feature representation, which is then predicted by linear mapping. Validated by many experiments on public datasets, the method has an AUC value as high as 0.983 and shows high accuracy and reliability in cell-penetrating peptide prediction.

Verification of Long‐Term Ensemble Evapotranspiration Hindcast Using a Conditional Nonlinear Optimal Parameter Perturbation Ensemble Prediction Method on the Tibetan Plateau

AbstractIn this research, a conditional nonlinear optimal perturbation related to parameters (CNOP‐P) method is employed to propose an ensemble prediction method titled as the conditional nonlinear optimal perturbation related to parameters ensemble prediction (CNOP‐PEP) method. Within the CNOP‐PEP method, all ensemble members are generated by the CNOP‐P method. To explore the operability and validity of the CNOP‐PEP method, long‐term evapotranspiration (ET) hindcast skill is evaluated at 13 sites on the Tibetan Plateau (TP) during the period of 2001–2018 with a land surface model. Two traditional ensemble prediction methods (the stochastically perturbed parameters (SPP) method and the one‐at‐a‐time (OAT) method) are also applied to assess the ET hindcast skills. The numerical results indicate that the CNOP‐PEP method shows the best hindcast skill for estimated ET at 13 stations on the TP among the three methods (CNOP‐PEP, OAT, and SPP methods) during the period of 2001–2018. This suggests that the CNOP‐PEP method is helpful and effective for long‐term and interannual hydrological predictions, such as long‐term and interannual ET predictions. The numerical results also indicate that ensemble members with certain special properties of fast‐growing types should be selected to implement ensemble prediction compared to these ordinary and traditional samples. The CNOP‐PEP method could supply special samples to improve the ensemble prediction and hindcast skill of hydrological cycle.

Machine learning for clinical outcome prediction in cerebrovascular and endovascular neurosurgery: systematic review and meta-analysis

BackgroundMachine learning (ML) may be superior to traditional methods for clinical outcome prediction. We sought to systematically review the literature on ML for clinical outcome prediction in cerebrovascular and endovascular...

Enhancing Soil Moisture Forecasting Accuracy with REDF-LSTM: Integrating Residual En-Decoding and Feature Attention Mechanisms

This study introduces an innovative deep learning model, Residual-EnDecode-Feedforward Attention Mechanism-Long Short-Term Memory (REDF-LSTM), designed to overcome the high uncertainty challenges faced by traditional soil moisture prediction methods. The REDF-LSTM model, by integrating a residual learning encoder–decoder LSTM layer, enhanced LSTM layers, and feedforward attention, not only captures the deep features of time series data but also optimizes the model’s ability to identify key influencing factors, including land surface features, atmospheric conditions, and other static environmental variables. Unlike existing methods, the innovation of this model lies in its first-time combination of the residual learning encoder–decoder and feedforward attention mechanisms in the soil moisture prediction field. It delves into the complex patterns of time series through the encoder–decoder structure and accurately locates key influencing factors through the feedforward attention mechanism, significantly improving predictive performance. The choice to combine the feedforward attention mechanism and encoder–decoder with the LSTM model is to fully leverage their advantages in processing complex data sequences and enhancing the model’s focus on important features, aiming for more accurate soil moisture prediction. After comparison with current advanced models such as EDLSTM, FAMLSTM, and GANBiLSTM, our REDF-LSTM demonstrated the best performance. Compared to traditional LSTM models, it achieved an average improvement of 13.07% in R2, 20.98% in RMSE, 24.86% in BIAS, and 11.1% in KGE key performance indicators, fully proving its superior predictive capability and potential application value in precision agriculture and ecosystem management.

Prediction Technology of a Reservoir Development Model While Drilling Based on Machine Learning and Its Application

In order to further understand the complex spatial distribution caused by the extremely strong heterogeneity of buried hill reservoirs, this paper proposes a new method for predicting the development pattern of buried hill reservoirs based on the traditional pre-drilling prediction and post-drilling evaluation methods that mainly rely on seismic, logging, and core data, which are difficult to meet the timeliness and accuracy of drilling operations. Firstly, the box method and normalization formula are used to process and normalize the abnormal data of element logging and engineering logging, and then the stepwise regression analysis method is used to optimize the sensitive parameters of element logging and engineering logging. The Light Gradient Boosting Machine (LightGBM) algorithm, deep neural network (DNN), and support vector machine (SVM) are used to establish a new method for predicting the development pattern of buried hill reservoirs. Lastly, a comprehensive evaluation index F1 score for the model is established to evaluate the prediction model for the development pattern of buried hill reservoirs. The F1 score value obtained from this model’s comprehensive evaluation index indicates that the LightGBM model achieves the highest accuracy, with 96.7% accuracy in identifying weathered zones and 95.8% accuracy in identifying interior zones. The practical application demonstrates that this method can rapidly and accurately predict the development mode of buried hill reservoirs while providing a new approach for efficient on-site exploration and decision-making in oil and gas field developments. Consequently, it effectively promotes exploration activities as well as enhances the overall process of oil and gas reservoir exploration.

MST-GAT: A multi-perspective spatial-temporal graph attention network for multi-sensor equipment remaining useful life prediction

Graph neural networks (GNNs) have been applied as an emerging technique for remaining useful life (RUL) prediction due to their ability to effectively exploit the dependencies between multi-sensor data. Despite existing GNN-based RUL prediction methods have achieved promising results, they do not consider the spatial-temporal dependencies of multi-sensor data from different perspectives (global and local), making it difficult to fully utilize the useful information in multi-sensor data. Secondly, traditional point prediction method can only provide the possible future RUL prediction value, but cannot quantify the uncertainty of RUL prediction, making it difficult to provide more information for decision-making. To address the above challenges, we propose a multi-perspective spatial-temporal graph attention network (MST-GAT) for multi-sensor equipment RUL prediction. MST-GAT constructs a multi-perspective graph attention network (MP-GAT) and a temporal convolutional network (TCN) to capture complex spatial-temporal dependencies in multi-sensor data. Specifically, MP-GAT employs a multi-head self-attention mechanism and two multi-head explicit sparse self-attention mechanisms to explicitly model global and local spatial dependencies, and feeds multi-perspective spatial information into TCN to capture temporal dependencies. Furthermore, a joint optimization network is constructed for predicting RUL and its prediction interval to quantify uncertainty. Experiments on three benchmark datasets demonstrate that MST-GAT outperforms state-of-the-art baselines in RUL prediction and can accurately quantify uncertainty.