Prediction Performance Of Model Research Articles

An attention-based deep learning model for phase-resolved wave prediction

Abstract Phase-resolved wave prediction capability, even if only over two wave periods in advance, is of value for optimal control of wave energy converters (WECs), resulting in a dramatic increase in power generation efficiency. Previous studies on wave-by-wave predictions have shown that an Artificial Neural Network (ANN) model can outperform the traditional linear wave theory-based model in terms of both prediction accuracy and prediction horizon when using synthetic wave data. However, the prediction performance of ANN models is significantly reduced by the varying wave conditions and buoy positions that occur in the field. To overcome these limitations, a novel wave prediction method is developed based on the neural network with an attention mechanism. This study validates the new model using wave data measured at sea. The model utilizes past time histories of three Sofar Spotter wave buoys at upwave locations to predict the vertical motion of a Datawell Waverider-4 at a downwave location. The results show that the attention-based neural network model is capable of capturing the slow variation in the displacement of the buoys, which reduces the prediction error compared to a standard ANN and Long Short-Term Memory (LSTM) model.

Seasonal-Spatial Distribution Variations and Predictions of Loliolus beka and Loliolus uyii in the East China Sea Region: Implications from Climate Change Scenarios

Global climate change profoundly impacts the East China Sea ecosystem and poses a major challenge to fishery management in this region. In addition, closely related species with low catches are often not distinguished in fishery production and relevant data are commonly merged in statistics and fishing logbooks, making it challenging to accurately predict their habitat distribution range. Here, merged fisheries-independent data of the closely related squid Loliolus beka (Sasaki, 1929) and Loliolus uyii (Wakiya and Ishikawa, 1921) were used to explore the construction and prediction performance of species distribution models. Data in 2018 to 2019 from the southern Yellow and East China Seas were used to identify the seasonal–spatial distribution characteristics of both species, revealing a boundary line at 29.00° N for L. uyii during the autumn, with the highest average individual weight occurring during the summer, with both larvae and juveniles occurring during the autumn. Thus, the life history of L. uyii can be divided into winter–spring nursery and summer–autumn spawning periods. L. beka showed a preference for inshore areas (15–60 m) during the summer and offshore areas (32.00–78.00 m) during the winter. High-value areas of both species included inshore areas of the southern Yellow and mid-East China Seas during the autumn, enlarging during the spring to include central areas of the survey region, before significantly decreasing during the summer. Therefore, this study provides both a novel perspective for modeling biological habitat distribution with limited data and a scientific basis for the adjustment of fishery resource management and conservation measures in the context of climate change.

Deep learning model based on endoscopic images predicting treatment response in locally advanced rectal cancer undergo neoadjuvant chemoradiotherapy: a multicenter study

PurposeNeoadjuvant chemoradiotherapy has been the standard practice for patients with locally advanced rectal cancer. However, the treatment response varies greatly among individuals, how to select the optimal candidates for neoadjuvant chemoradiotherapy is crucial. This study aimed to develop an endoscopic image-based deep learning model for predicting the response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer.MethodsIn this multicenter observational study, pre-treatment endoscopic images of patients from two Chinese medical centers were retrospectively obtained and a deep learning-based tumor regression model was constructed. Treatment response was evaluated based on the tumor regression grade and was defined as good response and non-good response. The prediction performance of the deep learning model was evaluated in the internal and external test sets. The main outcome was the accuracy of the treatment prediction model, measured by the AUC and accuracy.ResultsThis deep learning model achieved favorable prediction performance. In the internal test set, the AUC and accuracy were 0.867 (95% CI: 0.847–0.941) and 0.836 (95% CI: 0.818–0.896), respectively. The prediction performance was fully validated in the external test set, and the model had an AUC of 0.758 (95% CI: 0.724–0.834) and an accuracy of 0.807 (95% CI: 0.774–0.843).ConclusionThe deep learning model based on endoscopic images demonstrated exceptional predictive power for neoadjuvant treatment response, highlighting its potential for guiding personalized therapy.

The use of image classification to estimate flamingo abundance from aerial, drone and satellite imagery

The Lesser Flamingo Phoeniconaias minor is a Near Threatened species known to be highly gregarious and that can concentrate in large numbers at core foraging and breeding sites, yet is also known to disperse widely in search of suitable foraging habitat. The density of flamingos at occupied sites, together with frequent dispersal between foraging sites, poses significant challenges to obtaining reliable counts and associated population estimates. Our study tested the use of unmanned aerial vehicles (drones) imagery and high-resolution satellite imagery for producing density estimates of Lesser Flamingos at the only known breeding site in South Africa (Kamfers Dam, Northern Cape Province). We implemented a supervised classification approach using multiple supervised classification algorithms, including xgbTree, randomForest, nnet and GBM. The four supervised classification models performed very well at classifying flamingos from both drone and satellite imagery, with classification accuracies ranging between 0.98 and 0.99. The inclusion of an additional predictor (other than RGB bands) yielded the highest variable importance score and greatly increased model accuracy and predictive performance. Estimates of adult flamingos from drone imagery acquired across nesting mudflats on 5 February (n = 1 320) and 14 February (n = 330) 2019 suggested a 75% decline in incubating adults. Furthermore, a total of 16 134 (15 973–16 295) flamingos were estimated from drone imagery on 1 March 2019, and 17 291 (17 118–17 290) flamingos from satellite imagery on 19 March 2019. The modelled size of the creche across dates suggested a loss of 234 birds (4.5%) during the period of assessment, with the highest count estimating 5 218 juveniles. Our study successfully implemented machine learning classifiers that were able to detect and derive highly accurate counts of Lesser Flamingos from both drone and satellite imagery. This use of machine learning could greatly improve the efficiency of monitoring and associated conservation efforts underway for the Lesser Flamingo.

The use of image classification to estimate flamingo abundance from aerial, drone and satellite imagery

The Lesser Flamingo Phoeniconaias minor is a Near Threatened species known to be highly gregarious and that can concentrate in large numbers at core foraging and breeding sites, yet is also known to disperse widely in search of suitable foraging habitat. The density of flamingos at occupied sites, together with frequent dispersal between foraging sites, poses significant challenges to obtaining reliable counts and associated population estimates. Our study tested the use of unmanned aerial vehicles (drones) imagery and high-resolution satellite imagery for producing density estimates of Lesser Flamingos at the only known breeding site in South Africa (Kamfers Dam, Northern Cape Province). We implemented a supervised classification approach using multiple supervised classification algorithms, including xgbTree, randomForest, nnet and GBM. The four supervised classification models performed very well at classifying flamingos from both drone and satellite imagery, with classification accuracies ranging between 0.98 and 0.99. The inclusion of an additional predictor (other than RGB bands) yielded the highest variable importance score and greatly increased model accuracy and predictive performance. Estimates of adult flamingos from drone imagery acquired across nesting mudflats on 5 February (n = 1 320) and 14 February (n = 330) 2019 suggested a 75% decline in incubating adults. Furthermore, a total of 16 134 (15 973–16 295) flamingos were estimated from drone imagery on 1 March 2019, and 17 291 (17 118–17 290) flamingos from satellite imagery on 19 March 2019. The modelled size of the creche across dates suggested a loss of 234 birds (4.5%) during the period of assessment, with the highest count estimating 5 218 juveniles. Our study successfully implemented machine learning classifiers that were able to detect and derive highly accurate counts of Lesser Flamingos from both drone and satellite imagery. This use of machine learning could greatly improve the efficiency of monitoring and associated conservation efforts underway for the Lesser Flamingo.

Development of clinical prediction model to guide the use of CT head scans for non-traumatic Thai patient with seizure: A cross-sectional study.

The aim of this study was to develop clinical predictor tools for guiding the use of computed tomography (CT) head scans in non-traumatic Thai patients presented with seizure. A prediction model using a retrospective cross-sectional design was conducted. We recruited adult patients (aged ≥ 18 years) who had been diagnosed with seizures by their physicians and had undergone CT head scans for further investigation. Positive CT head defined as the presence of any new lesion that related to the patient's presented seizure officially reported by radiologist. A total of 9 candidate predictors were preselected. The prediction model was developed using a full multivariable logistic regression with backward stepwise elimination. We evaluated the model's predictive performance in terms of its discriminative ability and calibration via AuROC and calibration plot. The application was then constructed based on final model. A total of 362 patients were included into the analysis which comprising of 71 patients with positive CT head findings and 291 patients with normal results. Six final predictors were identified including: Glasgow coma scale, the presence of focal neurological deficit, history of malignancy, history of CVA, Epilepsy, and the presence of alcohol withdrawal symptom. In terms of discriminative ability, the final model demonstrated excellent performance (AuROC of 0.82 (95% CI: 0.76-0.87)). The calibration plot illustrated a good agreement between observed and predicted risks. This prediction model offers a reliable tool for effectively reduce unnecessary use and instill confidence in supporting physicians in determining the need for CT head scans in non-traumatic patients with seizures.

Evaluation of hydrogen production via steam reforming and partial oxidation of dimethyl ether using response surface methodology and artificial neural network

This study aims to develop two models for thermodynamic data on hydrogen generation from the combined processes of dimethyl ether steam reforming and partial oxidation, applying artificial neural networks (ANN) and response surface methodology (RSM). Three factors are recognized as important determinants for the hydrogen and carbon monoxide mole fractions. The RSM used the quadratic model to formulate two correlations for the outcomes. The ANN modeling used two algorithms, namely multilayer perceptron (MLP) and radial basis function (RBF). The optimum configuration for the MLP, employing the Levenberg–Marquardt (trainlm) algorithm, consisted of three hidden layers with 15, 10, and 5 neurons, respectively. The ideal RBF configuration contained a total of 80 neurons. The optimum configuration of ANN achieved the best mean squared error (MSE) performance of 3.95e−05 for the hydrogen mole fraction and 4.88e−05 for the carbon monoxide mole fraction after nine epochs. Each of the ANN and RSM models produced accurate predictions of the actual data. The prediction performance of the ANN model was 0.9994, which is higher than the RSM model's 0.9771. The optimal condition was obtained at O/C of 0.4, S/C of 2.5, and temperature of 250 °C to achieve the highest H2 production with the lowest CO emission.

Clinical value identification of RDW on in-hospital death in unruptured abdominal aortic aneurysm.

This study aimed to identify highly valuable blood indicators for predicting the clinical outcomes of patients with aortic aneurysms (AA). Baseline data of 1180 patients and 16 blood indicators were obtained from the public Medical Information Mart for Intensive Care-IV (MIMIC-IV) database. The association of blood indicators with 4 types of clinical outcomes was analyzed, and the prediction performance of core indicators on different outcomes was next evaluated. Then, we explored the detailed association between core indicators and key outcomes among subgroups. Finally, a machine learning model was established to improve the prediction performance. Generalized linear regression analysis indicated that only red cell volume distribution width (RDW) was commonly associated with 4 end-points including surgery requirement, ICU stay requirement, length of hospital stay, and in-hospital death (all P < .05). Further, RDW showed the best performance for predicting in-hospital death by receiver operating characteristic (ROC) analysis. The significant association between RDW and in-hospital death was then determined by 3 logistic regression models adjusting for different variables (all P < .05). Stratification analysis showed that their association was mainly observed in unruptured AA and abdominal AA (AAA, all P < .05). We subsequently established an RDW-based model for predicting the in-hospital death only in patients with unruptured AAA. The favorable prediction performance of the RDW-based model was verified in training, validation, and test sets. RDW was found to make the greatest contribution to in-hospital death within the model. RDW had favorable clinical value for predicting the in-hospital death of patients, especially in unruptured AAA.

A prediction model for stillbirth based on first trimester pre-eclampsia combined screening.

To evaluate the accuracy of combined models of maternal biophysical factors, ultrasound, and biochemical markers for predicting stillbirths. A retrospective cohort study of pregnant women undergoing first-trimester pre-eclampsia screening at 11-13 gestational weeks was conducted. Maternal characteristics and history, mean arterial pressure (MAP) measurement, uterine artery pulsatility index (UtA-PI) ultrasound, maternal ophthalmic peak ratio Doppler, and placental growth factor (PlGF) serum were collected during the visit. Stillbirth was classified as placental dysfunction-related when it occurred with pre-eclampsia or birth weight <10th percentile. Combined prediction models were developed from significant variables in stillbirths, placental dysfunction-related, and controls. We used the area under the receiver-operating-characteristics curve (AUC), sensitivity, and specificity based on a specific cutoff to evaluate the model's predictive performance by measuring the capacity to distinguish between stillbirths and live births. There were 13 (0.79%) cases of stillbirth in 1643 women included in the analysis. The combination of maternal factors, MAP, UtA-PI, and PlGF, significantly contributed to the prediction of stillbirth. This model was a good predictor for all (including controls) types of stillbirth (AUC 0.879, 95% CI: 0.799-0.959, sensitivity of 99.3%, specificity of 38.5%), and an excellent predictor for placental dysfunction-related stillbirth (AUC 0.984, 95% CI: 0.960-1.000, sensitivity of 98.5, specificity of 85.7). Screening at 11-13 weeks' gestation by combining maternal factors, MAP, UtA-PI, and PlGF, can predict a high proportion of stillbirths. Our model has good accuracy for predicting stillbirths, predominantly placental dysfunction-related stillbirths.

Fast retrieval of XCO2 over east Asia based on Orbiting Carbon Observatory-2 (OCO-2) spectral measurements

Abstract. The increase in greenhouse gas concentrations, particularly CO2, has significant implications for global climate patterns and various aspects of human life. Spaceborne remote sensing satellites play a crucial role in high-resolution monitoring of atmospheric CO2. However, the next generation of greenhouse gas monitoring satellites is expected to face challenges, particularly in terms of computational efficiency in atmospheric CO2 retrieval and analysis. To address these challenges, this study focuses on improving the speed of retrieving the column-averaged dry-air mole fraction of carbon dioxide (XCO2) using spectral data from the Orbiting Carbon Observatory-2 (OCO-2) satellite while still maintaining retrieval accuracy. A novel approach based on neural network (NN) models is proposed to tackle the nonlinear inversion problems associated with XCO2 retrievals. The study employs a data-driven supervised learning method and explores two distinct training strategies. Firstly, training is conducted using experimental data obtained from the inversion of the operational optimization model, which is released as the OCO-2 satellite products. Secondly, training is performed using a simulated dataset generated by an accurate forward calculation model. The inversion performance and prediction performance of the machine learning model for XCO2 are compared, analyzed, and discussed for the observed region over east Asia. The results demonstrate that the model trained on simulated data accurately predicts XCO2 in the target area. Furthermore, when compared to OCO-2 satellite product data, the developed XCO2 retrieval model not only achieves rapid predictions (&lt;1 ms) with good accuracy (1.8 ppm or approximately 0.45 %) but also effectively captures sudden increases in XCO2 plumes near industrial emission sources. The accuracy of the machine learning model retrieval results is validated against reliable data from Total Carbon Column Observing Network (TCCON) sites, demonstrating its ability to effectively capture CO2 seasonal variations and annual growth trends.

Homogeneous Ensemble Feature Selection for Mass Spectrometry Data Prediction in Cancer Studies

The identification of important proteins is critical for the medical diagnosis and prognosis of common diseases. Diverse sets of computational tools have been developed for omics data reduction and protein selection. However, standard statistical models with single-feature selection involve the multi-testing burden of low power with limited available samples. Furthermore, high correlations among proteins with high redundancy and moderate effects often lead to unstable selections and cause reproducibility issues. Ensemble feature selection in machine learning (ML) may identify a stable set of disease biomarkers that could improve the prediction performance of subsequent classification models and thereby simplify their interpretability. In this study, we developed a three-stage homogeneous ensemble feature selection (HEFS) approach for both identifying proteins and improving prediction accuracy. This approach was implemented and applied to ovarian cancer proteogenomics datasets comprising (1) binary putative homologous recombination deficiency (HRD)- positive or -negative samples; (2) multiple mRNA classes (differentiated, proliferative, immunoreactive, mesenchymal, and unknown samples). We conducted and compared various ML methods with HEFS including random forest (RF), support vector machine (SVM), and neural network (NN) for predicting both binary and multiple-class outcomes. The results indicated that the prediction accuracies varied for both binary and multiple-class classifications using various ML approaches with the proposed HEFS method. RF and NN provided better prediction accuracies than simple Naive Bayes or logistic models. For binary outcomes, with a sample size of 122 and nine selected prediction proteins using our proposed three-stage HEFS approach, the best ensemble ML (Treebag) achieved 83% accuracy, 85% sensitivity, and 81% specificity. For multiple (five)-class outcomes, the proposed HEFS-selected proteins combined with Principal Component Analysis (PCA) in NN resulted in prediction accuracies for multiple-class classifications ranging from 75% to 96% for each of the five classes. Despite the different prediction accuracies of the various models, HEFS identified consistent sets of proteins linked to the binary and multiple-class outcomes.

A deep neural network prediction method for diabetes based on Kendall's correlation coefficient and attention mechanism.

Diabetes is a chronic disease, which is characterized by abnormally high blood sugar levels. It may affect various organs and tissues, and even lead to life-threatening complications. Accurate prediction of diabetes can significantly reduce its incidence. However, the current prediction methods struggle to accurately capture the essential characteristics of nonlinear data, and the black-box nature of these methods hampers its clinical application. To address these challenges, we propose KCCAM_DNN, a diabetes prediction method that integrates Kendall's correlation coefficient and an attention mechanism within a deep neural network. In the KCCAM_DNN, Kendall's correlation coefficient is initially employed for feature selection, which effectively filters out key features influencing diabetes prediction. For missing values in the data, polynomial regression is utilized for imputation, ensuring data completeness. Subsequently, we construct a deep neural network (KCCAM_DNN) based on the self-attention mechanism, which assigns greater weight to crucial features affecting diabetes and enhances the model's predictive performance. Finally, we employ the SHAP model to analyze the impact of each feature on diabetes prediction, augmenting the model's interpretability. Experimental results show that KCCAM_DNN exhibits superior performance on both PIMA Indian and LMCH diabetes datasets, achieving test accuracies of 99.090% and 99.333%, respectively, approximately 2% higher than the best existing method. These results suggest that KCCAM_DNN is proficient in diabetes prediction, providing a foundation for informed decision-making in the diagnosis and prevention of diabetes.

Predicting risk of the subsequent early pregnancy loss in women with recurrent pregnancy loss based on preconception data

BackgroundFor women who have experienced recurrent pregnancy loss (RPL), it is crucial not only to treat them but also to evaluate the risk of recurrence. The study aimed to develop a risk predictive model to predict the subsequent early pregnancy loss (EPL) in women with RPL based on preconception data.MethodsA prospective, dynamic population cohort study was carried out at the Second Hospital of Lanzhou University. From September 2019 to December 2022, a total of 1050 non-pregnant women with RPL were participated. By December 2023, 605 women had subsequent pregnancy outcomes and were randomly divided into training and validation group by 3:1 ratio. In the training group, univariable screening was performed on RPL patients with subsequent EPL outcome. The least absolute shrinkage and selection operator (LASSO) regression and multivariate logistic regression were utilized to select variables, respectively. Subsequent EPL prediction model was constructed using generalize linear model (GLM), gradient boosting machine (GBM), random forest (RF), and deep learning (DP). The variables selected by LASSO regression and multivariate logistic regression were then established and compared using the best prediction model. The AUC, calibration curve, and decision curve (DCA) were performed to assess the prediction performances of the best model. The best model was validated using the validation group. Finally, a nomogram was established based on the best predictive features.ResultsIn the training group, the GBM model achieved the best performance with the highest AUC (0.805). The AUC between the variables screened by the LASSO regression (16-variables) and logistic regression (9-variables) models showed no significant difference (AUC: 0.805 vs. 0.777, P = 0.1498). Meanwhile, the 9-variable model displayed a well discrimination performance in the validation group, with an AUC value of 0.781 (95%CI 0.702, 0.843). The DCA showed the model performed well and was feasible for making beneficial clinical decisions. Calibration curves revealed the goodness of fit between the predicted values by the model and the actual values, the Hosmer–Lemeshow test was 7.427, and P = 0.505.ConclusionsPredicting subsequent EPL in RPL patients using the GBM model has important clinical implications. Future prospective studies are needed to verify the clinical applicability.Trial registrationThis study was registered in the Chinese Clinical Trial Registry with the registration number of ChiCTR2000039414 (27/10/2020).

Combining wavelet-enhanced feature selection and deep learning techniques for multi-step forecasting of urban water demand

Abstract Urban water demand (UWD) forecasting is essential for water supply network optimization and management, both in business-as-usual scenarios, as well as under external climate and socio-economic stressors. Different machine learning and deep learning models have shown promising forecasting skills in various areas of application. However, their potential to forecast multi-step ahead UWD has not been fully explored. Modelling uncertain UWD patterns and accounting for variations in water demand behaviors require techniques that can extract time-varying information and multi-scale changes. In this research, we comparatively investigate different state-of-the-art machine learning- and deep learning-based predictive models on 1-day- and 7-day-ahead UWD forecasting, using daily demand data from the city of Milan, Italy. The contribution of this paper is two-fold. First, we compare the forecasting performance of different machine learning and deep learning models on single- and multi-step daily UWD forecasting. These models include Artificial Neural Network (ANN), Support Vector Regression (SVR), Light Gradient Boosting Machine (LightGBM), and Long Short-Term Memory network with and without an attention mechanism (LSTM and AM-LSTM). We benchmark their prediction accuracy against autoregressive time series models. Second, we investigate the potential enhancement in predictive accuracy by incorporating the wavelet transform and feature selection performed by LightGBM into these models. Results show that, overall, wavelet-enhanced feature selection improves the model predictive performance. The hybrid model combining wavelet-enhanced feature selection via LightGBM with LSTM (WT-LightGBM-(AM)-LSTM) can achieve high levels of accuracy with Nash-Sutcliffe Efficiency larger than 0.95 and Kling–Gupta Efficiency higher than 0.93 for both 1-day- and 7-day-ahead UWD forecasts. Furthermore, performance is shown to be robust under the influence of external stressors causing sudden changes in UWD.

Computer vision for eye diseases detection using pre‐trained deep learning techniques and raspberry Pi

AbstractEarly diagnosis of eye diseases is very important to prevent visual impairment and guide appropriate treatment methods. This paper presents a unique approach that can detect numerous eye diseases automatically. Initially, this approach used the pre‐trained ImageNet models that provides various pre‐trained models for training the acquired data. The existing data sets are composed of 645 data images acquired clinically, represented by two groups of subjects as healthy and others holding the proposed eye defect like cataracts, foreign bodies, glaucoma, subconjunctival haemorrhage, and viral conjunctivitis. Followed by comparisons of the pre‐trained model's coefficients and prediction performance. Later, the first‐class execution model is integrated within the Raspberry Pi staging and the real‐time digital camera detection. The evaluation process used the confusion matrix, model accuracy, precision factor, recall coefficient, F1 score, and the Matthews Correlation Coefficient (MCC). Resulting in the performance of these pre‐trained ImageNet models used in this study represented by 93% (InceptionResNetV2), 90% (MobileNet), 86% (Residual Network ResNet50), 85% (InceptionV3), 78% (Visual Geometry Group VGG19), and 72% (Neural Architecture Search Network NASNetMobile). The results show that the InceptionResNetV2 achieved the highest performance. This proposed approach shows its efficiency and strength by early detection of the subject's unhealthy eyes through real‐time monitoring in the field of ophthalmology.

Multi-site solar irradiance prediction based on hybrid spatiotemporal graph neural network

Constructing accurate spatiotemporal correlations is a challenging task in joint prediction of multiple photovoltaic sites. Some advanced algorithms for incorporating other surrounding site information have been proposed, such as graph neural network-based methods, which are usually based on static or dynamic graphs to build spatial dependencies between sites. However, the possibility of the simultaneous existence of multiple spatial dependencies is not considered. This paper establishes a spatiotemporal prediction model based on hybrid spatiotemporal graph neural network. In this model, we apply adaptive hybrid graph learning to learn composite spatial correlations among multiple sites. A temporal convolution module with multi-subsequence temporal data input is used to extract local semantic information to better predict future nonlinear temporal dependencies. A spatiotemporal adaptive fusion module is added to address the issue of integrating diverse spatiotemporal trends among multiple sites. To assess the model's predictive performance, nine solar radiation observation stations were selected in two different climatic environments. The average root mean square error (RMSE) of the constructed model was 38.51 and 49.90 W/m2, with average mean absolute error (MAE) of 14.72 and 23.06 W/m2, respectively. Single-site and multi-site prediction models were selected as baseline models. Compared with the baseline models, the RMSE and MAE reduce by 3.1%–20.8% and 8.9%–32.8%, respectively, across all sites. The proposed model demonstrates the effectiveness of improving accuracy in forecasting solar irradiance through multi-site predictions.

Hybrid modeling of T-shaped partial least squares regression and transfer learning for formulation and manufacturing process development of new drug products

T-shaped partial least squares regression (T-PLSR) is a valuable machine learning technique for the formulation and manufacturing process development of new drug products. An accurate T-PLSR model requires experimental data with multiple formulations and process conditions. However, it is usually challenging to collect comprehensive experimental data using large-scale manufacturing equipment because of the cost, time, and large consumption of raw materials. This study proposes a hybrid modeling of T-PLSR and transfer learning (TL) to enhance the prediction performance of a T-PLSR model for large-scale manufacturing data by exploiting a large amount of small-scale manufacturing data for model building. The proposed method of T-PLSR+TL was applied to a practical case study focusing on scaling up the tableting process from an experienced compaction simulator to a less-experienced rotary tablet press. The T-PLSR+TL models achieved significantly better prediction performance for tablet quality attributes of new drug products than T-PLSR models without using large-scale manufacturing data with new drug products. The results demonstrated that T-PLSR+TL is more capable of addressing new drug products than T-PLSR by using small-scale manufacturing data to cover a scarcity of large-scale manufacturing data. Furthermore, T-PLSR+TL holds the potential to streamline formulation and manufacturing process development activities for new drug products using an extensive database.

Artificial neural-network-based subgrid-scale model for large-eddy simulation of isotropic turbulence

This study is concerned with accurately predicting the subgrid-scale (SGS) stress using an artificial neural network (ANN) with a linear eddy-viscosity term and a nonlinear term as the input variables. A priori and a posteriori tests are conducted to examine the prediction performance of the ANN-based SGS stress model in decaying homogeneous isotropic turbulence. In a priori test, the present ANN-based SGS model shows high correlation coefficients between the true and predicted SGS stresses, and excellent predictions of the SGS stress and dissipation. In a posteriori test, it is found that the ANN-based SGS model can predict the turbulence statistics more accurately than the traditional dynamic SGS models. The generalization capabilities of the model to untrained flow conditions and unstrained types of turbulent flow have been evaluated. It is found that the proposed ANN-based model can provide an accurate prediction of the SGS stress under different Reynolds numbers and flow types. A comparison among several existing ANN-based models with different input variables is presented, demonstrating a significant advantage of the present model.

Enhancing fluid classification using meta-learning and transformer through small-sample drilling data to interpret well logging data

As geological exploration and oil and gas development continue to advance, improvement in fluid prediction becomes increasingly crucial. Drilling data often suffer from limited sample size, challenging traditional machine learning methods to fully harness these data. Consequently, a more adaptable and versatile approach is required. In response to this issue, we introduce the meta-ViT (Vision Transformer) method—a novel framework that merges meta-learning with the ViT. Meta-learning's parameter updating mechanism refines the model's ability to discern patterns and nuances across tasks, while ViT, powered by meta-learning, achieves an enhanced grasp of geological exploration characteristics, boosting fluid detection efficiency. The support set supplies meta-learning insights, while the query set assesses generalization. ViT excels at identifying subterranean fluids. Meta-learning replicates varied tasks and data distributions, fortifying model adaptability. Meanwhile, Transformers' self-attention mechanism captures distant dependencies that traditional long short-term memory struggle to manage. Their residual connections and layer normalization also address gradient challenges, simplifying training. Hence, our model effectively interprets intricate drilling data features, improving predictive accuracy and adaptability. In our experiments using a small drilling data sample set, we compared meta-ViT against other models. The results reveal superior performance of our model with limited data, affirming its efficacy and prominence in fluid classification tasks. Overall, our proposed solution excels in fluid classification tasks involving small-sample drilling data by utilizing available samples to enhance model adaptability and predictive performance.

Hybrid machine learning approach to prediction of the compressive and flexural strengths of UHPC and parametric analysis Shapley additive explanations algorithm

Ultra-high-performance concrete (UHPC) is a sustainable construction material; it can be applied as a substitute for cement concrete. Artificial intelligence methods have been used to evaluate concrete composites to reduce time and money in the construction industries. So, this study applied machine learning (ML) and hybrid ML approaches to predict the compressive and flexural strength of UHPC. A dataset of 626 compressive strength and 317 flexural strength data points was collected from the published research articles, where fourteen important variables were selected as input parameters for the analysis of hybrid ML and ML algorithms. This research used XGBoost, LightGBM, and hybrid XGBoost- LightGBM algorithms to predict UHPC materials. Grid search (GS) techniques were used to adjust model hyper-parameters in search of improved high accuracy and efficiency. ML and hybrid ML models were train, and the test stage utilized statistical assessments such as R-square, root mean square error (RMSE), mean absolute error (MAE), and coefficient of efficiency (CE). The results presented hybrid ML algorithm was superior to the XGBoost and LightGBM algorithms in terms of R-square and RMSE values for both compressive strength and flexural strength prediction. A hybrid ML model and two ML models showed CS considerable R-square values above 0.94 at the testing stages and just over 0.97 at the training phase. Hybrid ML model performance accuracy for CS prediction was R-square value found that almost 0.996 for training and 0.963 for testing phases. At the same time, the FS prediction result showed that the R-square value of the Hybrid ML model and two traditional ML models were founded at almost 0.95 for the training phase and around 0.81 for the testing phase. But among them, the hybrid XGB-LGB model prediction performance was high accuracy and lowest error for CS and FS of UHPC trained and its hyperparameters optimized. Additionally, the SHAP investigation reveals the impact and relationship of the input variables with the output variables. SHAP analysis outcome reveals that curing age and steel fiber content input parameter had the highest positive impact on compressive strength and flexural strength of UHPC.