Robust Machine Learning Methods Research Articles

Short-term Classification of Strong Solar Energetic Particle Events Using Multivariate Time-series Classifiers

Solar energetic particle (SEP) events are one of the most crucial aspects of space weather that require continuous monitoring and forecasting. Their prediction depends on various factors, including source eruptions. In the present work, we use the Geostationary Solar Energetic Particle data set covering solar cycles 22, 23, and 24. We develop a framework using time-series-based machine-learning (ML) models with the aim of developing robust short-term forecasts by classifying SEP events. For this purpose, we introduce an ensemble learning approach that merges the results from univariate time series of three proton channels (E ≥10, 50, and 100 MeV) and the long-band X-ray flux (1–8 Å) channel from the Geostationary Operational Environmental Satellite missions and analyze their performance. We consider three models, namely, time series forest, supervised time series forest (STSF), and Bag-of-Symbolic Fourier Approximation Symbols. Our study also focuses on understanding and developing confidence in the predictive capabilities of our models. Therefore, we utilize multiple evaluation techniques and metrics. Based on that, we find STSF to perform well in all scenarios. The summary of metrics for the STSF model is as follows: the area under the ROC curve = 0.981, F 1-score = 0.960, true skill statistics = 0.919, Heidke skill score = 0.920, Gilbert skill score = 0.852, and Matthew’s correlation coefficient = 0.920. The Brier score loss of the STSF model is 0.077. This work lays the foundation for building near-real-time short-term SEP event predictions using robust ML methods.

Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods

Reproducibility is a desirable property of scientific research. On the one hand, it increases confidence in results. On the other hand, reproducible results can be extended on a solid basis. In rapidly developing fields such as machine learning, the latter is particularly important to ensure the reliability of research. In this paper, we present a systematic approach to reproducing (using the available implementation), replicating (using an alternative implementation) and reevaluating (using different datasets) state-of-the-art experiments. This approach enables the early detection and correction of deficiencies and thus the development of more robust and transparent machine learning methods. We detail the independent reproduction, replication, and reevaluation of the initially published experiments with a method that we want to extend. For each step, we identify issues and draw lessons learned. We further discuss solutions that have proven effective in overcoming the encountered problems. This work can serve as a guide for further reproducibility studies and generally improve reproducibility in machine learning.

Resilient back-propagation machine learning-based classification on fundus images for retinal microaneurysm detection.

The timely diagnosis of medical conditions, particularly diabetic retinopathy, relies on the identification of retinal microaneurysms. However, the commonly used retinography method poses a challenge due to the diminutive dimensions and limited differentiation of microaneurysms in images. Automated identification of microaneurysms becomes crucial, necessitating the use of comprehensive ad-hoc processing techniques. Although fluorescein angiography enhances detectability, its invasiveness limits its suitability for routine preventative screening. This study proposes a novel approach for detecting retinal microaneurysms using a fundus scan, leveraging circular reference-based shape features (CR-SF) and radial gradient-based texture features (RG-TF). The proposed technique involves extracting CR-SF and RG-TF for each candidate microaneurysm, employing a robust back-propagation machine learning method for training. During testing, extracted features from test images are compared with training features to categorize microaneurysm presence. The experimental assessment utilized four datasets (MESSIDOR, Diaretdb1, e-ophtha-MA, and ROC), employing various measures. The proposed approach demonstrated high accuracy (98.01%), sensitivity (98.74%), specificity (97.12%), and area under the curve (91.72%). The presented approach showcases a successful method for detecting retinal microaneurysms using a fundus scan, providing promising accuracy and sensitivity. This non-invasive technique holds potential for effective screening in diabetic retinopathy and other related medical conditions.

Sales Conversion Optimization Analysis Using the Random Forest Method

Sales conversion is a challenging field of work in sales and business. Companies are competing to be winners by improving their services and hoping that their product sales can increase in various ways, including by using optimization theory. However, the lack of data analysis is a problem that is often encountered in optimizing sales conversions. Various machine learning-based methods have also been used to help analyze sales conversion optimization. This research uses the Random Forest method which is one of the more robust machine learning methods compared to other methods, namely Adaptive Booster (AdaBoost) and K-Nearest Neighbor (KNN) in analyzing sales conversion optimization. The results showed that the Random Forest method had the best performance in classifying data, by using the 10 cross validation technique the results were obtained with a Mean Squared Error (MSE) value of 0.928 and a Root Mean Square Error (RMSE) of 0.963, better than the Adaptive Booster method. and K-Nearest Neighbor which has lower performance. Sales conversion optimization processing using Random Forest is proven to have the best performance as evidenced by the small Mean Squared Error and Root Mean Square Error which means it has an accurate level of performance compared to other methods.

Analysis of SEP Events and Their Possible Precursors Based on the GSEP Catalog

Solar energetic particle (SEP) events are one of the most crucial aspects of space weather. Their prediction depends on various factors including the source solar eruptions such as flares and coronal mass ejections (CMEs). The Geostationary Solar Energetic Particle (GSEP) events catalog was developed as an extensive data set toward this effort for solar cycles 22, 23, and 24. In the present work, we review and extend the GSEP data set by (1) adding “weak” SEP events that have proton enhancements from 0.5 to 10 pfu in the E >10 MeV channel and (2) improving the associated solar source eruptions information. We analyze and discuss spatiotemporal properties such as flare magnitudes, locations, rise times, and speeds and widths of CMEs. We check for the correlation of these parameters with peak proton fluxes and event fluences. Our study also focuses on understanding feature importance toward the optimal performance of machine-learning (ML) models for SEP event forecasting. We implement random forest, extreme gradient boosting, logistic regression, and support vector machine classifiers in a binary classification schema. Based on the evaluation of our best models, we find both the flare and CME parameters are requisites to predict the occurrence of an SEP event. This work is a foundation for our further efforts on SEP event forecasting using robust ML methods.

0537 Incident Hypertension Prediction in Obstructive Sleep Apnea using Machine Learning

Abstract Introduction Obstructive sleep apnea (OSA) is associated with hypertension due to intermittent hypoxia and sleep fragmentation. Due to the complex pathogenesis of hypertension, it is difficult to predict incident hypertension associated with OSA. A Machine Learning (ML) model to predict incident hypertension identified up to five years after the diagnosis of OSA by polysomnography developed. Methods Polysomnography provides time-series data on multiple physiological signals. We used the sleep heart health study (SHHS) cohort, where 4,797 participants had OSA. After excluding participants with pre-existing hypertension at baseline, the sample size was 2,652. 1,814 participants with follow-up data at 5 years were included (911/1,814, 50% with incident hypertension). In addition to clinical data (i.e. age and race), features extracted from polysomnography (heart rate variability, HRV calculated based on the electrocardiography R-R interval), electroencephalography delta power, statistical information (i.e., mean and standard deviation of signals), and heart rate periodicity functions fed to support vector machine (SVM) ML model to train and validate. The polysomnography features were calculated over the 30-second epochs identified based on respiratory events and EEG arousal and respiratory events annotation, and their corresponding parts in other signals based on sampling frequency. Technical artifacts in oxygen saturation and ECG were reconstructed with the interpolation method and removed from the signal respectively. The SVM is a robust ML method trained in an iterative fashion to find the global optimum. In comparison to the Deep Neural Network (DNN) approaches, SVMs results are interpretable. Each polysomnography signal and its corresponding features were trained on a separate SVM, followed by a fusion of the SVM results. The final results were fused by voting of individual SVM results. Results The SVM ML model thus far has achieved a test accuracy (area under the curve, AUC) of 66.06%, sensitivity 63.21%, and specificity 68.9%. Conclusion This proof-of-concept study suggests that supervised ML models, such as the SVM, may be useful in predicting incident hypertension associated with OSA. Further research is required regarding optimal input features to boost the accuracy, followed by external validation of the model in additional OSA cohorts. Support (if any) Research support 1R56HL157182, NIH/NHLBI

A data-driven strategy for industrial cracking furnace system scheduling under uncertainty

Cyclic scheduling of ethylene cracking furnace system (CSECFS) has a significant impact on raising economic performance of ethylene plants. However, in actual plants, optimal results of deterministic models may be suboptimal or ineffective because of various uncertainties. This paper proposes a novel data-driven adaptive robust optimization (DDARO) strategy that effectively bridges robust optimization and machine learning methods. A mixed-integer nonlinear programming (MINLP) model for CSECFS is developed initially. Second, data-driven uncertainty sets are generated using the historical data of processing flow rates: maximally correlated principal component analysis (MCPCA) is employed to partition the data into two subspaces, which are then delineated by support vector clustering (SVC) and generalized norm approaches, respectively. Third, a multi-stage DDARO model is established using the derived uncertainty sets and afterward transformed as a tractable single-stage model. Finally, a real-world case is performed to exemplify the effectiveness of the proposed framework.

Estimating Software Functional Size via Machine Learning

Measuring software functional size via standard Function Points Analysis (FPA) requires the availability of fully specified requirements and specific competencies. Most of the time, the need to measure software functional size occurs well in advance with respect to these ideal conditions, under the lack of complete information or skilled experts. To work around the constraints of the official measurement process, several estimation methods for FPA have been proposed and are commonly used. Among these, the International Function Point User Group (IFPUG) has adopted the ‘High-level FPA’ method (also known as NESMA method). This method avoids weighting each data and transaction function by using fixed weights instead. Applying High-level FPA, or similar estimation methods, is faster and easier than carrying out the official measurement process, but inevitably yields an approximation in the measures. In this paper, we contribute to the problem of estimating software functional size measures by using machine learning. To the best of our knowledge, machine learning methods were never applied to the early estimation of software functional size. Our goal is to understand whether machine learning techniques yield estimates of FPA measures that are more accurate than those obtained with High-level FPA or similar methods. An empirical study on a large dataset of functional size predictors was carried out to train and test three of the most popular and robust machine learning methods, namely: Random Forests, Support Vector Regression, and Neural Networks. A systematic experimental phase, with cycles of dataset filtering and splitting, parameters tuning, and model training and validation is presented. The estimation accuracy of the obtained models was then evaluated and compared to that of fixed-weight models (e.g., High-level FPA) and linear regression models, also using a second dataset as the test set. We found that Support Vector Regression yields quite accurate estimation models. However, the obtained level of accuracy does not appear significantly better with respect to High-level FPA or to models built via ordinary least squares regression. Noticeably, fairly good accuracy levels were obtained by models that do not even require discerning among different types of transactions and data.

Application of Machine Learning to Child Mode Choice with a Novel Technique to Optimize Hyperparameters.

Travel mode choice (TMC) prediction is crucial for transportation planning. Most previous studies have focused on TMC in adults, whereas predicting TMC in children has received less attention. On the other hand, previous children's TMC prediction studies have generally focused on home-to-school TMC. Hence, LIGHT GRADIENT BOOSTING MACHINE (LGBM), as a robust machine learning method, is applied to predict children's TMC and detect its determinants since it can present the relative influence of variables on children's TMC. Nonetheless, the use of machine learning introduces its own challenges. First, these methods and their performance are highly dependent on the choice of "hyperparameters". To solve this issue, a novel technique, called multi-objective hyperparameter tuning (MOHPT), is proposed to select hyperparameters using a multi-objective metaheuristic optimization framework. The performance of the proposed technique is compared with conventional hyperparameters tuning methods, including random search, grid search, and "Hyperopt". Second, machine learning methods are black-box tools and hard to interpret. To overcome this deficiency, the most influential parameters on children's TMC are determined by LGBM, and logistic regression is employed to investigate how these parameters influence children's TMC. The results suggest that MOHPT outperforms conventional methods in tuning hyperparameters on the basis of prediction accuracy and computational cost. Trip distance, "walkability" and "bikeability" of the origin location, age, and household income are principal determinants of child mode choice. Furthermore, older children, those who live in walkable and bikeable areas, those belonging low-income groups, and short-distance travelers are more likely to travel by sustainable transportation modes.

LANGaware: Robust Multimodal Machine Learning Methods for the Early Detection of Neurodegenerative and Psychiatric Diseases

AbstractBackgroundMultiple neurodegenerative and psychiatric diseases can affect speech, manifesting as subtle changes in spoken language. We perform a rigorous evaluation of a Machine Learning (ML) pipeline on a large‐scale multiaxial experimental setting, to investigate its capacity for detecting early signs of cognitive decline as manifested in speech. Using a diverse dataset of speech samples spanning different languages, patient cohorts and cognitive status diagnoses, we propose that the developed pipeline showcases robust performance, facilitating efficient decision support for early cognitive decline detection.MethodWe build upon previous research efforts on data‐driven cognitive decline prediction. Random forest models are constructed from a dataset of more than 3000 audio recordings and ASR‐generated transcripts, constituting cognitive assessment test tasks (e.g. picture description, everyday activity recounting, etc.). A multimodal NLP and audio analysis procedure derives sophisticated predictive biomarkers, using three methods for feature selection, feature decorrelation preprocessing and rigorous, grid search‐based hyperparameter optimization. We introduce novel experimental axes of investigation, including the spoken language (English and Greek), single and multi‐task verbal task settings, and different neurodegenerative diseases for the pathological cohorts (Alzheimer’s Disease or Dementia, along with age and education‐matched healthy cohorts). Four different NLP analysis tools for tokenization, syntax and dependency extraction are considered, using both ML and Deep Learning (DL) techniques. Model stability and generalization is supported via 5‐fold cross‐validation and the use of a separate test set.ResultResults are validated against medical expert diagnosis. Αverage performance across all investigative axes reaches high scores for accuracy (0.73), sensitivity (0.72), specificity (0.74), precision (0.77), F1 (0.74), and AUC (0.78). Additional findings suggest that single‐task models outperform multitask ones (e.g. reaching AUC scores of 0.85). Moreover, cognitive decline detection is more efficient for pathological cohorts of Alzheimer's, versus patients with Dementias.ConclusionThe broad empirical evaluation verifies the effectiveness of the deployed ML system, adding evidence in favor of multimodal AI solutions for predicting early cognitive decline and showcasing their robustness in diverse evaluation settings. Future work includes leveraging DL methods for classification and evaluation on differential diagnosis discrimination tasks.

Why it is Unfortunate that Linear Machine Learning "Works" so well in Electromechanical Switching of Ferroelectric Thin Films.

Machine learning (ML)is relied on for materials spectroscopy. It is challenging to make ML models fail because statistical correlations can mimic the physics without causality. Here, using a benchmark band-excitation piezoresponse force microscopy polarization spectroscopy (BEPS) dataset the pitfalls of the so-called "better", "faster", and "less-biased" ML of electromechanical switching are demonstrated and overcome. Using a toy and real experimental dataset, it is demonstrated how linear nontemporal ML methods result in physically reasonable embedding (eigenvalues) while producing nonsensical eigenvectors and generated spectra, promoting misleading interpretations. A new method of unsupervised multimodal hyperspectral analysis of BEPS is demonstrated using long-short-term memory (LSTM) β-variational autoencoders (β-VAEs) . By including LSTM neurons, the ordinal nature of ferroelectric switching is considered. To improve the interpretability of the latent space, a variational Kullback-Leibler-divergency regularization is imposed . Finally, regularization scheduling of β as a disentanglement metric is leveraged to reduce user bias. Combining these experiment-inspired modifications enables the automated detection of ferroelectric switching mechanisms, including a complex two-step, three-state one. Ultimately, this work provides a robust ML method for the rapid discovery of electromechanical switching mechanisms in ferroelectrics and is applicable to other multimodal hyperspectral materials spectroscopies.

Wearable Sensor Technology to Predict Core Body Temperature: A Systematic Review.

Heat-related illnesses, which range from heat exhaustion to heatstroke, affect thousands of individuals worldwide every year and are characterized by extreme hyperthermia with the core body temperature (CBT) usually > 40 °C, decline in physical and athletic performance, CNS dysfunction, and, eventually, multiorgan failure. The measurement of CBT has been shown to predict heat-related illness and its severity, but the current measurement methods are not practical for use in high acuity and high motion settings due to their invasive and obstructive nature or excessive costs. Noninvasive predictions of CBT using wearable technology and predictive algorithms offer the potential for continuous CBT monitoring and early intervention to prevent HRI in athletic, military, and intense work environments. Thus far, there has been a lack of peer-reviewed literature assessing the efficacy of wearable devices and predictive analytics to predict CBT to mitigate heat-related illness. This systematic review identified 20 studies representing a total of 25 distinct algorithms to predict the core body temperature using wearable technology. While a high accuracy in prediction was noted, with 17 out of 18 algorithms meeting the clinical validity standards. few algorithms incorporated individual and environmental data into their core body temperature prediction algorithms, despite the known impact of individual health and situational and environmental factors on CBT. Robust machine learning methods offer the ability to develop more accurate, reliable, and personalized CBT prediction algorithms using wearable devices by including additional data on user characteristics, workout intensity, and the surrounding environment. The integration and interoperability of CBT prediction algorithms with existing heat-related illness prevention and treatment tools, including heat indices such as the WBGT, athlete management systems, and electronic medical records, will further prevent HRI and increase the availability and speed of data access during critical heat events, improving the clinical decision-making process for athletic trainers and physicians, sports scientists, employers, and military officers.

Lung Cancer Prediction Using Robust Machine Learning and Image Enhancement Methods on Extracted Gray-Level Co-Occurrence Matrix Features

In the present era, cancer is the leading cause of demise in both men and women worldwide, with low survival rates due to inefficient diagnostic techniques. Recently, researchers have been devising methods to improve prediction performance. In medical image processing, image enhancement can further improve prediction performance. This study aimed to improve lung cancer image quality by utilizing and employing various image enhancement methods, such as image adjustment, gamma correction, contrast stretching, thresholding, and histogram equalization methods. We extracted the gray-level co-occurrence matrix (GLCM) features on enhancement images, and applied and optimized vigorous machine learning classification algorithms, such as the decision tree (DT), naïve Bayes, support vector machine (SVM) with Gaussian, radial base function (RBF), and polynomial. Without the image enhancement method, the highest performance was obtained using SVM, polynomial, and RBF, with accuracy of (99.89%). The image enhancement methods, such as image adjustment, contrast stretching at threshold (0.02, 0.98), and gamma correction at gamma value of 0.9, improved the prediction performance of our analysis on 945 images provided by the Lung Cancer Alliance MRI dataset, which yielded 100% accuracy and 1.00 of AUC using SVM, RBF, and polynomial kernels. The results revealed that the proposed methodology can be very helpful to improve the lung cancer prediction for further diagnosis and prognosis by expert radiologists to decrease the mortality rate.

A robust and parsimonious machine learning method to predict ICU admission of COVID-19 patients.

In this article, we discuss the development of prognostic machine learning (ML) models for COVID-19 progression, by focusing on the task of predicting ICU admission within (any of) the next 5 days. On the basis of 6,625 complete blood count (CBC) tests from 1,004 patients, of which 18% were admitted to intensive care unit (ICU), we created four ML models, by adopting a robust development procedure which was designed to minimize risks of bias and over-fitting, according to reference guidelines. The best model, a support vector machine, had an AUC of .85, a Brier score of .14, and a standardized net benefit of .69: these scores indicate that the model performed well over a variety of prediction criteria. We also conducted an interpretability study to back up our findings, showing that the data on which the developed model is based is consistent with the current medical literature. This also demonstrates that CBC data and ML methods can be used to predict COVID-19 patients’ ICU admission at a relatively low cost: in particular, since CBC data can be quickly obtained by means of routine blood exams, our models could be used in resource-constrained settings and provide health practitioners with rapid and reliable indications.Graphical abstract

A modeling approach for estimating hydrogen sulfide solubility in fifteen different imidazole-based ionic liquids

Absorption has always been an attractive process for removing hydrogen sulfide (H2S). Posing unique properties and promising removal capacity, ionic liquids (ILs) are potential media for H2S capture. Engineering design of such absorption process needs accurate measurements or reliable estimation of the H2S solubility in ILs. Since experimental measurements are time-consuming and expensive, this study utilizes machine learning methods to monitor H2S solubility in fifteen various ILs accurately. Six robust machine learning methods, including adaptive neuro-fuzzy inference system, least-squares support vector machine (LS-SVM), radial basis function, cascade, multilayer perceptron, and generalized regression neural networks, are implemented/compared. A vast experimental databank comprising 792 datasets was utilized. Temperature, pressure, acentric factor, critical pressure, and critical temperature of investigated ILs are the affecting parameters of our models. Sensitivity and statistical error analysis were utilized to assess the performance and accuracy of the proposed models. The calculated solubility data and the derived models were validated using seven statistical criteria. The obtained results showed that the LS-SVM accurately predicts H2S solubility in ILs and possesses R2, RMSE, MSE, RRSE, RAE, MAE, and AARD of 0.99798, 0.01079, 0.00012, 6.35%, 4.35%, 0.0060, and 4.03, respectively. It was found that the H2S solubility adversely relates to the temperature and directly depends on the pressure. Furthermore, the combination of OMIM+ and Tf2N-, i.e., [OMIM][Tf2N] ionic liquid, is the best choice for H2S capture among the investigated absorbents. The H2S solubility in this ionic liquid can reach more than 0.8 in terms of mole fraction.

SPLDExtraTrees: robust machine learning approach for predicting kinase inhibitor resistance.

Drug resistance is a major threat to the global health and a significant concern throughout the clinical treatment of diseases and drug development. The mutation in proteins that is related to drug binding is a common cause for adaptive drug resistance. Therefore, quantitative estimations of how mutations would affect the interaction between a drug and the target protein would be of vital significance for the drug development and the clinical practice. Computational methods that rely on molecular dynamics simulations, Rosetta protocols, as well as machine learning methods have been proven to be capable of predicting ligand affinity changes upon protein mutation. However, the severely limited sample size and heavy noise induced overfitting and generalization issues have impeded wide adoption of machine learning for studying drug resistance. In this paper, we propose a robust machine learning method, termed SPLDExtraTrees, which can accurately predict ligand binding affinity changes upon protein mutation and identify resistance-causing mutations. Especially, the proposed method ranks training data following a specific scheme that starts with easy-to-learn samples and gradually incorporates harder and diverse samples into the training, and then iterates between sample weight recalculations and model updates. In addition, we calculate additional physics-based structural features to provide the machine learning model with the valuable domain knowledge on proteins for these data-limited predictive tasks. The experiments substantiate the capability of the proposed method for predicting kinase inhibitor resistance under three scenarios and achieve predictive accuracy comparable with that of molecular dynamics and Rosetta methods with much less computational costs.

Gut Microbiota Exceeds Cervical Microbiota for Early Diagnosis of Endometriosis.

The diagnosis of endometriosis is typically delayed by years for the unexclusive symptom and the traumatic diagnostic method. Several studies have demonstrated that gut microbiota and cervical mucus potentially can be used as auxiliary diagnostic biomarkers. However, none of the previous studies has compared the robustness of endometriosis classifiers based on microbiota of different body sites or demonstrated the correlation among microbiota of gut, cervical mucus, and peritoneal fluid of endometriosis, searching for alternative diagnostic approaches. Herein, we enrolled 41 women (control, n = 20; endometriosis, n = 21) and collected 122 well-matched samples, derived from feces, cervical mucus, and peritoneal fluid, to explore the nature of microbiome of endometriosis patients. Our results indicated that microbial composition is remarkably distinguished between three body sites, with 19 overlapped taxa. Moreover, endometriosis patients harbor distinct microbial communities versus control group especially in feces and peritoneal fluid, with increased abundance of pathogens in peritoneal fluid and depletion of protective microbes in feces. Particularly, genera of Ruminococcus and Pseudomonas were identified as potential biomarkers in gut and peritoneal fluid, respectively. Furthermore, novel endometriosis classifiers were constructed based on taxa selected by a robust machine learning method. These results demonstrated that gut microbiota exceeds cervical microbiota in diagnosing endometriosis. Collectively, this study reveals important insights into the microbial profiling in different body sites of endometriosis, which warrant future exploration into the role of microbiota in endometriosis and highlighted values on gut microbiota in early diagnosis of endometriosis.

DPWSS: differentially private working set selection for training support vector machines.

Support vector machine (SVM) is a robust machine learning method and is widely used in classification. However, the traditional SVM training methods may reveal personal privacy when the training data contains sensitive information. In the training process of SVMs, working set selection is a vital step for the sequential minimal optimization-type decomposition methods. To avoid complex sensitivity analysis and the influence of high-dimensional data on the noise of the existing SVM classifiers with privacy protection, we propose a new differentially private working set selection algorithm (DPWSS) in this paper, which utilizes the exponential mechanism to privately select working sets. We theoretically prove that the proposed algorithm satisfies differential privacy. The extended experiments show that the DPWSS algorithm achieves classification capability almost the same as the original non-privacy SVM under different parameters. The errors of optimized objective value between the two algorithms are nearly less than two, meanwhile, the DPWSS algorithm has a higher execution efficiency than the original non-privacy SVM by comparing iterations on different datasets. To the best of our knowledge, DPWSS is the first private working set selection algorithm based on differential privacy.

Data-driven adaptive robust optimization for energy systems in ethylene plant under demand uncertainty

The operational optimization of energy systems is of great significance for improving the overall efficiency of industrial processes. Facing new challenges brought by widespread uncertainties, a data-driven adaptive robust industrial multi-type energy systems optimization framework was proposed by bridging robust optimization and machine learning methods in this paper. The industrial data were used to capture the demand uncertainty of the actual process. Hybrid models of units were first developed considering the operational characteristics, and the energy system optimization model was then formed as a mixed-integer nonlinear programming problem. The uncertain parameter set of process power demands was formed by the process models using historical data of a whole operating period. Afterward, the uncertainty set was constructed by applying the robust kernel density estimation method, which can reduce conservatism by considering the distributional information. By integrating the derived data-driven uncertainty set, a two-stage adaptive robust optimization model aiming at minimizing the weighted total energy consumption was developed. The multi-level robust optimization model was reformulated as a tractable single-level model by employing the affine decision rule. A case study on a plant-wide industrial energy system in the ethylene plant was performed, and the minimum optimal energy consumption was 25,350 kg/h, whose price of robustness was only 2.18%. The robust optimization results can guide the operational optimization of energy systems under uncertainty for the operators of the ethylene plant.

Application of robust machine learning methods to modeling hydrogen solubility in hydrocarbon fuels

Having accurate information about the hydrogen solubility in hydrocarbon fuels and feedstocks is very important in petroleum refineries and coal processing plants. In the present work, extreme gradient boosting (XGBoost), multi-layer perceptron (MLP) trained with Levenberg–Marquardt (LM) algorithm, adaptive boosting support vector regression (AdaBoost−SVR), and a memory-efficient gradient boosting tree system on adaptive compact distributions (LiteMORT) as four novel machine learning methods were used for estimating the hydrogen solubility in hydrocarbon fuels. To achieve this goal, a database containing 445 experimental data of hydrogen solubilities in 17 various hydrocarbon fuels/feedstocks was collected in wide-spread ranges of operating pressures and temperatures. These hydrocarbon fuels include petroleum fractions, refinery products, coal liquids, bitumen, and shale oil. Input parameters of the models are temperature and pressure along with density at 20 °C, molecular weight, and weight percentage of carbon (C) and hydrogen (H) of hydrocarbon fuels. XGBoost showed the highest accuracy compared to the other models with an overall mean absolute percent relative error of 1.41% and coefficient of determination (R2) of 0.9998. Also, seven equations of state (EOSs) were used to predict hydrogen solubilities in hydrocarbon fuels. The 2- and 3-parameter Soave-Redlich-Kwong EOS rendered the best estimates for hydrogen solubilities among the EOSs. Moreover, sensitivity analysis indicated that pressure owns the highest influence on hydrogen solubilities in hydrocarbon fuels and then temperature and hydrogen weight percent of the hydrocarbon fuels are ranked, respectively. Finally, Leverage approach results exhibited that the XGBoost model could be well trusted to estimate the hydrogen solubility in hydrocarbon fuels.