A machine learning approach to two-stage adaptive robust optimization

A transformation-proximal bundle algorithm for multistage adaptive robust optimization and application to constrained robust optimal control

This paper proposes a novel transformation-proximal bundle algorithmic framework to solve multistage adaptive robust optimization (ARO) problems. Different from existing solution methods, the proposed algorithmic framework partitions recourse decisions into state decisions and local decisions. It applies affine decision rule only to state decision variables and allows local decision variables to be fully adjustable. In this way, the original multistage ARO problem is proved to be transformed into a two-stage ARO problem. The proximal bundle algorithm with the Moreau- Yosida regularization is further developed for the exact solution of the resulting two-stage ARO problem. The transformation-proximal bundle algorithmic framework could generate less conservative solutions compared with the decision rule based approach, while enjoying a high computational efficiency. An application on multiperiod inventory control problem under demand uncertainty is presented to demonstrate the effectiveness and superiority of the proposed algorithm.

Adaptive robust optimization for lot-sizing under yield uncertainty

Data-driven adaptive robust optimization for energy systems in ethylene plant under demand uncertainty

Aim: The majority of machine learning (ML) models in healthcare are built on retrospective data, much of which is collected without explicit patient consent for use in artificial intelligence (AI) and ML applications. The primary aim of this study was to evaluate whether clinicians and scientific researchers themselves consent to provide their own data for the training of ML models. Materials and Methods: This survey was conducted through an anonymous online survey, utilizing platforms such as Telegram, LinkedIn, and Viber. The target audience comprised specific international groups, primarily Russian, German, and English-speaking, of clinicians and scientific researchers. These participants ranged in their levels of expertise and experience, from beginners to veterans. The survey centered on a singular, pivotal question: “Do You Consent to the Use of Your Biological and Private Data for Training Machine Learning and AI Models?” Respondents had the option to choose from three responses: “Yes” and “No”. Results: The survey was conducted in January 2024. A total of 119 unique and verified individuals participated in the survey. The results revealed that only 50% of respondents (63 persons) expressed consent to provide their own data for the training of ML and AI models. Conclusion: In the development of ML and AI models, particularly open-source ones, it is crucial to ascertain whether participants are willing to provide their private data. While ML algorithms can transform the nature of data, it is important to remember that the primary owner of this data is the individual. Our findings show that in 50% of the cases, even participants from scientific research and clinical backgrounds – individuals typically accountable for ensuring data quality in AI and ML model development – do not consent to the use of their data in AI and ML settings. This highlights the need for more stringent consent processes and ethical considerations in the utilization of personal data in AI and ML research.

This paper addresses the planning problem of residential micro combined heat and power (micro-CHP) systems (including micro-generation units, auxiliary boilers, and thermal storage tanks) considering the associated technical and economic factors. Since the accurate values of the thermal and electrical loads of this system cannot be exactly predicted for the planning horizon, the thermal and electrical load uncertainties are modeled using a two-stage adaptive robust optimization method based on a polyhedral uncertainty set. A solution method, which is composed of column-and-constraint generation (C&CG) algorithm and block coordinate descent (BCD) method, is proposed to efficiently solve this adaptive robust optimization model. Numerical results from a practical case study show the effective performance of the proposed adaptive robust model for residential micro-CHP planning and its solution method.

Machine Learning (ML) plays a key role in various intelligent systems, and building an effective ML model for a data set is a difficult task involving various steps including data cleaning, feature definition and extraction, ML algorithms development, model training and evaluation as well as others. One of the most important steps in the process is to compare generated substantial amounts of ML models to find the optimal one for the deployment. It is challenging to compare such models with dynamic number of features. This paper proposes a novel visualisation approach based on a radial net to compare ML models trained with a different number of features of a given data set while revealing implicit dependent relations. In the proposed approach, ML models and features are represented by lines and arcs respectively. The dependence of ML models with dynamic number of features is encoded into the structure of visualisation, where ML models and their dependent features are directly revealed from related line connections. ML model performance information is encoded with colour and line width in the innovative visualisation. Together with the structure of visualization, feature importance can be directly discerned to help to understand ML models.

This paper compares risk-averse optimization methods to address the self-scheduling and market involvement of a virtual power plant (VPP). The decision-making problem of the VPP involves uncertainty in the wind speed and electricity price forecast. We focus on two methods: risk-averse two-stage stochastic programming (SP) and two-stage adaptive robust optimization (ARO). We investigate both methods concerning formulations, uncertainty and risk, decomposition algorithms, and their computational performance. To quantify the risk in SP, we use the conditional value at risk (CVaR) because it can resemble a worst-case measure, which naturally links to ARO. We use two efficient implementations of the decomposition algorithms for SP and ARO; we assess (1) the operational results regarding first-stage decision variables, estimate of expected profit, and estimate of the CVaR of the profit and (2) their performance taking into consideration different sample sizes and risk management parameters. The results show that similar first-stage solutions are obtained depending on the risk parameterizations used in each formulation. Computationally, we identified three cases: (1) SP with a sample of 500 elements is competitive with ARO; (2) SP performance degrades comparing to the first case and ARO fails to converge in four out of five risk parameters; (3) SP fails to converge, whereas ARO converges in three out of five risk parameters. Overall, these performance cases depend on the combined effect of deterministic and uncertain data and risk parameters. Summary of Contribution: The work presented in this manuscript is at the intersection of operations research and computer science, which are intrinsically related with the scope and mission of IJOC. From the operations research perspective, two methodologies for optimization under uncertainty are studied: risk-averse stochastic programming and adaptive robust optimization. These methodologies are illustrated using an energy scheduling problem. The study includes a comparison from the point of view of uncertainty modeling, formulations, decomposition methods, and analysis of solutions. From the computer science perspective, a careful implementation of decomposition methods using parallelization techniques and a sample average approximation methodology was done . A detailed comparison of the computational performance of both methods is performed. Finally, the conclusions allow establishing links between two alternative methodologies in operations research: stochastic programming and robust optimization.

e14500 Background: Bexmarilimab, an investigational immunotherapeutic antibody targeting Clever-1, is currently investigated in phase I/II MATINS study (NCT03733990) for advanced solid tumors. Machine learning (ML) based models combining extensive data could be generated to predict treatment responses to this first-in-class macrophage checkpoint inhibitor. Methods: 52 baseline features from 138 patients included in the part 1/2 of phase I/II MATINS trial were included in ML modelling. 19 patients were classified as benefitting from the therapy by RECIST 1.1 defined clinical benefit rate (DCR) at cycle 4. Initial feature selection was done using both domain knowledge and removal of features with several missing values resulting in 44 features from 102 patients. The remaining data was standardized and feature selection using variance analysis (ANOVA) based on F-values between response and features was performed. With this approach, and by removing features with high multicollinearity, the number of features could be further reduced, and used sample size increased by removing features with missing values, until only the most important features were included in the data. Feature selection resulted in nine baseline features from 127 patients, of which 18 with DCR, to be used for ML model selection and training Several ML models were trained, and prediction performance evaluated using leave-one-out cross-validation (LOOCV). In LOOCV a ML model is trained as many times as there are samples in the data (127 times in this case), each time leaving one sample out from the training set as a test set. Finally, the out-of-sample test results are aggregated to form an overall view of the model performance with unseen data. Regularized Extreme gradient boosting (XGBoost) was found out to be the best performing prediction model. Results: Nine baseline features were associated with bexmarilimab treatment benefit, and the best ML prediction performance was obtained with five features. CBR was associated with low TNFalpha and neutrophils, and high Eotaxin, CK, and T-regs. ML model trained with these five features performed well in LOOCV as 15/18 (83%) DCR and 101/109 (93%) non-DCR were classified correctly, and all considered classification performance metrics were excellent. In feature importance analysis, high baseline T-regs and Eotaxin, and low TNFalpha were characterized as the most important predictors for treatment benefit with relative importances of 0.34, 0.25, and 0.24 (out of 1). Conclusions: This study highlights possibility of using ML models in predicting treatment benefit for novel cancer drugs such as bexmarilimab and boost the clinical development. The findings are in line with expected novel immune activating mode-of-action of bexmarilimab. Clinical trial information: NCT03733990.

This paper analyzes the optimization features of machine learning (ML) model training procedures using multi-GPU systems to enhance cyber security in telecommunication networks. A key aspect of the study is the use of data parallelism, which allows the distribution of the training load across multiple GPUs, significantly reducing training time and improving model accuracy-critical factors for rapid threat detection in cyberspace. A novel approach for optimizing data batch size using Mutual Information (MI) is proposed, which harmonizes the utilization of computational resources with the information content of the training data. MI helps to determine the optimal data batch size that minimizes training errors and improves model accuracy without a significant increase in training time. Experimental results demonstrate the substantial advantages of multi-GPU configurations compared to single-GPU setups, providing faster training and improved model accuracy. It was particularly emphasized that MI-guided batch size tuning significantly outperforms traditional manual tuning methods, ensuring higher validation accuracy and reducing training time. The study showed that the MI-based approach is an effective tool for addressing the problem of optimizing ML model training processes in real-world scenarios where cyber security is critical. The proposed methods allow ML models to train faster and more accurately identify potential threats, making them particularly relevant for telecommunication networks where a rapid response to new threats in real time is required. The implementation of modern computational technologies such as multi-GPU systems and MI-optimized training enhances the efficiency and accuracy of machine learning models. This, in turn, improves cyber security measures and ensures a more reliable defence of telecommunication networks against malicious attacks. It is noted that the proposed approaches can be adapted not only for cyber security but also for other areas where high model accuracy and fast training are important. Future research prospects include the development of new machine learning methods, particularly deep neural networks, the exploration of alternative computational architectures such as quantum computing or distributed systems, and their integration into real-time systems. Special attention should be paid to the ethical aspects of implementing automated cyber security systems, particularly in preventing bias in algorithms and ensuring fairness in their application.

Machine Learning (ML) will play a significant role in the success of the upcoming High-Luminosity LHC (HL-LHC) program at CERN. An unprecedented amount of data at the exascale will be collected by LHC experiments in the next decade, and this effort will require novel approaches to train and use ML models. In this paper, we discuss a Machine Learning as a Service pipeline for HEP (MLaaS4HEP) which provides three independent layers: a data streaming layer to read High-Energy Physics (HEP) data in their native ROOT data format; a data training layer to train ML models using distributed ROOT files; a data inference layer to serve predictions using pre-trained ML models via HTTP protocol. Such modular design opens up the possibility to train data at large scale by reading ROOT files from remote storage facilities, e.g., World-Wide LHC Computing Grid (WLCG) infrastructure, and feed the data to the user’s favorite ML framework. The inference layer implemented as TensorFlow as a Service (TFaaS) may provide an easy access to pre-trained ML models in existing infrastructure and applications inside or outside of the HEP domain. In particular, we demonstrate the usage of the MLaaS4HEP architecture for a physics use-case, namely, the t{bar{t}} Higgs analysis in CMS originally performed using custom made Ntuples. We provide details on the training of the ML model using distributed ROOT files, discuss the performance of the MLaaS and TFaaS approaches for the selected physics analysis, and compare the results with traditional methods.

Adaptive robust scheduling of a hydro/photovoltaic/pumped-storage hybrid system in day-ahead electricity and hydrogen markets

Robust transmission expansion planning with uncertain generations and loads using full probabilistic information

This article presents a two-stage adaptive robust optimization (ARO) for optimal sizing and operation of residential solar photovoltaic (PV) systems coupled with battery units. Uncertainties of PV generation and load are modeled by user-defined bounded intervals through polyhedral uncertainty sets. The proposed model determines the optimal size of PV–battery system while minimizing operating costs under the worst-case realization of uncertainties. The ARO model is proposed as a trilevel min–max–min optimization problem. The outer min problem characterizes sizing variables as “here-and-now” decisions to be obtained prior to uncertainty realization. The inner max–min problem, however, determines the operation variables in place of “wait-and-see” decisions to be obtained after uncertainty realization. An iterative decomposition methodology is developed by means of the column-and-constraint technique to recast the trilevel problem into a single-level master problem (the outer min problem) and a bilevel subproblem (the inner max–min problem). The duality theory and the Big-M linearization technique are used to transform the bilevel subproblem into a solvable single-level max problem. The immunization of the model against uncertainties is justified by testing the obtained solutions against 36 500 trial uncertainty scenarios in a postevent analysis. The proposed postevent analysis also determines the optimum robustness level of the ARO model to avoid over/under conservative solutions.

IntroductionArtificial intelligence (AI) applications in healthcare and medicine have increased in recent years. To enable access to personal data, Trusted Research Environments (TREs) (otherwise known as Safe Havens) provide safe and secure environments in which researchers can access sensitive personal data and develop AI (in particular machine learning (ML)) models. However, currently few TREs support the training of ML models in part due to a gap in the practical decision-making guidance for TREs in handling model disclosure. Specifically, the training of ML models creates a need to disclose new types of outputs from TREs. Although TREs have clear policies for the disclosure of statistical outputs, the extent to which trained models can leak personal training data once released is not well understood. BackgroundWe review, for a general audience, different types of ML models and their applicability within healthcare. We explain the outputs from training a ML model and how trained ML models can be vulnerable to external attacks to discover personal data encoded within the model. RisksWe present the challenges for disclosure control of trained ML models in the context of training and exporting models from TREs. We provide insights and analyse methods that could be introduced within TREs to mitigate the risk of privacy breaches when disclosing trained models. DiscussionAlthough specific guidelines and policies exist for statistical disclosure controls in TREs, they do not satisfactorily address these new types of output requests; i.e., trained ML models. There is significant potential for new interdisciplinary research opportunities in developing and adapting policies and tools for safely disclosing ML outputs from TREs.