Data-driven Setting Research Articles

Robust Conic Satisficing

Inspired by the principle of satisficing (Simon 1955), Long et al. (2021) propose an alternative framework for optimization under uncertainty, which we term as a robust satisficing model. Instead of sizing the uncertainty set in robust optimization, the robust satisficing model is specified by a target objective with the aim of delivering the solution that is least impacted by uncertainty in achieving the target. At the heart of this framework, we minimize the level of constraint violation under all possible realizations within the support set. Our framework is based on a constraint function that evaluates to the optimal objective value of a standard conic optimization problem, which can be used to model a wide range of constraint functions that are convex in the decision variables but can be either convex or concave in the uncertain parameters. We derive an exact semidefinite optimization formulation when the constraint is biconvex quadratic with quadratic penalty and the support set is ellipsoidal. We also show the equivalence between the more general robust satisficing problems and the classical adaptive robust linear optimization models with conic uncertainty sets, where the latter can be solved approximately using affine recourse adaptation. More importantly, under complete recourse, and reasonably chosen polyhedral support set and penalty function, we show that the exact reformulation and safe approximations do not lead to infeasible problems if the chosen target is above the optimum objective obtained when the nominal optimization problem is minimized. Finally, we extend our framework to the data-driven setting and showcase the modelling and the computational benefits of the robust satisficing framework over robust optimization with three numerical examples: portfolio selection, log-sum-exp optimization and adaptive lot-sizing problem.

Data-Driven Koopman Controller Synthesis Based on the Extended H₂ Norm Characterization

This paper presents a new data-driven controller synthesis based on the Koopman operator and the extended $\mathcal{H}_2$ norm characterization of discrete-time linear systems. We model dynamical systems as polytope sets which are derived from multiple data-driven linear models obtained by the finite approximation of the Koopman operator and then used to design robust feedback controllers combined with the $\mathcal{H}_2$ norm characterization. The use of the $\mathcal{H}_2$ norm characterization is aimed to deal with the model uncertainty that arises due to the nature of the data-driven setting of the problem. The effectiveness of the proposed controller synthesis is investigated through numerical simulations.

Randomized Assortment Optimization

When a firm selects an assortment of products to offer to customers, it uses a choice model to anticipate their probability of purchasing each product. In practice, the estimation of these models is subject to statistical errors, which may lead to significantly suboptimal assortment decisions. Recent work has addressed this issue using robust optimization, where the true parameter values are assumed unknown and the firm chooses an assortment that maximizes its worst-case expected revenues over an uncertainty set of likely parameter values, thus mitigating estimation errors. In this paper, we introduce the concept of randomization into the robust assortment optimization literature. We show that the standard approach of deterministically selecting a single assortment to offer is not always optimal in the robust assortment optimization problem. Instead, the firm can improve its worst-case expected revenues by selecting an assortment randomly according to a prudently designed probability distribution. We demonstrate this potential benefit of randomization across three popular choice models: the multinomial logit model, the Markov chain model, and the preference ranking model. We show how an optimal randomization strategy can be determined exactly and heuristically. Besides the superior in-sample performance of randomized assortments, we demonstrate improved out-of-sample performance in a data-driven setting that combines estimation with optimization. Our results suggest that more general versions of the assortment optimization problem—incorporating business constraints, more flexible choice models and/or more general uncertainty sets—tend to be more receptive to the benefits of randomization.

Offline Pricing and Demand Learning with Censored Data

We study a single product pricing problem with demand censoring in an offline data-driven setting. In this problem, a retailer is given a finite amount of inventory, and faces a random demand that is price sensitive in a linear fashion with unknown price sensitivity and base demand distribution. Any unsatisfied demand that exceeds the inventory level is lost and unobservable. We assume that the retailer has access to an offline data set consisting of triples of historical price, inventory level and potentially censored sales quantity. The retailer's objective is to use the offline data set to find an optimal price, maximizing her expected revenue with finite inventories. Due to demand censoring in the offline data, we show that the existence of near-optimal algorithms in a data-driven problem -- which we call problem identifiability -- is not always guaranteed. We develop a necessary and sufficient condition for problem identifiability, and show that it is directly related to some quality metrics of the censored data. We propose a novel data-driven algorithm that hedges against the distribution uncertainty arising from censored data, and admits provable finite-sample performance guarantees regardless of problem identifiability and offline data quality. Specifically, we prove that, for identifiable problems, the proposed algorithm is near-optimal, and, for unidentifiable problems, its worst-case revenue loss approaches the best-achievable performance guarantee among all data-driven algorithms. The numerical experiments demonstrate that our algorithm is highly effective and significantly improves both the expected revenue and worst-case revenue compared with three baseline algorithms.

Investigation of Melt Pool Geometry Control in Additive Manufacturing Using Hybrid Modeling

Metal additive manufacturing (AM) works on the principle of consolidating feedstock material in layers towards the fabrication of complex objects through localized melting and resolidification using high-power energy sources. Powder bed fusion and directed energy deposition are two widespread metal AM processes that are currently in use. During layer-by-layer fabrication, as the components continue to gain thermal energy, the melt pool geometry undergoes substantial changes if the process parameters are not appropriately adjusted on-the-fly. Although control of melt pool geometry via feedback or feedforward methods is a possibility, the time needed for changes in process parameters to translate into adjustments in melt pool geometry is of critical concern. A second option is to implement multi-physics simulation models that can provide estimates of temporal process parameter evolution. However, such models are computationally near intractable when they are coupled with an optimization framework for finding process parameters that can retain the desired melt pool geometry as a function of time. To address these challenges, a hybrid framework involving machine learning-assisted process modeling and optimization for controlling the melt pool geometry during the build process is developed and validated using experimental observations. A widely used 3D analytical model capable of predicting the thermal distribution in a moving melt pool is implemented and, thereafter, a nonparametric Bayesian, namely, Gaussian Process (GP), model is used for the prediction of time-dependent melt pool geometry (e.g., dimensions) at different values of the process parameters with excellent accuracy along with uncertainty quantification at the prediction points. Finally, a surrogate-assisted statistical learning and optimization architecture involving GP-based modeling and Bayesian Optimization (BO) is employed for predicting the optimal set of process parameters as the scan progresses to keep the melt pool dimensions at desired values. The results demonstrate that a model-based optimization can be significantly accelerated using tools of machine learning in a data-driven setting and reliable a priori estimates of process parameter evolution can be generated to obtain desired melt pool dimensions for the entire build process.

Variational framework for distance-minimizing method in data-driven computational mechanics

A variational framework for the distance-minimizing data-driven computing method is proposed. We provide a mathematical view on data-driven boundary value problems as well as the associated mathematical tools. The notion of function spaces is introduced into a data-driven boundary value problem and thus it can be formulated as a continuous optimization problem. An explanation of a data-driven boundary value problem as a double-minimization problem is given. This problem is subsequently broken down into two single-minimization problems, one of which is a constrained optimization problem. The associated constraint is given in terms of differential equations and the method of choice is the method of Lagrange multipliers. For the constrained optimization problem, we seek solutions by solving a system of equations for nodal degrees of freedom. The second is an unconstrained optimization problem whose solution is sought by working with the values of the solution at quadrature points and the supplied material data.The proposed variational formulation renders the high-order polynomial interpolation to straightforward implementation. In particular, spectral element methods are employed to reduce the computational cost while assuring the high accuracy of the data-driven solution. Within the spirit of the variational formulation, diffusion and dynamic problems can be efficiently studied in the data-driven setting with only few necessary modifications. Some representative numerical examples for parabolic and hyperbolic equations are presented. Numerical convergence studies show the reliability and robustness of the proposed formulation.

Sampling-Based Approximation Schemes for Capacitated Stochastic Inventory Control Models

We study the classical multiperiod capacitated stochastic inventory control problems in a data-driven setting. Instead of assuming full knowledge of the demand distributions, we assume that the demand distributions can only be accessed through drawing random samples. Such data-driven models are ubiquitous in practice, where the cumulative distribution functions of the underlying random demand are either unavailable or too complex to work with. We consider the sample average approximation (SAA) method for the problem and establish an upper bound on the number of samples needed for the SAA method to achieve a near-optimal expected cost, under any level of required accuracy and prespecified confidence probability. The sample bound is polynomial in the number of time periods as well as the confidence and accuracy parameters. Moreover, the bound is independent of the underlying demand distributions. However, the SAA requires solving the SAA problem, which is #P-hard. Thus, motivated by the SAA analysis, we propose a polynomial time approximation scheme that also uses polynomially many samples. Finally, we establish a lower bound on the number of samples required to solve this data-driven newsvendor problem to near-optimality.

Data-Driven Approximation Schemes for Joint Pricing and Inventory Control Models

We study the classic multiperiod joint pricing and inventory control problem in a data-driven setting. In this problem, a retailer makes periodic decisions on the prices and inventory levels of a product that she wishes to sell. The retailer's objective is to maximize the expected profit over a finite horizon by matching the inventory level with a random demand, which depends on the price in each period. In reality, the demand functions or random noise distributions are usually difficult to know exactly, whereas past demand data are relatively easy to collect. We propose a data-driven approximation algorithm that uses pre-collected demand data to solve the joint pricing and inventory control problem. We assume that the retailer does not know the noise distributions or the true demand functions; instead, we assume either she has access to demand hypothesis sets and the true demand functions can be represented by nonnegative combinations of candidate functions in the demand hypothesis sets, or the true demand function is generalized linear. We assume that the retailer does not know the noise distributions or true demand functions; instead, we assume that she has access only to demand hypothesis sets, while the true demand function is a nonnegative combination of the candidate demand functions in the demand hypothesis set or the true demand function is generalized linear. We prove the algorithm's sample complexity bound: the number of data samples needed in each period to guarantee a near-optimal profit is O(T6e −2 log T), where T is the number of periods, and e is the absolute difference between the expected profit of the data-driven policy and the expected optimal profit. In a numerical study, we demonstrate the construction of demand hypothesis sets from data, and show that the proposed data-driven algorithm solves the dynamic problem effectively and significantly improves the optimality gaps over the baseline algorithms.

Joint Pricing and Production: A Fusion of Machine Learning and Robust Optimization

We introduce a new distributionally robust optimization model to address a two-period, multi-item joint pricing and production problem, which can be implemented in a data-driven setting using historical demand and side information pertinent to the prediction of demands. Starting from an additive demand model we introduce a new partitioned-moment-based ambiguity set to characterize its residuals. Unlike the standard moment-based ambiguity set, we can adjust the level of robustness by varying the number of information clusters from being the most robust as the standard moment-based ambiguity set with one cluster to being the least robust as the empirical distribution. The partitioned-moment-based ambiguity set also addresses the key challenges in the stochastic dynamic optimization problem to determine how the second-period demand would evolve from the first-period information in a data-driven setting, without the need to impose additional assumptions on the distribution of demands such as independence. In addition, it also inspires a practicable non-anticipative policy that is adapted to the cluster. In particular, we investigate the joint pricing and production problem by proposing a cluster-adapted markdown policy and an affine recourse approximation, which allow us to reformulate the problem as a mixed-integer linear optimization problem that we can solve to optimality using commercial solvers. Both the numerical experiments and case study demonstrate that, with only a few number of clusters, the cluster-adapted markdown policy and the partitioned-moment-based ambiguity set can improve mean profit over the empirical model---when applied to most out-of-sample tests.

Joint Price and Inventory Optimization Under Minimax Regret

We study the problem of jointly optimizing the price and order quantity for a perishable product in a single selling period, also known as the pricing newsvendor problem, under demand ambiguity. Specifically, the demand is a function of the selling price and an uncertain factor where only the interval within which the uncertain factor lies with high confidence is known. We employ the minimax regret decision criterion to minimize the worst-case regret, where the regret is defined as the difference between the optimal profit that could be obtained with perfect information and the realized profit using the decision made with ambiguous demand information. First, we characterize the optimal pricing and ordering decisions under the minimax regret criterion and compare their properties with those in the classical models that seek to maximize the expected profit. In specific, we explore the impact of inventory risk by comparing the optimal price and the risk-free price, and study comparative statics with respect to the degree of demand ambiguity and the unit ordering cost. Second, we compare the minimax regret approach with two other approaches that are commonly used under demand ambiguity, namely the max-min robust approach and the regression-based profit maximization approach. In the demand ambiguity setting, we show that the minimax regret approach avoids the high degree of conservativeness that is often incurred in the max-min robust approach. In the data-driven setting, we show via a numerical study that the minimax regret approach outperforms the regression-based profit maximization approach when the data is scarce, when the demand has high volatility, or when the demand model is misspecified.

Multiple Illuminant Color Estimation via Statistical Inference on Factor Graphs.

This paper presents a method to recover a spatially varying illuminant color estimate from scenes lit by multiple light sources. Starting with the image formation process, we formulate the illuminant recovery problem in a statistically data-driven setting. To do this, we use a factor graph defined across the scale space of the input image. In the graph, we utilize a set of illuminant prototypes computed using a data driven approach. As a result, our method delivers a pixelwise illuminant color estimate being devoid of libraries or user input. The use of a factor graph also allows for the illuminant estimates to be recovered making use of a maximum a posteriori inference process. Moreover, we compute the probability marginals by performing a Delaunay triangulation on our factor graph. We illustrate the utility of our method for pixelwise illuminant color recovery on widely available data sets and compare against a number of alternatives. We also show sample color correction results on real-world images.

Information Fusion of Passive Sensors for Detection of Moving Targets in Dynamic Environments.

This paper addresses the problem of target detection in dynamic environments in a semi-supervised data-driven setting with low-cost passive sensors. A key challenge here is to simultaneously achieve high probabilities of correct detection with low probabilities of false alarm under the constraints of limited computation and communication resources. In general, the changes in a dynamic environment may significantly affect the performance of target detection due to limited training scenarios and the assumptions made on signal behavior under a static environment. To this end, an algorithm of binary hypothesis testing is proposed based on clustering of features extracted from multiple sensors that may observe the target. First, the features are extracted individually from time-series signals of different sensors by using a recently reported feature extraction tool, called symbolic dynamic filtering. Then, these features are grouped as clusters in the feature space to evaluate homogeneity of the sensor responses. Finally, a decision for target detection is made based on the distance measurements between pairs of sensor clusters. The proposed procedure has been experimentally validated in a laboratory setting for mobile target detection. In the experiments, multiple homogeneous infrared sensors have been used with different orientations in the presence of changing ambient illumination intensities. The experimental results show that the proposed target detection procedure with feature-level sensor fusion is robust and that it outperforms those with decision-level and data-level sensor fusion.

Data-driven chance constrained stochastic program

In this paper, we study data-driven chance constrained stochastic programs, or more specifically, stochastic programs with distributionally robust chance constraints (DCCs) in a data-driven setting to provide robust solutions for the classical chance constrained stochastic program facing ambiguous probability distributions of random parameters. We consider a family of density-based confidence sets based on a general $$\phi $$ź-divergence measure, and formulate DCC from the perspective of robust feasibility by allowing the ambiguous distribution to run adversely within its confidence set. We derive an equivalent reformulation for DCC and show that it is equivalent to a classical chance constraint with a perturbed risk level. We also show how to evaluate the perturbed risk level by using a bisection line search algorithm for general $$\phi $$ź-divergence measures. In several special cases, our results can be strengthened such that we can derive closed-form expressions for the perturbed risk levels. In addition, we show that the conservatism of DCC vanishes as the size of historical data goes to infinity. Furthermore, we analyze the relationship between the conservatism of DCC and the size of historical data, which can help indicate the value of data. Finally, we conduct extensive computational experiments to test the performance of the proposed DCC model and compare various $$\phi $$ź-divergence measures based on a capacitated lot-sizing problem with a quality-of-service requirement.

Sampling-Based Approximation Schemes for Capacitated Stochastic Inventory Control Models

We study the classical multi-period capacitated stochastic inventory control problems in a data-driven setting. The objective is to match the on-hand inventory level with the random demand in each period, subject to supply constraints, while minimizing expected cost over a finite time horizon. Instead of assuming a full knowledge of the demand distributions, we assume that they can only be accessed through drawing random samples. Such data-driven models are ubiquitous in practice, where the cumulative distribution functions of the underlying random variables are either unavailable or too complicated to work with. We apply the Sample Average Approximation (SAA) method to the capacitated inventory control problem and establish an upper bound on the number of samples needed for the SAA method to achieve a near-optimal expected cost, under any level of required accuracy and prespecified confidence probability. The sample bound is polynomial in the number of time periods as wel as the confidence and accuracy parameters. Moreover the bound is independent of the underlying demand distributions. We crucially use an inequality of Massart, which is a stronger version of Chernoff inequality, to ensure a uniform approximation on the right derivatives.While the SAA method uses polynomially many samples, we show that the underlying SAA problem is in fact #P-hard. Thus the SAA method, which solves the SAA problem to optimality, is not a polynomial time algorithm in general, unless #P = P. Nevertheless, motivated by the SAA analysis, we propose a randomized polynomial time approximation scheme, which also uses polynomially many samples. The approximation scheme involves a sparsification procedure on the right derivatives of the cost-to-go functions, which maintains the tractability of the underlying dynamic program. Finally, we establish a lower bound on the number of samples required to solving the data-driven problem to near-optimality.

Data-driven non-Markovian closure models

This paper has two interrelated foci: (i) obtaining stable and efficient data-driven closure models by using a multivariate time series of partial observations from a large-dimensional system; and (ii) comparing these closure models with the optimal closures predicted by the Mori–Zwanzig (MZ) formalism of statistical physics. Multilayer stochastic models (MSMs) are introduced as both a generalization and a time-continuous limit of existing multilevel, regression-based approaches to closure in a data-driven setting; these approaches include empirical model reduction (EMR), as well as more recent multi-layer modeling. It is shown that the multilayer structure of MSMs can provide a natural Markov approximation to the generalized Langevin equation (GLE) of the MZ formalism.A simple correlation-based stopping criterion for an EMR–MSM model is derived to assess how well it approximates the GLE solution. Sufficient conditions are derived on the structure of the nonlinear cross-interactions between the constitutive layers of a given MSM to guarantee the existence of a global random attractor. This existence ensures that no blow-up can occur for a broad class of MSM applications, a class that includes non-polynomial predictors and nonlinearities that do not necessarily preserve quadratic energy invariants.The EMR–MSM methodology is first applied to a conceptual, nonlinear, stochastic climate model of coupled slow and fast variables, in which only slow variables are observed. It is shown that the resulting closure model with energy-conserving nonlinearities efficiently captures the main statistical features of the slow variables, even when there is no formal scale separation and the fast variables are quite energetic.Second, an MSM is shown to successfully reproduce the statistics of a partially observed, generalized Lotka–Volterra model of population dynamics in its chaotic regime. The challenges here include the rarity of strange attractors in the model’s parameter space and the existence of multiple attractor basins with fractal boundaries. The positivity constraint on the solutions’ components replaces here the quadratic-energy–preserving constraint of fluid-flow problems and it successfully prevents blow-up.

Modeling Freedom From Progression for Standard-Risk Medulloblastoma: A Mathematical Tumor Control Model With Multiple Modes of Failure

As pediatric medulloblastoma (MB) is a relatively rare disease, it is important to extract the maximum information from trials and cohort studies. Here, a framework was developed for modeling tumor control with multiple modes of failure and time-to-progression for standard-risk MB, using published pattern of failure data. Outcome data for standard-risk MB published after 1990 with pattern of relapse information were used to fit a tumor control dose-response model addressing failures in both the high-dose boost volume and the elective craniospinal volume. Estimates of 5-year event-free survival from 2 large randomized MB trials were used to model the time-to-progression distribution. Uncertainty in freedom from progression (FFP) was estimated by Monte Carlo sampling over the statistical uncertainty in input data. The estimated 5-year FFP (95% confidence intervals [CI]) for craniospinal doses of 15, 18, 24, and 36 Gy while maintaining 54 Gy to the posterior fossa was 77% (95% CI, 70%-81%), 78% (95% CI, 73%-81%), 79% (95% CI, 76%-82%), and 80% (95% CI, 77%-84%) respectively. The uncertainty in FFP was considerably larger for craniospinal doses below 18 Gy, reflecting the lack of data in the lower dose range. Estimates of tumor control and time-to-progression for standard-risk MB provides a data-driven setting for hypothesis generation or power calculations for prospective trials, taking the uncertainties into account. The presented methods can also be applied to incorporate further risk-stratification for example based on molecular biomarkers, when the necessary data become available.

The Added Challenges of Bone Metastases Treatment in Elderly Patients

Most cancers occur in those older than the age of 65 years. As the population of the world ages and life expectancies continue to increase, it is important to address treatment challenges for elderly patients. This narrative review details the challenges of palliative radiotherapy treatment for elderly patients with bone metastases. We begin with the definition of elderly and its appropriateness, outlining recent demographic data of patients with cancer. The current status of elderly participation in clinical trials is discussed by reviewing the recent literature and clinical trial data. Factors affecting enrolment of the elderly are assessed, with a focus on palliative radiotherapy trials, and what we can do to improve accrual in this data-driven setting. At present, there is a lack of level 1 evidence that evaluates the optimal treatment for elderly patients with bone metastases. Therefore, a review of safety and efficacy is given based on previously published reports. Palliative radiotherapy for elderly patients is a worthwhile treatment and should be recommended regardless of age, as supported by available evidence. Patient, family and physician concerns about physical burden may be reduced as single treatments (that often can be done in a single visit) are as beneficial as multiple treatments for painful bone metastases. In elderly patients, radiotherapy may even be the best treatment for these cases as opioid-related adverse events are amplified in this group and often dosages are more difficult to titrate. Clinicians should continue to encourage the enrolment of elderly patients on to clinical trials as these data form the basis of optimal treatment guidelines. Radiation oncologists are encouraged to reduce the physical burden for elderly patients by offering single treatments where appropriate and completing consultation, treatment simulation and treatment in a single clinical visit.