Data-driven Tuning Research Articles

Small Tuning Parameter Selection for the Debiased Lasso

In this study, we investigate the bias and variance properties of the debiased Lasso in linear regression when the tuning parameter of the node-wise Lasso is selected to be smaller than in previous studies. We consider the case where the number of covariates p is bounded by a constant multiple of the sample size n. First, we show that the bias of the debiased Lasso can be reduced without diverging the asymptotic variance by setting the order of the tuning parameter to 1 / n . This implies that the debiased Lasso has asymptotic normality provided that the number of nonzero coefficients s 0 satisfies s 0 = o ( n / log p ) , whereas previous studies require s 0 = o ( n / log p ) if no sparsity assumption is imposed on the inverse of the second moment matrix of covariates. Second, we propose a data-driven tuning parameter selection procedure for the node-wise Lasso that is consistent with our theoretical results. Simulation studies show that our procedure yields confidence intervals with good coverage properties in various settings. We also present a real economic data example to demonstrate the efficacy of our selection procedure.

Experimentally-efficient data-driven rational feedforward control for multivariable systems

Feedforward control with task flexibility for MIMO systems is essential to meet the growing demands on throughput and accuracy of high-tech systems. The aim of this paper is to develop an experimentally efficient framework for data-driven tuning of rational feedforward controllers for general non-commutative MIMO systems. In the developed approach, the nonlinear terms in the non-convex optimisation problem of iterative learning control (ILC) are iteratively circumvented via approximation. This leads to a series of convex optimisation problems that can be solved offline to obtain the parameters of the rational feedforward controller. In addition, by limiting the number of offline iterations an experimentally intensive algorithm is derived, which could be beneficial in the case of severe model mismatch. A simulation study shows that the experimentally efficient approach converges fast through offline iterations and has improved convergence properties through the use of regularisation.

Practical one-shot data-driven design of fractional-order PID controller: Fictitious reference signal approach

This study proposes a one-shot data-driven tuning method for a fractional-order proportional-integral-derivative (FOPID) controller. The proposed method tunes the FOPID controller in the model-reference control formulation. A loss function is defined to evaluate the match between a given reference model and the closed-loop response while explicitly considering the closed-loop stability. A loss function value is based on the fictitious reference signal computed using the input/output data. Model matching is achieved via loss function minimization. The proposed method is simple and practical: it needs only one-shot input/output data of a plant (no plant model required), considers the bounded-input bounded-output stability of the closed-loop system from a bounded reference input to a bounded output, and automatically determines the appropriate parameter value via optimization. Numerical simulations show that the proposed approach facilitates good control performance, and destabilization can be avoided even if perfect model matching is unachievable.

Evaluating and improving health equity and fairness of polygenic scores

Polygenic scores (PGSs) are quantitative metrics for predicting phenotypic values, such as human height or disease status. Some PGS methods require only summary statistics of a relevant genome-wide association study (GWAS) for their score. One such method is Lassosum, which inherits the model selection advantages of Lasso to select a meaningful subset of the GWAS single-nucleotide polymorphisms as predictors from their association statistics. However, even efficient scores like Lassosum, when derived from European-based GWASs, are poor predictors of phenotype for subjects of non-European ancestry; that is, they have limited portability to other ancestries. To increase the portability of Lassosum, when GWAS information and estimates of linkage disequilibrium are available for both ancestries, we propose Joint-Lassosum (JLS). In the simulation settings we explore, JLS provides more accurate PGSs compared to other methods, especially when measured in terms of fairness. In analyses of UK Biobank data, JLS was computationally more efficient but slightly less accurate than a Bayesian comparator, SDPRX. Like all PGS methods, JLS requires selection of predictors, which are determined by data-driven tuning parameters. We describe a new approach to selecting tuning parameters and note its relevance for model selection for any PGS. We also draw connections to the literature on algorithmic fairness and discuss how JLS can help mitigate fairness-related harms that might result from the use of PGSs in clinical settings. While no PGS method is likely to be universally portable, due to the diversity of human populations and unequal information content of GWASs for different ancestries, JLS is an effective approach for enhancing portability and reducing predictive bias.

Adaptive data-driven selection of sequences of biological and cognitive markers in pre-clinical diagnosis of dementia

Effective clinical decision procedures must balance multiple competing objectives such as time-to-decision, acquisition costs, and accuracy. We describe and evaluate POSEIDON, a data-driven method for PrOspective SEquentIal DiagnOsis with Neutral zones to individualize clinical classifications. We evaluated the framework with an application in which the algorithm sequentially proposes to include cognitive, imaging, or molecular markers if a sufficiently more accurate prognosis of clinical decline to manifest Alzheimer’s disease is expected. Over a wide range of cost parameter data-driven tuning lead to quantitatively lower total cost compared to ad hoc fixed sets of measurements. The classification accuracy based on all longitudinal data from participants that was acquired over 4.8 years on average was 0.89. The sequential algorithm selected 14 percent of available measurements and concluded after an average follow-up time of 0.74 years at the expense of 0.05 lower accuracy. Sequential classifiers were competitive from a multi-objective perspective since they could dominate fixed sets of measurements by making fewer errors using less resources. Nevertheless, the trade-off of competing objectives depends on inherently subjective prescribed cost parameters. Thus, despite the effectiveness of the method, the implementation into consequential clinical applications will remain controversial and evolve around the choice of cost parameters.

Robust regression for electricity demand forecasting against cyberattacks

Standard methods for forecasting electricity loads are not robust to cyberattacks on electricity demand data, potentially leading to severe consequences such as major economic loss or a system blackout. Methods are required that can handle forecasting under these conditions and detect outliers that would otherwise go unnoticed. The key challenge is to remove as many outliers as possible while maintaining enough clean data to use in the regression. In this paper we investigate robust approaches with data-driven tuning parameters, and in particular present an adaptive trimmed regression method that can better detect outliers and provide improved forecasts. In general, data-driven approaches perform much better than their fixed tuning parameter counterparts. Recommendations for future work are provided.

Robust empirical likelihood inference for partially linear varying coefficient models with longitudinal data

This paper presents a robust empirical likelihood procedure based on the exponential squared loss (ESL) function and leverage-based weights for the partially linear varying coefficient model with longitudinal data. The proposed method simultaneously solves the problems of correlation structure of longitudinal data and the existence of outliers, and achieves robustness and efficiency by introducing an appropriate data-driven tuning parameter. More importantly, profit from the QR decomposition technique, our method allows the parametric and nonparametric parts of the models to be estimated separately, which can avoid the mutual influence between them and make the implementation easier. Under some mild conditions, the large sample theoretical properties of the robust empirical likelihood approach are established. Simulation studies and a real data analysis are also carried out to assess and illustrate the finite sample performance.

Data-driven tuning for chance constrained optimization: analysis and extensions

Data-driven tuning for chance constrained optimization: analysis and extensions

Tuning particle accelerators with safety constraints using Bayesian optimization

Tuning machine parameters of particle accelerators is a repetitive and time-consuming task that is challenging to automate. While many off-the-shelf optimization algorithms are available, in practice their use is limited because most methods do not account for safety-critical constraints in each iteration, such as loss signals or step-size limitations. One notable exception is safe Bayesian optimization, which is a data-driven tuning approach for global optimization with noisy feedback. We propose and evaluate a step-size limited variant of safe Bayesian optimization on two research facilities of the PSI: (a) the SwissFEL and (b) HIPA. We report promising experimental results on both machines, tuning up to 16 parameters subject to 224 constraints.

Non-Iterative Data-Driven Tuning of Model-Free Control Based on an Ultra-Local Model

In this paper, we present a data-driven tuning method for model-free control based on an ultra-local model (MFC-ULM), which is also called intelligent proportional-integral-derivative control. In industries, the control design must be easy, and it is important that the control law can be applied to nonlinear systems. The MFC-ULM has most of these features. However, in practice, trial-and-error tuning of MFC-ULM design parameters is necessary. To address this problem, we adopt a data-driven tuning approach. In the proposed method, the MFC-ULM design parameters can be tuned from single-experiment data without requiring system identification, and optimal parameters for the MFC-ULM are obtained using the least-squares method. Additionally, we adopt L2-norm regularization to avoid overlearning. The effectiveness of this method was examined using simulations of two nonlinear systems. The results revealed that the MFC-ULM design parameters can be obtained directly without knowing the characteristics of the controlled object.

ClassyTune: A Performance Auto-Tuner for Systems in the Cloud

Performance tuning can improve the system performance and thus enable the reduction of cloud computing resources needed to support an application. Due to the ever increasing number of parameters and complexity of systems, there is a necessity to automate performance tuning for the complicated systems in the cloud. The state-of-the-art tuning methods are adopting either the experience-driven tuning approach or the data-driven one. Data-driven tuning is attracting increasing attentions, as it has wider applicability. But existing data-driven methods cannot fully address the challenges of sample scarcity and high dimensionality simultaneously. We present ClassyTune, a data-driven automatic configuration tuning tool for cloud systems. ClassyTune exploits the machine learning model of classification for auto-tuning. This exploitation enables the induction of more training samples without increasing the input dimension. Experiments on seven popular systems in the cloud show that ClassyTune can effectively tune system performance to seven times higher for high-dimensional configuration space, outperforming expert tuning and the state-of-the-art auto-tuning solutions. We also describe a use case in which performance tuning enables the reduction of 33% computing resources needed to run an online stateless service.

Two-Degree-of-Freedom Controller Design Based on a Data-Driven Estimation Approach

Data-driven control design is a method to create and tune controllers directly from the initial experimental data without a mathematical model to be controlled. Tracking and disturbance suppression are necessary to control real systems. A two-degree-of-freedom (2DOF) control system is effective to simultaneously enhance the performances of both. This study proposes a direct data-driven tuning method for the controller parameters of a 2DOF control system using only one-shot initial experimental data without mathematical modeling of the controlled object. The proposed approach improves the tracking and disturbance suppression performances by utilizing an estimation method in which the sensitivity function and the closed-loop transfer function are identified after updating the parameters in the time domain. Specifically, the closed-loop response and control input are estimated after updating the parameters, realizing efficient control system design because the control performance can be evaluated prior to implementation. To validate the effectiveness of the proposed method, a simulation for a mechanical system and an experiment for motor control are performed. The proposed method can estimate the response and control input of a 2DOF control system after updating the parameters by offline computations. The optimal control parameters can be obtained to enhance the tracking and disturbance suppression performances.

Data-Driven Tuning for Chance-Constrained Optimization: Two Steps Towards Probabilistic Performance Guarantees

Parameters involved in the formulation of optimization problems are often partially unknown or random. A popular way to mitigate the effect of uncertainty is using joint chance constraints, which guarantee constraint satisfaction with high probability but are challenging to solve. In this letter, we analyze an approach for joint chance constrained problems that involves iteratively tuning problem parameters. We first show that existing naive approaches to tuning can lead to solutions without feasibility guarantees. We then introduce a two-step approach, where a tuning-based solution generation step is followed by an a posteriori solution verification step. A main challenge of the two-step approach is to guarantee that the solution generated in the first step has a high probability of being verified as feasible in the second step. We therefore analyze how the relationship between the feasibility criteria used in each step impacts the probability of obtaining a feasible solution. We demonstrate our results in a numerical case study of the optimal power flow problem.

Robust Self-Tuning Semiparametric PCA for Contaminated Elliptical Distribution

Principal component analysis (PCA) is one of the most popular dimension reduction methods. The usual PCA is known to be sensitive to the presence of outliers, and thus many robust PCA methods have been developed. Among them, the Tyler's M-estimator is shown to be the most robust scatter estimator under the elliptical distribution. However, when the underlying distribution is contaminated and deviates from ellipticity, Tyler's M-estimator might not work well. In this article, we apply the semiparametric theory to propose a robust semiparametric PCA. The merits of our proposal are twofold. First, it is robust to heavy-tailed elliptical distributions as well as robust to non-elliptical outliers. Second, it pairs well with a data-driven tuning procedure, which is based on active ratio and can adapt to different degrees of data outlyingness. Theoretical properties are derived, including the influence functions for various statistical functionals and asymptotic normality. Simulation studies and a data analysis demonstrate the superiority of our method.

Data-driven Tuning Based on Virtual Internal Model Tuning Considering Input Limitation

Data-driven Tuning Based on Virtual Internal Model Tuning Considering Input Limitation

Data-driven tuning of model-reference controllers for stable MIMO plants

A new data-driven tuning method of a linearly parametrized controller is proposed for stable multi-input multi-output plants. A criterion that evaluates the difference between responses of the reference model and those of the feedback system is employed for tracking control. Because the criterion is nonconvex with respect to controller parameters, its second order Taylor expansion, which is convex, is used for parameter tuning. To represent the approximated criterion using plant responses instead of a plant mathematical model, it is proposed to use the responses of all entries of the plant for a test function, typically a step function. The parameter value that minimizes the approximated data-based criterion is obtained adopting the linear least-squares method. The proposed tuning method is compared with virtual reference feedback tuning and its extended methods for multi-input multi-output plants.

A Survey on Data-driven Performance Tuning for Big Data Analytics Platforms

Many research works deal with big data platforms looking forward to data science and analytics. These are complex and usually distributed environments, composed of several systems and tools. As expected, there is a need for a closer look at performance issues. In this work, we review performance tuning strategies in the big data environment. We focus on data-driven tuning techniques, discussing the use of database inspired approaches. Concerning big data and NoSQL stores, performance tuning issues are quite different from the so-called conventional systems. Many existing solutions are mostly ad-hoc activities that do not fit for multiple situations. But there are some categories of data-driven solutions that can be taken as guidelines and incorporated into general-purpose auto-tuning modules for big data systems. We examine typical performance tuning actions, discussing available solutions to support some of the tuning process's primary activities. We also discuss recent implementations of data-driven performance tuning solutions for big data platforms. We propose an initial classification based on the domain state-of-the-art and present selected tuning actions for large-scale data processing systems. Finally, we organized existing works towards self-tuning big data systems based on this classification and presented general and system-specific tuning recommendations. We found that most of the literature pieces evaluate the use of tuning actions at the physical design perspective, and there is a lack of self-tuning machine-learning-based solutions for big data systems.

Limited-complexity controller tuning: A set membership data-driven approach

Data-driven tuning is an alternative to model-based controller design where controllers are directly identified from data, avoiding a plant identification step. In this paper, an approach to tune limited-complexity controllers from data for linear systems is proposed. The controller is parametrized as a linear combination of a large set of basis functions and the proposed algorithm allows to select a sparse subset of bases, guaranteeing a bounded approximation error. A feasibility condition allows to adjust the trade-off between accuracy and sparsity. The controller design is performed by solving a set of linear programming problems, allowing to handle large data-sets. The proposed strategy is evaluated by means of a Monte-Carlo simulation experiment on a flexible transmission benchmark model. Results show that the proposed solution offers similar results than previous approaches for large data-sets, requiring less adjustable parameters. However, for reduced data-sets, the presented algorithm shows better performance than the compared approaches.

Optimizing Regularized Cholesky Score for Order-Based Learning of Bayesian Networks.

Bayesian networks are a class of popular graphical models that encode causal and conditional independence relations among variables by directed acyclic graphs (DAGs). We propose a novel structure learning method, annealing on regularized Cholesky score (ARCS), to search over topological sorts, or permutations of nodes, for a high-scoring Bayesian network. Our scoring function is derived from regularizing Gaussian DAG likelihood, and its optimization gives an alternative formulation of the sparse Cholesky factorization problem from a statistical viewpoint. We combine simulated annealing over permutation space with a fast proximal gradient algorithm, operating on triangular matrices of edge coefficients, to compute the score of any permutation. Combined, the two approaches allow us to quickly and effectively search over the space of DAGs without the need to verify the acyclicity constraint or to enumerate possible parent sets given a candidate topological sort. The annealing aspect of the optimization is able to consistently improve the accuracy of DAGs learned by greedy and deterministic search algorithms. In addition, we develop several techniques to facilitate the structure learning, including pre-annealing data-driven tuning parameter selection and post-annealing constraint-based structure refinement. Through extensive numerical comparisons, we show that ARCS outperformed existing methods by a substantial margin, demonstrating its great advantage in structure learning of Bayesian networks from both observational and experimental data. We also establish the consistency of our scoring function in estimating topological sorts and DAG structures in the large-sample limit. Source code of ARCS is available at https://github.com/yeqiaoling/arcs_bn.

Cascade Control: Data-Driven Tuning Approach Based on Bayesian Optimization

Cascaded controller tuning is a multi-step iterative procedure that needs to be performed routinely upon maintenance and modification of mechanical systems. An automated data-driven method for cascaded controller tuning based on Bayesian optimization is proposed. The method is tested on a linear axis drive, modeled using a combination of first principles model and system identification. A custom cost function based on performance indicators derived from system data at different candidate configurations of controller parameters is modeled by a Gaussian process. It is further optimized by minimization of an acquisition function which serves as a sampling criterion to determine the subsequent candidate configuration for experimental trial and improvement of the cost model iteratively, until a minimum according to a termination criterion is found. This results in a data-efficient procedure that can be easily adapted to varying loads or mechanical modifications of the system. The method is further compared to several classical methods for auto-tuning, and demonstrates higher performance according to the defined data-driven performance indicators. The influence of the training data on a cost prior on the number of iterations required to reach optimum is studied, demonstrating the efficiency of the Bayesian optimization tuning method.