Smooth Penalty Research Articles

Distributed predefined-time optimal economic dispatch for microgrids

With the massive popularization of distributed generators, optimal economic dispatch has been a key optimization problem to maintain stable and efficient work of the whole system. In this paper, a new smooth reconstruction penalty function with continuous and piecewise linear differential is designed to deal with generation power constraints, which promotes to obtain a better suboptimal solution compared with the existing smooth penalty methods. A distributed predefined-time optimal economic dispatch strategy is presented by utilizing a time-based function. By employing the proposed strategy, the minimization of the generation cost with transmission loss under the power balance constraint and generation minimum/maximum constraints can be realized within a predefined settling time. The performance of the proposed optimization strategy is evaluated by simulations and hardware-in-the-loop experiments in terms of validity verification, robustness to load change and topology reconfiguration, plug-and-play functionality, and comparison with the existing results to illustrate the advantages of fast convergence and near optimal results.

Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding

Curriculum learning and self-paced learning are the training strategies that gradually feed the samples from easy to more complex. They have captivated increasing attention due to their excellent performance in robotic vision. Most recent works focus on designing curricula based on difficulty levels in input samples or smoothing the feature maps. However, smoothing labels to control the learning utility in a curriculum manner is still unexplored. In this work, we design a paced curriculum by label smoothing (P-CBLS) using paced learning with uniform label smoothing (ULS) for classification tasks and fuse uniform and spatially varying label smoothing (SVLS) for semantic segmentation tasks in a curriculum manner. In ULS and SVLS, a bigger smoothing factor value enforces a heavy smoothing penalty in the true label and limits learning less information. Therefore, we design the curriculum by label smoothing (CBLS). We set a bigger smoothing value at the beginning of training and gradually decreased it to zero to control the model learning utility from lower to higher. We also designed a confidence-aware pacing function and combined it with our CBLS to investigate the benefits of various curricula. The proposed techniques are validated on four robotic surgery datasets of multi-class, multi-label classification, captioning, and segmentation tasks. We also investigate the robustness of our method by corrupting validation data into different severity levels. Our extensive analysis shows that the proposed method improves prediction accuracy and robustness. The code is publicly available at https://github.com/XuMengyaAmy/P-CBLS. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this article is to improve the performance and robustness of deep neural networks in safety-critical applications such as robotic surgery by controlling the learning ability of the model in a curriculum learning manner and allowing the model to imitate the cognitive process of humans and animals. The designed approaches do not add parameters that require additional computational resources.

Fast generation of collision-free low-thrust trajectories for asteroid landing using deep neural networks

This study presents a fast generation method of time-optimal low-thrust trajectories based on deep neural networks (DNN) for the collision-free asteroid landings. An equivalent unconstrained differentiable problem is transformed via the introduction of smooth penalty function, so that it can be addressed with the optimal control framework. To efficiently solve the equivalent problem to generate sufficient dataset for training DNNs, a double homotopy method is developed. Specifically, the equivalent problem is connected to the time-optimal control problem without anti-collision through two consecutive simplifications, and a solving process with double backward continuation is also formulated. Besides, two DNN models are constructed to provide high-quality predictions for the minimum glide-slope constraint angle of an initial state, and the initial costates and landing time for time-optimal low-thrust trajectories with anti-collision constraints, respectively. As such, the efficiency of collision-free low-thrust landing trajectory generation is improved significantly. For the cases within the range of training dataset or within a certain extension, the low-thrust landing trajectories with a very small terminal landing error can be generated using the predicted initial costates. Meanwhile, with the high-quality initial costates, the high-precision landing trajectories can be obtained by the double homotopy method in only a few iterative steps. Finally, the promising results in a Bennu collision-free asteroid landing scenario validate the efficiency of the proposed method.

Advancing Glaucoma Diagnosis: Employing Confidence-Calibrated Label Smoothing Loss for Model Calibration

ObjectiveThe aim of our research is to enhance the calibration of machine learning models for glaucoma classification through a specialized loss function named Confidence-Calibrated Label Smoothing Loss (CC-LS). This approach is specifically designed to refine model calibration without compromising accuracy by integrating label smoothing and confidence penalty techniques, tailored to the specifics of glaucoma detection. DesignThis study focuses on the development and evaluation of a calibrated deep learning model. ParticipantsThe study employs fundus images from both external datasets-ORIGA (482 normal, 168 glaucoma) and REFUGE (720 normal, 80 glaucoma)-and an extensive internal dataset (4,639 images per category), aiming to bolster the model's generalizability. The model's clinical performance is validated using a comprehensive test set (47,913 normal, 1,629 glaucoma) from the internal dataset. MethodsThe CC-LS loss function seamlessly integrates label smoothing, which tempers extreme predictions to avoid overfitting, with confidence-based penalties. These penalties deter the model from expressing undue confidence in incorrect classifications. Our study aims at training models using the CC-LS and comparing their performance with those trained using conventional loss functions. Main Outcome MeasuresThe model's precision is evaluated using metrics like the Brier score, sensitivity, specificity, and the false positive rate, alongside qualitative heatmap analyses for a holistic accuracy assessment. ResultsPreliminary findings reveal that models employing the CC-LS mechanism exhibit superior calibration metrics, as evidenced by a Brier score of 0.098, along with notable accuracy measures: sensitivity of 81%, specificity of 80%, and weighted accuracy of 80%. Importantly, these enhancements in calibration are achieved without sacrificing classification accuracy. ConclusionsThe Confidence-Calibrated Label Smoothing Loss presents a significant advancement in the pursuit of deploying machine learning models for glaucoma diagnosis. By improving calibration, the CC-LS ensures that clinicians can interpret and trust the predictive probabilities, making AI-driven diagnostic tools more clinically viable. From a clinical standpoint, this heightened trust and interpretability can potentially lead to more timely and appropriate interventions, thereby optimizing patient outcomes and safety.

Robust Multi-Dimensional Time Series Forecasting.

Large-scale and high-dimensional time series data are widely generated in modern applications such as intelligent transportation and environmental monitoring. However, such data contains much noise, outliers, and missing values due to interference during measurement or transmission. Directly forecasting such types of data (i.e., anomalous data) can be extremely challenging. The traditional method to deal with anomalies is to cut out the time series with anomalous value entries or replace the data. Both methods may lose important knowledge from the original data. In this paper, we propose a multidimensional time series forecasting framework that can better handle anomalous values: the robust temporal nonnegative matrix factorization forecasting model (RTNMFFM) for multi-dimensional time series. RTNMFFM integrates the autoregressive regularizer into nonnegative matrix factorization (NMF) with the application of the L2,1 norm in NMF. This approach improves robustness and alleviates overfitting compared to standard methods. In addition, to improve the accuracy of model forecasts on severely missing data, we propose a periodic smoothing penalty that keeps the sparse time slices as close as possible to the time slice with high confidence. Finally, we train the model using the alternating gradient descent algorithm. Numerous experiments demonstrate that RTNMFFM provides better robustness and better prediction accuracy.

Statistical inference with regularized optimal transport

Abstract Optimal transport (OT) is a versatile framework for comparing probability measures, with many applications to statistics, machine learning and applied mathematics. However, OT distances suffer from computational and statistical scalability issues to high dimensions, which motivated the study of regularized OT methods like slicing, smoothing and entropic penalty. This work establishes a unified framework for deriving limit distributions of empirical regularized OT distances, semiparametric efficiency of the plug-in empirical estimator and bootstrap consistency. We apply the unified framework to provide a comprehensive statistical treatment of (i) average- and max-sliced $p$-Wasserstein distances, for which several gaps in existing literature are closed; (ii) smooth distances with compactly supported kernels, the analysis of which is motivated by computational considerations; and (iii) entropic OT, for which our method generalizes existing limit distribution results and establishes, for the first time, efficiency and bootstrap consistency. While our focus is on these three regularized OT distances as applications, the flexibility of the proposed framework renders it applicable to broad classes of functionals beyond these examples.

A Smooth Penalty Function Algorithm for Computing Nonlinear Inequality Constraint Optimization Problem

A Smooth Penalty Function Algorithm for Computing Nonlinear Inequality Constraint Optimization Problem

Research on the service condition monitoring method of rolling bearings based on adaptive deconvolution and transformer

Aiming at the complexity of the rolling bearing operation state in actual working conditions and the limitation of the feature extraction capability of single-scale model, this paper proposes a fault diagnosis method based on two-channel adaptive deconvolution and transformer module (TADAT). First, a frequency domain dataset is built and multiscale feature information is extracted using two-channel adaptive deconvolution at the beginning stage of the model; the global features of the signal are further extracted by combining them in series with the transformer (multi-head attention). Next, the loss function is optimized by introducing a regularized loss function with label smoothing and L2 regularization penalty term. Finally, the model is validated against an open-source dataset and operating condition data collected from an unbalanced bearing load test bed with an accuracy of over 98.5%. The model also maintains high accuracy under noisy samples. The test shows that the rolling bearing service condition monitoring method based on TADAT feature extraction model proposed by this method is characterized by simple model structure, high diagnostic accuracy and strong generalization performance, which provides a useful reference for the research of rolling bearing service condition monitoring.

Association of gait and cognition after surgery in patients with idiopathic normal pressure hydrocephalus

Idiopathic normal pressure hydrocephalus (iNPH) is a treatable disease in older adults. The association between gait and cognition has recently become a topic of interest. Sequential changes in this association were investigated in patients with iNPH using a newly developed statistical method. Data were extracted from the SINPHONI-2 multicenter study on iNPH. Fifty patients who underwent shunt surgery were included in this study. Gait and cognition were assessed using the Timed Up and Go (TUG) and Mini-Mental State Examination (MMSE) tests. In addition to the MMSE total score, changes in the sub-item scores were examined. The ordinal sub-items of the MMSE are usually treated as continuous or categorical; however, both are unsuitable. An ordinal smoothing penalty with a generalized additive model enables precise statistical inference of ordinal and binary predictors. The TUG time improved significantly at 3, 6, and 12 months after surgery. The MMSE total scores increased without statistical significance. Preoperatively, there was no association between TUG time and MMSE sub-items. At 3 months, the “Registration,” ”3-step command,” “Read,” and “Copy” sub-items were statistically significant. The number of significant sub-items increased after 12 months. Thus, the association between gait and cognition gradually increased after surgery in patients with iNPH.

An exact penalty function optimization method and its application in stress constrained topology optimization and scenario based reliability design problems

A smooth penalty function method which does not involve dual and slack implicit variables to formulate constrained optimization conditions is proposed. New active and loss functions are devised to independently and adaptively handle the violation of each constraint function. A single unconstrained minimization of the proposed penalty function produces a solution to the original optimization problem. The constrained optimization conditions depending only on the primal variable are derived and reduce to the canonical Karush-Kuhn-Tucker(KKT) conditions in a much more general framework. The derivative of the proposed loss functions with respect to constraint function can be interpreted as the Lagrange multiplier of the traditional Lagrangian function. The appearance of the first-order Hessian information is unveiled. This is in contrast to the existing works where the classical Lagrange Hessian contains only second order derivatives. Finally, some numerical examples including medium-scale stress constrained topology optimization and scenario based reliability design problems are presented to demonstrate the effectiveness of the proposed methodology.

Hierarchical Nash equilibrium seeking strategies of quadratic time‐varying games with Euler–Lagrange players

AbstractThis article investigates the distributed Nash equilibrium seeking problem of quadratic time‐varying games with Euler–Lagrange (EL) players, where external disturbances and parametric uncertainties are involved. A gradient‐based hierarchical algorithm consisting of a game layer and a control layer is proposed. Specifically, in the game layer, EL players communicate with neighbors through a graph to reach the consensus on potential aggregate values, which will be employed to calculate the gradient of each player's objective function, and then, a gradient‐based sliding mode controller is developed to track time‐varying gradient in the control layer. Thus, the convergence results are hierarchically obtained through the Lyapunov stability method. In addition, the hierarchical control strategy is extended to address the constrained problems through the utilization of a smooth penalty function. By appropriately choosing control parameters, the Nash equilibrium seeking errors can be arbitrarily small. The relation between the optimal solutions of the original problem and the dual one is further discussed. Finally, the proposed methods are numerically verified.

Group sparse optimization for inpainting of random fields on the sphere

Abstract We propose a group sparse optimization model for inpainting of a square-integrable isotropic random field on the unit sphere, where the field is represented by spherical harmonics with random complex coefficients. In the proposed optimization model, the variable is an infinite-dimensional complex vector and the objective function is a real-valued function defined by a hybrid of the $\ell _2$ norm and non-Lipschitz $\ell _p (0&amp;lt;p&amp;lt;1)$ norm that preserves rotational invariance property and group structure of the random complex coefficients. We show that the infinite-dimensional optimization problem is equivalent to a convexly-constrained finite-dimensional optimization problem. Moreover, we propose a smoothing penalty algorithm to solve the finite-dimensional problem via unconstrained optimization problems. We provide an approximation error bound of the inpainted random field defined by a scaled Karush–Kuhn–Tucker (KKT) point of the constrained optimization problem in the square-integrable space on the sphere with probability measure. Finally, we conduct numerical experiments on band-limited random fields on the sphere and images from Cosmic Microwave Background (CMB) data to show the promising performance of the smoothing penalty algorithm for inpainting of random fields on the sphere.

Global convergence of a class new smooth penalty algorithm for constrained optimization problem

Global convergence of a class new smooth penalty algorithm for constrained optimization problem

Morphological component analysis under non-convex smoothing penalty framework for gearbox fault diagnosis

The sparse representation methodology has been identified to be a promising tool for gearbox fault diagnosis. The core is how to precisely reconstruct the fault signal from noisy monitoring signals. The non-convex penalty has the ability to induce sparsity more efficiently than convex penalty. However, the introduction of non-convex penalty usually influences the convexity of the model, resulting in the unstable or sub-optimal solution. In this paper, we propose the non-convex smoothing penalty framework (NSPF) and combine it with morphological component analysis (MCA) for gearbox fault diagnosis. The proposed NSPF is a unify penalty construction framework, which contains many classical penalty while a new set of non-convex smoothing penalty functions can be generated. These non-convex penalty can guarantee the convexity of the objective function while enhancing the sparsity, thus the global optimal solution can be acquired. The simulation and engineering experiments validate that the NSPF enjoys more reconstruction precision compared to the existing penalties.

An exact penalty function approach for inequality constrained optimization problems based on a new smoothing technique

Exact penalty methods are one of the effective tools to solve nonlinear programming problems with inequality constraints. In this study, a new class of exact penalty functions is defined and a new family of smoothing techniques to exact penalty functions is introduced. Error estimations are presented among the original, non-smooth exact penalty and smoothed exact penalty problems. It is proved that an optimal solution of smoothed penalty problem is an optimal solution of original problem. A smoothing penalty algorithm based on the the new smoothing technique is proposed and the convergence of the algorithm is discussed. Finally, the efficiency of the algorithm on some numerical examples is illustrated.

Coherent Mortality Forecasting with a Model Averaging Approach: Evidence from Global Populations

Accurate forecasts and analyses of mortality rates are essential to many economic and finance practices, such as the designing of pension schemes. Recent studies have proposed advanced mortality models with age coherent forecasts, such that long-term predictions will not diverge infinitely among ages. Despite their effectiveness, individual models are inevitably misspecified in the empirical analysis, which reduces the reliableness. In this article, we propose a model averaging approach (MAA) that allows for age-specific weights and asymptotically achieves age coherence. Relevant technical details are also provided. The proposed method balances both the in-sample fitting and out-of-sample forecasting, with a uniquely designed smoothness penalty to resolve the potential overfitting and thus avoid abrupt changes in the long-term mortality forecasting. Using a large empirical dataset of 10 countries including 0 to 100 ages and spanning 1950 to 2018, the outstanding forecasting performance of MAA is presented. This robustly holds in various sensitivity analyses and is supported by simulation evidence. A case study is further conducted to discuss the improved forecasting efficiency of MAA, as well as its usefulness in economic and finance applications such as the annuity pricing. The proposed MAA approach is therefore a useful tool in forecasting mortality data for other practices.

Abstract 4274: Functional dissection of complex tissue microenvironment using spatial transcriptomics data

Abstract Spatial transcriptomics (ST) data holds enormous potential to provide new perspectives for biologists seeking to understand the tumor tissue microenvironments. Cells in the same microenvironment are always exposed to biochemical conditions displacing specific spatial patterns. To truly dismantle the tissue microenvironment heterogeneity as well as cells’ diverse responses to the variable microenvironment, we rely on spatial information to dissect the tissue into different functional regions, which are spatially dependent and biologically meaningful. We have developed a statistical method, smooth Low Rank approximation and Clustering (smoothLRC) to model the spatial transcriptomic data as its low-rank approximation with spatial smoothness. smoothLRC accomplishes this by extracting the latent low-rank structure of the expression profiles. We present a regularized maximum likelihood estimator approximating the noisy observed expression matrix. This estimator incorporates spatial information and addresses expression dropout. It uses a low-rank Poisson distribution to account for the noise that plagues ST-based gene expression data with severe dropout events and a spatial smoothness penalty to encourage neighboring cells to fall into the same functional region. A dropout is an event where genes expressed in a given cell or region are incorrectly measured as unexpressed. SmoothLRC has the following key features: 1) it is a statistically rigorous method; 2) the spatially smooth low-rank subspace could enable meaningful functional dissection, which is an unprecedented capability; 3) the low-rank structures could be used to recover the original expression, saving many expression values that are not captured. Citation Format: Xinyu Zhou, Alexander White, Tingbo Guo, Pengtao Dang, Yuhui Wei, Xiao Wang, Kaman So, Chi Zhang, Sha Cao. Functional dissection of complex tissue microenvironment using spatial transcriptomics data. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4274.

Local linear approximation with Laplacian smoothing penalty and application in biology.

Highly correlated structures appear in various fields, such as biology, biochemistry, and finance, with challenges of dimensionality and sparse estimation. To solve this problem, we propose an algorithm called local linear approximation with the Laplacian smoothing penalty (LLA-LSP). This method produces an accurate and smooth estimate that incorporates the correlation structure among predictors. We compare and discuss the difference between the Laplacian smoothing penalty and the total variance penalty. We prove that this algorithm converges to the oracle solution in a few iterations with a large probability. Numerical results show that the LLA-LSP has good performance in both variable selection and estimation. We apply the proposed algorithm to two biological datasets, a gene expression dataset and a chemical protein dataset, and provide meaningful insights.

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

Abstract This paper is concerned with a class of optimization problems with the non-negative orthogonal constraint, in which the objective function is $L$-smooth on an open set containing the Stiefel manifold $\textrm {St}(n,r)$. We derive a locally Lipschitzian error bound for the feasible points without zero rows when $n&amp;gt;r&amp;gt;1$, and when $n&amp;gt;r=1$ or $n=r$ achieve a global Lipschitzian error bound. Then, we show that the penalty problem induced by the elementwise $\ell _1$-norm distance to the non-negative cone is a global exact penalty, and so is the one induced by its Moreau envelope under a lower second-order calmness of the objective function. A practical penalty algorithm is developed by solving approximately a series of smooth penalty problems with a retraction-based nonmonotone line-search proximal gradient method, and any cluster point of the generated sequence is shown to be a stationary point of the original problem. Numerical comparisons with the ALM [Wen, Z. W. &amp; Yin, W. T. (2013, A feasible method for optimization with orthogonality constraints. Math. Programming, 142, 397–434),] and the exact penalty method [Jiang, B., Meng, X., Wen, Z. W. &amp; Chen, X. J. (2022, An exact penalty approach for optimization with nonnegative orthogonality constraints. Math. Programming. https://doi.org/10.1007/s10107-022-01794-8)] indicate that our penalty method has an advantage in terms of the quality of solutions despite taking a little more time.

Directional differentiability for shape optimization with variational inequalities as constraints

For equilibrium constrained optimization problems subject to nonlinear state equations, the property of directional differentiability with respect to a parameter is studied. An abstract class of parameter dependent shape optimization problems is investigated with penalty constraints linked to variational inequalities. Based on the Lagrange multiplier approach, on smooth penalties due to Lavrentiev regularization, and on adjoint operators, a shape derivative is obtained. The explicit formula provides a descent direction for the gradient algorithm identifying the shape of the breaking-line from a boundary measurement. A numerical example is presented for a nonlinear Poisson problem modeling Barenblatt’s surface energies and non-penetrating cracks.