Smooth Penalty Research Articles

A smooth penalty-based sample average approximation method for stochastic complementarity problems

Sample average approximation method is one of the effective methods in the stochastic optimization. A smooth penalty-based sample average approximation method for stochastic nonlinear complementarity problems is presented in this paper. Based on a smooth penalty function, a reformulation is proposed for the equivalent problem of EV formulation for stochastic complementary problems and it is proven that its solutions are existent under some mild assumptions. An implementable sample average approximation method for the reformulation is further established and its convergence is analyzed. The numerical results for some test examples are reported at last to show efficiency of the proposed method.

Exact differentiable penalty for a problem of quadratic programming with the use of a gradient-projective method

AbstractIt is known that the solution obtained for a problem of quadratic programming (QP) is not exact in the case of reduction of the original problem to a sequence of problems of unconditional minimization by a method of penalty functions with a differentiable penalty. In this paper we obtain an exact solution to a QP problem for a finite penalty constant and a smooth penalty with the use of a gradient-projective method.

The Partial Linear Model in High Dimensions

ABSTRACTPartial linear models have been widely used as flexible method for modelling linear components in conjunction with non‐parametric ones. Despite the presence of the non‐parametric part, the linear, parametric part can under certain conditions be estimated with parametric rate. In this paper, we consider a high‐dimensional linear part. We show that it can be estimated with oracle rates, using the least absolute shrinkage and selection operator penalty for the linear part and a smoothness penalty for the nonparametric part.

On Saddle Points in Semidefinite Optimization via Separation Scheme

This paper aims at investigating saddle point conditions for augmented Lagrangian functions for semidefinite optimization problems. By means of the image space analysis, the existence of a saddle point is shown to be equivalent to a regular weak nonlinear separation of two suitable subsets in the image space (IS) associated with the given problem. Especially, three classes of augmented Lagrangians based on smooth spectral penalty functions can be derived, as particular cases, from a nonlinear separation scheme in the IS. Without requiring the strict complementarity, it is proved that, under strong second-order sufficiency conditions, all these augmented Lagrangian functions admit a local saddle point, and their Hessians become positive definite in a neighborhood of a local optimal point of the original problem. The existence of global saddle points is then obtained under additional assumptions that do not require the compactness of the feasible set.

Return level estimation from non-stationary spatial data exhibiting multidimensional covariate effects

Careful modelling of non-stationarity is critical to reliable specification of marine and coastal design criteria. We present a spline based methodology to incorporate spatial, directional, temporal and other covariate effects in extreme value models for environmental variables such as storm severity. For storm peak significant wave height events, the approach uses quantile regression to estimate a suitable extremal threshold, a Poisson process model for the rate of occurrence of threshold exceedances, and a generalised Pareto model for size of threshold exceedances. Multidimensional covariate effects are incorporated at each stage using penalised (tensor products of) B-splines to give smooth model parameter variation as a function of multiple covariates. Optimal smoothing penalties are selected using cross-validation, and model uncertainty is quantified using a bootstrap re-sampling procedure. The method is applied to estimate return values for large spatial neighbourhoods of locations, incorporating spatial and directional effects. Extensions to joint modelling of multivariate extremes, incorporating extremal spatial dependence (using max-stable processes) or more general extremal dependence (using the conditional extremes approach) are outlined.

Surface reconstruction from microscopic images in optical lithography.

This paper presents a method to reconstruct 3D surfaces of silicon wafers from 2D images of printed circuits taken with a scanning electron microscope. Our reconstruction method combines the physical model of the optical acquisition system with prior knowledge about the shapes of the patterns in the circuit; the result is a shape-from-shading technique with a shape prior. The reconstruction of the surface is formulated as an optimization problem with an objective functional that combines a data-fidelity term on the microscopic image with two prior terms on the surface. The data term models the acquisition system through the irradiance equation characteristic of the microscope; the first prior is a smoothness penalty on the reconstructed surface, and the second prior constrains the shape of the surface to agree with the expected shape of the pattern in the circuit. In order to account for the variability of the manufacturing process, this second prior includes a deformation field that allows a nonlinear elastic deformation between the expected pattern and the reconstructed surface. As a result, the minimization problem has two unknowns, and the reconstruction method provides two outputs: 1) a reconstructed surface and 2) a deformation field. The reconstructed surface is derived from the shading observed in the image and the prior knowledge about the pattern in the circuit, while the deformation field produces a mapping between the expected shape and the reconstructed surface that provides a measure of deviation between the circuit design models and the real manufacturing process.

An analysis of finite element approximation in electrical impedance tomography

We present a finite element analysis of electrical impedance tomography for reconstructing the conductivity distribution from electrode voltage measurements by means of Tikhonov regularization. Two popular choices of the penalty term, i.e., the H1(Ω)-norm smoothness penalty and total variation seminorm penalty, are considered. A piecewise linear finite element method is employed for discretizing the forward model, i.e., the complete electrode model, the conductivity, and the penalty functional. The convergence of the finite element approximations for the Tikhonov model on both polyhedral and smooth curved domains is established. This provides rigorous justifications for the ad hoc discretization procedures. Numerical experiments confirm the convergence analysis.

Flexible Codebook Design for Limited Feedback Systems Via Sequential Smooth Optimization on the Grassmannian Manifold

Grassmannian quantization codebooks play a central role in a number of limited feedback schemes for single and multi-user multiple-input multiple-output (MIMO) communication systems. In practice, it is often desirable that these codebooks possess additional properties that facilitate their implementation, beyond the provision of good quantization performance. Although some good codebooks exist, their design tends to be a rather intricate task. The goal of this paper is to suggest a flexible approach to the design of Grassmannian codebooks based on sequential smooth optimization on the Grassmannian manifold and the use of smooth penalty functions to obtain additional desirable properties. As one example, the proposed approach is used to design rank-2 codebooks that have a nested structure and elements from a phase-shift keying (PSK) alphabet. In some numerical comparisons, codebooks designed using the proposed methods have larger minimum distances than some existing codebooks, and provide tangible performance gains when applied to a simple MIMO downlink scenario with zero-forcing beamforming, per-user unitary beamforming and rate control (PU 2RC), and block diagonalization signalling. Furthermore, the proposed approach yields codebooks that attain desirable additional properties without incurring a substantial degradation in performance.

CVaR-constrained stochastic programming reformulation for stochastic nonlinear complementarity problems

We reformulate a stochastic nonlinear complementarity problem as a stochastic programming problem which minimizes an expected residual defined by a restricted NCP function with nonnegative constraints and CVaR constraints which guarantee the stochastic nonlinear function being nonnegative with a high probability. By applying smoothing technique and penalty method, we propose a penalized smoothing sample average approximation algorithm to solve the CVaR-constrained stochastic programming. We show that the optimal solution of the penalized smoothing sample average approximation problem converges to the solution of the corresponding nonsmooth CVaR-constrained stochastic programming problem almost surely. Finally, we report some preliminary numerical test results.

Joint estimation of isoform expression and isoform-specific read distribution using multisample RNA-Seq data

RNA-sequencing technologies provide a powerful tool for expression analysis at gene and isoform level, but accurate estimation of isoform abundance is still a challenge. Standard assumption of uniform read intensity would yield biased estimates when the read intensity is in fact non-uniform. The problem is that, without strong assumptions, the read intensity pattern is not identifiable from data observed in a single sample. We develop a joint statistical model that accounts for non-uniform isoform-specific read distribution and gene isoform expression estimation. The main challenge is in dealing with the large number of isoform-specific read distributions, which potentially are as many as the number of splice variants in the genome. A statistical regularization via a smoothing penalty is imposed to control the estimation. Also, for identifiability reasons, the method uses information across samples from the same region. We develop a fast and robust computational procedure based on the iterated-weighted least-squares algorithm, and apply it to simulated data and two real RNA-Seq datasets with reverse transcription-polymerase chain reaction validation. Empirical tests show that our model performs better than existing methods in terms of increasing precision in isoform-level estimation. We have implemented our method in an R package called Sequgio as a pipeline for fast processing of RNA-Seq data.

A new class of exact penalty functions and penalty algorithms

For nonlinear programming problems, we propose a new class of smooth exact penalty functions, which includes both barrier-type and exterior-type penalty functions as special cases. We develop neces...

Penalized marginal likelihood estimation of finite mixtures of Archimedean copulas

This paper proposes finite mixtures of different Archimedean copula families as a flexible tool for modelling the dependence structure in multivariate data. A novel approach to estimating the parameters in this mixture model is presented by maximizing the penalized marginal likelihood via iterative quadratic programming. The motivation for the penalized marginal likelihood stems from an underlying Bayesian model that imposes a prior distribution on the parameter of each Archimedean copula family. An approximative marginal likelihood is obtained by a classical quadrature discretization of the integral w.r.t. each family-specific prior distribution, thus yielding a finite mixture model. Family-specific smoothness penalties are added and the penalized marginal likelihood is maximized using an iterative quadratic programming routine. For comparison purposes, we also present a fully Bayesian approach via simulation-based posterior computation. The performance of the novel estimation approach is evaluated by simulations and two examples involving the modelling of the interdependence of exchange rates and of wind speed measurements, respectively. For these examples, penalized marginal likelihood estimates are compared to the corresponding Bayesian estimates.

SU‐E‐J‐50: Maximum a Posteriori Proton CT Reconstruction Using Anatomic Prior

Purpose: It has been demonstrated that the information in one image modality can be used to significantly help the reconstruction of another image modality. Here we develop and evaluate a novel Maximum a Posteriori (MAP) approach to apply anatomical information in MRI or x‐ray CT image to facilitate the estimate of Proton CT (pCT) image. Method: To incorporate anatomical information of MRI or x‐ray CT into the reconstruction of pCT images through priors based on information theoretic similarity measures, we formulate a MAP optimization problem, which includes a least‐square data‐fitting term to account for the noise in the measurement and a penalty term to maximize the joint density of feature vectors extracted from the pCT image and anatomical image. We defined feature vectors that emphasize prominent boundaries in anatomical and functional images, and attach less importance to detail and noise that is less likely to be correlated in the two images. We use a non‐parametric, FFT‐based Parzen window approach to speed up the computation of the density function of the feature vector, the joint density of feature vector, and the gradients of joint entropy. The optimization problem is solved using conjugate gradient and Armijo‐backtracking line search. We performed realistic simulations using anthropomorphic phantom to validate our approach and compare its performance to MAP approach with a quadratic smoothness penalty function (without anatomic information from x‐ray). Results: The Preliminary results demonstrate the proposed MAP approach with MRI or x‐ray CT as prior can achieve better contrast recovery and biasvariance trade‐off curves compared with MAP approach without anatomical prior. Conclusions: The proposed MAP approach can achieve superior image quality by using anatomical information in the MRI/x‐ray CT image. In the future we will investigate how much data reduction we can achieve using the proposed method while maintain the same image quality.

Optimal Control of Switched Systems with an Inequality Constraint Based on Smooth Penalty Function Method

Optimal Control of Switched Systems with an Inequality Constraint Based on Smooth Penalty Function Method

Global convergence and finite termination of a class of smooth penalty function algorithms

In this paper, based on a new class of asymptotic l 1 exact penalty functions, we propose a smooth penalty function for solving nonlinear programming problems. One of the main features of our algorithm is that at each iteration, we do not need to solve the global minimum of penalty functions. Furthermore, global convergence property is established without requiring any constraint qualifications. By addressing perturbation functions, we obtain that the lower semi-continuity of the perturbation function at zero is a necessary and sufficient condition to ensure the convergence of the objective function values generated by the algorithm to the optimal value of the primal problem. Since the perturbation function is only dependent on the data of the primal problem, these results enable us to check the convergence property of the algorithm in advance. Finally, we discuss the finite termination of the algorithm when the solution set is non-degenerate or weakly sharp.

Estimation of Non-Parametric Regression Models: Production Profiles of Crude Oil

In many practical statistical situations, it is desirable to restrict the flexibility of nonparametric regression models to accommodate prior information. We propose an estimator for regression models with a smoothness penalty and constraints imposed by the nature of the problem. Our estimator is easy to implement and has an explicit algebraic structure. Alternative or additional constraints can be readily applied. We present production profiles of crude oil to demonstrate possible uses of the proposed estimator.

Adaptive evolutionary clustering

In many practical applications of clustering, the objects to be clustered evolve over time, and a clustering result is desired at each time step. In such applications, evolutionary clustering typically outperforms traditional static clustering by producing clustering results that reflect long-term trends while being robust to short-term variations. Several evolutionary clustering algorithms have recently been proposed, often by adding a temporal smoothness penalty to the cost function of a static clustering method. In this paper, we introduce a different approach to evolutionary clustering by accurately tracking the time-varying proximities between objects followed by static clustering. We present an evolutionary clustering framework that adaptively estimates the optimal smoothing parameter using shrinkage estimation, a statistical approach that improves a naive estimate using additional information. The proposed framework can be used to extend a variety of static clustering algorithms, including hierarchical, k-means, and spectral clustering, into evolutionary clustering algorithms. Experiments on synthetic and real data sets indicate that the proposed framework outperforms static clustering and existing evolutionary clustering algorithms in many scenarios.

An Adaptation Method for Morphological Opening Filters with a Smoothness Penalty on Structuring Elements

An Adaptation Method for Morphological Opening Filters with a Smoothness Penalty on Structuring Elements

A second-order smooth penalty function algorithm for constrained optimization problems

This paper introduces a second-order differentiability smoothing technique to the classical l 1 exact penalty function for constrained optimization problems(COP). Error estimations among the optimal objective values of the nonsmooth penalty problem, the smoothed penalty problem and the original optimization problem are obtained. Based on the smoothed problem, an algorithm for solving COP is proposed and some preliminary numerical results indicate that the algorithm is quite promising.

A modified exact smooth penalty function for nonlinear constrained optimization

In this paper, a modified simple penalty function is proposed for a constrained nonlinear programming problem by augmenting the dimension of the program with a variable that controls the weight of the penalty terms. This penalty function enjoys improved smoothness. Under mild conditions, it can be proved to be exact in the sense that local minimizers of the original constrained problem are precisely the local minimizers of the associated penalty problem. MSC:47H20, 35K55, 90C30.