Regularized Cost Function Research Articles

A regularization framework for robust dimensionality reduction with applications to image reconstruction and feature extraction

Dimensionality reduction has many applications in pattern recognition, machine learning and computer vision. In this paper, we develop a general regularization framework for dimensionality reduction by allowing the use of different functions in the cost function. This is especially important as we can achieve robustness in the presence of outliers. It is shown that optimizing the regularized cost function is equivalent to solving a nonlinear eigenvalue problem under certain conditions, which can be handled by the self-consistent field (SCF) iteration. Moreover, this regularization framework is applicable in unsupervised or supervised learning by defining the regularization term which provides some types of prior knowledge of projected samples or projected vectors. It is also noted that some linear projection methods can be obtained from this framework by choosing different functions and imposing different constraints. Finally, we show some applications of our framework by various data sets including handwritten characters, face images, UCI data, and gene expression data.

Missing boundary data reconstruction by the factorization method

We consider the data completion problem for the Laplace equation in a cylindrical domain. The Neumann and Dirichlet boundary conditions are given on one face of the cylinder while there is no condition on the other face. This Cauchy problem has been known since Hadamard (1953) to be ill-posed. Here it is set as an optimal control problem with a regularized cost function. We use the factorization method for elliptic boundary value problems. For each set of Cauchy data, to obtain the estimate of the missing data one has to solve a parabolic Cauchy problem in the cylinder and a linear equation. The operator appearing in these problems satisfy a Riccati equation that does not depend on the data. To cite this article: A. Ben Abda et al., C. R. Acad. Sci. Paris, Ser. I 347 (2009).

A New Value Picking Regularization Strategy—Application to the 3-D Electromagnetic Inverse Scattering Problem

The nonlinear electromagnetic inverse scattering problem of reconstructing a possibly quasi-piecewise constant inhomogeneous complex permittivity profile is solved by iterative minimization of a pixel-based data fit cost function. Because of the ill-posedness it is necessary to introduce some form of regularization. Many authors apply a smoothing constraint on the reconstructed permittivity profile, but such regularization smooths away sharp edges. In this paper, a simple yet effective regularization strategy, the value picking (VP) regularization, is proposed. This new technique is capable of reconstructing piecewise constant permittivity profiles without degrading the edges. It is based on the knowledge that only a few different permittivity values occur in such profiles, the values of which need not be known in advance. VP regularization does not impose this a priori information in a strict sense, such that it can be applied also to profiles that are only approximately piecewise constant. The VP regularization is introduced in the solution of the inverse problem by adding a choice function to the data fit cost function for every permittivity unknown in the discretized problem. When minimized, the choice function forces the corresponding permittivity unknown to be close to one member of a set of auxiliary variables, the VP values, which are continuously updated throughout the iterations. To minimize the regularized cost function, a half quadratic Gauss-Newton optimization technique is presented. Finally, a stepwise relaxed VP regularization scheme is proposed, in which the number of VP values is gradually increased. This scheme is tested with synthetic and measured scattering data, obtained from inhomogeneous 3D targets, and is shown to achieve high reconstruction quality.

FEATURE SELECTION VIA LEAST SQUARES SUPPORT FEATURE MACHINE

In many applications such as credit risk management, data are represented as high-dimensional feature vectors. It makes the feature selection necessary to reduce the computational complexity, improve the generalization ability and the interpretability. In this paper, we present a novel feature selection method — "Least Squares Support Feature Machine" (LS-SFM). The proposed method has two advantages comparing with conventional Support Vector Machine (SVM) and LS-SVM. First, the convex combinations of basic kernels are used as the kernel and each basic kernel makes use of a single feature. It transforms the feature selection problem that cannot be solved in the context of SVM to an ordinary multiple-parameter learning problem. Second, all parameters are learned by a two stage iterative algorithm. A 1-norm based regularized cost function is used to enforce sparseness of the feature parameters. The "support features" refer to the respective features with nonzero feature parameters. Experimental study on some of the UCI datasets and a commercial credit card dataset demonstrates the effectiveness and efficiency of the proposed approach.

Full-Wave Three-Dimensional Microwave Imaging With a Regularized Gauss–Newton Method— Theory and Experiment

A reconstruction algorithm is detailed for three-dimensional full-vectorial microwave imaging based on Newton-type optimization. The goal is to reconstruct the three-dimensional complex permittivity of a scatterer in a homogeneous background from a number of time-harmonic scattered field measurements. The algorithm combines a modified Gauss-Newton optimization method with a computationally efficient forward solver, based on the fast Fourier transform method and the marching-on-in-source-position extrapolation procedure. A regularized cost function is proposed by applying a multiplicative-additive regularization to the least squares datafit. This approach mitigates the effect of measurement noise on the reconstruction and effectively deals with the non-linearity of the optimization problem. It is furthermore shown that the modified Gauss-Newton method converges much faster than the Broyden-Fletcher-Goldfarb-Shanno quasi-Newton method. Promising quantitative reconstructions from both simulated and experimental data are presented. The latter data are bi-static polarimetric free-space measurements provided by Institut Fresnel, Marseille, France.

A Robust and Computationally Efficient Simultaneous Super-Resolution Scheme for Image Sequences

This work presents a new class of algorithms for super-resolution (SR) of image sequences. This class of algorithms estimates simultaneously all frames of a sequence by employing an iterative minimization of a regularized cost function. Similarly to other SR techniques, the proposed approach exploits the correlation among the frames of the sequence. This correlated information helps to improve the resolution of the captured images. By employing the motion information only in the prior term of the cost function, the proposed method achieves a better fidelity and more robust performance, when compared to other methods. This original approach provides a fidelity equivalent to the obtained by the simultaneous methods, while achieving a lower computational complexity. The isolation of the motion equations in the prior term allows an increased control over the motion errors, and as a consequence, provides an improved robustness. The performance of the proposed method is discussed and compared with other methods in the literature. In the comparative experiments, it is considered both the minimization over the Euclidean norm, which is employed to achieve low computational complexity, and the minimization over the Huber norm or norm, which is used to produce images with sharp edges and higher robustness to large motion errors.

A branch-and-price algorithm for scheduling parallel machines with sequence dependent setup times

We consider the problem of scheduling n independent jobs on m unrelated parallel machines with sequence-dependent setup times and availability dates for the machines and release dates for the jobs to minimize a regular additive cost function. In this work, we develop a new branch-and-price optimization algorithm for the solution of this general class of parallel machines scheduling problems. A new column generation accelerating method, termed “ primal box”, and a specific branching variable selection rule that significantly reduces the number of explored nodes are proposed. The computational results show that the approach solves problems of large size to optimality within reasonable computational time.

Super-Resolução Regularizada e Simultânea de Seqüência de Imagens

This work presents a new class of algorithms for super-resolution of images. In this new class of algorithms all frames of a sequence of images are simultaneously estimated by employing an iterative minimization of a regularized cost function. Similarly to other super-resolution techniques, the proposed approach exploits the existent correlation among the motion compensated frames. This correlated information helps to recover the details lost in the acquisition process and to obtain a image sequence with improved resolution. The proposed method achieved a better performance, when compared to other methods, by using the motion equations only in the prior term of the cost function. This original approach allows the production of an image sequence with higher fidelity and, at the same time, with lower computational complexity when compared with other algorithms in the literature. The isolation of the expressions that describes the correlation in the motion trajectory allows an increased control over the motion errors, and as a consequence, it was achieved an increased robustness. The performance of the proposed method were compared with other methods in the literature. In the comparative experiments it was considered the Euclidean norm, which is used to achieve low computational complexity, and the Huber norm, which is used to produce image with sharp edges and higher robustness to errors that occurs in the motion.

Efficient neighborhood search for the one-machine earliness–tardiness scheduling problem

This paper addresses the one-machine scheduling problem where the objective is to minimize a sum of costs such as earliness–tardiness costs. Since the sequencing problem is NP-hard, local search is very useful for finding good solutions. Unlike scheduling problems with regular cost functions, the scheduling (or timing) problem is not trivial when the sequence is fixed. Therefore, the local search approaches must deal with both job interchanges in the sequence and the timing of the sequenced jobs. We present a new approach that efficiently searches in a large neighborhood and always returns a solution for which the timing is optimal.

Second-order robust regularization cost function for detecting and reconstructing phase discontinuities

We propose a robust method for computing discontinuous phase maps from a fringe pattern with carrier frequency. Our algorithm is based on the minimization of an edge-preserving regularized cost function, specifically, on a robust regularized potential that uses a paradigm called the plate with adaptive rest condition, i.e., a second-order edge-preserving potential. Given that the proposed cost function is not convex, our method uses as its initial point an overly smoothed phase computed with a standard fringe analysis method and then reconstructs the phase discontinuities. Although the method is general purpose, it is introduced in the context of interferometric gauge-block calibration. The performance of the algorithm is demonstrated by numerical experiments with both synthetic and real data.

Half-quadratic cost function for computing arbitrary phase shifts and phase: Adaptive out of step phase shifting

We present a phase shifting robust method for irregular and unknown phase steps. The method is formulated as the minimization of a half-quadratic (robust) regularized cost function for simultaneously computing phase maps and arbitrary phase shifts. The convergence to, at least, a local minimum is guaranteed. The algorithm can be understood as a phase refinement strategy that uses as initial guess a coarsely computed phase and coarsely estimated phase shifts. Such a coarse phase is assumed to be corrupted with artifacts produced by the use of a phase shifting algorithm but with imprecise phase steps. The refinement is achieved by iterating alternated minimization of the cost function for computing the phase map correction, an outliers rejection map and the phase shifts correction, respectively. The method performance is demonstrated by comparison with standard filtering and arbitrary phase steps detecting algorithms.

Phase recovery from a single fringe pattern using an orientational vector-field-regularized estimator

Recent studies have demonstrated that the phase recovery from a single fringe pattern with closed fringes can be properly performed if the modulo 2pi fringe orientation is estimated. For example, the fringe pattern in quadrature can be efficiently obtained in terms of the orientational phase spatial operator using fast Fourier transformations and a spiral phase spectral operator in the Fourier space. The computation of the modulo 2pi fringe orientation, however, is by far the most difficult task in the global process of phase recovery. For this reason we propose the demodulation of fringe patterns with closed fringes through the computation of the modulo 2pi fringe orientation using an orientational vector-field-regularized estimator. As we will show, the phase recovery from a single pattern can be performed in an efficient manner using this estimator, provided that it requires one to solve locally in the fringe pattern a simple linear system to optimize a regularized cost function. We present simulated and real experiments applying the proposed methodology.

Robust phase demodulation of interferograms with open or closed fringes

We present two robust algorithms for fringe pattern analysis with partial-field and closed fringes. The algorithm for partial-field fringe patterns is presented as a refinement method for precomputed coarse phases. Such an algorithm consists of the minimization of a regularized cost function that incorporates an outlier rejection strategy, which causes the algorithm to become robust. On the basis of the phase refinement method, we propose a propagative scheme for phase retrieval from closed-fringe interferograms. The algorithm performance is demonstrated by demodulating closed-fringe interferograms with complex spatial distribution of stationary points and gradients in the illumination components.

Specification of the observation model for regularized image up-sampling

Regularization is one of the most promising methods for image up-sampling, which is an ill-posed inverse problem. A key element of such a regularization approach is the observation model relating the observed lower resolution (LR) image to the desired higher resolution (HR) up-sampled image, used in the data-fidelity term of the regularization cost function. This paper presents an algorithm to determine this observation model based on a model of the physical acquisition process for the LR image, and the ideal acquisition process for the desired HR image, both from the same underlying continuous image. The method is illustrated with typical scenarios corresponding to LR and HR cameras modeled by either Gaussian or rectangular apertures. Experiments with some regularized image up-samplers demonstrate the importance of using the correct, adapted observation model as determined by our algorithm. Index Terms-Camera aperture, data fidelity, image up-sampling, interpolation, multidimensional signal processing, observation model, power spectral density (PSD), super-resolution.

Single machine batch scheduling with sequential job processing

The problem of scheduling n jobs on a single machine in batches to minimize some regular cost functions is studied. Jobs within each batch are processed sequentially so that the processing time of a batch is equal to the sum of the processing times of the jobs contained in it. Jobs in the same batch are completed at the same time when the last job of the batch has finished its processing. A constant set-up time precedes the processing of each batch. The number of jobs in each batch is bounded by some value b. If b < n, then the problem is called bounded. Otherwise, it is unbounded. For both the bounded and unbounded problems, dynamic programming algorithms are presented for minimizing the maximum lateness, the number of late jobs, the total tardiness, the total weighted completion time, and the total weighted tardiness when all due dates are equal, which are polynomial if there is a fixed number of distinct due dates or processing times. More efficient algorithms are derived for some special cases of both the bounded and unbounded problems in which all due dates and/or processing times are equal. Several special cases of the bounded problem are shown to be NP-hard. Thus, a comprehensive classification of the computational complexities of the special cases is provided.

Single machine batch scheduling with sequential job processing

The problem of scheduling n jobs on a single machine in batches to minimize some regular cost functions is studied. Jobs within each batch are processed sequentially so that the processing time of a batch is equal to the sum of the processing times of the jobs contained in it. Jobs in the same batch are completed at the same time when the last job of the batch has finished its processing. A constant set-up time precedes the processing of each batch. The number of jobs in each batch is bounded by some value b. If b < n, then the problem is called bounded. Otherwise, it is unbounded. For both the bounded and unbounded problems, dynamic programming algorithms are presented for minimizing the maximum lateness, the number of late jobs, the total tardiness, the total weighted completion time, and the total weighted tardiness when all due dates are equal, which are polynomial if there is a fixed number of distinct due dates or processing times. More efficient algorithms are derived for some special cases of both the bounded and unbounded problems in which all due dates and/or processing times are equal. Several special cases of the bounded problem are shown to be NP-hard. Thus, a comprehensive classification of the computational complexities of the special cases is provided.

Positioning automated guided vehicles in a loop layout

We address the problem of determining the home positions for m automated guided vehicles (AGVs) in a loop layout where n pickup points are positioned along the circumference ( m< n). A home position is the location where idle AGVs are held until they are assigned to the next transportation task. The home positions need to be selected so as to minimize an objective function of the response times, where the response time for a pickup point is defined as the travel time to the pickup point from the nearest home location. For the unidirectional flow system, where all AGVs can move in one direction only, we first point out that the problem of minimizing an arbitrary regular cost function can quite straightforwardly be solved in O( n 2) time if m=1 and in O( mn m ) time if m⩾2, which is polynomial for a fixed number m of AGVs. For m⩾3, we can do better, however: we derive a generic O( mn 3) time and O( mn) space dynamic programming algorithm for minimizing any regular function of the response times. For minimizing maximum response time a further gain in efficiency is possible: this problem can be solved in O( n 2) time if m=2 and O(n 2 logn) time if m⩾3. Our results improve on earlier published work, where it was suggested that problems with m⩾2 are NP-hard. For the bidirectional flow system, where the AGVs can move in both directions, the problem of determining the home locations is inherently much more difficult. Important objective functions like average response time and maximum response time can nonetheless still be minimized by the same types of algorithms and in the same amount of time as their unidirectional counterparts, once restrictive conditions apply such that the case m=1 can be solved in polynomial time. One such restrictive condition is that each AGV travels at constant speed.

Discontinuous Mayer control problem under state-constraints

This paper deals with Mayer's problem for control systems with state constraints and, possibly, discontinuous terminal cost. The main result of this paper consists in the characterization of the value function as the unique solution to an Hamilton-Jacobi equation. The above characterization extends results already obtained in the case of regular cost functions and under some controlability assumptions on the boundary of the set of constraints.

On finding the number of clusters

We present a novel approach to finding the number of clusters in data based on the minimization of a regularized cost function. Minimization of the proposed cost function results in the minimization of the sum-of-squared distances of the data points from the respective nearest cluster center as well as the sum-of-squared distances of the individual cluster centers from neighborhood cluster centers. Smaller values of the neighborhood encourage the formation of more distinct cluster centers, while larger values of the neighborhood encourage the formation of fewer distinct cluster centers. We identify the neighborhood as a scale parameter and obtain the number of cluster centers at varying values of the scale parameter. The number of cluster centers in the data is then obtained based on persistence over the largest range of the scale parameter. Four simulations are presented to illustrate the efficacy of the proposed algorithm.

Scheduling jobs on unrelated parallel machines to minimize regular total cost functions

In this paper we consider unrelated parallel machine scheduling problems that involve the minimization of regular total cost functions. We first present some properties of optimal solutions and then provide a lower bound. These mechanisms are tested on the well-known practical problem of minimizing total weighted flow time on unrelated parallel machines. In doing so, we design a branch and bound algorithm incorporating the mechanisms derived for the general total cost function along with the ones derived specifically for the total weighted flow time criterion. Computational experience indicates that incoiporating reduction and bounding mechanisms significantly improves the performance of the branch and bound algorithm.