Penalized Objective Function Research Articles

A parallel fuzzy GMM-algorithm for approximate VGARCH-modeling with a multi-modal discontinuous merit function

General modeling of multivariate stochastic volatility with exact methods is coupled with computational difficulties. Over-fitting and outliers are two central problems in time series modeling, where fuzzy mathematical approaches can be successfully applied. The risk for over-fitting is present also in General Method of Moments-estimation (GMM), but the mathematical flexibility of the approach – in essence based on formulating and solving a consistent system of equations – allows counteracting measures that are difficult or impossible to implement successfully in traditional maximum likelihood based methods. We show that fuzzy mathematical programming can be applied to cope with the problem of over-fitting in GMM-estimation. We introduce a fuzzy component in the equation system derived from the moment conditions. The fuzzy augmented equation system allows an approximate solution that counteracts over-fitting. The penalized objective function is similar to the penalty functions normally used in constrained nonlinear optimization. In order to reduce the risk for over-estimation, the quadratic objective function of the ordinary GMM-formulation is replaced by an augmented exponential fuzzy penalty function. The barrier coefficient and fuzzy tolerance level are estimated via a multi-modal, discontinuous merit function that models hypothetical real world conditions. We show in a large set of simulations, that the fuzzy GMM-algorithm (FGMM) yields estimates mostly closer to the true values than those generated by the crisp GMM. The fuzzy GMM-algorithm is generalized to a multi-computer setting and solved by parallel processors. The multi-computer formulation is fully scalable and independent of mesh size. The study combines high performance computing, fuzzy mathematical programming and GMM in the estimation of volatility.

Investigation of statistical iterative reconstruction for dedicated breast CT

Dedicated breast CT has great potential for improving the detection and diagnosis of breast cancer. Statistical iterative reconstruction (SIR) in dedicated breast CT is a promising alternative to traditional filtered backprojection (FBP). One of the difficulties in using SIR is the presence of free parameters in the algorithm that control the appearance of the resulting image. These parameters require tuning in order to achieve high quality reconstructions. In this study, the authors investigated the penalized maximum likelihood (PML) method with two commonly used types of roughness penalty functions: hyperbolic potential and anisotropic total variation (TV) norm. Reconstructed images were compared with images obtained using standard FBP. Optimal parameters for PML with the hyperbolic prior are reported for the task of detecting microcalcifications embedded in breast tissue. Computer simulations were used to acquire projections in a half-cone beam geometry. The modeled setup describes a realistic breast CT benchtop system, with an x-ray spectra produced by a point source and an a-Si, CsI:Tl flat-panel detector. A voxelized anthropomorphic breast phantom with 280 μm microcalcification spheres embedded in it was used to model attenuation properties of the uncompressed woman's breast in a pendant position. The reconstruction of 3D images was performed using the separable paraboloidal surrogates algorithm with ordered subsets. Task performance was assessed with the ideal observer detectability index to determine optimal PML parameters. The authors' findings suggest that there is a preferred range of values of the roughness penalty weight and the edge preservation threshold in the penalized objective function with the hyperbolic potential, which resulted in low noise images with high contrast microcalcifications preserved. In terms of numerical observer detectability index, the PML method with optimal parameters yielded substantially improved performance (by a factor of greater than 10) compared to FBP. The hyperbolic prior was also observed to be superior to the TV norm. A few of the best-performing parameter pairs for the PML method also demonstrated superior performance for various radiation doses. In fact, using PML with certain parameter values results in better images, acquired using 2 mGy dose, than FBP-reconstructed images acquired using 6 mGy dose. A range of optimal free parameters for the PML algorithm with hyperbolic and TV norm-based potentials is presented for the microcalcification detection task, in dedicated breast CT. The reported values can be used as starting values of the free parameters, when SIR techniques are used for image reconstruction. Significant improvement in image quality can be achieved by using PML with optimal combination of parameters, as compared to FBP. Importantly, these results suggest improved detection of microcalcifications can be obtained by using PML with lower radiation dose to the patient, than using FBP with higher dose.

Application of cellular automata to sewer network optimization problems

A Cellular Automata approach is introduced in this paper for the optimal design of sewer network problems. The solution of sewer network optimization problems requires the determination of pipe diameters and average pipe cover depths, minimizing the total cost of the sewer network subject to operational constraints. One constraint of the problem, namely, the maximum relative depth of flow, is first used to define the pipe diameters in terms of the pipe slope and, hence, the nodal elevation of the network. As a result, the sewer network optimization problem is redefined, in terms of the network nodal elevations, and the resulting problem is solved using a Cellular Automata (CA) approach. The nodes of the network are used as the CA cell, with the corresponding elevations as CA cell states. The updating rule of the proposed CA is then derived by requiring the stationary total penalized objective function of the problem, with respect to the cell state. The proposed method is used to solve two benchmark examples and the results are compared to those obtained by other methods. The results indicate that the proposed method is highly efficient compared to alternative methods, while producing comparable results.

Majorization-Minimization algorithms for nonsmoothly penalized objective functions

In this paper, we propose a general class of algorithms for optimizing an extensive variety of nonsmoothly penalized objective functions that satisfy certain regularity conditions. The proposed framework utilizes the majorization-minimization (MM) algorithm as its core optimization engine. The resulting algorithms rely on iterated soft-thresholding, implemented componentwise, allowing for fast, stable updating that avoids the need for any high-dimensional matrix inversion. We establish a local convergence theory for this class of algorithms under weaker assumptions than previously considered in the statistical literature. We also demonstrate the exceptional effectiveness of new acceleration methods, originally proposed for the EM algorithm, in this class of problems. Simulation results and a microarray data example are provided to demonstrate the algorithm's capabilities and versatility.

A Penalty Function Approach to Bias Reduction in Nonlinear Panel Models with Fixed Effects

We consider estimation of nonlinear panel data models with individual specific fixed effects. Estimation of these models is complicated since estimation of the fixed effects when the time dimension is short generally results in inconsistent estimates of all model parameters. We present a penalized objective function that reduces the bias in the resulting point estimates. The penalty function is simple to construct and requires no modification for models with multiple individual specific parameters. We illustrate the approach through a series of simulations that suggest the approach is effective in reducing bias and in an empirical study of insider trading activity.

Optimization of phenol degradation in a combined photochemical–biological wastewater treatment system

A UV/H 2O 2 process and an activated sludge bioreactor were integrated to achieve a cost-effective process of aqueous degradation of phenol. By applying the two-step Haldane approach for the activated sludge bioreactor and a validated kinetic model for the UV/H 2O 2 process, an integrated model was developed for the phenol degradation in a combined photochemical–biological system. In this study, a kinetic model was employed to study the effects of different parameters in the combined photochemical–biological system, where the optimum conditions in terms of retention time and cost were determined. Three penalized objective functions were developed to find the best operating conditions including the minimization of the retention time, the power consumption, and the total cost. The least retention time for this system was determined to be 99 h and the optimal electrical energy consumption occurred at a photochemical retention time of 15 h with the biological retention time of 92 h. Total cost for different retention times was calculated demonstrating the fact that the incurred cost by the photochemical unit was considerably higher than that of the biological unit; however, the minimum total cost was evaluated to occur at 15.5 h of photochemical retention time and 90 h of biological retention time.

Solving Rank-Constrained LMI Problems With Application to Reduced-Order Output Feedback Stabilization

This paper presents an iterative penalty function method for solving rank-constrained linear matrix inequality (LMI) problems and illustrates its application to reduced-order output feedback stabilization. We propose a penalized objective function to replace the rank condition, so that a solution to the original nonconvex LMI feasibility problem can be obtained by solving a series of convex LMI optimization subproblems. Numerical experiments were performed to demonstrate the proposed method.

Bandwidth optimization for Internet traffic in generalized processor sharing servers

Bandwidth optimization is considered when several classes of Internet traffic are served in generalized processor sharing (GPS) servers. Internet traffic shows self-similar patterns that make it difficult to obtain analytical performance in GPS. Thus, for performance estimation of different classes of traffic, we use fluid simulation techniques that can reduce the simulation complexity, compared to packet-level simulation. Using the relationship between the guaranteed bandwidth vector and the corresponding performance, we propose a bandwidth optimization problem to minimize the total bandwidth such that performance requirements are satisfied. We use an exterior penalty function method to solve the optimization problem. However, a penalized objective function may have local minimum which is not a global minimum. Thus, we propose a new methodology to circumvent the limitation of the exterior penalty function method.

Docking of Atomic Clusters Through Nonlinear Optimization

The problem of molecular docking is defined as that of finding a minimum energy configuration of a pair of molecular structures (usually consisting of proteins, DNA or RNA fragments). It is often assumed that the two interacting structures can be considered as rigid bodies and that it is of interest to researchers to develop methods which enable to discover the potential binding sites. Many different models have been proposed in the literature for the definition of the potential energy between two molecular structures, most of which contain at least a term (known as Van Der Waals interaction) which accounts for pairwise attraction between atoms, a repulsion term and a term which takes into account electrostatic forces (Coulomb interaction). Some well known models, and in particular those used in rigid docking, are based on the assumption that the only terms which are relevant in the process of docking are pairwise interactions between atoms belonging to the two different parts of the structure. In this paper the problem of finding the lowest energy configuration of a pair of biomolecular structures, considered as rigid bodies, is defined and formulated as a global optimization problem. In terms of dimension of the search space this formulation is not 'high-dimensional', as there are only six degrees of freedom: 3 translation and 3 rotation parameters. However the energy surface of the docking problem is characterized by a huge number of local minima; moreover each function evaluation is quite expensive (interesting structures usually possess a few thousand atoms each). So there is a strong need both of local and of global optimization procedures. In this paper a local optimization technique, based upon standard non linear programming software and a penalized objective function, is introduced and its potential usefulness in the context of global optimization is outlined.

Complexity control in the topology optimization of continuum structures

A general mesh independent filter as a mean to control the complexity of topology optimization designed structures is discussed. A new mesh-independent filter, applied over the move-limits of the sequential linear programming is proposed, and it is shown that its use alleviates common problems in the continuum topology optimization, like checkerboarding, mesh dependency, as well as effects associated to non-structured meshes, like numerical anisotropy. The structural optimization formulation adopted in this work is the minimization of a penalized function of the volume, with constraints on the compliance of each load case. Aspects of this penalized objective function are discussed, and several numerical examples are shown.

Asymptotic properties of penalized objective functions in regular mathematical programming problems

Asymptotic properties of penalized objective functions in regular mathematical programming problems

An improved 'complex' method of function minimization

At present a large number of fast convergent optimizers are available. They are usually based on gradient information coupled with repeated quadratic and/or cubic fit during the linear minimization process. However, when a design engineer is faced with objective functions with no such continuity of the gradient, these efficient methods are of little use. Further, if a penalty function method is used for handling constraints, there is every possibility of a constraint breakthrough, in which case the penalized objective function becomes discontinuous and the method is likely to fail. The method described in this article is based on evolutionary operation and is capable of handling constraints directly. The method does not demand continuity of the objective function gradient and has a high probability of locating the global minima. Moreover, the same feasible starting point can be re-used for checking the nature of the predicted minima.