Subgradient-based Optimization Research Articles

N-PSO: endmember extraction using advance particle swarm optimization for NLMM

This paper presents a fully unsupervised endmember extraction technique for hyperspectral image unmixing using nonlinear mixing model. The underlying idea of the model is that the pixel reflectances are nonlinear functions of pure spectral components contaminated by an additive noise. These nonlinear functions are approximated using polynomial functions, leading to a polynomial post-nonlinear mixing model (PPNM). In an unknown environment, the evaluation of the parameters involved in PPNM model is a tedious task, which is categorized as an NP hard problem. A method based on the combination of swarm intelligence, least-square (LS) and sub-gradient-based optimization (SO) is proposed to estimate the parameters involved in the model. The particle swarm optimization (PSO) is used to search the optimal endmember combination in the feasible solution space. The nonlinearity and respective abundances are evaluated using the LS and SO method, respectively. The proposed method is equipped with an adaptive tuning parameter-free mechanism and modified updating strategy. This strategy not only improves the result in terms of overall accuracy but also maintains physical constraints on the value of the resultant endmember set. The proposed method has been evaluated using simulated and real hyperspectral scenes. The experimental results on the hyperspectral scenes show that the proposed method obtains a higher extraction precision than those of the existing endmember extraction algorithms. Statistical analysis on a real hyperspectral image shows that the results obtain using N-PSO are 20–40% better than those from the existing approaches.

Diagonal discrete gradient bundle method for derivative free nonsmooth optimization

Typically, practical nonsmooth optimization problems involve functions with hundreds of variables. Moreover, there are many practical problems where the computation of even one subgradient is either a difficult or an impossible task. In such cases, the usual subgradient-based optimization methods cannot be used. However, the derivative free methods are applicable since they do not use explicit computation of subgradients. In this paper, we propose an efficient diagonal discrete gradient bundle method for derivative-free, possibly nonconvex, nonsmooth minimization. The convergence of the proposed method is proved for semismooth functions, which are not necessarily differentiable or convex. The method is implemented using Fortran 95, and the numerical experiments confirm the usability and efficiency of the method especially in case of large-scale problems.

Direct conditional probability density estimation with sparse feature selection

Regression is a fundamental problem in statistical data analysis, which aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional probability density is multi-modal, asymmetric, and heteroscedastic. To overcome this limitation, various estimators of conditional densities themselves have been developed, and a kernel-based approach called least-squares conditional density estimation (LS-CDE) was demonstrated to be promising. However, LS-CDE still suffers from large estimation error if input contains many irrelevant features. In this paper, we therefore propose an extension of LS-CDE called sparse additive CDE (SA-CDE), which allows automatic feature selection in CDE. SA-CDE applies kernel LS-CDE to each input feature in an additive manner and penalizes the whole solution by a group-sparse regularizer. We also give a subgradient-based optimization method for SA-CDE training that scales well to high-dimensional large data sets. Through experiments with benchmark and humanoid robot transition datasets, we demonstrate the usefulness of SA-CDE in noisy CDE problems.

An algorithm for LQ optimal actuator location

The locations of the control hardware are typically a design variable in controller design for distributed parameter systems. In order to obtain the most efficient control system, the locations of control hardware as well as the feedback gain should be optimized. These optimization problems are generally non-convex. In addition, the models for these systems typically have a large number of degrees of freedom. Consequently, existing optimization schemes for optimal actuator placement may be inaccurate or computationally impractical. In this paper, the feedback control is chosen to be an optimal linear quadratic regulator. The optimal actuator location problem is reformulated as a convex optimization problem. A subgradient-based optimization scheme which leads to the global solution of the problem is used to optimize actuator locations. The optimization algorithm is applied to optimize the placement of piezoelectric actuators in vibration control of flexible structures. This method is compared with a genetic algorithm, and is observed to be faster and more accurate. Experiments are performed to verify the efficacy of optimal actuator placement.

A novel subgradient-based optimization algorithm for blockmodel functional module identification

Functional module identification in biological networks may provide new insights into the complex interactions among biomolecules for a better understanding of cellular functional organization. Most of existing functional module identification methods are based on the optimization of network modularity and cluster networks into groups of nodes within which there are a higher-than-expectation number of edges. However, module identification simply based on this topological criterion may not discover certain kinds of biologically meaningful modules within which nodes are sparsely connected but have similar interaction patterns with the rest of the network. In order to unearth more biologically meaningful functional modules, we propose a novel efficient convex programming algorithm based on the subgradient method with heuristic path generation to solve the problem in a recently proposed framework of blockmodel module identification. We have implemented our algorithm for large-scale protein-protein interaction (PPI) networks, including Saccharomyces cerevisia and Homo sapien PPI networks collected from the Database of Interaction Proteins (DIP) and Human Protein Reference Database (HPRD). Our experimental results have shown that our algorithm achieves comparable network clustering performance in comparison to the more time-consuming simulated annealing (SA) optimization. Furthermore, preliminary results for identifying fine-grained functional modules in both biological networks and the comparison with the commonly adopted Markov Clustering (MCL) algorithm have demonstrated the potential of our algorithm to discover new types of modules, within which proteins are sparsely connected but with significantly enriched biological functionalities.

Combinatorial individual differences scaling within the city-block metric

A new method is proposed for conducting individual differences scaling within the city-block metric that does not rely on gradient- or subgradient-based optimization. Instead, a combinatorial optimization scheme is utilized for identifying object coordinates minimizing the least-squares loss function. The illustrative application of combinatorial individual differences scaling within the city-block metric to schematic face stimuli suggests that the new method offers a promising alternative to gradient-based attempts for fitting city-block scaling models, which suffer from the well-documented difficulty of local minima.

A subgradient-based optimization for reservoirs system management

The problem of determining the optimal operating policy for a system of reservoirs equipped with hydroelectric power plants is a nonlinear programming problem. In general, practitioners have found it hard to solve even a problem for a system with a small number of planning periods and hydroelectric plants using current nonlinear programming techniques. This is due, in large measure, to complex nonlinearities of the power generating functions and a large number of linear equality constraints involved in the model. Furthermore, most reservoirs systems in arid regions have periods of low water level which often results in an empty feasible region adding considerable difficulties to the operational feasibilities of the solutions. In this paper, we provide a general formulation of the water resource allocation problem with explicit engineering details generally omitted or oversimplified in the published literature, and investigate several solution procedures for their applicabilities, and we develop an efficient algorithmic framework exploiting the general water resource system special network structure to solve the linear equality constraints. Practical feasibility of this approach is demonstrated in several applications with a real water resource system.