Original Objective Function Research Articles

Variational Bayesian Learning for Dirichlet Process Mixture of Inverted Dirichlet Distributions in Non-Gaussian Image Feature Modeling.

In this paper, we develop a novel variational Bayesian learning method for the Dirichlet process (DP) mixture of the inverted Dirichlet distributions, which has been shown to be very flexible for modeling vectors with positive elements. The recently proposed extended variational inference (EVI) framework is adopted to derive an analytically tractable solution. The convergency of the proposed algorithm is theoretically guaranteed by introducing single lower bound approximation to the original objective function in the EVI framework. In principle, the proposed model can be viewed as an infinite inverted Dirichlet mixture model that allows the automatic determination of the number of mixture components from data. Therefore, the problem of predetermining the optimal number of mixing components has been overcome. Moreover, the problems of overfitting and underfitting are avoided by the Bayesian estimation approach. Compared with several recently proposed DP-related methods and conventional applied methods, the good performance and effectiveness of the proposed method have been demonstrated with both synthesized data and real data evaluations.

Beampattern synthesis with minimal dynamic range ratio

Abstract The excitations with a small value of dynamic range ratio (DRR) are helpful to control the mutual coupling between array elements, reduce the output power loss, and simplify the feeding network design. Unlike the commonly specified DRR, this communication focuses on finding the minimal DRR of the excitations while synthesizing a desired beampattern. To address this issue, a nonconvex and nonlinear optimization problem is formulated to synthesize array pattern while minimizing the DRR of the excitations, which is solved by extending the penalty method. Firstly, violations of all the inequality constraints are augmented into the original objective function as penalty functions which are nonsmooth. Secondly, a new smooth function is introduced to replace the nonsmooth penalty functions. These operators convert the constrained optimization problem to a smooth unconstrained one which can be easily solved. Theoretical analysis on the approximation error between the constrained optimization problem and the transformed one is provided. Numerical results are also given to verify the proposed scheme in achieving the minimal DRR.

A Regularized Variable Projection Algorithm for Separable Nonlinear Least Squares Problems

Separable nonlinear least-squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNLLS problems based on the VP method. Selecting an appropriate regularization parameter is difficult because of the nonlinear optimization procedure. We propose to determine the regularization parameter using the weighted generalized cross-validation method at every iteration. This makes the original objective function changing during the optimization procedure. To circumvent this problem, we use an inequation to produce a consistent demand of decreasing at successive iterations. The approximation of the Jacobian of the regularized problem is also discussed. The proposed regularized VP algorithm is tested by the parameter estimation problem of several statistical models. Numerical results demonstrate the effectiveness of the proposed algorithm.

Interactive decision making with fuzzy goals for simple recourse in multiobjective stochastic programming problems

In this paper, we focus on multiobjective stochastic programming problems, and propose two interactive fuzzy decision-making methods to obtain a satisfactory solution of a decision maker. In the proposed methods, equality constraints with random variables are formulated on the basis of the two-stage programming method, in which two kinds of random variables, i.e., continuous ones and discrete ones, are considered, respectively. Under the assumption that the decision maker has fuzzy goals not only for the original objective functions but also for the expectations of shortages and excesses for the violation of the equality constraints, the M-α-Pareto optimality concept is introduced, and two interactive fuzzy decision-making methods are proposed to obtain a satisfactory solution from among an M-α-Pareto optimal solution set. The proposed method is applied to a farm planning problem in the Philippines, in which it is assumed that an amount supplied of water resource in dry season is represented as a continuous or discrete random variable.

Energy efficient resource allocation in timesharing multiuser systems with hybrid energy harvesting transmitter

An energy efficient resource allocation scheme in timesharing multiuser system with a hybrid energy harvesting transmitter is studied in this paper. Specially, the operation energy of system is supplied by constant energy and energy harvesting, which harvests energy from external environment. Our goal is to maximize the energy efficiency of timesharing multiuser systems by considering jointly allocation of transmission time and power control in an off-line manner. The original nonconvex objective function is transformed into convex optimization problem via the fractional programming approach. Then, we solve the convex problem by Lagrange dual decomposition method. Simulation results show that the proposed energy efficient resource allocation scheme has a better performance than the scheme which decomposes optimization problem into two parts (power allocation, time allocation) to solve iteratively.

A simple globally convergent algorithm for the nonsmooth nonconvex single source localization problem

We study the single source localization problem which consists of minimizing the squared sum of the errors, also known as the maximum likelihood formulation of the problem. The resulting optimization model is not only nonconvex but is also nonsmooth. We first derive a novel equivalent reformulation as a smooth constrained nonconvex minimization problem. The resulting reformulation allows for deriving a delightfully simple algorithm that does not rely on smoothing or convex relaxations. The proposed algorithm is proven to generate bounded iterates which globally converge to critical points of the original objective function of the source localization problem. Numerical examples are presented to demonstrate the performance of our algorithm.

A Robust Iteratively Reweighted $\ell _2$ Approach for Spectral Compressed Sensing in Impulsive Noise

This letter concentrates on the problem of spectral compressed sensing in impulsive noise, which aims to recover a spectrally sparse signal from its contaminated and undersampled measurements. We propose a robust formulation for joint sparse signal and frequency recovery, which includes the generalized $\ell _p$ -norm $(0 data-fidelity fitting term added to a log-sum sparsity-promoting regularizer. To handle this intractable issue, we develop an iteratively reweighted $\ell _2$ approach via majorizing the original objective function by a quadratic surrogate function. Simulation results illustrate that the proposed approach attains a significant performance improvement over the existing methods under impulsive noise.

Efficient local optimisation‐based approach for non‐convex and non‐smooth source localisation problems

This study proposes a novel approach to solve the source localisation problem with noisy range measurements. Since the objective function of the range-based least square problem is non-convex and non-smooth, it is challenging to achieve an accurate estimation. Unlike previous methods that took non-convexity property of the objective function into account, this approach addresses the problem considering the non-smoothness property of the objective function. The problem is solved by a two-stage local optimisation technique which is based on a smooth non-convex approximation of the original objective function. First, the least square method is utilised to obtain a coarse estimation of the problem. Then, the estimation is refined by the trust region method which converges quadratically. Simulation results are presented to show that the proposed approach outperforms other existing methods in terms of the mean squared localisation error and convergence speed.

An Interval Programming Approach for Multi-period and Multi-product Aggregate Production Planning by Considering the Decision Maker’s Preference

This paper presents an interval programming approach for solving a typical multi-period and multi-product aggregate production planning (MPMP-APP) problem. Firstly, a MPMP-APP model based on interval programming is developed, in which the decision maker’s risk preference is taken into consideration. Next, to solve the MPMP-APP model based on interval numbers, the original uncertain objective function is replaced by two crisp objective functions which are equivalent to minimizing the interval value and the deviation of the uncertain objective function, respectively, and uncertain constraints are transformed into their corresponding crisp equivalents by using the possibility degree based on six possible relations between two intervals. And then, the linear weighted sum method is adopted to transform the above two-objective model into a single one which can be solved by LINGO software. Finally, an industrial example is used to illustrate the validity and flexibility of the proposed method. It is expected that this study can provide a useful reference for decision makers to make a rational production plan in uncertain environment.

The Moreau envelope based efficient first-order methods for sparse recovery

Sparse recovery from indirectly under-sampled or possibly noisy data is a burgeoning topic drawing the attention of many researchers. Since sparse recovery problems can be cast as a class of the constrained convex optimization models which minimize a nonsmooth convex objection function in a convex closed set, fast and efficient methods for solving the constrained optimization models are highly needed. By introducing the indicator functions related to constrained sets in sparse recovery models, we reformulate these models as two general unconstrained optimization problems. To develop fast first-order methods, two smoothing approaches are proposed based on the Moreau envelope: smoothing the related indicator functions or the objection functions. By using the first smoothing approach, we obtain more proper unconstrained models for sparse recovery from noisy data. Fast iterative shrinkage-thresholding algorithm (FISTA) is applied to solve the smoothed models. When smoothing the objection functions, we propose an efficient first-order method based on FISTA to establish the rate of convergence of order O(logkk) for the iterative sequence of values of the original objective functions. Numerical experiments have demonstrated that the two proposed smoothing methods are comparable to the state-of-the-art first-order methods with respect to accuracy and speed when applied to the sparse recovery problems such as compressed sensing, matrix completion, and robust and stable principal component analysis.

Fixed-point algorithms for frequency estimation and structured low rank approximation

We develop fixed-point algorithms for the approximation of structured matrices with rank penalties. In particular we use these fixed-point algorithms for making approximations by sums of exponentials, i.e., frequency estimation. For the basic formulation of the fixed-point algorithm we show that it converges to the solution of a related minimization problem, namely the one obtained by replacing the original objective function with its convex envelope and keeping the structured matrix constraint unchanged. It often happens that this solution agrees with the solution to the original minimization problem, and we provide a simple criterion for when this is true. We also provide more general fixed-point algorithms that can be used to treat the problems of making weighted approximations by sums of exponentials given equally or unequally spaced sampling. We apply the method to the case of missing data, although the above mentioned convergence results do not hold in this case. However, it turns out that the method often gives perfect reconstruction (up to machine precision) in such cases. We also discuss multidimensional extensions, and illustrate how the proposed algorithms can be used to recover sums of exponentials in several variables, but when samples are available only along a curve.

An interior affine scaling cubic regularization algorithm for derivative-free optimization subject to bound constraints

In this paper, we introduce an affine scaling cubic regularization algorithm for solving optimization problem without available derivatives subject to bound constraints employing a polynomial interpolation approach to handle the unavailable derivatives of the original objective function. We first define an affine scaling cubic model of the approximate objective function which is obtained by the polynomial interpolation approach with an affine scaling method. At each iteration a candidate search direction is determined by solving the affine scaling cubic regularization subproblem and the new iteration is strictly feasible by way of an interior backtracking technique. The global convergence and local superlinear convergence of the proposed algorithm are established under some mild conditions. Preliminary numerical results are reported to show the effectiveness of the proposed algorithm.

Newton-Type Greedy Selection Methods for $\ell _0$ -Constrained Minimization.

We introduce a family of Newton-type greedy selection methods for -constrained minimization problems. The basic idea is to construct a quadratic function to approximate the original objective function around the current iterate and solve the constructed quadratic program over the cardinality constraint. The next iterate is then estimated via a line search operation between the current iterate and the solution of the sparse quadratic program. This iterative procedure can be interpreted as an extension of the constrained Newton methods from convex minimization to non-convex -constrained minimization. We show that the proposed algorithms converge asymptotically and the rate of local convergence is superlinear up to certain estimation error. Our methods compare favorably against several state-of-the-art greedy selection methods when applied to sparse logistic regression and sparse support vector machines.

Dual Control and Online Optimal Experimental Design

Dual control refers to strategies that strike a balance between control and estimation. Combined with nonlinear model predictive control, dual control offers advanced feedback methods for optimal control problems under uncertainties. We present dual control from a new perspective, namely, the interplay between the performance control task and the information gain task in connection with optimal experimental design. A new approach to dual control is proposed in which the covariance matrix of the estimates is weighted by the derivatives of the nominal objective value with respect to unknown parameters and initial states. This quantity can be interpreted as a predictive variance. Applying the idea of biobjective formulation, we use a weighted sum of the original objective function and the predictive variance as a modified objective function. Most notably, we discuss the theoretical background of this approach and provide probabilistic bounds for the controller performance with respect to the original objective function. As applications, we investigate a moon lander problem and a batch bioreactor problem. Numerical results demonstrate the superiority of dual control over nominal control. We also carry out an analysis of the relationship between the performance control task and the information gain task in order to assess when the extra effort for dual control is justified.

Quadratic Convex Reformulations for Semicontinuous Quadratic Programming

We consider in this paper a class of semi-continuous quadratic programming problems which arises in many real-world applications such as production planning, portfolio selection and subset selection in regression. We propose a lift-and-convexification approach to derive an equivalent reformulation of the original problem. This lift-and-convexification approach lifts the quadratic term involving $x$ only in the original objective function $f(x,y)$ to a quadratic function of both $x$ and $y$ and convexifies this equivalent objective function. While the continuous relaxation of our new reformulation attains the same tight bound as achieved by the continuous relaxation of the well known perspective reformulation, the new reformulation also retains the linearly constrained quadratic programming structure of the original mix-integer problem. This prominent feature improves the performance of branch-and-bound algorithms by providing the same tightness at the root node as the state-of-the-art perspective reformulation and offering much faster processing time at children nodes. We further combine the lift-and-convexification approach and the quadratic convex reformulation approach in the literature to form an even tighter reformulation. Promising results from our computational tests in both portfolio selection and subset selection problems numerically verify the benefits from these theoretical features of our new reformulations.

Secrecy Energy Efficiency Maximization in Cognitive Radio Networks

In this paper, we investigate a tradeoff between the secrecy rate (SR) and energy efficiency (EE) in an underlay cognitive radio network that consists of a cognitive base station (CBS), a cognitive user (CU), a primary user (PU), and multiple eavesdroppers (EDs). By using a so-called secrecy EE (SEE), which is defined as the ratio of SR to the total power consumption of CBS, as the design criterion, we formulate an SEE maximization (SEEM) problem for the CBS-CU transmission under the constraints of the transmit power of CBS, the SR of CU, and the quality-of-service (QoS) requirement of PU. Since the formulated optimization problem with a fractional objective function is non-convex and mathematically intractable, we first convert the original fractional objective function into an equivalent subtractive form, and then develop a method of combining the penalty function and the difference of two-convex functions (D.C.) approach to obtain an approximate convex problem. Based on this, an optimal beamforming (OBF) scheme is finally proposed to obtain the optimal solution. Furthermore, to reduce the computational complexity, we design a zero-forcing-based beamforming (ZFBF) scheme to achieve a sub-optimal solution to the SEEM problem. Simulation results are given to illustrate the effectiveness and advantage of the proposed SEE oriented OBF and ZFBF schemes over conventional SR maximization and EE maximization schemes.

A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes

In this paper, we develop a version of the bundle method to solve unconstrained difference of convex (DC) programming problems. It is assumed that a DC representation of the objective function is available. Our main idea is to utilize subgradients of both the first and second components in the DC representation. This subgradient information is gathered from some neighborhood of the current iteration point and it is used to build separately an approximation for each component in the DC representation. By combining these approximations we obtain a new nonconvex cutting plane model of the original objective function, which takes into account explicitly both the convex and the concave behavior of the objective function. We design the proximal bundle method for DC programming based on this new approach and prove the convergence of the method to an $$\varepsilon $$ź-critical point. The algorithm is tested using some academic test problems and the preliminary numerical results have shown the good performance of the new bundle method. An interesting fact is that the new algorithm finds nearly always the global solution in our test problems.

EAF Management Optimization

The article presents the main advantages of electric arc furnace (EAF): it has a great contribution to reintroduce significant quantities of reusable metallic materials in the economic circuit, it constitutes itself as an important part in the Primary Materials and Energy Recovery (PMER), good productivity, good quality / price ratio, the possibility of developing a wide variety of classes and types of steels, including special steels and high alloy. In this paper it is presented some important developments of electric arc furnace: vacuum electric arc furnace, artificial intelligence expert systems for pollution control Steelworks. Another important aspect presented in the article is an original block diagram for optimization the EAF management system. This scheme is based on the original objective function (criterion function) represented by the price / quality ratio. The article presents an original block diagram for optimization the control system of the EAF. For designing this concept of EAF management system, many principles were used.

Partial evaluation in Rank Aggregation Problems

Solving a problem by using metaheuristic algorithms requires the evaluation of a large number of potential solutions. This paper presents a theoretical and experimental study of the application of partial evaluation in the Rank Aggregation Problem (RAP). Partial evaluation just computes the part of the objective function that is affected by the modifications introduced by certain operators. In particular, we study some of the most common mutation/neighboring operators that are used in problems defined in the space of permutations, namely insertion, interchange and inversion. The theoretical study shows that the complexity of evaluating the original objective function can be reduced from quadratic to linear by using partial evaluation after applying insertion and interchange. Regarding the experimental study, it shows that for the insertion and interchange operators, the running time for the evaluation of the objective function may be reduced by a factor of 10 (30) for problems with more than 50 (100) items to be ranked. In the case of the inversion operator the amount of time saved time is not so large, although it is still significant.

Undersampled Sparse Phase Retrieval via Majorization–Minimization

In the undersampled phase retrieval problem, the goal is to recover an $N$-dimensional complex signal $\mathbf{x}$ from only $M<N$ noisy intensity measurements without phase information. This problem has drawn a lot of attention to reduce the number of required measurements since a recent theory established that $M\approx4N$ intensity measurements are necessary and sufficient to recover a generic signal $\mathbf{x}$. In this paper, we propose to exploit the sparsity in the original signal and develop low-complexity algorithms with superior performance based on the majorization-minimization (MM) framework. The proposed algorithms are preferred to existing benchmark methods since at each iteration a simple surrogate problem is solved with a closed-form solution that monotonically decreases the original objective function. Experimental results validate that our algorithms outperform existing up-to-date methods in terms of recovery probability and accuracy, under the same settings.