DC Algorithm Research Articles

A sub-additive DC approach to the complementarity problem

In this article, we propose a new merit function based on sub-additive functions for solving a general complementarity problem. This leads to consider an optimization problem that is equivalent to the NCP. In the case of a concave NCP this optimization problem is a Difference of Convex (DC) program and we can therefore use DC Algorithm to locally solve it. We prove that in the case of a monotone NCP, it is sufficient to compute a stationary point of the optimization problem to get a solution of the complementarity problems. In the case of a general NCP, assuming that a DC decomposition of the complementarity problem is known, we propose a penalization technique to reformulate the optimization problem as a DC program. Numerical results on linear complementarity problems and absolute value equations show that our method is promising.

A new nonconvex approach for image restoration with Gamma noise

In this paper, a new total generalized variational(TGV) model for restoring images with multiplicative noise is proposed, which contains a nonconvex fidelity term and a TGV term. We use a difference of convex functions algorithm (DCA) to deal with the proposed model. For multiplicative noise removal, there exist many models and algorithms, most of which focus on convex approximation so that numerical algorithms with guaranteed convergence can be designed. Unlike these algorithms, we use the DCA algorithm to remove multiplicative noise. By numerical experiments, it is shown that the proposed approach leads to a better solution compared with the gradient projection algorithm for solving the classic multiplicative noise removal models. We prove that the sequence generated by the DCA algorithm converges to a stationary point, which satisfies the first order optimality condition. Finally, we demonstrate the performance of our whole scheme by numerical examples. A comparison with other methods is provided as well. Numerical results demonstrate that the proposed algorithm significantly outperforms some previous methods for multiplicative Gamma noise removal.

A refined convergence analysis of $$\hbox {pDCA}_{e}$$ pDCA e with applications to simultaneous sparse recovery and outlier detection

We consider the problem of minimizing a difference-of-convex (DC) function, which can be written as the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous possibly nonsmooth concave function. We refine the convergence analysis in Wen et al. (Comput Optim Appl 69, 297–324, 2018) for the proximal DC algorithm with extrapolation ( $$\hbox {pDCA}_e$$ ) and show that the whole sequence generated by the algorithm is convergent without imposing differentiability assumptions in the concave part. Our analysis is based on a new potential function and we assume such a function is a Kurdyka–Łojasiewicz (KL) function. We also establish a relationship between our KL assumption and the one used in Wen et al. (2018). Finally, we demonstrate how the $$\hbox {pDCA}_e$$ can be applied to a class of simultaneous sparse recovery and outlier detection problems arising from robust compressed sensing in signal processing and least trimmed squares regression in statistics. Specifically, we show that the objectives of these problems can be written as level-bounded DC functions whose concave parts are typically nonsmooth. Moreover, for a large class of loss functions and regularizers, the KL exponent of the corresponding potential function are shown to be 1/2, which implies that the $$\hbox {pDCA}_e$$ is locally linearly convergent when applied to these problems. Our numerical experiments show that the $$\hbox {pDCA}_e$$ usually outperforms the proximal DC algorithm with nonmonotone linesearch (Liu et al. in Math Program, 2018. https://doi.org/10.1007/s10107-018-1327-8 , Appendix A) in both CPU time and solution quality for this particular application.

Iterative Reweighted Minimization for Generalized Norm/Quasi-Norm Difference Regularized Unconstrained Nonlinear Programming

In this paper, we study the minimization of l p-q (0 <; p ≤ 1, q ≥ 1, p=6q), the general difference of l p and l q norms/quasi-norms, as a nonconvex metric for solving unconstrained nonlinear programming. We first establish an exact (stable) sparse recovery condition for the l p-q constrained problem under an extended restricted p-isometry property and then propose an iterative algorithm for the l p-q regularized unconstrained minimization based on the t-variant of the iterative reweighted minimization method (t ≥ 1) and e-approximation. We theoretically prove that the proposed algorithm converges to a stationary point satisfying the first-order optimality condition. In particular, we provide a convergence rate analysis of the method and show that the local convergence is superlinear under a certain condition. Our extensive experimental results demonstrate that if the sensing matrix satisfies the restricted p-isometry property, the proposed iterative reweighted minimization method for the l p-q unconstrained problem generally outperforms the existing methods (especially for those methods based on the difference of norms). For the ill-conditioned sensing matrix, a variant of our method via the difference of convex functions algorithm (DCA) shows better performance on the frequency of success for signal sparse recovery. Likewise, our methods are illustrated to be valid and generally outperform the existing methods for real images.

Joint Pushing and Caching for Bandwidth Utilization Maximization in Wireless Networks

Joint pushing and caching is recognized as an efficient remedy to the problem of spectrum scarcity incurred by tremendous mobile data traffic. In this paper, we design the optimal joint pushing and caching policy to maximize bandwidth utilization, which is of fundamental importance to mobile telecom carriers. In particular, we consider a multiuser wireless network with multicast opportunities where each user is equipped with a cache of limited size. First, we formulate the stochastic optimization problem as an infinite horizon average cost Markov decision process. By the structural analysis, we show that how the optimal policy achieves a balance between the current transmission cost and the future average transmission cost. We also show that the optimal average transmission cost decreases with the cache sizes, revealing a tradeoff between storage and bandwidth. Then, due to the fact that obtaining a numerical optimal solution suffers the curse of dimensionality and implementing it requires a centralized controller and global system information, we develop a low-complexity decentralized policy (LDP) by using a linear approximation of the value function and transforming challenging discrete optimization problems into difference of convex (DC) problems, which can be efficiently solved by using DC algorithms. We also obtain an upper bound on the performance gap between the average cost of LDP and the minimum average cost, which can be easily evaluated. Next, we propose an online decentralized algorithm to implement the proposed LDP, when priori knowledge of user demand processes is not available. Finally, using numerical results, we demonstrate the advantage of the proposed solutions over some existing designs. The results in this paper offer useful guidelines for designing practical cache-enabled multiuser wireless networks.

Easy Hall position sensor fault-tolerant control algorithm for brushless DC drives

Easy Hall position sensor fault-tolerant control algorithm for brushless DC drives

Integration of Magnified Alternating Current in Battery Fast Chargers Based on DC–DC Converters Using Transformerless Resonant Filter Design

For safety and longevity reasons, in subzero temperatures, lithium-ion batteries (LIBs) can only be charged after precommissioning their temperature. Therefore, in such conditions, fast charging depends on fast heating. Recently, the injection of ac currents into LIBs has been reported as a technique with the potential to decrease heating time. This paper proposes a method based on a multi-objective algorithm for dc–dc converter design using transformerless resonant filters. The method enables the dc–dc converters to produce magnified ac current in addition to the dc current. Using the proposed design method, a topological survey of dc–dc converters with magnified ac current capability composed of either half-or full-bridge switch arrangements is carried out. In the presented experimental setup, it is demonstrated that by using an $LCL$ circuit with specific component values and a full-bridge switch arrangement, magnifications of up to 15.7 may be reached. Furthermore, by matching the switching frequency with the frequency where the $LCL$ and the battery resonate, for the same injected ac current, the current flowing in the semiconductors and the switching frequency could be reduced. This allowed a loss reduction in the semiconductors of up to 75%, when compared with an equivalent dc–dc converter enabled to produce a nonmagnified ac current.

Nonmonotone Enhanced Proximal DC Algorithms for a Class of Structured Nonsmooth DC Programming

In this paper we consider a class of structured nonsmooth difference-of-convex (DC) minimization in which the first convex component is the sum of a smooth and a nonsmooth function while the second...

Solving nonmonotone affine variational inequalities problem by DC programming and DCA

In this paper, we consider an optimization model for solving the nonmonotone affine variational inequalities problem (AVI). It is formulated as a DC (Difference of Convex functions) program for which DCA (DC Algorithms) are applied. The resulting DCA are simple: it consists of solving successive convex quadratic program. Numerical experiments on several test problems illustrate the efficiency of the proposed approach in terms of the quality of the obtained solutions and the speed of convergence.

A generalized proximal linearized algorithm for DC functions with application to the optimal size of the firm problem

The purpose of this paper is twofold. First, we examine convergence properties of an inexact proximal point method with a quasi distance as a regularization term in order to find a critical point (in the sense of Toland) of a DC function (difference of two convex functions). Global convergence of the sequence and some convergence rates are obtained with additional assumptions. Second, as an application and its inspiration, we study in a dynamic setting, the very important and difficult problem of the limit of the firm and the time it takes to reach it (maturation time), when increasing returns matter in the short run. Both the formalization of the critical size of the firm in term of a recent variational rationality approach of human dynamics and the speed of convergence results are new in Behavioral Sciences.

An Inertial Algorithm for DC Programming

We consider nonsmooth optimization problems whose objective function is defined by the Difference of Convex (DC) functions. With the aim of computing critical points that are also d(irectional)-stationary for such a class of nonconvex programs we propose an algorithmic scheme equipped with an inertial-force procedure. In contrast to the classical DC algorithm of P. D. Tao and L. T. H. An, the proposed inertial DC algorithm defines trial points whose sequence of functional values is not necessary monotonically decreasing, a property that proves useful to prevent the algorithm from converging to a critical point that is not d-stationary. Moreover, our method can handle inexactness in the solution of convex subproblems yielding trial points. This is another property of practical interest that substantially reduces the computational burden to compute d-stationary/critical points of DC programs. Convergence analysis of the proposed algorithm yields global convergence to critical points, and convergence rate is established for the considered class of problems. Numerical experiments on large-scale (nonconvex and nonsmooth) image denoising models show that the proposed algorithm outperforms the classic one in this particular application, specifically in the case of piecewise constant images with neat edges such as QR codes.

Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization

In this paper we consider a class of structured nonsmooth difference-of-convex (DC) minimization in which the first convex component is the sum of a smooth and nonsmooth functions while the second convex component is the supremum of possibly infinitely many convex smooth functions. We first propose an inexact enhanced DC algorithm for solving this problem in which the second convex component is the supremum of finitely many convex smooth functions, and show that every accumulation point of the generated sequence is an $$(\alpha ,\eta )$$ -D-stationary point of the problem, which is generally stronger than an ordinary D-stationary point. In addition, inspired by the recent work (Pang et al. in Math Oper Res 42(1):95–118, 2017; Wen et al. in Comput Optim Appl 69(2):297–324, 2018), we propose two proximal DC algorithms with extrapolation for solving this problem. We show that every accumulation point of the solution sequence generated by them is an $$(\alpha ,\eta )$$ -D-stationary point of the problem, and establish the convergence of the entire sequence under some suitable assumption. We also introduce a concept of approximate $$(\alpha ,\eta )$$ -D-stationary point and derive iteration complexity of the proposed algorithms for finding an approximate $$(\alpha ,\eta )$$ -D-stationary point. In contrast with the DC algorithm (Pang et al. 2017), our proximal DC algorithms have much simpler subproblems and also incorporate the extrapolation for possible acceleration. Moreover, one of our proximal DC algorithms is potentially applicable to the DC problem in which the second convex component is the supremum of infinitely many convex smooth functions. In addition, our algorithms have stronger convergence results than the proximal DC algorithm in Wen et al. (2018).

A unified DC programming framework and efficient DCA based approaches for large scale batch reinforcement learning

We investigate a powerful nonconvex optimization approach based on Difference of Convex functions (DC) programming and DC Algorithm (DCA) for reinforcement learning, a general class of machine learning techniques which aims to estimate the optimal learning policy in a dynamic environment typically formulated as a Markov decision process (with an incomplete model). The problem is tackled as finding the zero of the so-called optimal Bellman residual via the linear value-function approximation for which two optimization models are proposed: minimizing the $$\ell _{p}$$ -norm of a vector-valued convex function, and minimizing a concave function under linear constraints. They are all formulated as DC programs for which attractive DCA schemes are developed. Numerical experiments on various examples of the two benchmarks of Markov decision process problems—Garnet and Gridworld problems, show the efficiency of our approaches in comparison with two existing DCA based algorithms and two state-of-the-art reinforcement learning algorithms.

Maximum-Margin Model for Nearest Prototype Classifiers

In this paper, we study nearest prototype classifiers, which classify data instances into the classes to which their nearest prototypes belong. We propose a maximum-margin model for nearest prototype classifiers. To provide the margin, we define a class-wise discriminant function for instances by the negatives of distances of their nearest prototypes of the class. Then, we define the margin by the minimum of differences between the discriminant function values of instances with respect to the classes they belong to and the values of the other classes. The optimization problem corresponding to the maximum-margin model is a difference of convex functions (DC) program. It is solved using a DC algorithm, which is ak-means-like algorithm, i.e., the members and positions of prototypes are alternately optimized. Through a numerical study, we analyze the effects of hyperparameters of the maximum-margin model, especially considering the classification performance.

DC-Programming versus ℓ0-Superiorization for Discrete Tomography

Abstract In this paper we focus on the reconstruction of sparse solutions to underdetermined systems of linear equations with variable bounds. The problem is motivated by sparse and gradient-sparse reconstruction in binary and discrete tomography from limited data. To address the ℓ0-minimization problem we consider two approaches: DC-programming and ℓ0-superiorization. We show that ℓ0-minimization over bounded polyhedra can be equivalently formulated as a DC program. Unfortunately, standard DC algorithms based on convex programming often get trapped in local minima. On the other hand, ℓ0-superiorization yields comparable results at significantly lower costs.

Exact recovery low-rank matrix via transformed affine matrix rank minimization

Abstract The goal of affine matrix rank minimization problem is to reconstruct a low-rank or approximately low-rank matrix under linear constraints. In general, this problem is combinatorial and NP-hard. In this paper, a nonconvex fraction function is studied to approximate the rank of a matrix and translate this NP-hard problem into a transformed affine matrix rank minimization problem. The equivalence between these two problems is established, and we proved that the uniqueness of the global minimizer of transformed affine matrix rank minimization problem also solves affine matrix rank minimization problem if some conditions are satisfied. Moreover, we also proved that the optimal solution to the transformed affine matrix rank minimization problem can be approximately obtained by solving its regularization problem for some proper smaller λ > 0. Lastly, the DC algorithm is utilized to solve the regularization transformed affine matrix rank minimization problem and the numerical experiments on image inpainting problems show that our method performs effectively in recovering low-rank images compared with some state-of-art algorithms.

Joint Virtual Computing and Radio Resource Allocation in Limited Fronthaul Green C-RANs

We consider the virtualization technique in the downlink transmission of limited fronthaul capacity cloud-radio access networks. A novel virtual computing resource allocation (VCRA) method which can dynamically split the users workload into smaller fragments to be served by virtual machines is presented. Under the proposed scheme, we aim at maximizing the network energy efficiency by a joint design of virtual computing resources, transmit beamforming, remote radio head (RRH) selection, and RRH-user association. Moreover, we construct a more realistic fronthaul power consumption model, which is directly proportional to users’ rate transmitted by the corresponding RRHs. The formulated problem is combinatorial and difficult to solve in general. Our first contribution is to customize a branch-and-reduce-and-bound method to attain a globally optimal solution. To compute a high-quality approximate solution, a standard routine is used to deal with the continuous relaxation of the original problem. However, the proposed continuous relaxation is non-convex which implies another challenge. For a practically appealing solution approach, we resort to a local optimization method, namely the difference of convex algorithm. Our second contribution is on the use of Lipschitz continuity to arrive at a sequence of convex quadratic programs, which can be solved efficiently by modern convex solvers. Finally, a post-processing procedure is proposed to obtain a high-performance feasible solution from the continuous relaxation. Extensive numerical results demonstrate that the proposed algorithms converge rapidly and achieve near-optimal performance as well as outperform other known methods. Moreover, we numerically show that the VCRA scheme significantly improves the system energy efficiency compared to the existing schemes.

Minimization of transformed $$L_1$$ L 1 penalty: theory, difference of convex function algorithm, and robust application in compressed sensing

We study the minimization problem of a non-convex sparsity promoting penalty function, the transformed $$l_1$$ (TL1), and its application in compressed sensing (CS). The TL1 penalty interpolates $$l_0$$ and $$l_1$$ norms through a nonnegative parameter $$a \in (0,+\infty )$$ , similar to $$l_p$$ with $$p \in (0,1]$$ , and is known to satisfy unbiasedness, sparsity and Lipschitz continuity properties. We first consider the constrained minimization problem, and discuss the exact recovery of $$l_0$$ norm minimal solution based on the null space property (NSP). We then prove the stable recovery of $$l_0$$ norm minimal solution if the sensing matrix A satisfies a restricted isometry property (RIP). We formulated a normalized problem to overcome the lack of scaling property of the TL1 penalty function. For a general sensing matrix A, we show that the support set of a local minimizer corresponds to linearly independent columns of A. Next, we present difference of convex algorithms for TL1 (DCATL1) in computing TL1-regularized constrained and unconstrained problems in CS. The DCATL1 algorithm involves outer and inner loops of iterations, one time matrix inversion, repeated shrinkage operations and matrix-vector multiplications. The inner loop concerns an $$l_1$$ minimization problem on which we employ the Alternating Direction Method of Multipliers. For the unconstrained problem, we prove convergence of DCATL1 to a stationary point satisfying the first order optimality condition. In numerical experiments, we identify the optimal value $$a=1$$ , and compare DCATL1 with other CS algorithms on two classes of sensing matrices: Gaussian random matrices and over-sampled discrete cosine transform matrices (DCT). Among existing algorithms, the iterated reweighted least squares method based on $$l_{1/2}$$ norm is the best in sparse recovery for Gaussian matrices, and the DCA algorithm based on $$l_1$$ minus $$l_2$$ penalty is the best for over-sampled DCT matrices. We find that for both classes of sensing matrices, the performance of DCATL1 algorithm (initiated with $$l_1$$ minimization) always ranks near the top (if not the top), and is the most robust choice insensitive to the conditioning of the sensing matrix A. DCATL1 is also competitive in comparison with DCA on other non-convex penalty functions commonly used in statistics with two hyperparameters.

Linear Feature Transform and Enhancement of Classification on Deep Neural Network

A weighted and convex regularized nuclear norm model is introduced to construct a rank constrained linear transform on feature vectors of deep neural networks. The feature vectors of each class are modeled by a subspace, and the linear transform aims to enlarge the pairwise angles of the subspaces. The weight and convex regularization resolve the rank degeneracy of the linear transform. The model is computed by a difference of convex function algorithm whose descent and convergence properties are analyzed. Numerical experiments are carried out in convolutional neural networks on CAFFE platform for 10 class handwritten digit images (MNIST) and small object color images (CIFAR-10) in the public domain. The transformed feature vectors improve the accuracy of the network in the regime of low dimensional features subsequent to dimensional reduction via principal component analysis. The feature transform is independent of the network structure, and can be applied to reduce complexity of the final fully-connected layer without retraining the feature extraction layers of the network.

Solving the degree-concentrated fault-tolerant spanning subgraph problem by DC programming

In this paper, we consider the maximum and minimum versions of degree-concentrated fault-tolerant spanning subgraph problem which has many applications in network communications. We prove that both this two problems are NP-hard. For the maximum version, we use DC programming relaxation to propose a heuristic algorithm. Numerical tests indicate that the proposed algorithm is efficient and effective. For the minimum version, we also formulate it as a DC program, and show that the DC algorithm does not perform well for this problem.