DC Algorithm Research Articles

Structure-Guided L1-2 Minimization for Stable Multichannel Seismic Attenuation Compensation

Absorption in subsurface media severely degrades seismic data quality. Seismic attenuation compensation as an important processing method can effectively improve the resolution and fidelity of seismic data. Based on sparse reflectivity model and attenuated convolution function, inversion-based compensation approaches show better stability and accuracy over traditional direct compensation schemes. However, conventional inversion-based compensation methods are conducted on single trace, which ignore the subsurface spatial continuity and make the compensated result contaminated with high-frequency noise. In this paper, we develop a structurally constrained multichannel L1-2 minimization for seismic attenuation compensation. We first estimate structure tensors from migrated seismic images. The structure tensors can be decomposed by eigenvalues and eigenvectors, which can reflect the structural orientations. Then, we introduce the estimated orientations as a regularization term to the L1-2 inversion-based compensation objective function. In this way, we can improve the stability of the compensation result and enhance the spatial continuity of the compensated seismic reflectors. The structure-guided L1-2 regularized compensation objective function can be efficiently solved via difference of convex algorithm and alternating direction method of multipliers. Synthetic and field data examples demonstrate that the proposed method possesses superior performance over conventional L1-2 regularized inversion-based compensation.

Stabilization of DC–DC buck converter with unknown constant power load via passivity‐based control plus proportion‐integration

It is known that constant power load (CPL) may cause a negative impedance, which seriously affects the stability of power system. In this paper, a new control algorithm for DC–DC buck converter feeding unknown CPL is proposed. First, under the assumption of known extracted power load, the standard passivity–based control (PBC) is presented to reshape the system energy and compensate for the negative impedance and a proportion-integration (PI) action around passive output is added to improve disturbance rejection performance, which forms the PBC plus PI (PBC+PI). Then, a parameter estimation algorithm is developed, based on immersion and invariance (I&I) technique, in order to online estimate the extracted power load. In the next step, the online estimation scheme is adopted to construct an adaptive strategy. Finally, the stability analysis of the cascaded system containing a closed-loop control system and observer error dynamics is conducted. Simulation and experimental results are demonstrated to validate the performance of the proposed controller.

Deep Reinforcement Learning-Based Resource Allocation in Cooperative UAV-Assisted Wireless Networks

We consider the downlink of an unmanned aerial vehicle (UAV) assisted cellular network consisting of multiple cooperative UAVs, whose operations are coordinated by a central ground controller using wireless fronthaul links, to serve multiple ground user equipments (UEs). A problem of jointly designing UAVs’ positions, transmit beamforming, as well as UAV-UE association is formulated in the form of mixed integer nonlinear programming (MINLP) to maximize the sum UEs’ achievable rate subject to limited fronthaul capacity constraints. Solving the considered problem is hard owing to its non-convexity and the unavailability of channel state information (CSI) due to the movement of UAVs. To tackle these effects, we propose a novel algorithm comprising of two distinguishing features: (i) exploiting a deep Q-learning approach to tackle the issue of CSI unavailability for determining UAVs’ positions, (ii) developing a difference of convex algorithm (DCA) to efficiently solve for the UAV’s transmit beamforming and UAV-UE association. The proposed algorithm recursively solves the problem of interest until convergence, where each recursion executes two steps. In the first step, the deep Q-learning (DQL) algorithm allows UAVs to learn the overall network state and account for the joint movement of all UAVs to adapt their locations. In the second step, given the determined UAVs’ positions from the DQL algorithm, the DCA iteratively solves a convex approximate subproblem of the original non-convex MINLP problem with the updated parameters, where the problem’s variables are transmit beamforming and UAV-UE association. Numerical results show that our design outperforms the existing algorithms in terms of algorithmic convergence and network performance with a gain of up to 70%.

A LogTVSCAD Nonconvex Regularization Model for Image Deblurring in the Presence of Impulse Noise

This paper proposes a nonconvex model (called LogTVSCAD) for deblurring images with impulsive noises, using the log-function penalty as the regularizer and adopting the smoothly clipped absolute deviation (SCAD) function as the data-fitting term. The proposed nonconvex model can effectively overcome the poor performance of the classical TVL1 model for high-level impulsive noise. A difference of convex functions algorithm (DCA) is proposed to solve the nonconvex model. For the model subproblem, we consider the alternating direction method of multipliers (ADMM) algorithm to solve it. The global convergence is discussed based on Kurdyka–Lojasiewicz. Experimental results show the advantages of the proposed nonconvex model over existing models.

On the Secrecy Capacity of MIMO Wiretap Channels: Convex Reformulation and Efficient Numerical Methods

This paper presents novel numerical approaches to finding the secrecy capacity of the multiple-input multiple-output (MIMO) wiretap channel subject to multiple linear transmit covariance constraints, including sum power constraint, per antenna power constraints and interference power constraint. An analytical solution to this problem is not known and existing numerical solutions suffer from slow convergence rate and/or high per-iteration complexity. Deriving computationally efficient solutions to the secrecy capacity problem is challenging since the secrecy rate is expressed as a difference of convex functions (DC) of the transmit covariance matrix, for which its convexity is only known for some special cases. In this paper we propose two low-complexity methods to compute the secrecy capacity along with a convex reformulation for degraded channels. In the first method we capitalize on the accelerated DC algorithm which requires solving a sequence of convex subproblems, for which we propose an efficient iterative algorithm where each iteration admits a closed-form solution. In the second method, we rely on the concave-convex equivalent reformulation of the secrecy capacity problem which allows us to derive the so-called partial best response algorithm to obtain an optimal solution. Notably, each iteration of the second method can also be done in closed form. The simulation results demonstrate a faster convergence rate of our methods compared to other known solutions. We carry out extensive numerical experiments to evaluate the impact of various parameters on the achieved secrecy capacity.

DCA based approaches for bi-level variable selection and application for estimate multiple sparse covariance matrices

Variable selection plays an important role in analyzing high dimensional data and is a fundamental problem in machine learning. When the data possesses certain group structures in which individual variables are also meaningful scientifically, we are naturally interested in selecting important groups as well as important variables within the selected groups. This is referred to as the bi-level variable selection which is much more complex than the selection of individual variables. In recent years, research on the topic of variable selection is very active, but the majority of the work is focused on the individual variable selection. There is therefore a need to further develop more effective approaches for bi-level variable selection. Since DC (Difference of Convex functions) programming and DCA (DC Algorithm), powerful tools in nonconvex programming framework, have been successfully investigated for individual variable selection, we believe that they could be efficiently exploited for the more difficult bi-level variable selection task. In that direction, we investigate in this work DC approximations of the mixed zero norm (ℓ0,0) and the combined norm (ℓ0+ℓq,0). The resulting approximate problems are then formulated as DC programs for which DCA based algorithms are introduced. As an application, these DCA schemes are developed for estimating multiple sparse covariance matrices sharing some common structures such as the locations or weights of non-zero elements. The experimental results on both simulated and real datasets indicate the efficiency of our algorithms.

Resources optimization for secure transmission in wireless powered communication networks

This paper studies the secrecy communication on wireless powered communication network (WPCN) in an orthogonal frequency division multiplexing (OFDM) system, where an energy harvest (EH) user sends confidential information to the hybrid access point (HAP) in the presence of a potential eavesdropper. We introduce a wireless powered cooperative jammer to improve the secrecy performance of this system. For the user and the jammer, we both consider the linear energy harvesting model and the nonlinear energy harvesting model. More specifically, the half-duplex WPCN is considered first. We study secrecy rate maximization problem by jointly optimizing the time allocation and the transmit power allocation for the HAP, the jammer and the user over sub-carriers. The nested form algorithm is proposed to solve the original non-convex problem, where one-dimension search method is used at the outer layer and a convex approximation algorithm by leveraging DC-based optimization is adopted at the inner layer. Then, the full-duplex WPCN is considered. We formulated the secrecy rate maximization problem and optimized the transmit power by using the DC algorithm and convex optimization. Simulation results demonstrate that the proposed scheme dramatically enhances the secrecy rate compared to the benchmark schemes.

Unified SVM algorithm based on LS-DC loss

Over the past two decades, support vector machine (SVM) has become a popular supervised machine learning model, and plenty of distinct algorithms are designed separately based on different KKT conditions of the SVM model for classification/regression with different losses, including the convex loss or nonconvex loss. In this paper, we propose an algorithm that can train different SVM models in a \emph{unified} scheme. First, we introduce a definition of the \emph{LS-DC} (\textbf{l}east \textbf{s}quares type of \textbf{d}ifference of \textbf{c}onvex) loss and show that the most commonly used losses in the SVM community are LS-DC loss or can be approximated by LS-DC loss. Based on DCA (difference of convex algorithm), we then propose a unified algorithm, called \emph{UniSVM}, which can solve the SVM model with any convex or nonconvex LS-DC loss, in which only a vector is computed, especially by the specifically chosen loss. Particularly, for training robust SVM models with nonconvex losses, UniSVM has a dominant advantage over all existing algorithms because it has a closed-form solution per iteration, while the existing algorithms always need to solve an L1SVM/L2SVM per iteration. Furthermore, by the low-rank approximation of the kernel matrix, UniSVM can solve the large-scale nonlinear problems with efficiency. To verify the efficacy and feasibility of the proposed algorithm, we perform many experiments on some small artificial problems and some large benchmark tasks with/without outliers for classification and regression for comparison with state-of-the-art algorithms. The experimental results demonstrate that UniSVM can achieve comparable performance in less training time. The foremost advantage of UniSVM is that its core code in Matlab is less than 10 lines; hence, it can be easily grasped by users or researchers.

Indefinite twin support vector machine with DC functions programming

Twin support vector machine (TWSVM) is an efficient algorithm for binary classification. However, the lack of the structural risk minimization principle restrains the generalization of TWSVM and the guarantee of convex optimization constraints TWSVM to only use positive semi-definite kernels (PSD). In this paper, we propose a novel TWSVM for indefinite kernel called indefinite twin support vector machine with difference of convex functions programming (ITWSVM-DC). The indefinite TWSVM (ITWSVM) leverages a maximum margin regularization term to improve the generalization of TWSVM and a smooth quadratic hinge loss function to make the model continuously differentiable. The representer theorem is applied to the ITWSVM and the convexity of the ITWSVM is analyzed. In order to address the non-convex optimization problem when the kernel is indefinite, a difference of convex functions (DC) is used to decompose the non-convex objective function into the subtraction of two convex functions and a line search method is applied in the DC algorithm to accelerate the convergence rate. A theoretical analysis illustrates that ITWSVM-DC can converge to a local optimum and extensive experiments on indefinite and positive semi-definite kernels show the superiority of ITWSVM-DC.

Low-rank matrix recovery with Ky Fan 2-k-norm

Low-rank matrix recovery problem is difficult due to its non-convex properties and it is usually solved using convex relaxation approaches. In this paper, we formulate the non-convex low-rank matrix recovery problem exactly using novel Ky Fan 2-k-norm-based models. A general difference of convex functions algorithm (DCA) is developed to solve these models. A proximal point algorithm (PPA) framework is proposed to solve sub-problems within the DCA, which allows us to handle large instances. Numerical results show that the proposed models achieve high recoverability rates as compared to the truncated nuclear norm method and the alternating bilinear optimization approach. The results also demonstrate that the proposed DCA with the PPA framework is efficient in handling larger instances.

Improving neural network using a sine tree-seed algorithm for tuning motor DC

A DC motor is applied to delicate speed and position in the industry. The stability and productivity of a system are keys for tuning of a DC motor speed. Stabilized speed is influenced by load sway and environmental factors. In this paper, a comparison study in diverse techniques to tune the speed of the DC motor with parameter uncertainties is showed. The research has discussed the application of the feed-forward neural network (FFNN) which is enhanced by a sine tree-seed algorithm (STSA). STSA is a hybrid method of the tree-seed algorithm (TSA) and Sine Cosine algorithm. The STSA method is aimed to improve TSA performance based on the sine cosine algorithm (SCA) method. A feed-forward neural network (FFNN) is popular and capable of nonlinear issues. The focus of the research is on the achievement speed of DC motor. In addition, the proposed method will be compared with proportional integral derivative (PID), FFNN, marine predator algorithm-feed-forward neural network (MPA-NN) and atom search algorithm-feed-forward neural network (ASO-NN). The performance of the speed from the proposed method has the best result. The settling time value of the proposed method is more stable than the PID method. The ITAE value of the STSA-NN method was 31.3% better than the PID method. Meanwhile, the ITSE value is 29.2% better than the PID method.

Alternating DC algorithm for partial DC programming problems

Alternating DC algorithm for partial DC programming problems

An Accelerated Proximal Algorithm for the Difference of Convex Programming

In this paper, we propose an accelerated proximal point algorithm for the difference of convex (DC) optimization problem by combining the extrapolation technique with the proximal difference of convex algorithm. By making full use of the special structure of DC decomposition and the information of stepsize, we prove that the proposed algorithm converges at rate of O 1 / k 2 under milder conditions. The given numerical experiments show the superiority of the proposed algorithm to some existing algorithms.

Outcome weighted ψ-learning for individualized treatment rules.

An individualized treatment rule is often employed to maximize a certain patient-specific clinical outcome based on his/her clinical or genomic characteristics as well as heterogeneous response to treatments. Although developing such a rule is conceptually important to personalized medicine, existing methods such as the partial least squares Qian and Murphy (2011) suffers from the difficulty of indirect maximization of a patient's clinical outcome, while the outcome weighted learning Y. Zhao, Zeng, Rush, and Kosorok (2012) is not robust against any perturbation of the outcome. In this article, we propose a weighted ψ-learning method to optimize an individualized treatment rule, which is robust against any data perturbation near the decision boundary by seeking the maximum separation. To solve nonconvex minimization, we employ a difference convex algorithm to relax the non-convex minimization iteratively based on a decomposition of the cost function into a difference of two convex functions. On this ground, we also introduce a variable selection method for further removing redundant variables for a higher performance. Finally, we illustrate the proposed method by simulations and a lung health study and demonstrate that it yields higher performances in terms of accuracy of prediction of individualized treatment.

Alternating DCA for reduced-rank multitask linear regression with covariance matrix estimation

We study a challenging problem in machine learning that is the reduced-rank multitask linear regression with covariance matrix estimation. The objective is to build a linear relationship between multiple output variables and input variables of a multitask learning process, taking into account the general covariance structure for the errors of the regression model in one hand, and reduced-rank regression model in another hand. The problem is formulated as minimizing a nonconvex function in two joint matrix variables (X,Θ) under the low-rank constraint on X and positive definiteness constraint on Θ. It has a double difficulty due to the non-convexity of the objective function as well as the low-rank constraint. We investigate a nonconvex, nonsmooth optimization approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) for this hard problem. A penalty reformulation is considered which takes the form of a partial DC program. An alternating DCA and its inexact version are developed, both algorithms converge to a weak critical point of the considered problem. Numerical experiments are performed on several synthetic and benchmark real multitask linear regression datasets. The numerical results show the performance of the proposed algorithms and their superiority compared with three classical alternating/joint methods.

Sparse signal recovery via infimal convolution based penalty

In this paper, we propose a non-convex penalty function for sparse signal recovery by using infimal convolution approximation. First, we show that this penalty function is between the ℓ1 norm and the difference of ℓ1 and ℓ2 norm (ℓ1−2 norm), thus it can retain the advantages of these two norms at the same time, which means that it can induce the sparsity effectively for the low-amplitude components as the ℓ1 norm and relieve underestimating the high-amplitude components as the ℓ1−2 norm. Second, we employ two iterative methods to solve the non-convex penalty minimization. One is based on the difference of convex algorithm (DCA), which uses the alternating direction method of multipliers (ADMM) to solve the subproblem. And the other one employs the forward–backward splitting (FBS) algorithm, which can be solved by using the derived closed-form solutions. We also show that these two algorithms converge to a stationary point satisfying the first-order optimality condition. The experimental results demonstrate the effectiveness of the proposed method by comparing with some other penalty functions.

DCA for online prediction with expert advice

We investigate DC (Difference of Convex functions) programming and DCA (DC Algorithm) for a class of online learning techniques, namely prediction with expert advice, where the learner’s prediction is made based on the weighted average of experts’ predictions. The problem of predicting the experts’ weights is formulated as a DC program for which an online version of DCA is investigated. The two so-called approximate/complete variants of online DCA based schemes are designed, and their regrets are proved to be logarithmic/sublinear. The four proposed algorithms for online prediction with expert advice are furthermore applied to online binary classification. Experimental results tested on various benchmark datasets showed their performance and their superiority over three standard online prediction with expert advice algorithms—the well-known weighted majority algorithm and two online convex optimization algorithms.

A difference-of-convex programming approach with parallel branch-and-bound for sentence compression via a hybrid extractive model

Sentence compression is an important problem in natural language processing with wide applications in text summarization, search engine and human-AI interaction system etc. In this paper, we design a hybrid extractive sentence compression model combining a probability language model and a parse tree language model for compressing sentences by guaranteeing the syntax correctness of the compression results. Our compression model is formulated as an integer linear programming problem, which can be rewritten as a Difference-of-Convex (DC) programming problem based on the exact penalty technique. We use a well-known efficient DC algorithm -- DCA to handle the penalized problem for local optimal solutions. Then a hybrid global optimization algorithm combining DCA with a parallel branch-and-bound framework, namely PDCABB, is used for finding global optimal solutions. Numerical results demonstrate that our sentence compression model can provide excellent compression results evaluated by F-score, and indicate that PDCABB is a promising algorithm for solving our sentence compression model.

A proximal DC approach for quadratic assignment problem

In this paper, we show that the quadratic assignment problem (QAP) can be reformulated to an equivalent rank constrained doubly nonnegative (DNN) problem. Under the framework of the difference of convex functions (DC) approach, a semi-proximal DC algorithm is proposed for solving the relaxation of the rank constrained DNN problem whose subproblems can be solved by the semi-proximal augmented Lagrangian method. We show that the generated sequence converges to a stationary point of the corresponding DC problem, which is feasible to the rank constrained DNN problem under some suitable assumptions. Moreover, numerical experiments demonstrate that for most QAP instances, the proposed approach can find the global optimal solutions efficiently, and for others, the proposed algorithm is able to provide good feasible solutions in a reasonable time.

Sparse Phase Retrieval Via PhaseLiftOff

The aim of sparse phase retrieval is to recover a $k$-sparse signal $\mathbf{x}_0\in \mathbb{C}^{d}$ from quadratic measurements $|\langle \mathbf{a}_i,\mathbf{x}_0\rangle|^2$ where $\mathbf{a}_i\in \mathbb{C}^d, i=1,\ldots,m$. Noting $|\langle \mathbf{a}_i,\mathbf{x}_0\rangle|^2={\text{Tr}}(A_iX_0)$ with $A_i=\mathbf{a}_i\mathbf{a}_i^*\in \mathbb{C}^{d\times d}, X_0=\mathbf{x}_0\mathbf{x}_0^*\in \mathbb{C}^{d\times d}$, one can recast sparse phase retrieval as a problem of recovering a rank-one sparse matrix from linear measurements. Yin and Xin introduced PhaseLiftOff which presents a proxy of rank-one condition via the difference of trace and Frobenius norm. By adding sparsity penalty to PhaseLiftOff, in this paper, we present a novel model to recover sparse signals from quadratic measurements. Theoretical analysis shows that the solution to our model provides the stable recovery of $\mathbf{x}_0$ under almost optimal sampling complexity $m=O(k\log(d/k))$. The computation of our model is carried out by the difference of convex function algorithm (DCA). Numerical experiments demonstrate that our algorithm outperforms other state-of-the-art algorithms used for solving sparse phase retrieval.