Convex Algorithm Research Articles

Blind image deblurring with a difference of the mixed anisotropic and mixed isotropic total variation regularization

This paper proposes a simple model for image deblurring with a new total variation regularization. Classically, the L1-21 regularizer represents a difference of anisotropic (i.e. L1) and isotropic (i.e. L21) total variation, so we define a new regularization as Le-2e, which is the weighted difference of the mixed anisotropic (i.e. L0 + L1 = Le) and mixed isotropic (i.e. L0 + L21 = L2e), and it is characterized by sparsity-promotingand robustness in image deblurring. Then, we merge the L0-gradient into the model for edge-preserving and detail-removing. The union of the Le-2e regularization and L0-gradient improves the performance of image deblurring and yields high-quality blur kernel estimates. Finally, we design a new solution format that alternately iterates the difference of convex algorithm, the split Bregman method, and the approach of half-quadratic splitting to optimize the proposed model. Experimental results on quantitative datasets and real-world images show that the proposed method can obtain results comparable to state-of-the-art works.

Impact of Hardware Impairment on the Joint Reconfigurable Intelligent Surface and Robust Transceiver Design in MU-MIMO System

Reconfigurable intelligent surface (RIS) is a revolutionary passive radio technique to facilitate capacity enhancement beyond the current massive multiple-input multiple-output (MIMO) transmission. However, the potential hardware impairment (HWI) of the RIS usually causes inevitable performance degradation and the amplification of imperfect CSI. These impacts still lack full investigation in the RIS-assisted wireless network. This paper developed a robust joint RIS and transceiver design algorithm to minimize the worst-case mean square error (MSE) of the received signal under the HWI effect and imperfect channel state information (CSI) in the RIS-assisted multi-user MIMO (MU-MIMO) wireless network. Specifically, since the proposed robust joint RIS and transceiver design problem yields non-convex characteristics under severe HWI, an iterative three-step convex algorithm is developed to approach the optimality by relaxation and convex transformation. Compared with the state-of-the-art baselines that ignore the HWI, the proposed robust algorithm inhibits the destruction of HWI while raising the worst-case MSE effectively in several numerical simulations. Moreover, due to the properties of the HWI, the performance loss is notable under the magnification of the number of reflected elements in the RIS-assisted MU-MIMO wireless network.

Large-scale robust regression with truncated loss via majorization-minimization algorithm

The utilization of regression methods employing truncated loss functions is widely praised for its robustness in handling outliers and representing the solution in the sparse form of the samples. However, due to the non-convexity of the truncated loss, the commonly used algorithms such as difference of convex algorithm (DCA) fail to maintain sparsity when dealing with non-convex loss functions, and adapting DCA for efficient optimization also incurs additional development costs. To address these challenges, we propose a novel approach called truncated loss regression via majorization-minimization algorithm (TLRM). TLRM employs a surrogate function to approximate the original truncated loss regression and offers several desirable properties: (i) Eliminating outliers before the training process and encapsulating general convex loss regression within its structure as iterative subproblems, (ii) Solving the convex loss problem iteratively thereby facilitating the use of a well-established toolbox for convex optimization. (iii) Converging to a truncated loss regression and providing a solution with sample sparsity. Extensive experiments demonstrate that TLRM achieves superior sparsity without sacrificing robustness, and it can be several tens of thousands of times faster than traditional DCA on large-scale problems. Moreover, TLRM is also applicable to datasets with millions of samples, making it a practical choice for real-world scenarios. The codebase for methods with truncated loss functions is accessible at https://i-do-lab.github.io/optimal-group.org/Resources/Code/TLRM.html.

Stabilized BB projection algorithm for large-scale convex constrained nonlinear monotone equations to signal and image processing problems

The Barzilai–Borwein (BB) method is a popular and efficient gradient method for solving large-scale unconstrained optimization problems. In general, it converges much faster than the steepest descent (Cauchy) method. However, it may not converge, even when the objective function is strongly convex. To overcome this, by virtue of bounding the distance between sequential iterates, [Burdakov et al. (2019)] proposed a stabilized BB method. In this paper, by combining the stabilized BB method with a projection approach, we develop a stabilized BB projection (SBBP) method to solve nonlinear monotone equations with convex constraints. SBBP does not involve computing matrices proving the usefulness for solving large-scale problems. Its global convergence and Q-linear convergence rate are also established. We report numerical experiments on recovering sparse signals and restoring blurred images arising from compressive sensing, demonstrating the usefulness of SBBP.

Plug‐and‐play algorithms for convex non‐convex regularization: Convergence analysis and applications

When the sparse regularizer is convex and its proximal operator has a closed‐form, first‐order iterative algorithms based on proximal operators can effectively solve the sparse optimization problems. Recently, plug‐and‐play (PnP) algorithms have achieved significant success by incorporating advanced denoisers to replace the proximal operators in iterative algorithms. However, convex sparse regularizers such as the ‐norm tend to underestimate the large values within the sparse solutions. In contrast, the convex non‐convex (CNC) sparse regularization enables the non‐convex regularizer while preserving the convexity of the objective function. In this paper, we propose several PnP algorithms for solving the CNC sparse regularization model and discuss their convergence properties. Specifically, we first derive the proximal operator for CNC sparse regularization in iterative form and subsequently integrate it with several first‐order algorithms to yield different PnP algorithms. Then, based on the monotone operator theory, we prove that the proposed PnP algorithms are convergent under the condition that the data‐fidelity term is strongly convex and the residuals of the denoisers are contractive. We also emphasized the significance of strong convexity in the data‐fidelity term for the CNC sparse regularization model. Additionally, we demonstrate the superiority of the proposed PnP algorithms through extensive image restoration tasks.

Super-resolution of positive near-colliding point sources

Abstract In this paper, we analyze the capacity of super-resolution (SR) of one-dimensional positive sources. In particular, we consider a similar setting as in Batenkov et al. (2020, Inf. Inference, 10, 515–572) and restrict the results to the specific case of super-resolving positive sources. To be more specific, we consider resolving $d$ positive point sources with $p \leqslant d$ nodes closely spaced and forming a cluster, while the rest of the nodes are well separated. Our results show that when the noise level $\epsilon \lesssim \mathrm{SRF}^{-2 p+1}$, where $\mathrm{SRF}=(\varOmega \varDelta )^{-1}$ with $\varOmega $ being the cutoff frequency and $\varDelta $ the minimal separation between the nodes, the minimax error rate for reconstructing the cluster nodes is of order $\frac{1}{\varOmega } \mathrm{SRF}^{2 p-2} \epsilon $, while for recovering the corresponding amplitudes $\{a_j \}$, the rate is of order $\mathrm{SRF}^{2 p-1} \epsilon $. For the non-cluster nodes, the corresponding minimax rates for the recovery of nodes and amplitudes are of order $\frac{\epsilon }{\varOmega }$ and $\epsilon $, respectively. Compared with results for complex sources in Batenkov et al. (2020, Inf. Inference, 10, 515–572), our findings reveal that the positivity of point sources actually does not mitigate the ill-posedness of the SR problem. Although surprising, this fact does not contradict positivity’s significant role in the convex algorithms. In fact, our findings are consistent with existing convex algorithms’ stability results for resolving separation-free positive sources, validating their superior SR capabilities. Moreover, our numerical experiments demonstrate that the Matrix Pencil method perfectly meets the minimax rates for resolving positive sources.

Six-Degree-of-Freedom Rocket Landing Optimization via Augmented Convex–Concave Decomposition

In this paper an augmented convex–concave decomposition (ACCD) method for treating nonlinear equality constraints in an otherwise convex problem is proposed. This augmentation improves greatly the feasibility of the problem when compared to the original convex–concave decomposition approach. The effectiveness of the ACCD is demonstrated by solving a fuel-optimal six-degree-of-freedom rocket landing problem in atmosphere, subject to multiple nonlinear equality constraints. Compared with known approaches such as sequential convex programming, where conventional linearization (with or without slack variables) is employed to treat those nonlinear equality constraints, it is shown that the proposed ACCD leads to more robust convergence of the solution process and a more interpretable behavior of the sequential convex algorithm. The methodology can also be applied effectively in the case where the determination of discrete controls or decision-making variables needs to be made, such as the on–off use of reaction control system thrusters in the rocket landing problem, without the need for a mixed integer solver. Numerical results are shown for a representative, reusable rocket benchmark problem.

Joint Task Offloading and Resource Allocation for Space–Air–Ground Collaborative Network

The space–air–ground collaborative network can provide computing service for ground users in remote areas by deploying edge servers on satellites and high-altitude platform (HAP) drones. However, with the growing number of ground devices required to be severed, it becomes imperative to address the issue of spectrum demand for the HAP drone to meet the access of a large number of users. In addition, the long propagation distance between devices and the HAP drone, and between the HAP drone and LEO satellites, will lead to high data transmission energy consumption. Motivated by these factors, we introduce a space–air–ground collaborative network that employs the non-orthogonal multiple access (NOMA) technique, enabling all ground devices to access the HAP drone. Therefore, all devices can share the same communication spectrum. Furthermore, the HAP drone can process part of the ground devices’ tasks locally, and offload the rest to satellites within the visible range for processing. Based on this system, we formulate a weighted energy consumption minimization problem considering power control, computing frequency allocation, and task-offloading decision. The problem is solved by the proposed low-complexity iterative algorithm. Specifically, the original problem is decomposed into interconnected coupled subproblems using the block coordinate descent (BCD) method. The first subproblem is to optimize power control and computing frequency allocation, which is solved by a convex algorithm after a series of transformations. The second subproblem is to make an optimal task-offloading strategy, and we solve it using the concave–convex procedure (CCP)-based algorithm after penalty-based transformation on binary variables. Simulation results verify the convergence and performance of the proposed iterative algorithm compared with the two benchmark algorithms.

Infrared Dim Small Target Detection Based on Nonconvex Constraint with L1–L2 Norm and Total Variation

Infrared dim small target detection has received a lot of attention, because it is a crucial component of the IR search and track systems (IRST). The robust principal component analysis (RPCA) is a common detection framework, which works with poor performance with complex background edges and sparse clutters due to the inappropriate approximation of sparse items. A nonconvex constraint detection method based on the difference between the L1 and L2 (L1–L2) norm and total variation (TV) is presented. The L1–L2 norm is a more accurate sparse item approximation of L0 norm, which can achieve a better description of the sparse item to separate the target from the complex backgrounds. Then, the total variation norm is conducted on the target image to suppress the sparse clutters. The new model is solved using the alternating direction method of multipliers (ADMM) method. Then, the subproblems in the model are tackled by the difference of convex algorithm (DCA) and the Newton conjugate gradient (Newton-CG) solving L1–L2 norm and TV norm, respectively. In the experiment, we conducted experiments on multiple and single target datasets, and the proposed model outperforms the state-of-the-art (SOTA) methods in terms of background suppression and robustness to accurately detect the target. It can achieve a higher true position rate (TPR) with a low false position rate (FPR).

Towards Faster Training Algorithms Exploiting Bandit Sampling From Convex to Strongly Convex Conditions

The training process for deep learning and pattern recognition normally involves the use of convex and strongly convex optimization algorithms such as AdaBelief and SAdam to handle lots of “uninformative” samples that should be ignored, thus incurring extra calculations. To solve this open problem, we propose to design bandit sampling method to make these algorithms focus on “informative” samples during training process. Our contribution is twofold: first, we propose a convex optimization algorithm with bandit sampling, termed AdaBeliefBS, and prove that it converges faster than its original version; second, we prove that bandit sampling works well for strongly convex algorithms, and propose a generalized SAdam, called SAdamBS, that converges faster than SAdam. Finally, we conduct a series of experiments on various benchmark datasets to verify the fast convergence rate of our proposed algorithms.

Distributed projection‐free algorithm for constrained aggregative optimization

AbstractIn this paper, we focus on solving a distributed convex aggregative optimization problem in a network, where each agent has its own cost function which depends not only on its own decision variables but also on the aggregated function of all agents' decision variables. The decision variable is constrained within a feasible set. In order to minimize the sum of the cost functions when each agent only knows its local cost function, we propose a distributed Frank–Wolfe algorithm based on gradient tracking for the aggregative optimization problem where each node maintains two estimates, namely an estimate of the sum of agents' decision variable and an estimate of the gradient of global function. The algorithm is projection‐free, but only involves solving a linear optimization to get a search direction at each step. We show the convergence of the proposed algorithm for convex and smooth objective functions over a time‐varying network. Finally, we demonstrate the convergence and computational efficiency of the proposed algorithm via numerical simulations.

A new difference of anisotropic and isotropic total variation regularization method for image restoration.

Total variation (TV) regularizer has diffusely emerged in image processing. In this paper, we propose a new nonconvex total variation regularization method based on the generalized Fischer-Burmeister function for image restoration. Since our model is nonconvex and nonsmooth, the specific difference of convex algorithms (DCA) are presented, in which the subproblem can be minimized by the alternating direction method of multipliers (ADMM). The algorithms have a low computational complexity in each iteration. Experiment results including image denoising and magnetic resonance imaging demonstrate that the proposed models produce more preferable results compared with state-of-the-art methods.

Cross Layer Resource Allocation in H-CRAN With Spectrum and Energy Cooperation

5G and beyond wireless networks are the upcoming evolution for the current cellular networks to provide the essential requirement of future demands such as high data rate, low energy consumption, and low latency to provide seamless communication for the emerging applications. Heterogeneous cloud radio access network (H-CRAN) is envisioned as a new trend of 5G that uses the advantages of heterogeneous and cloud radio access networks to enhance both spectral and energy efficiency. In this paper, building on the notion of effective capacity (EC), we propose a framework in orthogonal frequency division multiple access (OFDMA)- non-orthogonal multiple access (NOMA) based H-CRAN to meet these demands simultaneously. Our proposed approach is to maximize the effective energy efficiency (EEE) while considering spectrum and power cooperation between a macro base station (MBS) and radio remote heads (RRHs). To solve the formulated problem and to make it more tractable, we transform the original problem into an equivalent subtractive form via Dinkelbach algorithm. Afterward, the combinational framework of distributed stable matching and successive convex algorithm (SCA) is then adopted to obtain the solution of the equivalent problem. Hereby, we propose an efficient resource allocation scheme to maximize energy efficiency while maintaining the delay quality of service (QoS) requirements for all users. The simulation results show that the proposed algorithm can provide a non-trivial trade-off between delay and energy efficiency in OFDMA-NOMA based H-CRAN systems in terms of EC and EEE and the spectrum and power cooperation improves EEE of the proposed network. Moreover, our proposed solution complexity is much lower than the optimal solution and it suffers a very limited gap compared to the optimal method.

MAP inference algorithms without approximation for collective graphical models on path graphs via discrete difference of convex algorithm

Collective graphical model (CGM) is a probabilistic model that provides a framework for analyzing aggregated count data. Maximum a posteriori (MAP) inference of unobserved variables under given observations is one of the essential operations in CGM. Because the MAP inference problem is known to be NP-hard in general, the current mainstream approach is to solve an alternative problem obtained by approximating the objective function and applying continuous relaxation. However, this approach has two significant drawbacks. First, the quality of the solution deteriorates when the values in the count data are negligible due to the inaccuracy of Stirling’s approximation. Second, the application of continuous relaxation causes the violation of integrality constraints. This paper proposes novel algorithms for MAP inference in CGMs on path graphs to overcome these problems. Our method is based on the discrete difference of convex algorithm (DCA); DCA is a general framework to minimize the sum of a convex function and a concave function by repeatedly minimizing surrogate functions. Utilizing the particular structure of path graphs, we efficiently solve the surrogate function minimization by minimum convex cost flow algorithms. Furthermore, our approach also leads to a new method of solving another important task; MAP inference of the sample size in CGM on path graphs. Our method is naturally applicable to this task, allowing us to design very efficient algorithms. Experimental results on synthetic and real-world datasets show the effectiveness of the proposed algorithms.

Adioc loss: An Auxiliary descent IoC loss function

Object detection based on deep learning has progressed significantly hitherto. Intersection over Union (IoU) is a wildly adopted evaluation metric in this field, which is also used as a loss function to constrain training. To address the problem of IoU zero gradient under the non-overlap circumstance, most improved loss functions tend to change penalty terms while few loss functions tried to improve IoU itself. In this paper, we proposed an intersection over convex (IoC) algorithm via analysis of IoU series loss functions. IoC can provide a gradient when a bounding box (also called a predicted box) and a ground truth (also called a target box) share no region. This characteristic will accelerate the training phase. In consideration of the comprehensiveness of the loss function, we constructed Auxiliary descent intersection over convex (Adioc) loss. Adioc loss function was tested on a one-stage network named Yolov5 and a two-stage network named Faster-RCNN. For the one-stage network, the results showed that under the same training batch, the accuracy of the Yolov5s network on VOC datasets increased by 0.1% ∼ 0.4%, and the accuracy of the Yolov5s network on COCO datasets increased by 0.1% ∼ 0.4% The relative improvement of Yolov5 m is 0.6% ∼ 0.7% on COCO dataset. For the two-stage network, the improvement of Faster-RCNN on COCO datasets is about 0.3% ∼ 0.5%.

Joint Resource Allocation for Full-Duplex Ambient Backscatter Communication: A Difference Convex Algorithm

Nowadays, Ambient Backscatter Communication (AmBC) systems have emerged as a green communication technology to enable massive self-sustainable wireless networks by leveraging Radio Frequency (RF) Energy Harvesting (EH) capability. A Full-duplex Ambient Backscatter Communication (FAmBC) network with a Full-duplex Access Point (AP), a dedicated Legacy User (LU), and several Backscatter Devices (BDs) is considered in this study. The AP with two antennas transfers downlink Orthogonal Frequency Division Multiplexing (OFDM) information and energy to the dedicated LU and several BDs, respectively, while receiving uplink backscattered information from BDs at the same time. One of the key aims in AmBC networks is to ensure fairness among BDs. To address this, we propose the Multi-objective Lexicographical Optimization Problem (MLOP), which aims to maximize the minimum BD’s throughput while enhancing overall BDs’ throughput, subject to the AP’s subcarrier power, BDs’ reflection coefficients, and backscatter time allocation. Owe to the MLOP is non-convex, we propose Difference Convex Algorithm (DCA) using Exterior Penalty Function Method (EPFM)—an inventive non-convex optimization method— to reach the optimal solution. The most critical advantage of applying this proposed approach is finding the globally optimal solution. The effectiveness of the proposed method supported by theoretical analysis confirms its superiority compared to some of the investigated suboptimal algorithms with the same computational complexity.

Structured Sparse Coding With the Group Log-regularizer for Key Frame Extraction

Key frame extraction based on sparse coding can reduce the redundancy of continuous frames and concisely express the entire video. However, how to develop a key frame extraction algorithm that can automatically extract a few frames with a low reconstruction error remains a challenge. In this paper, we propose a novel model of structured sparse-coding-based key frame extraction, wherein a nonconvex group log-regularizer is used with strong sparsity and a low reconstruction error. To automatically extract key frames, a decomposition scheme is designed to separate the sparse coefficient matrix by rows. The rows enforced by the nonconvex group log-regularizer become zero or nonzero, leading to the learning of the structured sparse coefficient matrix. To solve the nonconvex problems due to the log-regularizer, the difference of convex algorithm (DCA) is employed to decompose the log-regularizer into the difference of two convex functions related to the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$I_{1}$</tex> norm, which can be directly obtained through the proximal operator. Therefore, an efficient structured sparse coding algorithm with the group log-regularizer for key frame extraction is developed, which can automatically extract a few frames directly from the video to represent the entire video with a low reconstruction error. Experimental results demonstrate that the proposed algorithm can extract more accurate key frames from most SumMe videos compared to the state-of-the-art methods. Furthermore, the proposed algorithm can obtain a higher compression with a nearly 18% increase compared to sparse modeling representation selection (SMRS) and an 8% increase compared to SC-det on the VSUMM dataset.

Robust tensor recovery with nonconvex and nonsmooth regularization

This paper considers the least squares loss minimization problem regularized by tensor average rank and zero norm, to decompose the noisy observation into a low-rank tensor, a sparse tensor and a noise tensor. Due to the nonconvex and nonsmooth of the average rank and the zero norm of tensors, it is not easy to solve the problem directly. We construct the variational characterizations of the tensor average rank and the tensor zero norm, get an equivalent mathematical program with an equilibrium constraint, and then propose an equivalent Lipschitz surrogate for the regularization problem by adding the equality constraint into the objective. Moreover, we design a convex algorithm with multi-stage to verify the efficiency of the proposed method. Numerical experiments are reported for simulation data and actual data to show the performance of the new method.

Difference of convex algorithms for bilevel programs with applications in hyperparameter selection

In this paper, we present difference of convex algorithms for solving bilevel programs in which the upper level objective functions are difference of convex functions, and the lower level programs are fully convex. This nontrivial class of bilevel programs provides a powerful modelling framework for dealing with applications arising from hyperparameter selection in machine learning. Thanks to the full convexity of the lower level program, the value function of the lower level program turns out to be convex and hence the bilevel program can be reformulated as a difference of convex bilevel program. We propose two algorithms for solving the reformulated difference of convex program and show their convergence to stationary points under very mild assumptions. Finally we conduct numerical experiments to a bilevel model of support vector machine classification.

Energy-Efficient Online Data Sensing and Processing in Wireless Powered Edge Computing Systems

Wireless powered multi-access edge computing (MEC) has emerged as a promising paradigm to enable high-performance computation of energy-constrained wireless devices (WDs) in internet of things (IoT) systems. However, to overcome the severe path loss of both energy transfer and data communications, wireless powered MEC suffers from high operating power consumption. To achieve sustainable and economic system operation, this paper focuses on developing energy-efficient online data processing strategy for wireless powered MEC systems under stochastic fading channels. In particular, we consider a hybrid access point (HAP) transmitting RF energy to and processing the sensing data offloaded from multiple WDs. Under an average power constraint of the HAP, we target at maximizing the long-term average data sensing rate of the WDs while maintaining task data queue stability. To this end, we formulate a multi-stage stochastic optimization problem to control the energy transfer and task data processing in sequential time slots. Without the knowledge of future channel fading, it is very challenging to determine the sequential control actions that are tightly coupled by the battery and data buffer dynamics. To solve the problem, we propose a Lyapunov optimization-based online algorithm named LEESE, which decomposes the multi-stage stochastic problem into per-slot deterministic optimization problems. We show that each per-slot problem can be equivalently transformed into a convex optimization problem. To facilitate online implementation in large-scale MEC systems, instead of solving the per-slot problem with off-the-shelf convex algorithms, we propose a block coordinate descent (BCD)-based method that produces a close-to-optimal solution in less than 0.04% of the computation delay. Simulation results demonstrate that the proposed LEESE algorithm can provide 18% higher data sensing rate than the representative benchmark methods considered, while incurring sub-millisecond computation delay suitable for real-time control under fading channel.