Biconvex Problem Research Articles

Bicriteria Sparse Nonnegative Matrix Factorization via Two-Timescale Duplex Neurodynamic Optimization.

In this article, sparse nonnegative matrix factorization (SNMF) is formulated as a mixed-integer bicriteria optimization problem for minimizing matrix factorization errors and maximizing factorized matrix sparsity based on an exact binary representation of l0 matrix norm. The binary constraints of the problem are then equivalently replaced with bilinear constraints to convert the problem to a biconvex problem. The reformulated biconvex problem is finally solved by using a two-timescale duplex neurodynamic approach consisting of two recurrent neural networks (RNNs) operating collaboratively at two timescales. A Gaussian score (GS) is defined as to integrate the bicriteria of factorization errors and sparsity of resulting matrices. The performance of the proposed neurodynamic approach is substantiated in terms of low factorization errors, high sparsity, and high GS on four benchmark datasets.

A Neurodynamic Approach for SWIPT Power Splitting Optimization

Simultaneous wireless information and power transfer (SWIPT) systems using energy from RF signals can effectively solve the energy shortage of wireless devices. However, the existing SWIPT optimization methods using numerical algorithms are difficult to solve the non-convex problem and to adapt to the dynamic communication circumstances. In this paper, a duplex neurodynamic optimization method is used to address the SWIPT system’s power partitioning issue. The information rate maximization problem of the SWIPT system is framed as a biconvex problem. A duplex recurrent neural network is used to concurrently execute local search and update the initial state of the neural network by a particle swarm optimization method to get the global optimum. The experimental results demonstrate that the duplex neurodynamic-based SWIPT system maximizes information rate while satisfying the minimal harvesting energy requirement in a variety of channel states.

An Online Inference-Aided Incentive Framework for Information Elicitation Without Verification

We study the design of incentive mechanisms for the problem of information elicitation without verification (IEWV). In IEWV, a data requester seeks to design proper incentives to optimize the tradeoff between the quality of information (collected from distributed crowd workers) and the total cost of incentives (provided to crowd workers) without verifiable ground truth. While prior work often relies on sufficient knowledge of worker information, we study a scenario where the data requester cannot access workers’ heterogeneous information quality and costs ex-ante. We propose a continuum-armed bandit-based incentive mechanism that dynamically learns the optimal reward level from workers’ reported information. A key challenge is that the data requester cannot evaluate the workers’ information quality without verification, which motivates the design of an inference algorithm. The inference problem is non-convex, yet we reformulate it as a bi-convex problem and derive an approximate solution with a performance guarantee, which ensures the effectiveness of our online reward design. We further enhance the inference algorithm using part of the workers’ historical reports. We also propose a novel rule for the data requester to aggregate workers’ solutions more effectively. We show that our mechanism achieves a sub-linear regret <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\tilde {O}(T^{1/2})$ </tex-math></inline-formula> and outperforms several celebrated benchmarks.

Beampattern synthesis with nulling for sub-arrayed MIMO radar under constant modulus constraints

Sub-arrayed multiple-input multiple-output (MIMO) radar holds the benefits of waveform diversity and coherent processing gain jointly. Based on prior knowledge, we exploit the sub-arrayed MIMO radar to form multiple beams in the desired directions and null the power toward the interference or clutter. Meanwhile, to maximize the radiation efficiency of nonlinear amplifiers in radar systems, the transmit weight vectors are required to have constant modulus properties, which leads to a complicated nonconvex problem. To that end, an iteration method with high calculation efficiency based on the alternating direction penalty method (ADPM) framework is presented to resolve this problem. The original problem is transformed into a general bi-convex problem with bi-affine constraints via introducing auxiliary variables. A high convergence rate can be achieved by selectively increasing the penalty parameters in each iteration. For any initialization, the ADPM-based method is proved to converge. And the obtained convergent solution fulfills the Karush-Kuhn-Tucker (KKT) conditions. Finally, the effectiveness of the proposed method is demonstrated by simulation results.

ADJUST: a dictionary-based joint reconstruction and unmixing method for spectral tomography

Advances in multi-spectral detectors are causing a paradigm shift in x-ray computed tomography (CT). Spectral information acquired from these detectors can be used to extract volumetric material composition maps of the object of interest. If the materials and their spectral responses are known a priori, the image reconstruction step is rather straightforward. If they are not known, however, the maps as well as the responses need to be estimated jointly. A conventional workflow in spectral CT involves performing volume reconstruction followed by material decomposition, or vice versa. However, these methods inherently suffer from the ill-posedness of the joint reconstruction problem. To resolve this issue, we propose ‘A Dictionary-based Joint reconstruction and Unmixing method for Spectral Tomography’ (ADJUST). Our formulation relies on forming a dictionary of spectral signatures of materials common in CT and prior knowledge of the number of materials present in an object. In particular, we decompose the spectral volume linearly in terms of spatial material maps, a spectral dictionary, and the indicator of materials for the dictionary elements. We propose a memory-efficient accelerated alternating proximal gradient method to find an approximate solution to the resulting bi-convex problem. From numerical demonstrations on several synthetic phantoms, we observe that ADJUST performs exceedingly well compared to other state-of-the-art methods. Additionally, we address the robustness of ADJUST against limited and noisy measurement patterns. The demonstration of the proposed approach on a spectral micro-CT dataset shows its potential for real-world applications. Code is available at https://github.com/mzeegers/ADJUST.

Optimizing Write Fidelity of MRAMs by Alternating Water-Filling Algorithm

Magnetic random-access memory (MRAM) is a promising memory technology due to its high density, non-volatility, and high endurance. However, achieving high memory fidelity incurs high write-energy costs, which should be reduced for large-scale deployment of MRAMs. In this paper, we formulate a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">biconvex</i> optimization problem to optimize write fidelity given energy and latency constraints. The basic idea is to allocate non-uniform write pulses depending on the importance of each bit position. The fidelity measure we consider is mean squared error (MSE), for which we optimize write pulses via alternating convex search (ACS). We derive analytic solutions and propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">alternating water-filling</i> algorithm by casting the MRAM’s write operation as communication over parallel channels. Hence, the proposed alternating water-filling algorithm is computationally more efficient than the original ACS while their solutions are identical. Since the formulated biconvex problem is non-convex, both the original ACS and the proposed algorithm do not guarantee global optimality. However, the MSEs obtained by the proposed algorithm are comparable to the MSEs by complicated global nonlinear programming solvers. Furthermore, we prove that our algorithm can reduce the MSE exponentially with the number of bits per word. For an 8-bit accessed word, the proposed algorithm reduces the MSE by a factor of 21. We also evaluate MNIST dataset classification supposing that the model parameters of deep neural networks are stored in MRAMs. The numerical results show that the optimized write pulses can achieve 40% write-energy reduction for the same classification accuracy.

UAV-Assisted Physical Layer Security in Multi-Beam Satellite-Enabled Vehicle Communications

In this paper, we investigate unmanned aerial vehicle (UAV) assisted physical layer security in multi-beam satellite enabled vehicle communications. Particularly, the UAV is exploited as a relay to improve the secure satellite-to-vehicle link, and simultaneously serves as a jammer by deliberately generating artificial noise (AN) to confuse Eve. The satellite beamforming (BF) and UAV power allocation (PA) are jointly optimized to maximize the secrecy rate of the legitimate user within a target beam while guaranteeing the quality of service (QoS) of users within other beams. Since the problem is nonconvex, we first convert it into an equivalent two-stage problem. Then, the outer-stage problem is solved by using one-dimensional search, and the inner-stage problem is transformed to a bi-convex problem by using the semi-definite relaxation (SDR) and Charnes Cooper transformation. To solve the inner-stage bi-convex problem, we propose an iterative alternating optimization algorithm, where the optimal BF is obtained by semi-definite programming (SDP), and the optimal UAV PA is subsequently obtained by solving the reformulated fractional programming problem with an iterative Dinkelbach method. The tightness of SDR and the complexity of our proposed approach are analyzed, and extensive simulations are carried out to evaluate the effectiveness of our proposed approach.

Constrained principal component analysis with stochastically ordered scores for high-dimensional mass spectrometry data

In this paper, we consider a constrained principal component analysis (PCA) for the projection of high-dimensional samples from different groups to a lower-dimensional space for which the principal scores are stochastically ordered over the groups. We express the problem as the minimization of a constrained biconvex problem and develop an iterative algorithm to solve it. We numerically show that the solution to our constrained PCA problem approximately rotates the principal coordinates of the ordinary PCA to achieve ordered scores. Consequently, our approach significantly improves the scores and the corresponding loading matrix compared to the original PCA if their true values are ordered over groups. We finally apply our method to two data examples: (i) the direct infusion multiple reaction monitoring mass spectrometry (DI-MRM-MS) data of white rice to verify the authenticity of adulterated Japonica rice and (ii) high-dimensional lipidomics data from ultra-performance liquid chromatography-mass spectrometry/mass spectrometry (UPLC-MS/MS) analysis of patients with liver diseases.

Service-Oriented Energy-Latency Tradeoff for IoT Task Partial Offloading in MEC-Enhanced Multi-RAT Networks

The development of the 5G network is envisioned to offer various types of services like virtual reality/augmented reality and autonomous vehicles applications with low-latency requirements in Internet-of-Things (IoT) networks. Mobile-edge computing (MEC) has become a promising solution for enhancing the computation capacity of mobile devices at the edge of the network in a 5G wireless network. Additionally, multiple radio access technologies (multi-RATs) have been verified with the potential in lowering the transmission latency and energy consumption, while improving the Quality of Services (QoS). Benefiting from the cooperation of multi-RATs, large latency-sensitive computing service tasks (L2SC) can be offloaded by different RATs simultaneously, which has great practical significance for data partitioned oriented applications with large task sizes. In this article, to enhance the L2SC offloading services for satisfying low-latency requirements with low energy consumption, we investigate the energy-latency tradeoff problem for partial task offloading in the MEC-enhanced multi-RAT network, considering the limitation of energy and computing in capability-constrained end devices in IoT networks. Specifically, we formulated the L2SC task computation offloading problem to minimize the weighted sum of the latency cost and the energy consumption by jointly optimizing the local computing frequency, task splitting, and transmit power, while guaranteeing the stringent latency requirement and the residual energy constraint. Due to the nonsmoothness and nonconvexity of the formulated problem with high complexity, we convert the tradeoff problem into a smooth biconvex problem and propose an alternate convex search-based algorithm, which can greatly reduce the computational complexity. Numerical simulation results show the effectiveness of the proposed algorithm with various performance parameters.

Joint Spatial-Spectral Smoothing in a Minimum-Volume Simplex for Hyperspectral Image Super-Resolution

The limitations of hyperspectral sensors usually lead to coarse spatial resolution of acquired images. A well-known fusion method called coupled non-negative matrix factorization (CNMF) often amounts to an ill-posed inverse problem with poor anti-noise performance. Moreover, from the perspective of matrix decomposition, the matrixing of remotely-sensed cubic data results in the loss of data’s structural information, which causes the performance degradation of reconstructed images. In addition to three-dimensional tensor-based fusion methods, Craig’s minimum-volume belief in hyperspectral unmixing can also be utilized to restore the data structure information for hyperspectral image super-resolution. To address the above difficulties simultaneously, this article incorporates the regularization of joint spatial-spectral smoothing in a minimum-volume simplex, and spatial sparsity—into the original CNMF, to redefine a bi-convex problem. After the convexification of the regularizers, the alternating optimization is utilized to decouple the regularized problem into two convex subproblems, which are then reformulated by separately vectorizing the variables via vector-matrix operators. The alternating direction method of multipliers is employed to split the variables and yield the closed-form solutions. In addition, in order to solve the bottleneck of high computational burden, especially when the size of the problem is large, complexity reduction is conducted to simplify the solutions with constructed matrices and tensor operators. Experimental results illustrate that the proposed algorithm outperforms state-of-the-art fusion methods, which verifies the validity of the new fusion approach in this article.

C-RAN With Hybrid RF/FSO Fronthaul Links: Joint Optimization of Fronthaul Compression and RF Time Allocation

This paper considers the uplink of a cloud radio access network (C-RAN) comprised of several multi-antenna remote radio units (RUs) which compress the signals that they receive from multiple mobile users (MUs) and forward them to a CU via wireless fronthaul links. To enable reliable high rate fronthaul links, we employ a hybrid radio frequency (RF)/free space optical (FSO) system for fronthauling. Moreover, to strike a balance between complexity and performance, we consider three different quantization schemes at the RUs, namely per-antenna vector quantization (AVQ), per-RU vector quantization (RVQ), and distributed source coding (DSC), two different RF fronthaul transmission modes, namely orthogonal transmission and non-orthogonal transmission, and two different detectors at the CU, namely the linear minimum mean square error detector and the optimal successive interference cancellation detector. For this network architecture, we investigate the joint optimization of the quantization noise covariance matrices at the RUs and the RF time allocation to the multiple-access and fronthaul links for rate region maximization. To this end, we formulate a unified weighted sum rate maximization problem valid for each possible combination of the considered quantization, RF fronthaul transmission, and detection schemes. To handle the non-convexity of the unified problem, we transform it into a bi-convex problem which facilitates the derivation of an efficient suboptimal solution using alternating convex optimization and golden section search. Moreover, by introducing a backoff parameter to reduce the probability of infeasibility, we generalize the proposed optimization framework to account for imperfect channel estimation. Our simulation results show that for each combination of the considered quantization, RF fronthaul transmission, and detection schemes, C-RAN with hybrid RF/FSO fronthauling can achieve a considerable sum rate gain compared to conventional systems employing pure FSO fronthauling, especially under unfavorable atmospheric conditions. In addition, employing a more sophisticated quantization scheme can significantly improve the system performance under adverse atmospheric conditions. In contrast, in clear weather conditions, when the FSO link capacity is high, the simple AVQ scheme performs close to the optimal DSC scheme. Furthermore, our simulation results suggest that the proposed algorithm can be adapted to the quality of the channel estimates by tuning the backoff parameter.

Distributed Green Offloading and Power Optimization in Virtualized Small Cell Networks With Mobile Edge Computing

Virtualized small cell networks (SCNs) integrated with mobile edge computing (MEC) is a promising paradigm to provide both wideband access and intensive computation economically for user equipments (UEs) in the scenario of multiple mobile virtual network operators and infrastructure providers. However, the model of offloading in this case is often of high complexity and lacks effective solution. In this paper, by jointly considering offloading, time slice and power allocation, we formulate the energy consumption reduction of the UEs in virtualized SCNs with MEC as a mixed integer nonlinear programming. Our aim is to minimize the total energy consumption of the UEs subject to minimum overall throughput of the network. To solve the problem efficiently, we convert it into a biconvex problem by adding auxiliary variable, which enables the derivation of an efficient iterative algorithm by two subproblems. Towards the first subproblem, we introduce local variables to handle the coupling constraint, and propose an alternating direction method of multipliers (ADMM)-based distributed algorithm, where the closed-form expressions of the optimal solutions in variables updating are derived. For the second subproblem, the closed-form expressions of the optimal solution is also derived. Finally, the effectiveness of the proposed algorithm is demonstrated by extensive simulations with different system configurations.

A general coordinated voltage regulation method in distribution networks with soft open points

Due to the high flexibility and fast response capability, power electronic devices are versatile, and increasingly being considered to mitigate voltage violations and improve system efficiency in distribution networks with high penetration of distributed generation (DG). This paper proposes a novel voltage regulation framework in distribution systems through the general types of Soft Open Points (SOPs) and reactive powers of DG inverters. By using the linearized DistFlow equations, we establish a biconvex problem that minimizes voltage deviation via a well-coordinated scheme for SOPs and DGs. The coordination of different types of power electronic devices can be simultaneously modelled into this problem which can be efficiently solved by alternative convex search algorithm. The proposed approach is validated by numerical simulations on modified IEEE 15-bus system, IEEE 33-bus system, and IEEE 69-bus system with different types of power electronic compensators.

Blind Hyperspectral Unmixing Considering the Adjacency Effect

This paper focuses on the blind unmixing technique for analyzing hyperspectral images (HSIs). A joint deconvolution and blind hyperspectral unmixing (DBHU) algorithm is proposed, which is aimed at eliminating the impact of the adjacency effect (AE) on unmixing. In remote sensing imagery, the AE occurs in the presence of atmospheric scattering over a heterogeneous surface. The AE leads to blurring and additional mixing of HSIs and makes it difficult to estimate endmembers and abundances accurately. In this paper, we first model the blurred HSIs by the use of a bilinear mixing model (BMM), where a blurring kernel is used to model the mixing caused by the AE. Based on the BMM, the DBHU problem is formulated as a constrained and biconvex optimization problem. Specifically, the minimum-volume simplex (MVS) is incorporated to deal with the additional mixing caused by the AE, and 3-D total variation (TV) priors are adopted to model the spectral–spatial correlation of the data. In DBHU, the biconvex problem is efficiently solved by a nonstandard application of the alternating direction method of multipliers (ADMM) algorithm, where a block coordinate descent scheme is applied by splitting the original problem into two saddle point subproblems, and then minimizing the subproblems alternately via the ADMM until convergence. The experimental results obtained with both simulated and real data confirm the viability of the proposed algorithm, and DBHU works well, even where both blurring and noise are present in the scene.

Low-Rank Matrix Learning Using Biconvex Surrogate Minimization.

Many machine learning problems involve learning a low-rank positive semidefinite matrix. However, existing solvers for this low-rank semidefinite program (SDP) are often expensive. In this paper, by factorizing the target matrix as a product of two matrices and using a Courant penalty to penalize for their difference, we reformulate the SDP as a biconvex optimization problem. This allows the use of multiconvex optimization techniques to define simple surrogates, which can be minimized easily by block coordinate descent. Moreover, while traditionally this biconvex problem approaches the original problem only when the penalty parameter is infinite, we show that the two problems are equivalent when the penalty parameter is sufficiently large. Experiments on a number of SDP applications in machine learning show that the proposed algorithm is as accurate as other state-of-the-art algorithms, but is much faster, especially on large data sets.

A new optimization in SDP-based learning

Many machine learning problems can be reduced to learning a low-rank positive semidefinite matrix (denoted as Z), which encounters semidefinite program (SDP). Existing SDP solvers are often expensive for large-scale learning. In order to avoid directly solving SDP, some works convert SDP into a nonconvex program by factorizing Z as XX⊤. However, this would bring higher-order nonlinearity, resulting in scarcity of structure in subsequent optimization. In this paper, we propose a novel surrogate for SDP-based learning, in which the structure of subproblem is exploited. More specifically, we surrogate unconstrained SDP by a biconvex problem, through factorizing Z as XY⊤ and using a quadratic penalty to penalize the difference of X and Y, in which the resultant subproblems are convex. Furthermore, we provide a theory bound for the associated penalty parameter under the assumption that the objective function is Lipschitz-smooth, such that the proposed surrogate will solve the original SDP when the penalty parameter is larger than this bound. Experiments on three SDP-based machine learning problems demonstrate that the proposed algorithm is as accurate as the state-of-the-art, but is faster on large-scale learning.

Worst case fair beamforming for multiple multicast groups in multicell networks

The design problem associated with robust downlink beamforming in multicast, multigroup, multicell wireless systems is addressed. The channel state information (CSI) of users is assumed to be imperfect and the uncertainty of CSI is modelled using the Frobenius norm. The objective is to optimise the signal-to-interference-plus-noise ratio over all users with a constraint on the maximum total transmitted power. This was achieved through a robust solution using the successive convex approximation (SCA) method. The beamforming problem is treated as a bi-convex problem, which is solved using the iterate-alternative convex technique. Here, the CSI uncertainty is addressed using a convex package through the non-monotone spectral projected gradient method and the beamforming vector is extracted using the SCA method. Also, the authors offer the required condition to extract the beamform vector using the SCA method through a suboptimal solution that always addressed before using different beamforming methods. Their simulation results examine all proposed system parameters in order to show convergence and feasibility of the solution. They also compare the solution with a suboptimal solution and the quality of service method for imperfect CSI in downlink beamforming. Numerical results show that the robust solution achieves the best power efficiency for practical solutions.

Slicing the Edge: Resource Allocation for RAN Network Slicing

Network slicing is considered to be a crucial feature of fifth generation cellular systems. By dividing the network infrastructure into multiple logical segments, network slicing can support parallel services with different requirements. While technology developments focus on slicing the core networks, there are limited studies in network slicing for radio access networks (RAN). Hence, in this letter, we study the RAN slicing and slice coordination, which is formulated as a bi-convex problem. Even though there are complicated couplings between the RAN resource allocation for each slice and the coordination of the slices that share the same resources, we design two algorithms addressing these couplings of this bi-convex problem. Simulation results validate the efficacy of our proposed algorithms.

A Biconvex Analysis for Lasso &lt;inline-formula&gt; &lt;tex-math notation="LaTeX"&gt;$\ell _1$&lt;/tex-math&gt; &lt;/inline-formula&gt; Reweighting

Iterative $\ell _1$ reweighting algorithms are very popular in sparse signal recovery and compressed sensing, since in the practice they have been observed to outperform classical $\ell _1$ methods. Nevertheless, the theoretical analysis of their convergence is a critical point, and generally is limited to the convergence of the functional to a local minimum or to subsequence convergence. In this letter, we propose a new convergence analysis of a Lasso $\ell _1$ reweighting method, based on the observation that the algorithm is an alternated convex search for a biconvex problem. Based on that, we are able to prove the numerical convergence of the sequence of the iterates generated by the algorithm. Furthermore, we propose an alternative iterative soft thresholding procedure, which is faster than the main algorithm.

HCLR: A hybrid clustering and low-rank regularization-based method for photon-limited image restoration

A photon-limited image can be represented as a pixel matrix limited by the relatively small number of collected photons. The image can also be seen as being contaminated by Poisson noise because the total number of photons follows the Poisson distribution. Through exploitation of the inherent properties of observation combined with application of a denoising method, an image can be significantly restored. In this paper, a hybrid clustering and low-rank regularization-based model (HCLR) is proposed based on the essential features of patch clustering and noise. An efficient Newton-type method is designed to optimize this biconvex problem. Experimental results demonstrate that HCLR achieves competitive denoising performance, especially for high noise levels, compared with state-of-the-art Poisson denoising algorithms.