Alternate Convex Search Research Articles

Inverse optimization in semi-definite programs to impute unknown constraint matrices

In a typical (forward) optimization problem, a decision-maker uses given values of model parameters to compute the values of decision variables. The goal in inverse optimization (IO) is instead to infer parameters that render given values of decision variables optimal. Most papers on IO utilize duality to impute objective function parameters. A corresponding literature for imputing constraint parameters is essentially non-existent, even for linear programs. The difficulty is that these IO problems include nonconvex bilinear constraints and/or objectives. We study inverse semi-definite programs (SDPs) where matrices on the left-hand-sides of constraints are unknown. These unknown matrices must satisfy some side constraints. We consider two types of such problems, depending on which SDP decision variable values are given: primal or dual. In each situation, we study three sub-cases based on whether both the right-hand-side coefficients and the objective function matrix are known or only one of these two is known. This creates six variants of the inverse problem. For each variant, we first look for sufficient conditions, either on the side constraints or on problem structure, under which the nonconvex bilinear inverse problem can be simplified substantially. In particular, it either has a trivial solution or can be reformulated as a (set of) convex program(s). When such conditions are not evident, we propose three tailored heuristics called the Convex–Concave Procedure, Sequential Semi-definite Programming, and Alternate Convex Search. We perform computational experiments to gain insights into the performance of these three methods.

SCMA-Enabled Multi-Cell Edge Computing Networks: Design and Optimization

Multi-access edge computing (MEC) is regarded as a promising approach for providing resource-constrained mobile devices with computing resources through task offloading. Sparse code multiple access (SCMA) is a code-domain non-orthogonal multiple access (NOMA) scheme that can meet the demands of multi-cell MEC networks for high data transmission rates and massive connections. In this paper, we propose an optimization framework for SCMA-enabled multi-cell MEC networks. The joint resource allocation and computation offloading problem is formulated to minimize the system cost, which is defined as the weighted energy cost and latency. Due to the nonconvexity of the proposed optimization problem induced by the coupled optimization variables, we first propose an algorithm based on the block coordinate descent (BCD) method to iteratively optimize the transmit power and edge computing resources allocation by deriving closed-form solutions, and further develop an improved low-complexity simulated annealing (SA) algorithm to solve the computation offloading and multi-cell SCMA codebook allocation problem. To solve the problem of partial state observation and timely decision-making in long-term optimization environment, we put forward a multiagent deep deterministic policy gradient (MADDPG) algorithm with centralized training and distributed execution. Furthermore, we extend the framework to the partial offloading case and propose an algorithm based on alternating convex search for solving the task offloading ratio. Numerical results show that the proposed multi-cell SCMA-MEC scheme achieves lower energy consumption and system latency in comparison to the orthogonal frequency division multiple access (OFDMA) and power-domain (PD) NOMA techniques.

Task offloading based on edge collaboration in MEC-enabled IoV networks

Benefiting from its abundant computing resources and low computing latency, mobile edge computing (MEC) is a promising approach for enhancing the computing capacity of the 5G Internet of vehicles (IoV). Because of the high mobility, handover is frequent and inevitable in IoV networks. In this paper, we investigate an edge collaborative task offloading and splitting strategy in MEC-enabled IoV networks, in which the task is splitted on the edge and paralleling executed by each part of the task on several MEC servers when handover is occured. Applications in IoV networks have flexible requirements on latency and energy consumption. To realize the tradeoff between latency and energy consumption, we formulate the task offloading and splitting as an optimization problem with the aim of minimizing the total cost of latency and energy consumption by jointly optimizing the task splitting ratio and uplink transmit power of vehicle terminal (VT). Because the proposed problem is non-smooth and non-convex, we divide the original problem into two convex subproblems, and apply an alternate convex search (ACS) algorithm to obtain the optimized solution with low computational complexity. Numerical simulation results show that the proposed method can adjust the offloading strategy properly according to task preference, and obtain a lower total cost compared with the baseline algorithms.

Distilling sub-space structure across views for cardiac indices estimation.

Cardiac indices estimation in multi-view images attracts great attention due to its capability for cardiac function assessment. However, the variation of the cardiac indices across views causes that most cardiac indices estimation methods can only be trained separately in each view, resulting in low data utilization. To solve this problem, we have proposed distilling the sub-space structure across views to explore the multi-view data fully for cardiac indices estimation. In particular, the sub-space structure is obtained via building a n×n covariance matrix to describe the correlation between the output dimensions of all views. Then, an alternate convex search algorithm is proposed to optimize the cross-view learning framework by which: (i) we train the model with regularization of sub-space structure in each view; (ii) we update the sub-space structure based on the learned parameters from all views. In the end, we have conducted a series of experiments to verify the effectiveness of our proposed framework. The model is trained on three views (short axis, 2-chamber view and 4-chamber view) with two modalities (magnetic resonance imaging and computed tomography). Compared to the state-of-the-art methods, our method has demonstrated superior performance on cardiac indices estimation tasks.

Optimizing Write Fidelity of MRAMs by Alternating Water-Filling Algorithm

Magnetic random-access memory (MRAM) is a promising memory technology due to its high density, non-volatility, and high endurance. However, achieving high memory fidelity incurs high write-energy costs, which should be reduced for large-scale deployment of MRAMs. In this paper, we formulate a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">biconvex</i> optimization problem to optimize write fidelity given energy and latency constraints. The basic idea is to allocate non-uniform write pulses depending on the importance of each bit position. The fidelity measure we consider is mean squared error (MSE), for which we optimize write pulses via alternating convex search (ACS). We derive analytic solutions and propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">alternating water-filling</i> algorithm by casting the MRAM’s write operation as communication over parallel channels. Hence, the proposed alternating water-filling algorithm is computationally more efficient than the original ACS while their solutions are identical. Since the formulated biconvex problem is non-convex, both the original ACS and the proposed algorithm do not guarantee global optimality. However, the MSEs obtained by the proposed algorithm are comparable to the MSEs by complicated global nonlinear programming solvers. Furthermore, we prove that our algorithm can reduce the MSE exponentially with the number of bits per word. For an 8-bit accessed word, the proposed algorithm reduces the MSE by a factor of 21. We also evaluate MNIST dataset classification supposing that the model parameters of deep neural networks are stored in MRAMs. The numerical results show that the optimized write pulses can achieve 40% write-energy reduction for the same classification accuracy.

Learning adaptive weighted response consistency correlation filters for real-time UAV tracking

Due to the high efficiency of discriminative correlation filter (DCF), it has attracted widespread attention in the field of UAV object tracking. To handle the problem of filter degradation, many trackers usually introduce temporal regularization to enhance the discriminative power of the filter. However, these temporal regularization methods only utilize the limited information between two consecutive frames, which are susceptible interference by previous corrupted information. Besides, regularization terms with predefined hyperparameters can not well adapt to the variations of the target across sequent frames, which may cause the model degradation or drift. We propose a tracker based on DCF framework to fully exploit the information hidden in the historical response map, namely adaptive weighted response consistency-based DCF tracking. Specifically, carefully selected historical response maps with fixed weight distribution are introduced in training phase to increase the robustness of the filter. Further, we present a unified loss for jointly learning the filter and the weight distribution, which can be solved by the alternate convex search method. The joint loss guarantees that reliable response maps contribute more to filter learning, leading to a more discriminative and adaptive filter for tracking the target. Extensive experiments show that the proposed method achieves state-of-the-art results on two datasets.

Loss Decomposition and Centroid Estimation for Positive and Unlabeled Learning.

This paper studies Positive and Unlabeled learning (PU learning), of which the target is to build a binary classifier where only positive data and unlabeled data are available for classifier training. To deal with the absence of negative training data, we first regard all unlabeled data as negative examples with false negative labels, and then convert PU learning into the risk minimization problem in the presence of such one-side label noise. Specifically, we propose a novel PU learning algorithm dubbed "Loss Decomposition and Centroid Estimation" (LDCE). By decomposing the loss function of corrupted negative examples into two parts, we show that only the second part is affected by the noisy labels. Thereby, we may estimate the centroid of corrupted negative set via an unbiased way to reduce the adverse impact of such label noise. Furthermore, we propose the "Kernelized LDCE" (KLDCE) by introducing the kernel trick, and show that KLDCE can be easily solved by combining Alternative Convex Search (ACS) and Sequential Minimal Optimization (SMO). Theoretically, we derive the generalization error bound which suggests that the generalization risk of our model converges to the empirical risk with the order of O(1/√k+1/√{n-k}+1/√n) ( n and k are the amounts of training data and positive data correspondingly). Experimentally, we conduct intensive experiments on synthetic dataset, UCI benchmark datasets and real-world datasets, and the results demonstrate that our approaches (LDCE and KLDCE) achieve the top-level performance when compared with both classic and state-of-the-art PU learning methods.

Simultaneous wireless information and power transfer optimization via alternating convex-concave procedure with imperfect channel state information

Simultaneous Wireless Information and Power Transfer (SWIPT) systems are viable models to implement wireless charging in Broadcast Channel (BC) systems. With the emergence of Internet of Things (IOT) and the 5th generation of telecommunication systems (5G), to make lifelong powered devices, wireless charging gets wider attention in scientific and industrial community. To optimally design such a system, we optimize a multi-user multi-antenna BC by simultaneously maximizing the sum of harvested energy and minimizing the sum Mean Square Error (MSE) for symbol detection at each receiver node. To have a tractable design problem, we recast this multi-objective problem as a Difference of Convex (DC) bilinear problem. The derived design problem is solved using the Alternating Convex Search (ACS) method and the Penalty Convex-Concave Program (PCCP) procedure. We solve this problem under two conditions: the perfect and the imperfect Channel State Information (CSI). Simulation results prove that the proposed algorithm outperforms reported recent research studies in this field.

Probabilistic Structure Learning for EEG/MEG Source Imaging With Hierarchical Graph Priors.

Brain source imaging is an important method for noninvasively characterizing brain activity using Electroencephalogram (EEG) or Magnetoencephalography (MEG) recordings. Traditional EEG/MEG Source Imaging (ESI) methods usually assume the source activities at different time points are unrelated, and do not utilize the temporal structure in the source activation, making the ESI analysis sensitive to noise. Some methods may encourage very similar activation patterns across the entire time course and may be incapable of accounting the variation along the time course. To effectively deal with noise while maintaining flexibility and continuity among brain activation patterns, we propose a novel probabilistic ESI model based on a hierarchical graph prior. Under our method, a spanning tree constraint ensures that activity patterns have spatiotemporal continuity. An efficient algorithm based on an alternating convex search is presented to solve the resulting problem of the proposed model with guaranteed convergence. Comprehensive numerical studies using synthetic data on a realistic brain model are conducted under different levels of signal-to-noise ratio (SNR) from both sensor and source spaces. We also examine the EEG/MEG datasets in two real applications, in which our ESI reconstructions are neurologically plausible. All the results demonstrate significant improvements of the proposed method over benchmark methods in terms of source localization performance, especially at high noise levels.

Generalized co-sparse factor regression

Multivariate regression techniques are commonly applied to explore the associations between large numbers of outcomes and predictors. In real-world applications, the outcomes are often of mixed types, including continuous measurements, binary indicators, and counts, and the observations may also be incomplete. Building upon the recent advances in mixed-outcome modeling and sparse matrix factorization, generalized co-sparse factor regression (GOFAR) is proposed, which utilizes the flexible vector generalized linear model framework and encodes the outcome dependency through a sparse singular value decomposition (SSVD) of the integrated natural parameter matrix. To avoid the estimation of the notoriously difficult joint SSVD, GOFAR proposes both sequential and parallel unit-rank estimation procedures. By combining the ideas of alternating convex search and majorization–minimization, an efficient algorithm is developed to solve the sparse unit-rank problem and implemented in the R package gofar. Extensive simulation studies and two real-world applications demonstrate the effectiveness of the proposed approach.

An Efficient Calibration of MIMO Channel Sounders With Internal Crosstalk

In this paper, a calibration of multiple-input multiple-output (MIMO) channel sounders with internal crosstalk is considered.The objective is to minimize the number of back-to-back (B2B) connections required to estimate the transmitter (Tx) and receiver (Rx) response matrices that convey the information about linear distortion and internal crosstalk. A signal and system model is developed for the B2B measurements, where only some pairs of the Tx and Rx ports of the sounder are utilized among all pairs of the ports. Using the measurement model, a least-square estimation problem is then formulated and converted in the frequency domain to weighted rank-one approximation problems. The notion of system identifiability of a MIMO channel sounder is introduced and some optimal sets of B2B connections are proposed. Given a proposed optimal set of B2B connections, the alternate convex search (ACS) algorithm with a proper initialization is also proposed to solve the weighted rank-one approximation problems. Finally, it is shown how to calibrate field measurement data by using the estimated response matrices. Numerical results show that, only after a couple of iterations, the ACS algorithm with the proposed initialization achieves a comparable identification and calibration performance to the conventional method that requires the B2B connections of all port pairs.

A multi-task transfer learning method with dictionary learning

Transfer learning is a problem that samples are generated from more than one domains, which focuses on transferring knowledge from source tasks to target tasks. A variety of methodologies are proposed for transfer learning. And a number of them concentrate on the inner relationship among each domain while some pay more attention to knowledge transfer. In this paper, based on the hinge loss and SVM, a new dictionary learning with multi-task transfer learning method(DMTTL) is proposed. The dictionary learning method is utilized to learn sparse representation of the given samples. Moreover, a regularization term for two dictionaries are exploited so that the similarity of samples can be well determined. Besides, a new optimization method based on alternate convex search is proposed with convergence analysis, which indicates that the DMTTL is a reasonable approach. After that, the comparison of DMTTL with the state-of-the-art approaches manifests the feasibility and the competitive performance for multi-task classification problem. And the statistic results show that the proposed method outperforms the previous methods.

Generalized Benders Decomposition to Secure Energy-Efficient Resource Allocation for Multiuser Full-Duplex Relay Cooperative Networks

In this paper, we investigate the issue of resource allocation for secure energy efficient communication in a multiuser orthogonal frequency division multiplexing (OFDM) based full-duplex (FD) relaying network in the presence of a passive eavesdropper whose channel state information (CSI) is not perfectly known. Our goal is to maximize the overall secure energy efficiency (SEE), which presents the relationship between energy consumption and secrecy performance. In the context of multiuser communications, such a resource allocation strategy jointly combines subcarrier permutation, subcarrier pair allocation, as well as power allocation altogether. The considered optimization problem is formulated as a mixed integer nonconvex programming problem, which is generally NP hard. Analyzing the property of such a problem, we first use the Dinkelbach's method to eliminate the fractional form and then exploit Generalized Benders decomposition to decouple the original problem into a master problem for pure integer programming and a primal problem for nonlinear programming. More specific, given the nonconvexity of the primal problem, we accordingly transform it into an equivalent relaxed convex problem by applying dual decomposition, alternative convex search, and difference of convex function programming. The numerical results are provided to validate the theoretical analysis and to demonstrate the effectiveness of the proposed algorithm.

A general coordinated voltage regulation method in distribution networks with soft open points

Due to the high flexibility and fast response capability, power electronic devices are versatile, and increasingly being considered to mitigate voltage violations and improve system efficiency in distribution networks with high penetration of distributed generation (DG). This paper proposes a novel voltage regulation framework in distribution systems through the general types of Soft Open Points (SOPs) and reactive powers of DG inverters. By using the linearized DistFlow equations, we establish a biconvex problem that minimizes voltage deviation via a well-coordinated scheme for SOPs and DGs. The coordination of different types of power electronic devices can be simultaneously modelled into this problem which can be efficiently solved by alternative convex search algorithm. The proposed approach is validated by numerical simulations on modified IEEE 15-bus system, IEEE 33-bus system, and IEEE 69-bus system with different types of power electronic compensators.

Deterministic and Bayesian Sparse Signal Processing Algorithms for Coherent Multipath Directions-of-Arrival (DOAs) Estimation

In this paper, we propose novel deterministic and Bayesian methods to identify the directions-of-arrival (DOAs) of coherent multipath signals assuming that a very few number of multipath signals are present and multiple snapshots are available. For both types of methods, we exploit both sparsity and the underlying structure of coherent signal propagation. For the deterministic case, we formulate the problem of estimating the DOAs of coherent multipath signals as a biconvex optimization problem and then solve it by an alternating convex search approach. This is in contrast to the widely used 11 sparse signal recovery convex optimization problem, which only exploits sparsity of the signal under consideration. For the Bayesian case, we propose two novel algorithms based on the mean field variational Bayesian expectation maximization approach. The first algorithm assumes that the true scenario DOAs of the multipath signals are exactly aligned with the angular grids. We next extend the on-grid model to deal with the off-grid problem, which is the second proposed Bayesian algorithm of this paper. These two Bayesian algorithms are in contrast to the widely used sparse Bayesian learning relevance vector machine algorithm, which only exploits sparsity in spatial domain. A simulation study is carried out in terms of root-mean-squared error in the DOA estimates to compare the performances of different algorithms. Finally, we demonstrate the application of the proposed off-grid Bayesian algorithm by analyzing data from the shallow water HF97 ocean acoustic experiment.

Sequential Subspace Clustering via Temporal Smoothness for Sequential Data Segmentation.

This paper develops a novel sequential subspace clustering method for sequential data. Inspired by the state-of-the-art methods, ordered subspace clustering, and temporal subspace clustering, we design a novel local temporal regularization term based on the concept of temporal predictability. Through minimizing the short-term variance on historical data, it can recover the temporal smoothness relationships in sequential data. Moreover, we claim that the local temporal regularization is more important than the global structural regularization for a specific task, such as sequential subspace clustering, which leads to a concise minimization objective function. To solve the bi-convex objective function, a simple and efficient optimization algorithm based on the alternate convex search method is devised to jointly learn the coding matrix and the dictionary. Furthermore, five baseline methods are also devised for comparison with our proposed method from different aspects. Extensive experimental results and comparisons with the state-of-the-art methods on three data sets demonstrate the effectiveness of the proposed temporal smoothness sequential subspace clustering method for sequential data.

Relay selection for SWIPT: Performance analysis of optimization problems and the trade-off between ergodic capacity and energy harvesting

In this paper, we focus on the optimal relay selection (RS) and ergodic capacity of an energy harvesting (EH) amplify-and-forward (AF) multi-relay network, where multiple relays cooperate with the source node and the destination node. We begin the work with the optimization of a single relay. In the first step, after the average end-to-end signal-to-noise ratio (SNR) is obtained, the high end-to-end SNR is approximated to achieve the PDF and CDF, and we jointly optimize time switching (TS) and power splitting (PS) ratios by using Alternate Convex Search algorithm, second derivatives and Hessian matrices. Following that, closed-form expressions for the outage probability and throughput in both delay-restricted and delay-tolerant transmission mode in the high SNR regime are derived. Most importantly, we achieve not only ergodic capacity in the considered EH multi-relay network but also the trade-off between it and the average EH. Our theoretical results are proved by numerical simulations, in which delay-tolerant throughput outperforms delay-restricted throughput thanks to the change of number of relays. In addition, the data rate can be significantly enhanced thanks to the achievement of optimal TS and PS ratios.

Subspace-based identification of discrete time-delay system

We investigate the identification problems of a class of linear stochastic time-delay systems with unknown delayed states in this study. A time-delay system is expressed as a delay differential equation with a single delay in the state vector. We first derive an equivalent linear time-invariant (LTI) system for the time-delay system using a state augmentation technique. Then a conventional subspace identification method is used to estimate augmented system matrices and Kalman state sequences up to a similarity transformation. To obtain a state-space model for the time-delay system, an alternate convex search (ACS) algorithm is presented to find a similarity transformation that takes the identified augmented system back to a form so that the time-delay system can be recovered. Finally, we reconstruct the Kalman state sequences based on the similarity transformation. The time-delay system matrices under the same state-space basis can be recovered from the Kalman state sequences and input-output data by solving two least squares problems. Numerical examples are to show the effectiveness of the proposed method.

Distributed multi-objective cross-layer optimization with joint hyperlink and transmission mode scheduling in network coding-based wireless networks

In this work, we address a cross-layer multi-objective optimization problem of maximizing network lifetime and optimizing aggregate system utility with intra-flow network coding, solved in a distributed manner. Based on the network utility maximization (NUM) framework, we resolve this problem to accommodate routing, scheduling, and stream control from different layers in the coded networks. Specially, we consider that there are two scheduling primitives, namely hyperlink and transmission mode, to be concurrently activated for the multi-objective optimization. Given the constraints with respect to these primitives, the optimization problem is specifically formulated as a quadratically constrained quadratic programming (QCQP) problem that is NP-hard in general, and its scheduling subproblem even when reduced to account for only one of these primitives is a maximum weighted independent set (MWIS) problem that is NP-hard already. To alleviate this complex problem in a distributed manner, we resort to alternate convex search (ACS) and primal decomposition (PD) to approximate the optimal results by using biconvex programming model and subgradient-based algorithm that can iteratively approach to the optimal solution. For the wireless multihop networks, wherein an optimal solution could be practically approximated as its validity would be out-of-date soon in the error-prone wireless environment, our simulation results show that the distributed method can fulfill our requirements, and can make a good trade-off on the heterogeneous objectives with well computational efficiency.

Moving off the grid in an experimental, compressively sampled photonic link.

Perhaps the largest obstacle to practical compressive sampling is an inability to accurately, and sparsely describe the data one seeks to recover due to poor choice of signal model parameters. In such cases the recovery process will yield artifacts, or in many cases, fail completely. This work represents the first demonstration of a solution to this so-called "off-grid" problem in an experimental, compressively sampled system. Specifically, we show that an Alternating Convex Search algorithm is able to significantly reduce these data model errors in harmonic signal recovery.