Convex Formulation Research Articles

Optimal Joint User Association and Multi-Pattern Resource Allocation in Heterogeneous Networks

This paper studies the joint user association and resource allocation in heterogeneous networks (HetNets) from a novel perspective, motivated by and generalizing the idea of fractional frequency reuse. By treating the multi-cell multi-user resource allocation as resource partitioning among multiple reuse patterns, we propose a unified framework to analyze and compare a wide range of user association and resource allocation strategies for HetNets, and provide an optimal benchmark for network performance. The enabling mechanisms are a novel formulation to consider all possible interference patterns or any pre-defined subset of patterns, and efficient sparsity-pursuit algorithms to find the solution. A notable feature of this formulation is that the patterns remain fixed during the resource optimization process. This creates a favorable opportunity for convex formulations while still considering interference coupling. More important, in view of the fact that multi-cell resource allocation is very computational demanding, our framework provides a systematic way to trade off performance for the reduction of computational complexity by restricting the candidate patterns to a small number of feature patterns. Relying on the sparsity-pursuit capability of the proposed algorithms, we develop a practical guideline to identify the feature patterns. Numerical results show that the identified feature patterns can significantly improve the existing strategies, and jointly optimizing the user association and resource allocation indeed brings considerable gain.

Multiresolution convex variational model for multiphase image segmentation

This paper presents a convex variational model for multiphase image segmentation by incorporating a multiresolution approach. We extend our previous work to formulate the energy functional which is robust response to image variations. In contrast to our previous work, which can lead to local minima, a global solution is proposed to minimize the segmentation energy with some constraint conditions. By incorporating edge-based information, a non-convex energy functional is first introduced on the membership functions, which are used as indicators of different homogeneous regions. Then the non-convex problem is converted into a continuous convex formulation. An efficient dual minimization implementation of our binary partitioning function model accurately describes disjoint regions using stable segmentation. Experiments results show the proposed model is robust to noise, independent of initialization and unambiguous segmentation. Compared with the traditional variational models, the proposed model can get more accurate results and higher computational efficiency.

SU-F-T-340: Direct Editing of Dose Volume Histograms: Algorithms and a Unified Convex Formulation for Treatment Planning with Dose Constraints

Purpose: To develop a procedure for including dose constraints in convex programming-based approaches to treatment planning, and to support dynamic modification of such constraints during planning. Methods: We present a mathematical approach that allows mean dose, maximum dose, minimum dose and dose volume (i.e., percentile) constraints to be appended to any convex formulation of an inverse planning problem. The first three constraint types are convex and readily incorporated. Dose volume constraints are not convex, however, so we introduce a convex restriction that is related to CVaR-based approaches previously proposed in the literature. To compensate for the conservatism of this restriction, we propose a new two-pass algorithm that solves the restricted problem on a first pass and uses this solution to form exact constraints on a second pass. In another variant, we introduce slack variables for each dose constraint to prevent the problem from becoming infeasible when the user specifies an incompatible set of constraints. We implement the proposed methods in Python using the convex programming package cvxpy in conjunction with the open source convex solvers SCS and ECOS. Results: We show, for several cases taken from the clinic, that our proposed method meets specified constraints (often with margin) when they are feasible. Constraints are met exactly when we use the two-pass method, and infeasible constraints are replaced with the nearest feasible constraint when slacks are used. Finally, we introduce ConRad, a Python-embedded free software package for convex radiation therapy planning. ConRad implements the methods described above and offers a simple interface for specifying prescriptions and dose constraints. Conclusion: This work demonstrates the feasibility of using modifiable dose constraints in a convex formulation, making it practical to guide the treatment planning process with interactively specified dose constraints. This work was supported by the Stanford BioX Graduate Fellowship and NIH Grant 5R01CA176553.

An augmented Lagrangian approach to Wasserstein gradient flows and applications

Taking advantage of the Benamou-Brenier dynamic formulation of optimal transport, we propose a convex formulation for each step of the JKO scheme for Wasserstein gradient flows which can be attacked by an augmented Lagrangian method which we call the ALG2-JKO scheme. We test the algorithm in particular on the porous medium equation. We also consider a semi implicit variant which enables us to treat nonlocal interactions as well as systems of interacting species. Regarding systems, we can also use the ALG2-JKO scheme for the simulation of crowd motion models with several species.

Convex biclustering.

In the biclustering problem, we seek to simultaneously group observations and features. While biclustering has applications in a wide array of domains, ranging from text mining to collaborative filtering, the problem of identifying structure in high-dimensional genomic data motivates this work. In this context, biclustering enables us to identify subsets of genes that are co-expressed only within a subset of experimental conditions. We present a convex formulation of the biclustering problem that possesses a unique global minimizer and an iterative algorithm, COBRA, that is guaranteed to identify it. Our approach generates an entire solution path of possible biclusters as a single tuning parameter is varied. We also show how to reduce the problem of selecting this tuning parameter to solving a trivial modification of the convex biclustering problem. The key contributions of our work are its simplicity, interpretability, and algorithmic guarantees-features that arguably are lacking in the current alternative algorithms. We demonstrate the advantages of our approach, which includes stably and reproducibly identifying biclusterings, on simulated and real microarray data.

Enhanced Sparsity by Non-Separable Regularization

This paper develops a convex approach for sparse one-dimensional deconvolution that improves upon L1-norm regularization, the standard convex approach. We propose a sparsity-inducing non-separable non-convex bivariate penalty function for this purpose. It is designed to enable the convex formulation of ill-conditioned linear inverse problems with quadratic data fidelity terms. The new penalty overcomes limitations of separable regularization. We show how the penalty parameters should be set to ensure that the objective function is convex, and provide an explicit condition to verify the optimality of a prospective solution. We present an algorithm (an instance of forward-backward splitting) for sparse deconvolution using the new penalty.

Exploring the Genetic Patterns of Complex Diseases via the Integrative Genome-Wide Approach.

Genome-wide association studies (GWASs), which assay more than a million single nucleotide polymorphisms (SNPs) in thousands of individuals, have been widely used to identify genetic risk variants for complex diseases. However, most of the variants that have been identified contribute relatively small increments of risk and only explain a small portion of the genetic variation in complex diseases. This is the so-called missing heritability problem. Evidence has indicated that many complex diseases are genetically related, meaning these diseases share common genetic risk variants. Therefore, exploring the genetic correlations across multiple related studies could be a promising strategy for removing spurious associations and identifying underlying genetic risk variants, and thereby uncovering the mystery of missing heritability in complex diseases. We present a general and robust method to identify genetic patterns from multiple large-scale genomic datasets. We treat the summary statistics as a matrix and demonstrate that genetic patterns will form a low-rank matrix plus a sparse component. Hence, we formulate the problem as a matrix recovering problem, where we aim to discover risk variants shared by multiple diseases/traits and those for each individual disease/trait. We propose a convex formulation for matrix recovery and an efficient algorithm to solve the problem. We demonstrate the advantages of our method using both synthesized datasets and real datasets. The experimental results show that our method can successfully reconstruct both the shared and the individual genetic patterns from summary statistics and achieve comparable performances compared with alternative methods under a wide range of scenarios. The MATLAB code is available at: http://www.comp.hkbu.edu.hk/~xwan/iga.zip.

Optimal Quality-of-Service Scheduling for Energy-Harvesting Powered Wireless Communications

In this paper, a new dynamic string tautening algorithm is proposed to generate the most energy-efficient off-line schedule for delay-limited traffic of transmitters with non-negligible circuit power. The algorithm is based on two key findings that we derive through judicious convex formulation and resultant optimality conditions, specifies a set of simple but optimal rules, and generates the optimal schedule with a low complexity of $\mathcal{O}(N^2)$ in the worst case. The proposed algorithm is also extended to on-line scenarios, where the transmit schedule is generated on-the-fly. Simulation shows that the proposed algorithm requires substantially lower average complexity by almost two orders of magnitude to retain optimality than general convex solvers. The effective transmit region, specified by the tradeoff of the data arrival rate and the energy harvesting rate, is substantially larger using our algorithm than using other existing alternatives. Significantly more data or less energy can be supported in the proposed algorithm.

Convex Low Rank Approximation

Low rank approximation is an important tool in many applications. Given an observed matrix with elements corrupted by Gaussian noise it is possible to find the best approximating matrix of a given rank through singular value decomposition. However, due to the non-convexity of the formulation it is not possible to incorporate any additional knowledge of the sought matrix without resorting to heuristic optimization techniques. In this paper we propose a convex formulation that is more flexible in that it can be combined with any other convex constraints and penalty functions. The formulation uses the so called convex envelope, which is the provably best possible convex relaxation. We show that for a general class of problems the envelope can be efficiently computed and may in some cases even have a closed form expression. We test the algorithm on a number of real and synthetic data sets and show state-of-the-art results.

Convex Banding of the Covariance Matrix

We introduce a new sparse estimator of the covariance matrix for high-dimensional models in which the variables have a known ordering. Our estimator, which is the solution to a convex optimization problem, is equivalently expressed as an estimator that tapers the sample covariance matrix by a Toeplitz, sparsely banded, data-adaptive matrix. As a result of this adaptivity, the convex banding estimator enjoys theoretical optimality properties not attained by previous banding or tapered estimators. In particular, our convex banding estimator is minimax rate adaptive in Frobenius and operator norms, up to log factors, over commonly studied classes of covariance matrices, and over more general classes. Furthermore, it correctly recovers the bandwidth when the true covariance is exactly banded. Our convex formulation admits a simple and efficient algorithm. Empirical studies demonstrate its practical effectiveness and illustrate that our exactly banded estimator works well even when the true covariance matrix is only close to a banded matrix, confirming our theoretical results. Our method compares favorably with all existing methods, in terms of accuracy and speed. We illustrate the practical merits of the convex banding estimator by showing that it can be used to improve the performance of discriminant analysis for classifying sound recordings. Supplementary materials for this article are available online.

Learning Model-Based Sparsity via Projected Gradient Descent

Several convex formulation methods have been proposed previously for statistical estimation with structured sparsity as the prior. These methods often require a carefully tuned regularization parameter, often a cumbersome or heuristic exercise. Furthermore, the estimate that these methods produce might not belong to the desired sparsity model, albeit accurately approximating the true parameter. Therefore, greedy-type algorithms could often be more desirable in estimating structured-sparse parameters. So far, these greedy methods have mostly focused on linear statistical models. In this paper we study the projected gradient descent with non-convex structured-sparse parameter model as the constraint set. Should the cost function have a Stable Model-Restricted Hessian the algorithm produces an approximation for the desired minimizer. As an example we elaborate on application of the main results to estimation in Generalized Linear Model.

Beyond Low Rank + Sparse: Multiscale Low Rank Matrix Decomposition

We present a natural generalization of the recent low rank + sparse matrix decomposition and consider the decomposition of matrices into components of multiple scales. Such decomposition is well motivated in practice as data matrices often exhibit local correlations in multiple scales. Concretely, we propose a multi-scale low rank modeling that represents a data matrix as a sum of block-wise low rank matrices with increasing scales of block sizes. We then consider the inverse problem of decomposing the data matrix into its multi-scale low rank components and approach the problem via a convex formulation. Theoretically, we show that under various incoherence conditions, the convex program recovers the multi-scale low rank components either exactly or approximately. Practically, we provide guidance on selecting the regularization parameters and incorporate cycle spinning to reduce blocking artifacts. Experimentally, we show that the multi-scale low rank decomposition provides a more intuitive decomposition than conventional low rank methods and demonstrate its effectiveness in four applications, including illumination normalization for face images, motion separation for surveillance videos, multi-scale modeling of the dynamic contrast enhanced magnetic resonance imaging and collaborative filtering exploiting age information.

Hierarchical Clustering Multi-Task Learning for Joint Human Action Grouping and Recognition.

This paper proposes a hierarchical clustering multi-task learning (HC-MTL) method for joint human action grouping and recognition. Specifically, we formulate the objective function into the group-wise least square loss regularized by low rank and sparsity with respect to two latent variables, model parameters and grouping information, for joint optimization. To handle this non-convex optimization, we decompose it into two sub-tasks, multi-task learning and task relatedness discovery. First, we convert this non-convex objective function into the convex formulation by fixing the latent grouping information. This new objective function focuses on multi-task learning by strengthening the shared-action relationship and action-specific feature learning. Second, we leverage the learned model parameters for the task relatedness measure and clustering. In this way, HC-MTL can attain both optimal action models and group discovery by alternating iteratively. The proposed method is validated on three kinds of challenging datasets, including six realistic action datasets (Hollywood2, YouTube, UCF Sports, UCF50, HMDB51 & UCF101), two constrained datasets (KTH & TJU), and two multi-view datasets (MV-TJU & IXMAS). The extensive experimental results show that: 1) HC-MTL can produce competing performances to the state of the arts for action recognition and grouping; 2) HC-MTL can overcome the difficulty in heuristic action grouping simply based on human knowledge; 3) HC-MTL can avoid the possible inconsistency between the subjective action grouping depending on human knowledge and objective action grouping based on the feature subspace distributions of multiple actions. Comparison with the popular clustered multi-task learning further reveals that the discovered latent relatedness by HC-MTL aids inducing the group-wise multi-task learning and boosts the performance. To the best of our knowledge, ours is the first work that breaks the assumption that all actions are either independent for individual learning or correlated for joint modeling and proposes HC-MTL for automated, joint action grouping and modeling.

T2 shuffling: Sharp, multicontrast, volumetric fast spin-echo imaging.

A new acquisition and reconstruction method called T2 Shuffling is presented for volumetric fast spin-echo (three-dimensional [3D] FSE) imaging. T2 Shuffling reduces blurring and recovers many images at multiple T2 contrasts from a single acquisition at clinically feasible scan times (6-7 min). The parallel imaging forward model is modified to account for temporal signal relaxation during the echo train. Scan efficiency is improved by acquiring data during the transient signal decay and by increasing echo train lengths without loss in signal-to-noise ratio (SNR). By (1) randomly shuffling the phase encode view ordering, (2) constraining the temporal signal evolution to a low-dimensional subspace, and (3) promoting spatio-temporal correlations through locally low rank regularization, a time series of virtual echo time images is recovered from a single scan. A convex formulation is presented that is robust to partial voluming and radiofrequency field inhomogeneity. Retrospective undersampling and in vivo scans confirm the increase in sharpness afforded by T2 Shuffling. Multiple image contrasts are recovered and used to highlight pathology in pediatric patients. A proof-of-principle method is integrated into a clinical musculoskeletal imaging workflow. The proposed T2 Shuffling method improves the diagnostic utility of 3D FSE by reducing blurring and producing multiple image contrasts from a single scan. Magn Reson Med 77:180-195, 2017. © 2016 Wiley Periodicals, Inc.

QoE optimization of video multicast with heterogeneous channels and playback requirements

We propose an application-layer forward error correction (AL-FEC) code rate allocation scheme to maximize the quality of experience (QoE) of a video multicast. The allocation dynamically assigns multicast clients to the quality layers of a scalable video bitstream, based on their heterogeneous channel qualities and video playback capabilities. Normalized mean opinion score (NMOS) is employed to value the client's quality of experience across various possible adaptations of a multilayer video, coded using mixed spatial-temporal-amplitude scalability. The scheme provides assurance of reception of the video layers using fountain coding and effectively allocates coding rates across the layers to maximize a multicast utility measure. An advantageous feature of the proposed scheme is that the complexity of the optimization is independent of the number of clients. Additionally, a convex formulation is proposed that attains close to the best performance and offers a reliable alternative when further reduction in computational complexity is desired. The optimization is extended to perform suppression of QoE fluctuations for clients with marginal channel qualities. The scheme offers a means to trade-off service utility for the entire multicast group and clients with the worst channels. According to the simulation results, the proposed optimization framework is robust against source rate variations and limited amount of client feedback.

Bit‐error‐rate minimisation in multiuser transmission schemes for multiple‐input – multiple‐output communication with increasing number of base station antennas

Multiuser transmission (MuT) can be carried out at the base station of a multiple-input–multiple-output system for mitigating the downlink multiuser interference, achieving a near-single-user performance at the mobile terminals. These MuT systems can be designed for optimising different performance metrics, such as the bit-error-rate (BER), which is the focus of this study. In order to mitigate the MuT's complexity, a convex formulation for the minimum BER (MinBER) optimisation problem is proposed, by employing some slight modifications on its original formulation. The authors have shown that the novel convex formulation makes the application of some optimisation algorithms very efficient, leading to a reduced complexity and enabling the system operation in higher dimensions. Indeed, the authors also show that in these conditions the MinBER–MuT system presents the most noticeable gains in performance. The results demonstrate that, among the investigated techniques, the proposed Line Search Quasi-Newton algorithm relying on the penalty function approach results in the best performance versus complexity trade-off in the context of high capacity multiuser systems.

NuMax: A Convex Approach for Learning Near-Isometric Linear Embeddings

We propose a novel framework for the deterministic construction of linear, near-isometric embeddings of a finite set of data points. Given a set of training points X ⊂ \BBR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> , we consider the secant set S(X) that consists of all pairwise difference vectors of X, normalized to lie on the unit sphere. We formulate an affine rank minimization problem to construct a matrix Ψ that preserves the norms of all the vectors in S(X) up to a distortion parameter δ. While affine rank minimization is NP-hard, we show that this problem can be relaxed to a convex formulation that can be solved using a tractable semidefinite program (SDP). In order to enable scalability of our proposed SDP to very large-scale problems, we adopt a two-stage approach. First, in order to reduce compute time, we develop a novel algorithm based on the Alternating Direction Method of Multipliers (ADMM) that we call Nuclear norm minimization with Max-norm constraints (NuMax) to solve the SDP. Second, we develop a greedy, approximate version of NuMax based on the column generation method commonly used to solve large-scale linear programs. We demonstrate that our framework is useful for a number of signal processing applications via a range of experiments on large-scale synthetic and real datasets.

On the Trace of Scatter Matrix Difference: A Convex Approximate Discriminant Analysis Formulation and Its Mechanism Research in Imperial Examination System

In this paper, we present a convex discriminant analysis formulation, which is extended to solve multi-label classification problems. The original Linear Discriminant Analysis energy optimization function is turned into another form as a convex formulation (namely, convex Approximate LDA, denoted as “convexALDA” for short) using the generalized eigen-decomposition. We give applications by incorporating convexALDA as a regularizer into discriminant regression analysis. Extensive experimental results on multi-label classification tasks and an extensive application scenario on communication characteristics of imperial examination system are provided. In this way we have a brand-new comprehension for it, and a new idea and method was also put forward for studying the system.

Direct Synthesis of Fixed-Order &lt;inline-formula&gt; &lt;tex-math notation="TeX"&gt;${\cal H}_{\infty}$&lt;/tex-math&gt;&lt;/inline-formula&gt; Controllers

This technical note considers the fixed-order H ∞ output feedback control design problem for linear time invariant (LTI) systems. The objective is to design a fixed-order controller with guaranteed stability and closed-loop H ∞ performance. This problem is NP-hard due to the non-convex rank constraint which appears in the formulation. We propose an algorithm for noniterative direct synthesis (NODS) of reduced order robust controllers. NODS entails initial computation of two positive-definite matrices via full-order convex LMI conditions. These are then utilized by appropriate eigenvalue decomposition to directly obtain a suboptimal convex formulation for the fixed-order controller.

A convex formulation for joint RNA isoform detection and quantification from multiple RNA-seq samples.

BackgroundDetecting and quantifying isoforms from RNA-seq data is an important but challenging task. The problem is often ill-posed, particularly at low coverage. One promising direction is to exploit several samples simultaneously.ResultsWe propose a new method for solving the isoform deconvolution problem jointly across several samples. We formulate a convex optimization problem that allows to share information between samples and that we solve efficiently. We demonstrate the benefits of combining several samples on simulated and real data, and show that our approach outperforms pooling strategies and methods based on integer programming.ConclusionOur convex formulation to jointly detect and quantify isoforms from RNA-seq data of multiple related samples is a computationally efficient approach to leverage the hypotheses that some isoforms are likely to be present in several samples. The software and source code are available at http://cbio.ensmp.fr/flipflop.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0695-9) contains supplementary material, which is available to authorized users.