Convex Penalty Research Articles

Age Minimization in Outdoor and Indoor Communications With Relay-Aided Dual RIS

In this letter, we investigate an outdoor and indoor wireless communication network with the assistance of a relay-aided double-sided reconfigurable intelligent surface (RIS). A scheduling problem is considered at the outdoor access point (AP) to minimize the sum of age of information (AoI). To serve the indoor users and further enhance the wireless link quality, a double-sided RIS with relay is utilized. Since the formulated problem is non-convex with highly-coupled variables, a successive convex approximation (SCA) and penalty based alternating optimization (AO) algorithm with difference of convex (DC) functions is proposed to solve it in an iterative manner. Finally, simulation results show the effectiveness and significant performance improvement in terms of AoI of the proposed algorithm compared with other baselines.

Improving the accuracy and internal consistency of regression-based clustering of high-dimensional datasets.

Component-wise Sparse Mixture Regression (CSMR) is a recently proposed regression-based clustering method that shows promise in detecting heterogeneous relationships between molecular markers and a continuous phenotype of interest. However, CSMR can yield inconsistent results when applied to high-dimensional molecular data, which we hypothesize is in part due to inherent limitations associated with the feature selection method used in the CSMR algorithm. To assess this hypothesis, we explored whether substituting different regularized regression methods (i.e. Lasso, Elastic Net, Smoothly Clipped Absolute Deviation (SCAD), Minmax Convex Penalty (MCP), and Adaptive-Lasso) within the CSMR framework can improve the clustering accuracy and internal consistency (IC) of CSMR in high-dimensional settings. We calculated the true positive rate (TPR), true negative rate (TNR), IC and clustering accuracy of our proposed modifications, benchmarked against the existing CSMR algorithm, using an extensive set of simulation studies and real biological datasets. Our results demonstrated that substituting Adaptive-Lasso within the existing feature selection method used in CSMR led to significantly improved IC and clustering accuracy, with strong performance even in high-dimensional scenarios. In conclusion, our modifications of the CSMR method resulted in improved clustering performance and may thus serve as viable alternatives for the regression-based clustering of high-dimensional datasets.

Estimation of high-dimensional vector autoregression via sparse precision matrix

Summary We consider the problem of estimating sparse vector autoregression (VAR) via penalized precision matrices. This matrix is the output of the underlying directed acyclic graph of the VAR process, whose zero components correspond to the zero coefficients of the graphical representation of the VAR. The sparsity-based precision matrix estimator is deduced from the D-trace loss with convex and nonconvex penalty functions. We establish the consistency of the penalized estimator and provide the conditions for which all true zero entries of the precision matrix are actually estimated as zero with probability tending to one. The relevance of the method is supported by simulated experiments and a real data application.

Communication-efficient and Byzantine-robust distributed learning with statistical guarantee

Communication efficiency and robustness are two major issues in modern distributed learning frameworks. This is due to the practical situations where some computing nodes may have limited communication power or may behave adversarial behaviors. To address the two issues simultaneously, this paper develops two communication-efficient and robust distributed learning algorithms for convex problems. Our motivation is based on surrogate likelihood framework and the median and trimmed mean operations. Particularly, the proposed algorithms are provably robust against Byzantine failures, and also achieve optimal statistical rates for strong convex losses and convex (non-smooth) penalties. For typical statistical models such as generalized linear models, our results show that statistical errors dominate optimization errors in finite iterations. Simulated and real data experiments are conducted to demonstrate the numerical performance of our algorithms.

Versatile Time-Frequency Representations Realized by Convex Penalty on Magnitude Spectrogram

Versatile Time-Frequency Representations Realized by Convex Penalty on Magnitude Spectrogram

Nonstationary iterated Tikhonov regularization: convergence analysis via Hölder stability

In this paper, we study the nonstationary iterated Tikhonov regularization method (NITRM) proposed by Jin and Zhong (2014 Numer. Math. 127 485–513) to solve the inverse problems, where the inverse mapping fulfills a Hölder stability estimate. The iterates of NITRM are defined through certain minimization problems in the settings of Banach spaces. In order to study the various important characteristics of the sought solution, we consider the non-smooth uniformly convex penalty terms in the minimization problems. In the case of noisy data, we terminate the method via a discrepancy principle and show the strong convergence of the iterates as well as the convergence with respect to the Bregman distance. For noise free data, we show the convergence of the iterates to the sought solution. Additionally, we derive the convergence rates of NITRM method for both the noisy and noise free data that are missing from the literature. In order to derive the convergence rates, we solely utilize the Hölder stability of the inverse mapping that opposes the standard analysis which requires a source condition as well as a nonlinearity estimate to be satisfied by the inverse mapping. Finally, we discuss three numerical examples to show the validity of our results.

Inference for low‐ and high‐dimensional inhomogeneous Gibbs point processes

AbstractGibbs point processes (GPPs) constitute a large and flexible class of spatial point processes with explicit dependence between the points. They can model attractive as well as repulsive point patterns. Feature selection procedures are an important topic in high‐dimensional statistical modeling. In this paper, a composite likelihood (in particular pseudo‐likelihood) approach regularized with convex and nonconvex penalty functions is proposed to handle statistical inference for possibly high‐dimensional inhomogeneous GPPs. We particularly investigate the setting where the number of covariates diverges as the domain of observation increases. Under some conditions provided on the spatial GPP and on penalty functions, we show that the oracle property, consistency and asymptotic normality hold. Our results also cover the low‐dimensional case which fills a large gap in the literature. Through simulation experiments, we validate our theoretical results and finally, an application to a tropical forestry dataset illustrates the use of the proposed approach.

Graphiclasso: Graphical lasso for learning sparse inverse-covariance matrices

In modern multivariate statistics, where high-dimensional datasets are ubiquitous, learning large (inverse-) covariance matrices is imperative for data analysis. A popular approach to estimating a large inverse-covariance matrix is to regularize the Gaussian log-likelihood function by imposing a convex penalty function. In a seminal article, Friedman, Hastie, and Tibshirani (2008, Biostatistics 9: 432–441) proposed a graphical lasso (Glasso) algorithm to efficiently estimate sparse inverse-covariance matrices from the convex regularized log-likelihood function. In this article, I first explore the Glasso algorithm and then introduce a new graphiclasso command for the large inverse-covariance matrix estimation. Moreover, I provide a useful command for tuning parameter selection in the Glasso algorithm using the extended Bayesian information criterion, the Akaike information criterion, and cross-validation. I demonstrate the use of Glasso using simulation results and real-world data analysis.

A Fast Averaged Kaczmarz Iteration with Convex Penalty for Inverse Problems in Hilbert Spaces

A Fast Averaged Kaczmarz Iteration with Convex Penalty for Inverse Problems in Hilbert Spaces

Convergence rates of a dual gradient method for constrained linear ill-posed problems

In this paper we consider a dual gradient method for solving linear ill-posed problems Ax = y, where A : X rightarrow Y is a bounded linear operator from a Banach space X to a Hilbert space Y. A strongly convex penalty function is used in the method to select a solution with desired feature. Under variational source conditions on the sought solution, convergence rates are derived when the method is terminated by either an a priori stopping rule or the discrepancy principle. We also consider an acceleration of the method as well as its various applications.

Optimal management of DC pension fund under the relative performance ratio and VaR constraint

This paper investigates the optimal management of defined contribution pension plan under the Omega ratio and Value-at-Risk (VaR) constraint. Interest and inflation risks are considered, and the financial market consists of cash, a zero-coupon bond, an inflation-indexed zero-coupon bond, and a stock. The goal of the pension manager is to maximize the performance ratio of the real terminal wealth under the VaR constraint. An auxiliary process is introduced to transform the original problem into a self-financing problem. We obtain the optimal terminal wealth under different cases by combining the linearization method, the Lagrange dual method, the martingale method, and the concavification method. There are fourteen cases for the convex penalty function, and there are six cases for the concave penalty function. Besides, when the penalty and reward functions are both power functions, the explicit forms of the optimal investment strategies are obtained. Numerical examples are shown to illustrate the impacts of the performance ratio and VaR constraint.

Reconfigurable Intelligent Surface-Assisted Secure Mobile Edge Computing Networks

Mobile edge computing (MEC) has been recognized as a viable technology to satisfy low-delay computation requirements for resource-constrained Internet of things (IoT) devices. Nevertheless, the broadcast feature of wireless electromagnetic communications may lead to the security threats to IoT devices. In order to enhance the task offloading security, this paper proposes a reconfigurable intelligent surface (RIS)-assisted secure MEC network framework. Furthermore, we investigate the max-min computation efficiency problem under the secure computation rate requirements, by jointly optimizing the local computing frequencies and transmission power of IoT devices, time-slot assignment, and phase beamforming of the RIS. To solve the formulated non-convex problem, we further develop an iterative algorithm, in which the Dinkelbach-type method and block coordinate descent (BCD) technique are utilized to tackle the fractional objective function and coupled optimization variables, respectively. In particular, the successive convex approximation (SCA) and penalty function-based methods are exploited to solve the transmit power control and reflecting beamforming optimization subproblems, respectively, and the closed-form expression for local computing frequencies optimization subproblem is derived. Numerical results quantify the performance gain achieved by the proposed RIS-assisted secure MEC networks, when compared to existing benchmark methods.

Two-point Landweber-type method with convex penalty terms for nonsmooth nonlinear inverse problems

Abstract In this work, we propose a two-point Landweber-type method with general convex penalty terms for solving nonsmooth nonlinear inverse problems. The design of our method can cope with nonsmooth nonlinear inverse problems or the nonlinear inverse problems whose data are contaminated by various types of noise. The method consists of the two-point acceleration strategy and inner solvers. Inner solvers are used to solve the minimization problems with respect to the penalty terms in each steps. If the minimization problems can be solved explicitly, the inner solvers will be chosen to be the exact solvers. Otherwise, we will use inexact solvers as inner solvers. Convergence results are given without utilizing the Gâteaux differentiability of the forward operator or the reflexivity of the image space. Numerical simulations are given to test the performance of the proposed method.

Communication-Efficient Distributed Learning for High-Dimensional Support Vector Machines

Distributed learning has received increasing attention in recent years and is a special need for the era of big data. For a support vector machine (SVM), a powerful binary classification tool, we proposed a novel efficient distributed sparse learning algorithm, the communication-efficient surrogate likelihood support vector machine (CSLSVM), in high-dimensions with convex or nonconvex penalties, based on a communication-efficient surrogate likelihood (CSL) framework. We extended the CSL for distributed SVMs without the need to smooth the hinge loss or the gradient of the loss. For a CSLSVM with lasso penalty, we proved that its estimator could achieve a near-oracle property for l1 penalized SVM estimators on whole datasets. For a CSLSVM with smoothly clipped absolute deviation penalty, we showed that its estimator enjoyed the oracle property, and that it used local linear approximation (LLA) to solve the optimization problem. Furthermore, we showed that the LLA was guaranteed to converge to the oracle estimator, even in our distributed framework and the ultrahigh-dimensional setting, if an appropriate initial estimator was available. The proposed approach is highly competitive with the centralized method within a few rounds of communications. Numerical experiments provided supportive evidence.

Fundamental barriers to high-dimensional regression with convex penalties

In high-dimensional regression, we attempt to estimate a parameter vector β0∈Rp from n≲p observations {(yi,xi)}i≤n, where xi∈Rp is a vector of predictors and yi is a response variable. A well-established approach uses convex regularizers to promote specific structures (e.g., sparsity) of the estimate βˆ while allowing for practical algorithms. Theoretical analysis implies that convex penalization schemes have nearly optimal estimation properties in certain settings. However, in general the gaps between statistically optimal estimation (with unbounded computational resources) and convex methods are poorly understood. We show that when the statistican has very simple structural information about the distribution of the entries of β0, a large gap frequently exists between the best performance achieved by any convex regularizer satisfying a mild technical condition and either: (i) the optimal statistical error or (ii) the statistical error achieved by optimal approximate message passing algorithms. Remarkably, a gap occurs at high enough signal-to-noise ratio if and only if the distribution of the coordinates of β0 is not log-concave. These conclusions follow from an analysis of standard Gaussian designs. Our lower bounds are expected to be generally tight, and we prove tightness under certain conditions.

Data Rate Maximization in RIS-Assisted D2D Communication with Transceiver Hardware Impairments

We maximize the transmit rate of device-to-device (D2D) in a reconfigurable intelligent surface (RIS) assisted D2D communication system by satisfying the unit-modulus constraints of reflectin elements, the transmit power limit of base station (BS) and the transmitter in a D2D pair. Since it is a non-convex optimization problem, the block coordinate descent (BCD) technique is adopted to decouple this problem into three subproblems. Then, the non-convex subproblems are approximated into convex problems by using successive convex approximation (SCA) and penalty convex-concave procedure (CCP) techniques. Finally, the optimal solution of original problem is obtained by iteratively optimizing the subproblems. Simulation results reveal the validity of the algorithm that we proposed to solve the optimization problem and illustrate the effectiveness of RIS to improve the transmit rate of the D2D pair even with hardware impairments.

Asymptotic normality of robust M-estimators with convex penalty

This paper develops asymptotic normality results for individual coordinates of robust M-estimators with convex penalty in high-dimensions, where the dimension p is at most of the same order as the sample size n, i.e, p∕n≤γ for some fixed constant γ>0. The asymptotic normality requires a bias correction and holds for most coordinates of the M-estimator for a large class of loss functions including the Huber loss and its smoothed versions regularized with a strongly convex penalty. The asymptotic variance that characterizes the width of the resulting confidence intervals is estimated with data-driven quantities. This estimate of the variance adapts automatically to low (p∕n→0) or high (p∕n≤γ) dimensions and does not involve the proximal operators seen in previous works on asymptotic normality of M-estimators. For the Huber loss, the estimated variance has a simple expression involving an effective degrees-of-freedom as well as an effective sample size. The case of the Huber loss with Elastic-Net penalty is studied in details and a simulation study confirms the theoretical findings. The asymptotic normality results follow from Stein formulae for high-dimensional random vectors on the sphere developed in the paper which are of independent interest.

Joint power-trajectory-scheduling optimization in a mobile UAV-enabled network via alternating iteration

This work focuses on an unmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC) system based on device-to-device (D2D) communication. In this system, the UAV exhibits caching, computing and relaying capabilities to periodically provide specific service to cellular users and D2D receiver nodes in the appointed time slot. Besides, the D2D transmitter can provide additional caching services to D2D receiver to reduce the pressure of the UAV. Note that communication between multi-type nodes is mutually restricted and different links share spectrum resources. To achieve an improved balance between different types of node, we aim to maximize the overall energy efficiency while satisfying the quality-of-service requirements of the cellular nodes. To address this problem, we propose an alternating iteration algorithm to jointly optimize the scheduling strategies of the user, transmitting power of the UAV and D2D-TX nodes, and UAV trajectory. The successive convex approximation, penalty function, and Dinkelbach method are employed to transform the original problem into a group of solvable subproblems and the convergence of the method is proved. Simulation results show that the proposed scheme performs better than other benchmark algorithms, particularly in terms of balancing the tradeoff between minimizing UAV energy consumption and maximizing throughput.

Distributed Anytime-Feasible Resource Allocation Subject to Heterogeneous Time-Varying Delays

This paper considers distributed allocation strategies, formulated as a distributed sum-preserving (fixed-sum) allocation of resources over a multi-agent network in the presence of heterogeneous arbitrary time-varying delays. We propose a double time-scale scenario for unknown delays and a faster single time-scale scenario for known delays. Further, the links among the nodes are considered subject to certain nonlinearities (e.g, quantization and saturation/clipping). We discuss different models for nonlinearities and how they may affect the convergence, sum-preserving feasibility constraint, and solution optimality over general weight-balanced uniformly strongly connected networks and, further, time-delayed undirected networks. Our proposed scheme works in a variety of applications with general non-quadratic strongly-convex smooth objective functions. The non-quadratic part, for example, can be due to additive convex penalty or barrier functions to address the local box constraints. The network can change over time, is not necessarily connected at all times, but is only assumed to be uniformly-connected. The novelty of this work is to address all-time feasible Laplacian gradient solutions in presence of nonlinearities, switching digraph topology (not necessarily all-time connected), and heterogeneous time-varying delays.

Upper and lower degree-constrained graph orientation with minimum penalty

In the degree-constrained graph orientation problem, the input is an unweighted, undirected graph G=(V,E) and nonnegative integers av and bv (with av≤bv) for each v∈V, and the objective is to assign a direction to every edge in E in such a way that for each v∈V, the number of outgoing edges in the resulting directed graph lies in the interval [av,bv]. When such an orientation does not exist, it is desirable to find an orientation that best fits the condition instead. In this paper, we consider the problem of finding an orientation that minimizes the total penalty ∑v∈Vcv, where cv is a penalty incurred whenever a vertex v violates its degree constraints. As penalty functions, convex, concave, and step functions are considered in this paper. We show that the problem with any convex penalty function can be solved in O(|E|1.5min⁡{log⁡(|E|⋅C),|E|0.5log⁡Δlog⁡|E|}) time, where Δ and C are the maximum degree and the largest magnitude of a penalty, respectively. In contrast, we show APX-hardness of the problem with step or concave functions. For trees and graphs with treewidth τ, the problem with any penalty function can be solved exactly in O(|V|log⁡Δ) time and O(τ2Δ2τ+2|V|) time, respectively. Finally, we consider the generalization of the problem to edge-weighted graphs and establish strong bounds on its inapproximability that hold even in the special case of stars. On the positive side, we can extend our algorithms for unweighted version of the problem to obtain pseudo-polynomial-time algorithms for the edge-weighted problem variant when restricted to trees and graphs with bounded treewidth. Also, we design a PTAS and a linear-time algorithm for stars with further restrictions on the degree constraints and edge weights.