Coordinate Descent Optimization Research Articles

Implementation Aspects in Regularized Structural Equation Models

This article reviews several implementation aspects in estimating regularized single-group and multiple-group structural equation models (SEM). It is demonstrated that approximate estimation approaches that rely on a differentiable approximation of non-differentiable penalty functions perform similarly to the coordinate descent optimization approach of regularized SEMs. Furthermore, using a fixed regularization parameter can sometimes be superior to an optimal regularization parameter selected by the Bayesian information criterion when it comes to the estimation of structural parameters. Moreover, the widespread penalty functions of regularized SEM implemented in several R packages were compared with the estimation based on a recently proposed penalty function in the Mplus software. Finally, we also investigate the performance of a clever replacement of the optimization function in regularized SEM with a smoothed differentiable approximation of the Bayesian information criterion proposed by O’Neill and Burke in 2023. The findings were derived through two simulation studies and are intended to guide the practical implementation of regularized SEM in future software pieces.

System error estimation for sensor network with integrated sensing and communication application

The integrated sensing and communication base station can perform both communication and sensing tasks simultaneously. The key premise of accurate localization and tracking of targets is sensor registration process. If system errors of multi-sensor system are not corrected properly, it can lead to significant sensing errors and reduce the overall system performance. In this paper, we propose a system error estimation method for a 3-dimensional asynchronous base station network to achieve satisfactory localization and tracking performance. Specifically, by making use of the priori information of the cooperative targets, we propose to minimize the sum of squares of the position mismatch errors of the targets, this leads to a nonlinear error estimation model for the system errors. To handle the high nonlinearity and ensure fast convergence of online registration, an efficient inexact block coordinate descent optimization algorithm incorporating the proximal method is proposed. The algorithm divides the estimation problem into a linear least square subproblem and several quadratic constrained quadratic program subproblems, enabling different kinds of system errors to be alternatively updated. Finally, the effectiveness and feasibility of the proposed estimation algorithm are demonstrated through numerical simulation.

Massive Parallelization of Massive Sample-Size Survival Analysis

Large-scale observational health databases are increasingly popular for conducting comparative effectiveness and safety studies of medical products. However, increasing number of patients poses computational challenges when fitting survival regression models in such studies. In this article, we use Graphics Processing Units (GPUs) to parallelize the computational bottlenecks of massive sample-size survival analyses. Specifically, we develop and apply time- and memory-efficient single-pass parallel scan algorithms for Cox proportional hazards models and forward-backward parallel scan algorithms for Fine-Gray models for analysis with and without a competing risk using a cyclic coordinate descent optimization approach. We demonstrate that GPUs accelerate the computation of fitting these complex models in large databases by orders of magnitude as compared to traditional multi-core CPU parallelism. Our implementation enables efficient large-scale observational studies involving millions of patients and thousands of patient characteristics. The above implementation is available in the open-source R package Cyclops.

Fairness-aware data offloading of IoT applications enabled by heterogeneous UAVs

With the widespread application of IoT devices, some new scenarios with insufficient coverage of communication infrastructure have arisen, such as wildlife habit monitoring. For these IoT applications, how to efficiently backhaul data collected by IoT devices has become a key technology. As a new data collecting mode for environments lacking communication infrastructure, UAV-enabled data offloading has attracted increasing research interest. Nevertheless, the fairness of data offloading is rarely studied, which directly influences the quality of service. Therefore, we investigate the fairness-aware task scheduling for data offloading enabled by heterogeneous UAVs. To reduce the uncertainty of user status information caused by poor communication, a service time related data arrival rate estimation model is proposed. The fairness-aware heterogeneous UAV-enabled data offloading model is proposed to ensure that all users share system resources fairly. Then, we creatively propose a dynamic incentive-based coordinate descent optimization framework-DISCO, which can progressively improve resource utilization while maximizing service fairness. Besides, we theoretically analyze and experimentally demonstrate its convergence and low time complexity. The performance of DISCO is verified by a series of simulation experiments. Compared with four baseline algorithms, DISCO shows distinct advantages in terms of service fairness, energy efficiency, adaptability and data throughput.

Simulation and Majorization of Multi-source Relatively Complemented Heating System Based on TRNSYS

In order to make up for the shortcomings of solar energy, ground source heat pump as well as air source heat pump during heating, a new multi-complementary heating system with three heat sources is proposed. Take the Tumeteyouqi item as the object of study for program devise and load simulation on the basis of the set scheme, and TRNSYS hybrid heating system is established. The main parameters of the air source heat pump and the solar collector are optimized, and the hooke-Jeewes and Coordinate descent optimization algorithms are used to optimize the annual value of system costs, and the two algorithms are compared. Results are included the Collector area of 80m2, the Collector of 42.1875 ° inclination, Azimuth Angle of 10°. The unit area is homologous and the volume of the water box is 0.10125. The capacity of the water tank is 8.1 m3, and the heat quantity of the air source heat pump is 286200kj/h. The system reaches the optimum, and the annual value of the heating cost is reduced by 3.8%. Then the supply heating cost is reduced by 4.96%. The Hooke-Jeewes algorithm is superior to the Coordinate descent algorithm.

Joint Transmit Power and Trajectory Optimization for Two-Way Multihop UAV Relaying Networks

Unmanned aerial vehicle (UAV) has been more and more widely used in military and civilian, with its unique advantages of flexibility, convenience, and wide coverage. As flying stations, UAVs can quickly set up relay communication links for different missions, to enhance the receiving signal power, increase the system capacity, and expand the communication coverage. In this article, we investigate a two-way multihop UAV relaying network, where there are two ground users as sources and multiple UAVs as relays to help the two ground sources exchange information. For the purpose of enhancing the efficiency of the investigated UAV-assisted relaying, we come up with a productive two-way multihop UAV relaying pattern, which can achieve a data rate of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$({1}/{2})$ </tex-math></inline-formula> data packets per time slot with the decode-and-forward protocol. Then, we further formulate a joint transmit power and trajectory optimization problem for the UAVs in this two-way multihop relaying scenario. The formulated problem is nonconvex which makes it difficult to solve directly; hence, we propose an iterative algorithm to obtain an approximate optimal solution based on block coordinate descent and successive convex optimization techniques. Numerical results demonstrate that our proposed two-way multihop UAV relaying network achieves significant throughput gains compared with other benchmark schemes.

Intelligent Omni Surface-Assisted Secure MIMO Communication Networks With Artificial Noise

Intelligent omni surface (IOS) technology that can provide both reflection and refraction for signals impinging on its surface, was recently proposed to overcome the coverage area limitation of reconfigurable intelligent surfaces (RISs). This letter investigates the efficacy of IOS in the context of physical layer security (PLS) over wireless channels. In particular, we consider a scenario where the IOS is utilized to enhance the secrecy performance of a legitimate receiver in the presence of a multi-antenna eavesdropper. In addition, artificial noise (AN) aided beamforming is implemented to provide additional security robustness. The resultant optimization problem is non-convex and difficult to solve. Accordingly, the block coordinate descent (BCD) optimization approach is adopted, and the Lagrangian dual method is applied to reduce the complexity of the AN-aided beamforming design. Furthermore, the reflecting and refracting phase shifts are optimized via the quadratically constrained quadratic programming (QCQP) method. Simulation results validate the efficacy of the proposed algorithm and confirm the superiority of IOS over traditional RIS.

Variable Selection with Multiply-Imputed Datasets: Choosing Between Stacked and Grouped Methods

Penalized regression methods are used in many biomedical applications for variable selection and simultaneous coefficient estimation. However, missing data complicates the implementation of these methods, particularly when missingness is handled using multiple imputation. Applying a variable selection algorithm on each imputed dataset will likely lead to different sets of selected predictors. This article considers a general class of penalized objective functions which, by construction, force selection of the same variables across imputed datasets. By pooling objective functions across imputations, optimization is then performed jointly over all imputed datasets rather than separately for each dataset. We consider two objective function formulations that exist in the literature, which we will refer to as “stacked” and “grouped” objective functions. Building on existing work, we (i) derive and implement efficient cyclic coordinate descent and majorization-minimization optimization algorithms for continuous and binary outcome data, (ii) incorporate adaptive shrinkage penalties, (iii) compare these methods through simulation, and (iv) develop an R package miselect. Simulations demonstrate that the “stacked” approaches are more computationally efficient and have better estimation and selection properties. We apply these methods to data from the University of Michigan ALS Patients Biorepository aiming to identify the association between environmental pollutants and ALS risk. Supplementary materials for this article are available online.

Joint Design of Fronthaul and Access Links in Massive MIMO Multi-UAV-Enabled CRANs

This letter proposes a novel architecture of multi-unmanned aerial vehicle (UAV)-enabled cloud radio access network (CRAN). In particular, we propose to deploy the UAVs as flying remote radio heads (RRHs) to serve ground user equipments (UEs). The baseband unit (BBU) is equipped with a large-scale antenna array to serve the flying RRHs and affords all the baseband signal processing. To optimize the proposed architecture, we consider the maximization of the minimum rate of UEs by jointly optimizing UAVs placement, quantization noise variance, and power control. The corresponding optimization problem is not convex and to solve it we devise an efficient iterative algorithm combining the block coordinate descent and successive convex optimization methods. Numerical results demonstrate the superior performance of the proposed algorithm compared to two benchmark schemes.

Two-dimensional variable selection and its applications in the diagnostics of product quality defects

The root cause diagnostics of product quality defects in multistage manufacturing processes often requires a joint identification of crucial stages and process variables. To meet this requirement, this article proposes a novel penalized matrix regression methodology for two-dimensional variable selection. The method regresses a scalar response variable against a matrix-based predictor using a generalized linear model. The unknown regression coefficient matrix is decomposed as a product of two factor matrices. The rows of the first factor matrix and the columns of the second factor matrix are simultaneously penalized to inspire sparsity. To estimate the parameters, we develop a Block Coordinate Proximal Descent (BCPD) optimization algorithm, which cyclically solves two convex sub-optimization problems. We have proved that the BCPD algorithm always converges to a critical point with any initialization. In addition, we have also proved that each of the sub-optimization problems has a closed-form solution if the response variable follows a distribution whose (negative) log-likelihood function has a Lipschitz continuous gradient. A simulation study and a dataset from a real-world application are used to validate the effectiveness of the proposed method.

3D Location and Resource Allocation Optimization for UAV-Enabled Emergency Networks Under Statistical QoS Constraint

Unmanned aerial vehicles (UAVs) as aerial base stations have attracted great attention in emergency communication networks due to flexible deployment. With the popularization of smart devices, the demand for multimedia services is increasing in disaster relief. Therefore, it is very important to significantly improve throughput of unmanned aerial vehicle base station (UAV-BS) while ensuring quality-of-service (QoS) of multimedia traffic. In this paper, we consider a UAV-BS to serve a group of users in the downlink who have different statistical delay-bound QoS requirements in an emergency situation. We formulate a problem to maximize the sum statistical-QoS-guaranteed throughput (effective capacity) of all users by jointly optimizing the UAV's 3D location, power and bandwidth allocation under each user's statistical QoS requirement constraint. The formulated problem is a non-linear non-convex optimization problem, which is very difficult to solve. To this end, we propose an efficient iteration algorithm based on the block coordinate descent and successive convex optimization techniques to solve it. Specifically, we decouple the primary problem into three sub-problems, which can be approximated as easy-to-solve convex optimization problems. In each iteration, three sub-problems are alternately optimized. Finally, numerical simulation results prove the effectiveness of our proposed algorithm compared with benchmarks.

Joint trajectory and power optimization for securing UAV communications against active eavesdropping

Due to both of jamming and eavesdropping, active eavesdroppers can induce more serious security threats to unmanned aerial vehicle (UAV)-enabled communications. This paper considers a secure UAV communication system including both the downlink (DL) and uplink (UL) transmissions, where the confidential information is transmitted between a UAV and a ground node in the presence of an active eavesdropper. We aim to maximize the average secrecy rates of the DL and UL communications, respectively, by jointly optimizing the UAV trajectory and the UAV/ground node's transmit power control over a given flight period. Due to the non-convexity of the formulated problems, it is difficult to obtain globally optimal solutions. However, we propose efficient iterative algorithms to obtain high-quality suboptimal solutions by applying the block coordinate descent and successive convex optimization methods. Simulation results show that the joint optimization algorithms can effectively improve the secrecy rate performance for both the DL and UL communications, as compared with other baseline schemes. The proposed schemes can be considered as special cases of UAV-assisted non-orthogonal multiple access (NOMA) networks.

Mission-Critical Resource Allocation With Puncturing in Industrial Wireless Networks Under Mixed Services

As the manufacturing industry gradually expands and tends to be intelligent, the industrial networks have been greatly applied and are of paramount importance. Furthermore, taking the advantages of wireless networks into account, such as flexible deployment, low cost, and easy maintenance, etc., this paper is dedicated to the research of industrial wireless networks. Facing the problem of limited radio resource, in order to make full use of frequency band, this paper proposes a resource scheduling scheme with puncturing, which guarantees the reliability of subsequent ultra-reliable and low-latency communications (URLLC) services under the conditions of existing enhanced mobile broadband (eMBB) services. By optimizing the bandwidth pre-allocation for eMBB services, puncture weight and transmit power, this scheme can effectively meet the needs of the system. Specifically, the purpose of this paper is to minimize the decoding error rate of the devices carrying URLLC service while ensuring the demand for throughput of eMBB services. The problem proposed is a nonlinear stochastic optimization problem involving complex Q-function. The closed-form expression of the objective function is obtained through linear approximation and probability theory, which is further decoupled into three sub-problems. To this end, a block coordinate descent optimization (BCDO) algorithm is proposed to obtain the optimal bandwidth allocation, puncture weight and transmit power. The simulation results verify the rationality of the theoretical analysis and the effectiveness of the proposed algorithm, and also elaborate the influence of different parameters on the decoding error rate of the URLLC devices.

Improving Localization of Cardiac Geometry Using ECGI.

Electrocardiographic imaging (ECGI) requires a model of the torso, and inaccuracy in the position of the heart is a known source of error. We previously presented a method to localize the heart when body and heart surface potentials are known. The goal of this study is to extend this approach to only use body surface potentials. We used an iterative coordinate descent optimization to estimate the positions of the heart for several consecutive heartbeats relying on the assumption that the epicardial potential sequence is the same in each beat. The method was tested with data synthesized using measurements from a isolated-heart, torso-tank preparation. Improvement was evaluated in terms of both heart localization and ECGI accuracy. The geometric correction resulted in cardiac geometries closely matching ground truth geometry. ECGI accuracy increased dramatically by all metrics using the corrected geometry. Future studies will employ more realistic animal models and then human subjects. Success could impact clinical ECGI by reducing errors from respiratory movement and perhaps decrease imaging requirements, reducing both cost and logistical difficulty of ECGI, widening clinical applicability.

ADDRESSING IMBALANCED INSURANCE DATA THROUGH ZERO-INFLATED POISSON REGRESSION WITH BOOSTING

AbstractA machine learning approach to zero-inflated Poisson (ZIP) regression is introduced to address common difficulty arising from imbalanced financial data. The suggested ZIP can be interpreted as an adaptive weight adjustment procedure that removes the need for post-modeling re-calibration and results in a substantial enhancement of predictive accuracy. Notwithstanding the increased complexity due to the expanded parameter set, we utilize a cyclic coordinate descent optimization to implement the ZIP regression, with adjustments made to address saddle points. We also study how various approaches alleviate the potential drawbacks of incomplete exposures in insurance applications. The procedure is tested on real-life data. We demonstrate a significant improvement in performance relative to other popular alternatives, which justifies our modeling techniques.

3D Placement for Multi-UAV Relaying: An Iterative Gibbs-Sampling and Block Coordinate Descent Optimization Approach

In this paper, we consider an unmanned aerial vehicle (UAV) enabled relaying system where multiple UAVs are deployed as aerial relays to support simultaneous communications from a set of source nodes to their destination nodes on the ground. An optimization problem is formulated under practical channel models to maximize the minimum achievable expected rate among all pairs of ground nodes by jointly designing UAVs' three-dimensional (3D) placement as well as the bandwidth-and-power allocation. This problem, however, is non-convex and thus difficult to solve. As such, we propose a new method, called iterative Gibbs-sampling and block- coordinate-descent (IGS-BCD), to efficiently obtain a high-quality suboptimal solution by synergizing the advantages of both the deterministic (BCD) and stochastic (GS) optimization methods. Specifically, our proposed method alternates between two optimization phases until convergence is reached, namely, one phase that uses the BCD method to find locally-optimal UAVs' 3D placement and the other phase that leverages the GS method to generate new UAVs' 3D placement for exploration. Moreover, we present an efficient method for properly initializing UAVs' placement that leads to faster convergence of the proposed IGS-BCD algorithm. Numerical results show that the proposed IGS-BCD and initialization methods outperform the conventional BCD or GS method alone in terms of convergence-and-performance trade-off, as well as other benchmark schemes.

Full-Duplex UAV Relaying for Multiple User Pairs

Based on the advantages of small size, lightweight, as well as flexible deployment and recycling, unmanned aerial vehicle (UAV) has been more and more widely used in military and civilian. As flying relays, UAVs can quickly set up relay communication links for different missions, to enhance the receiving signal power, increase the system capacity, and expand the communication coverage. In this article, we investigate full-duplex (FD) UAV relaying for multiple source–destination pairs. To fully exploit the flying flexibility of the UAV in serving multiple source–destination pairs, we propose a scheduling protocol that exploits time-division multiple access (TDMA) to serve different source–destination pairs in turns when flying along an optimized trajectory. Then, we further formulate a joint optimization problem of the TDMA-based user scheduling, the dynamic UAV trajectory, and the UAV transmit power to maximize the system throughput. The formulated problem is nonconvex that makes it difficult to solve directly, hence we propose an iterative algorithm to obtain an approximate optimal solution based on block coordinate descent and successive convex optimization techniques. Simulation results demonstrate that our proposed FD-based UAV relaying network achieves significant throughput gains compared with the half-duplex (HD) baseline, and the TDMA-based protocol outperforms the OFDMA-based ones with fixed UAV position/trajectory when the UAV helps relay information for multiple source–destination pairs.

Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms

In several scientific and industrial applications, it is desirable to build compact, interpretable learning models where the output depends on a small number of input features. Recent work has shown that such best-subset selection-type problems can be solved with modern mixed integer optimization solvers. Despite their promise, such solvers often come at a steep computational price when compared with open-source, efficient specialized solvers based on convex optimization and greedy heuristics. In “Fast Best-Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms,” Hussein Hazimeh and Rahul Mazumder push the frontiers of computation for best-subset-type problems. Their algorithms deliver near-optimal solutions for problems with up to a million features—in times comparable with the fast convex solvers. Their work suggests that principled optimization methods play a key role in devising tools central to interpretable machine learning, which can help in gaining a deeper understanding of their statistical properties.

Coordinate descent optimization for one-to-one correspondence and supervised classification of 3D shapes

Recent developments in shape analysis and retrieval play an important role in a wide variety of applications that potentially require matching of objects with different geometries. In shape classification, there is no natural way to represent an object but the similarity measure, distance between representations or descriptors, depends heavily on the strategy of computing optimal correspondences. In this paper we introduce a new numerical method for registering surfaces. Thus, finding an optimal one-to-one correspondence between their shapes. Unfortunately, solving this type of optimization problem is generally hard because the solutions space is nonlinear with no natural manifold structure on it. To overcome such limitations we make use of recent methods to represent objects then we find optimal correspondences using a discretized approximation of the search space. The proposed method has the advantage of extending the Riemannian analysis of 3D curves in a natural way for surfaces. We demonstrate the proposed algorithms using different public datasets for 2D and 3D objects matching and classification.

Optimizing the Estimation of a Histogram-Bin Width—Application to the Multivariate Mixture-Model Estimation

A maximum-likelihood estimation of a multivariate mixture model’s parameters is a difficult problem. One approach is to combine the REBMIX and EM algorithms. However, the REBMIX algorithm requires the use of histogram estimation, which is the most rudimentary approach to an empirical density estimation and has many drawbacks. Nevertheless, because of its simplicity, it is still one of the most commonly used techniques. The main problem is to estimate the optimum histogram-bin width, which is usually set by the number of non-overlapping, regularly spaced bins. For univariate problems it is usually denoted by an integer value; i.e., the number of bins. However, for multivariate problems, in order to obtain a histogram estimation, a regular grid must be formed. Thus, to obtain the optimum histogram estimation, an integer-optimization problem must be solved. The aim is therefore the estimation of optimum histogram binning, alone and in application to the mixture model parameter estimation with the REBMIX&amp;EM strategy. As an estimator, the Knuth rule was used. For the optimization algorithm, a derivative based on the coordinate-descent optimization was composed. These proposals yielded promising results. The optimization algorithm was efficient and the results were accurate. When applied to the multivariate, Gaussian-mixture-model parameter estimation, the results were competitive. All the improvements were implemented in the rebmix R package.