Coordinate Descent Research Articles

A safe screening rule with bi-level optimization of [formula omitted] support vector machine

Support vector machine (SVM) has achieved many successes in machine learning, especially for a small sample problem. As a famous extension of the traditional SVM, the ν support vector machine (ν-SVM) has shown outstanding performance due to its great model interpretability. However, it still faces challenges in training overhead for large-scale problems. To address this issue, we propose a safe screening rule with bi-level optimization for ν-SVM (SRBO-ν-SVM) which can screen out inactive samples before training and reduce the computational cost without sacrificing the prediction accuracy. Our SRBO-ν-SVM is strictly deduced by integrating the Karush–Kuhn–Tucker (KKT) conditions, the variational inequalities of convex problems and the ν-property. Furthermore, we develop an efficient dual coordinate descent method (DCDM) to further improve computational speed. Finally, a unified framework for SRBO is proposed to accelerate many SVM-type models, and it is successfully applied to one-class SVM. Experimental results on 6 artificial data sets and 30 benchmark data sets have verified the effectiveness and safety of our proposed methods in supervised and unsupervised tasks.

Spreading code optimization for low-earth orbit satellites via mixed-integer convex programming

Optimizing the correlation properties of spreading codes is critical for minimizing inter-channel interference in satellite navigation systems. By improving the codes’ correlation sidelobes, we can enhance navigation performance while minimizing the required spreading code lengths. In the case of low-earth orbit (LEO) satellite navigation, shorter code lengths (on the order of a hundred) are preferred due to their ability to achieve fast signal acquisition. Additionally, the relatively high signal-to-noise ratio in LEO systems reduces the need for longer spreading codes to mitigate inter-channel interference. In this work, we propose a two-stage block coordinate descent (BCD) method which optimizes the codes’ correlation properties while enforcing the autocorrelation sidelobe zero property. In each iteration of the BCD method, we solve a mixed-integer convex program over a block of 25 binary variables. Our method is applicable to spreading code families of arbitrary sizes and lengths, and we demonstrate its effectiveness for a problem with 66 length-127 codes and a problem with 130 length-257 codes.

Coordinate descent algorithm for generalized group fused Lasso

AbstractWe deal with a model with discrete varying coefficients to consider modeling for heterogeneity and clustering for homogeneity, and estimate the varying coefficients by generalized group fused Lasso (GGFL). GGFL allows homogeneous groups to be joined together based on one-to-many relationships among groups. This makes GGFL a powerful technique, but to date there has been no effective algorithm for obtaining the solutions. Here we propose an algorithm for obtaining a GGFL solution based on the coordinate descent method, and show that a solution for each coordinate direction converges to the optimal solution. In a simulation, we show our algorithm is superior to ADMM, which is one of the popular algorithms. We also present an application to a spatial data analysis.

Generalized fused Lasso for grouped data in generalized linear models

Generalized fused Lasso (GFL) is a powerful method based on adjacent relationships or the network structure of data. It is used in a number of research areas, including clustering, discrete smoothing, and spatio-temporal analysis. When applying GFL, the specific optimization method used is an important issue. In generalized linear models, efficient algorithms based on the coordinate descent method have been developed for trend filtering under the binomial and Poisson distributions. However, to apply GFL to other distributions, such as the negative binomial distribution, which is used to deal with overdispersion in the Poisson distribution, or the gamma and inverse Gaussian distributions, which are used for positive continuous data, an algorithm for each individual distribution must be developed. To unify GFL for distributions in the exponential family, this paper proposes a coordinate descent algorithm for generalized linear models. To illustrate the method, a real data example of spatio-temporal analysis is provided.

Iterative reconstruction methods and the resolution principle for fast-ion loss detector measurements

Fast-ion loss detectors (FILDs) are crucial for analyzing fast-ion dynamics in magnetically confined fusion plasmas. A core challenge is to derive an accurate ion velocity distribution, requiring treatment of thousands of remapped camera frames for a full discharge. The ill-posed nature of this task necessitates regularization with a well-chosen regularization parameter and computationally efficient methods. In this work, we introduce the ‘resolution principle,’ a novel criterion for selecting the optimal regularization parameter, providing a distinction between genuine features and artefacts smaller than the diagnostic resolution in the reconstruction, thereby preventing misinterpretations. This principle, coupled with three iterative reconstruction techniques—Kaczmarz’s method, coordinate descent, and Cimmino’s method—demonstrates enhanced reconstruction capabilities compared to conventional methods like Tikhonov regularization. Utilizing these techniques allows rapid processing of measurements from full discharges, removing the computational bottleneck and facilitating between-discharge reconstructions. By reconstructing 6000 camera frames from an ELMy H-mode discharge at ASDEX Upgrade, we capture the temporal evolution of gyroradii and pitch angles, unveiling a direct correlation between pitch-angle behavior and changes in the toroidal magnetic field for a specific subset of lost ions accelerated by edge-localized modes (ELMs) to energies approximately twice that of the injection energy.

The Droplet Search Algorithm for Kernel Scheduling

Kernel scheduling is the problem of finding the most efficient implementation for a computational kernel. Identifying this implementation involves experimenting with the parameters of compiler optimizations, such as the size of tiling windows and unrolling factors. This article shows that it is possible to organize these parameters as points in a coordinate space. The function that maps these points to the running time of kernels, in general, will not determine a convex surface. However, this article provides empirical evidence that the origin of this surface (an unoptimized kernel) and its global optimum (the fastest kernel) reside on a convex region. We call this hypothesis the “droplet expectation.” Consequently, a search method based on the Coordinate Descent algorithm tends to find the optimal kernel configuration quickly if the hypothesis holds. This approach—called Droplet Search—has been available in Apache TVM since April of 2023. Experimental results with six large deep learning models on various computing devices (ARM, Intel, AMD, and NVIDIA) indicate that Droplet Search is not only as effective as other AutoTVM search techniques but also 2 to 10 times faster. Moreover, models generated by Droplet Search are competitive with those produced by TVM’s AutoScheduler (Ansor), despite the latter using 4 to 5 times more code transformations than AutoTVM.

Riemannian Preconditioned Coordinate Descent for Low Multilinear Rank Approximation

Riemannian Preconditioned Coordinate Descent for Low Multilinear Rank Approximation

Joint Transmit and Receive Beamforming Design for DPC-Based MIMO DFRC Systems

This paper proposes an optimal beamforming strategy for a downlink multi-user multi-input–multi-output (MIMO) dual-function radar communication (DFRC) system with dirty paper coding (DPC) adopted at the transmitter. We aim to achieve the maximum weighted sum rate of communicating users while adhering to a predetermined transmit covariance constraint for radar performance assurance. To make the intended problem trackable, we leverage the equivalence of the weighted sum rate and the weighted minimum mean squared error (MMSE) to reframe the issue and devise a block coordinate descent (BCD) approach to iteratively calculate transmit and receive beamforming solutions. Through this methodology, we demonstrate that the optimal receive beamforming aligns with the traditional MMSE approach, whereas the optimal transmit beamforming design can be cast into a quadratic optimization problem defined on a complex Stiefel manifold. Based on the majorization–minimization (MM) method, an iterative algorithm is then developed to compute the optimal transmit beamforming design by solving a series of orthogonal Procrustes problems (OPPs) that admit closed-form optimal solutions. Numerical findings serve to validate the efficacy of our scheme. It is demonstrated that our approach can achieve at least 73% higher spectral efficiency than the existing methods in a high signal-to-noise ratio (SNR) regime.

Accelerating regularized tensor decomposition using the alternating direction method of multipliers with multiple Nesterov’s extrapolations

Tensor decomposition is an essential tool for multiway signal processing. At present, large-scale high-order tensor data require fast and efficient decomposing algorithms. In this paper, we propose accelerated regularized tensor decomposition algorithms using the alternating direction method of multipliers with multiple Nesterov’s extrapolations in the block coordinate descent framework. We implement the acceleration in three cases: only in the inner loop, only in the outer loop, and in both the inner and outer loops. Adaptive safeguard strategies are developed following the acceleration to guarantee monotonic convergence. Afterwards, we utilize the proposed algorithms to accelerate two types of conventional decomposition: nonnegative CANDECOMP/PARAFAC (NCP) and sparse CANDECOMP/PARAFAC (SCP). The experimental results on synthetic and real-world tensors demonstrate that the proposed algorithms achieve significant acceleration effects and outperform state-of-the-art algorithms. The accelerated algorithm with extrapolations in both the inner and outer loops has the fastest convergence speed and takes almost one-third of the running time of typical algorithms.

Establishment of a mortality risk nomogram for predicting in-hospital mortality of sepsis: cohort study from a Chinese single center.

To establish a mortality risk nomogram for predicting in-hospital mortality of sepsis patients in the Chinese population. Data were obtained from the medical records of sepsis patients enrolled at the Affiliated Huadu Hospital, Southern Medical University, between 2019 and 2021. A total of 696 sepsis patients were initially included in our research, and 582 cases were finally enrolled after screening and divided into the survival group (n = 400) and the non-survival group (n = 182) according to the incidence of mortality during hospitalization. Twenty-eight potential sepsis-related risk factors for mortality were identified. Least absolute shrinkage and selection operator (LASSO) regression was used to optimize variable selection by running cyclic coordinate descent with k-fold (tenfold in this case) cross-validation. We used binary logistic regression to build a model for predicting mortality from the variables based on LASSO regression selection. Binary logistic regression was used to establish a nomogram based on independent mortality risk factors. To validate the prediction accuracy of the nomogram, receiver operating characteristic curve (ROC) analysis, decision curve analysis (DCA) and restricted cubic spline (RCS) analysis were employed. Eventually, the Hosmer-Lemeshow test and calibration curve were used for nomogram calibration. LASSO regression identified a total of ten factors, namely, chronic heart disease (CHD), lymphocyte count (LYMP), neutrophil-lymphocyte ratio (NLR), red blood cell distribution width (RDW), C reactive protein (CRP), Procalcitonin (PCT), lactic acid, prothrombin time (PT), alanine aminotransferase (ALT), total bilirubin (Tbil), interleukin-6 (IL6), that were incorporated into the multivariable analysis. Finally, a nomogram including CHD, LYMP, NLR, RDW, lactic acid, PT, CRP, PCT, Tbil, ALT, and IL6 was established by multivariable logistic regression. The ROC curves of the nomogram in the training and validation sets were 0.9836 and 0.9502, respectively. DCA showed that the nomogram could be applied clinically if the risk threshold was between 29.52 and 99.61% in the training set and between 31.32 and 98.49% in the testing set. RCS showed that when the value of independent risk factors from the predicted model exceeded the median, the mortality hazard ratio increased sharply. The results of the Hosmer-Lemeshow test (χ2 = 0.1901, df = 2, p = 0.9091) and the calibration curves of the training and validation sets showed good agreement with the actual results, which indicated good stability of the model. Our nomogram, including CHD, LYMP, NLR, RDW, lactic acid, PT, CRP, PCT, Tbil, ALT, and IL6, exhibits good performance for predicting mortality risk in adult sepsis patients.

Enhanced multi-model multi-scan data association and tracking algorithm via convex variational inference

Tracking multiple targets with unknown measurement-to-target association and uncertain target dynamics is a significant problem that arises in various applications such as surveillance monitoring and intelligent transportation systems. In this paper, we propose an enhanced multi-model multi-scan data association algorithm to address the problem of tracking multiple maneuvering targets. First, we use a probabilistic graphical model to represent the joint distribution of the dynamic model indices, target state, and multi-scan data association variables. This formulation transforms the inference of marginal distributions into a Bethe free energy (BFE) problem. Next, to transform the BFE problem into a convex one, we demonstrate that the BFE function can be made convex through re-weighting. Additionally, we decompose the re-weighted BFE function into a block-wise sum form. We prove that under certain regularization conditions, each block of the re-weighted BFE is convex, ensuring convergence of the primal–dual coordinate ascent algorithm to the minimum of the overall re-weighted BFE. Finally, we provide a particle implementation of the proposed algorithm, accompanied by an analysis of its complexity. Simulation results indicate that the proposed algorithm exhibits favorable performance when compared to both the single-model multi-scan algorithm and the multi-model single-scan algorithm.

Singular Value Decomposition-Based Penalized Multinomial Regression for Classifying Imbalanced Medulloblastoma Subgroups Using Methylation Data.

Medulloblastoma (MB) is a molecularly heterogeneous brain malignancy with large differences in clinical presentation. According to genomic studies, there are at least four distinct molecular subgroups of MB: sonic hedgehog (SHH), wingless/INT (WNT), Group 3, and Group 4. The treatment and outcomes depend on appropriate classification. It is difficult for the classification algorithms to identify these subgroups from an imbalanced MB genomic data set, where the distribution of samples among the MB subgroups may not be equal. To overcome this problem, we used singular value decomposition (SVD) and group lasso techniques to find DNA methylation probe features that maximize the separation between the different imbalanced MB subgroups. We used multinomial regression as a classification method to classify the four different molecular subgroups of MB using the reduced DNA methylation data. Coordinate descent is used to solve our loss function associated with the group lasso, which promotes sparsity. By using SVD, we were able to reduce the 321,174 probe features to just 200 features. Less than 40 features were successfully selected after applying the group lasso, which we then used as predictors for our classification models. Our proposed method achieved an average overall accuracy of 99% based on fivefold cross-validation technique. Our approach produces improved classification performance compared with the state-of-the-art methods for classifying MB molecular subgroups.

Establishment of a Risk Prediction Model for Metabolic Syndrome in High Altitude Areas in Qinghai Province, China: A Cross-Sectional Study.

The prevalence of metabolic syndrome (MetS) is increasing worldwide, and early prediction of MetS risk is highly beneficial for health outcomes. This study aimed to develop and validate a nomogram to predict MetS risk in Qinghai Province, China, and it provides a methodological reference for MetS prevention and control in Qinghai Province, China. A total of 3073 participants living between 1900 and 3710 meters above sea level in Qinghai Province participated in this study between March 2014 and March 2016. We omitted 12 subjects who were missing diagnostic component data for MetS, ultimately resulting in 3061 research subjects, 70% of the subjects were assigned randomly to the training set, and the remaining subjects were assigned to the validation set. The least absolute shrinkage and selection operator (LASSO) regression analysis method was used for variable selection via running cyclic coordinate descent with 10-fold cross-validation. Multivariable logistic regression was then performed to develop a predictive model and nomogram. The receiver operating characteristic (ROC) curves was used for model evaluation, and calibration plot and decision curve analysis (DCA) were used for model validation. Of 24 variables studied, 6 risk predictors were identified by LASSO regression analysis: hyperlipidaemia, hyperglycemia, abdominal obesity, systolic blood pressure (SBP), diastolic blood pressure (DBP), and body mass index (BMI). A prediction model including these 6 risk factors was constructed and displayed good predictability with an area under the ROC curve of 0.914 for the training set and 0.930 for the validation set. DCA revealed that if the threshold probability of MetS is less than 82%, the application of this nomogram is more beneficial than both the treat-all or treat-none strategies. The nomogram developed in our study demonstrated strong discriminative power and clinical applicability, making it a valuable reference for meets prevention and control in the plateau areas of Qinghai Province.

An Optimal Choice of Characteristic Polynomial Roots for Pole Placement Control Design

The problem of finding the arrangement of closed-loop control system poles that minimizes an objective function is considered. The system optimality criterion is the value of the H∞ norm of the frequency transfer function relative to the disturbance with constraints imposed on the system pole placement and the values of the H∞ norm of the sensitivity function and the transfer function from measurement noise to control. An optimization problem is formulated as follows: the vector of variables consists of the characteristic polynomial roots of the closed loop system with the admissible values restricted to a given pole placement region; in addition to the optimality criterion, the objective function includes penalty elements for other constraints. It is proposed to use a logarithmic scale for the moduli of the characteristic polynomial roots as elements of the vector of variables. The multi-extremality problem of the objective function is solved using the multiple start procedure. A coordinate descent modification with a pair of coordinates varied simultaneously is used for search.

Vehicle Selection and Resource Allocation for Federated Learning-Assisted Vehicular Network

To exploit the massive amounts of onboard data in vehicular networks while protecting data privacy and security, federated learning (FL) is regarded as a promising technology to support enormous vehicular applications. Despite that FL has great potential to improve the architecture of intelligent vehicular networks, the mobility of the vehicles and the dynamic nature of wireless channels make the integration of FL and vehicular networks more challenging. In this paper, we propose a vehicle mobility- and channel dynamic-aware FL (MADCA-FL) scheme to fit vehicular networks and enhance learning performances. This novel scheme enables the RSU to select appropriate vehicles and weightedly average the local models. Afterward, MADCA-FL formulates a problem to maximize the model accuracy while assuring the latency and energy restrictions, by jointly optimizing the computation and communication resources. With a mixed- integer non-linear programming structure, the problem is NP-hard. Firstly, we utilize the successive convex approximation algorithm to handle the non-convexity, and then apply the Lagrange multiplier method and the block coordinate descent method to obtain the optimal solution. Extensive experiments are conducted to confirm the effectiveness of our proposed scheme.

Pruning Deep Neural Network Models via Minimax Concave Penalty Regression

In this study, we propose a filter pruning method based on MCP (Minimax Concave Penalty) regression. The convolutional process is conceptualized as a linear regression procedure, and the regression coefficients serve as indicators to assess the redundancy of channels. In the realm of feature selection, the efficacy of sparse penalized regression gradually outperforms that of Lasso regression. Building upon this insight, MCP regression is introduced to screen convolutional channels, coupled with the coordinate descent method, to effectuate model compression. In single-layer pruning and global pruning analyses, the Top1 loss value associated with the MCP regression compression method is consistently smaller than that of the Lasso regression compression method across diverse models. Specifically, when the global pruning ratio is set to 0.3, the Top1 accuracy of the MCP regression compression method, in comparison with that of the Lasso regression compression method, exhibits improvements of 0.21% and 1.67% under the VGG19_Simple and VGG19 models, respectively. Similarly, for ResNet34, at two distinct pruning ratios, the Top1 accuracy demonstrates enhancements of 0.33% and 0.26%. Lastly, we compare and discuss the novel methods introduced in this study, considering both time and space resource consumption.

Feature selection by Universum embedding

Feature selection in classification is an important task in machine learning. Inspired by the success of Universum support vector machine proposed by Weston et al. on improving the classification ability of classical support vector machine, this paper considers a special type of Universum and further lets it play its role in both useful feature identification and separating hyperplane construction, aiming to improve both the feature selection ability and classification performance of Universum support vector machine. By introducing this special Universum, a redundant feature can be identified by observing whether some Universum sample is useful. In fact, we prove that by observing the dual solution of the optimization problem, useful features can be selected from a set satisfying some properties. Due to the introduction of these extra Universum samples, it needs to cope with a large-scale optimization problem. To improve the training efficiency, we modify the sequential minimal optimization algorithm and further combine it with the coordinate descent technique to solve the proposed model. Experimental results on artificial datasets, benchmark datasets, and text classification datasets demonstrate that the proposed method improves the classification performance of support vector machine and Universum support vector machine, and also has good feature selection ability.

Impact Analysis and Compensation Methods of Frequency Synchronization Errors in Distributed Geosynchronous Synthetic Aperture Radar

Frequency synchronization error, as one of the inevitable technical challenges in distributed synthetic aperture radar (SAR), has different impacts on different SAR systems. Multi-monostatic SAR is a typical distributed configuration where frequency synchronization errors are tiny in distributed airborne and low earth orbit (LEO) SAR systems. However, due to the long time delay and long synthetic aperture time, the imaging performance of a multi-monostatic geosynchronous (GEO) SAR system is affected by frequency oscillator errors. In this paper, to investigate the frequency synchronization problem in this configuration, we firstly model the echo signals with the frequency synchronization errors, which can be divided into fixed frequency errors and random phase noise. Secondly, we talk about the impacts of the two kinds of errors on imaging performance. To solve the problem, we thirdly propose an autofocus back-projection (ABP) algorithm, which adopts the coordinate descent method and iteratively adjusts the phase error estimation until the image reaches its maximum sharpness. Based on the characteristics of the frequency synchronization errors, we further propose the Node ABP (NABP) algorithm, which greatly reduces the amount of storage and computation compared to the ABP algorithm. Finally, simulations are carried out to validate the effectiveness of the ABP and NABP algorithms.

Wasserstein distance regularized graph neural networks

Distribution shift widely exists in graph representation learning and often reduces model performance. This work investigates how to improve the performance of a graph neural network (GNN) in a single graph by controlling distribution shift between embedding spaces. Specifically, we provide an upper error-bound estimation, which quantitatively analyzes how distribution shift affects GNNs' performance in a single graph. Considering that there is no natural domain division in a single graph, we propose PW-GNN to simultaneously learn discriminative embedding and reduce distribution shift. PW-GNN measures distribution discrepancy using the distance between test embeddings and prototypes, and transfers minimizing distribution shift to minimizing the power of Wasserstein distance, which is introduced into GNNs as a regularizer. A series of theoretical analyses are carried out to demonstrate the effectiveness of PW-GNN. Besides, a low-complexity training algorithm is designed by exploring entropy-regularized strategy and block coordinate descent method. Extensive numerical experiments are conducted on different datasets with both biased and unbiased splits. We empirically test our model equipped with four backbone models. Results show that PW-GNN outperforms state-of-the-art baselines and mitigates up to 8% of negative effects off distribution shift on backbones.

Iterative Trajectory Planning and Resource Allocation for UAV-Assisted Emergency Communication with User Dynamics

The demand for air-to-ground communication has surged in recent years, underscoring the significance of unmanned aerial vehicles (UAVs) in enhancing mobile communication, particularly in emergency scenarios due to their deployment efficiency and flexibility. In situations such as emergency cases, UAVs can function as efficient temporary aerial base stations and enhance communication quality in instances where terrestrial base stations are incapacitated. Trajectory planning and resource allocation of UAVs continue to be vital techniques, while a relatively limited number of algorithms account for the dynamics of ground users. This paper focuses on emergency communication scenarios such as earthquakes, proposing an innovative path planning and resource allocation algorithm. The algorithm leverages a multi-stage subtask iteration approach, inspired by the block coordinate descent technique, to address the challenges presented in such critical environments. In this study, we establish an air-to-ground communication model, subsequently devising a strategy for user dynamics. This is followed by the introduction of a joint scheduling process for path and resource allocation, named ISATR (iterative scheduling algorithm of trajectory and resource). This process encompasses highly interdependent decision variables, such as location, bandwidth, and power resources. For mobile ground users, we employ the cellular automata (CA) method to forecast the evacuation trajectory. This algorithm successfully maintains data communication in the emergency-stricken area and enhances the communication quality through bandwidth division and power control which varies with time. The effectiveness of our algorithm is validated by evaluating the average throughput with different parameters in various simulation conditions and by using several heuristic methods as a contrast.