Coordinate Descent Method Research Articles

Spreading code optimization for low-earth orbit satellites via mixed-integer convex programming

Optimizing the correlation properties of spreading codes is critical for minimizing inter-channel interference in satellite navigation systems. By improving the codes’ correlation sidelobes, we can enhance navigation performance while minimizing the required spreading code lengths. In the case of low-earth orbit (LEO) satellite navigation, shorter code lengths (on the order of a hundred) are preferred due to their ability to achieve fast signal acquisition. Additionally, the relatively high signal-to-noise ratio in LEO systems reduces the need for longer spreading codes to mitigate inter-channel interference. In this work, we propose a two-stage block coordinate descent (BCD) method which optimizes the codes’ correlation properties while enforcing the autocorrelation sidelobe zero property. In each iteration of the BCD method, we solve a mixed-integer convex program over a block of 25 binary variables. Our method is applicable to spreading code families of arbitrary sizes and lengths, and we demonstrate its effectiveness for a problem with 66 length-127 codes and a problem with 130 length-257 codes.

Coordinate descent algorithm for generalized group fused Lasso

AbstractWe deal with a model with discrete varying coefficients to consider modeling for heterogeneity and clustering for homogeneity, and estimate the varying coefficients by generalized group fused Lasso (GGFL). GGFL allows homogeneous groups to be joined together based on one-to-many relationships among groups. This makes GGFL a powerful technique, but to date there has been no effective algorithm for obtaining the solutions. Here we propose an algorithm for obtaining a GGFL solution based on the coordinate descent method, and show that a solution for each coordinate direction converges to the optimal solution. In a simulation, we show our algorithm is superior to ADMM, which is one of the popular algorithms. We also present an application to a spatial data analysis.

Generalized fused Lasso for grouped data in generalized linear models

Generalized fused Lasso (GFL) is a powerful method based on adjacent relationships or the network structure of data. It is used in a number of research areas, including clustering, discrete smoothing, and spatio-temporal analysis. When applying GFL, the specific optimization method used is an important issue. In generalized linear models, efficient algorithms based on the coordinate descent method have been developed for trend filtering under the binomial and Poisson distributions. However, to apply GFL to other distributions, such as the negative binomial distribution, which is used to deal with overdispersion in the Poisson distribution, or the gamma and inverse Gaussian distributions, which are used for positive continuous data, an algorithm for each individual distribution must be developed. To unify GFL for distributions in the exponential family, this paper proposes a coordinate descent algorithm for generalized linear models. To illustrate the method, a real data example of spatio-temporal analysis is provided.

Vehicle Selection and Resource Allocation for Federated Learning-Assisted Vehicular Network

To exploit the massive amounts of onboard data in vehicular networks while protecting data privacy and security, federated learning (FL) is regarded as a promising technology to support enormous vehicular applications. Despite that FL has great potential to improve the architecture of intelligent vehicular networks, the mobility of the vehicles and the dynamic nature of wireless channels make the integration of FL and vehicular networks more challenging. In this paper, we propose a vehicle mobility- and channel dynamic-aware FL (MADCA-FL) scheme to fit vehicular networks and enhance learning performances. This novel scheme enables the RSU to select appropriate vehicles and weightedly average the local models. Afterward, MADCA-FL formulates a problem to maximize the model accuracy while assuring the latency and energy restrictions, by jointly optimizing the computation and communication resources. With a mixed- integer non-linear programming structure, the problem is NP-hard. Firstly, we utilize the successive convex approximation algorithm to handle the non-convexity, and then apply the Lagrange multiplier method and the block coordinate descent method to obtain the optimal solution. Extensive experiments are conducted to confirm the effectiveness of our proposed scheme.

Pruning Deep Neural Network Models via Minimax Concave Penalty Regression

In this study, we propose a filter pruning method based on MCP (Minimax Concave Penalty) regression. The convolutional process is conceptualized as a linear regression procedure, and the regression coefficients serve as indicators to assess the redundancy of channels. In the realm of feature selection, the efficacy of sparse penalized regression gradually outperforms that of Lasso regression. Building upon this insight, MCP regression is introduced to screen convolutional channels, coupled with the coordinate descent method, to effectuate model compression. In single-layer pruning and global pruning analyses, the Top1 loss value associated with the MCP regression compression method is consistently smaller than that of the Lasso regression compression method across diverse models. Specifically, when the global pruning ratio is set to 0.3, the Top1 accuracy of the MCP regression compression method, in comparison with that of the Lasso regression compression method, exhibits improvements of 0.21% and 1.67% under the VGG19_Simple and VGG19 models, respectively. Similarly, for ResNet34, at two distinct pruning ratios, the Top1 accuracy demonstrates enhancements of 0.33% and 0.26%. Lastly, we compare and discuss the novel methods introduced in this study, considering both time and space resource consumption.

Impact Analysis and Compensation Methods of Frequency Synchronization Errors in Distributed Geosynchronous Synthetic Aperture Radar

Frequency synchronization error, as one of the inevitable technical challenges in distributed synthetic aperture radar (SAR), has different impacts on different SAR systems. Multi-monostatic SAR is a typical distributed configuration where frequency synchronization errors are tiny in distributed airborne and low earth orbit (LEO) SAR systems. However, due to the long time delay and long synthetic aperture time, the imaging performance of a multi-monostatic geosynchronous (GEO) SAR system is affected by frequency oscillator errors. In this paper, to investigate the frequency synchronization problem in this configuration, we firstly model the echo signals with the frequency synchronization errors, which can be divided into fixed frequency errors and random phase noise. Secondly, we talk about the impacts of the two kinds of errors on imaging performance. To solve the problem, we thirdly propose an autofocus back-projection (ABP) algorithm, which adopts the coordinate descent method and iteratively adjusts the phase error estimation until the image reaches its maximum sharpness. Based on the characteristics of the frequency synchronization errors, we further propose the Node ABP (NABP) algorithm, which greatly reduces the amount of storage and computation compared to the ABP algorithm. Finally, simulations are carried out to validate the effectiveness of the ABP and NABP algorithms.

Wasserstein distance regularized graph neural networks

Distribution shift widely exists in graph representation learning and often reduces model performance. This work investigates how to improve the performance of a graph neural network (GNN) in a single graph by controlling distribution shift between embedding spaces. Specifically, we provide an upper error-bound estimation, which quantitatively analyzes how distribution shift affects GNNs' performance in a single graph. Considering that there is no natural domain division in a single graph, we propose PW-GNN to simultaneously learn discriminative embedding and reduce distribution shift. PW-GNN measures distribution discrepancy using the distance between test embeddings and prototypes, and transfers minimizing distribution shift to minimizing the power of Wasserstein distance, which is introduced into GNNs as a regularizer. A series of theoretical analyses are carried out to demonstrate the effectiveness of PW-GNN. Besides, a low-complexity training algorithm is designed by exploring entropy-regularized strategy and block coordinate descent method. Extensive numerical experiments are conducted on different datasets with both biased and unbiased splits. We empirically test our model equipped with four backbone models. Results show that PW-GNN outperforms state-of-the-art baselines and mitigates up to 8% of negative effects off distribution shift on backbones.

Robust condition identification against label noise in industrial processes based on trusted connection dictionary learning

In the era of big data, the pervasive use of artificial intelligence (AI) technology has revolutionized various industries. AI-powered systems, particularly those utilizing data-driven interpretable and reliable methods for accurate condition identification, have emerged as a crucial factor in achieving intelligent industrial systems. However, in real industrial sites, the labels of offline data often contain incorrect labels, i.e. label noise, due to human error, sensor failure, and other sources, which reduces the accuracy of the model and affects the performance and safety of the industrial system. To tackle the challenge of precarious performance caused by label noise, this paper proposes a robust method called Trusted Connection Dictionary Learning (TCDL) which effectively mitigates the adverse effects of label noise through interpretable constraints and guarantees the trustworthiness of the model for industrial applications. Specifically, to address the issue of data structure and label inconsistency arising from label noise, this paper presents a dynamic graph model that jointly optimizes data features and label information to obtain consistent representations of them. Subsequently, to effectively fit the labeled data, TCDL leverages the label information from the nearest neighbors through the optimized graph structure and facilitates the self-correction of label noise, leading to robust representations of process features and a reliable condition classifier. Finally, a novel iterative optimization algorithm that combines the Block Coordinate Descent (BCD) method and the Alternating Direction Multiplier Method (ADMM) is also developed to ensure the convergence of the algorithm. Due to simple constraints, it can obtain accurately optimized variables with few iterations. Numerical simulations and industrial experiments demonstrate the superiority of the proposed method compared to some state-of-the-art methods.

More communication-efficient distributed sparse learning

In a modern distributed learning framework, the speeds of intra-worker calculation and inter-worker communication may differ by 1,000 times. It is advisable to perform expensive calculations on worker machines and communicate as few rounds as possible. In this paper, we propose three novel distributed sparse learning algorithms (EDSL-ET, EDSL-IBCD, and EDSL-IBCD-ET) in high dimensions, which are efficient in both computation and communication. Algorithm EDSL-ET utilizes the Topk sparse technique to reduce communication costs and adopts error feedback to guarantee convergence. Algorithm EDSL-IBCD greatly reduces communication costs by introducing an independent block coordinate descent method. Algorithm EDSL-IBCD-ET takes advantage of EDSL-ET and EDSL-IBCD algorithms for extra-high dimensional feature learning, which is the best one whether in communication, computation, and stability. We give theoretical guarantees of the three algorithms under mild conditions, which match the performance of the centralized algorithm. Extensive experiments on simulated and real data validate our theoretical analysis and demonstrate that the proposed algorithms perform well with just a few rounds of communications.

Free vibrations of variablesection beams taking rotational and frictional forces into account

Introduction. Beams are widely used in the construction industry as load-bearing structures of bridges, overpasses, coverings, slabs, stairs, equipment platforms, etc. In order to fully utilize the bearing capacity of such structures and to reduce the rate of material consumption, beams of variable cross-section along the length can be used. During operation, such structural elements are subjected to various types of vibrations, which determines the relevance of studying various aspects of vibrational motion.Aim. To apply numerical methods for studying the influence of rotational inertial forces in the presence of viscous frictional forces on free vibrations of variable-section beams. This area of research presents interest, since the calculations are directly related to the determination of frequencies and forms of natural vibrations of structures.Materials and methods. Free vibrations were described by a homogeneous partial differential equation of hyperbolic type. The methods of variable separation and finite differences were used. A discrete domain in the form of a set of uniform grid nodes and a homogeneous system of algebraic equations were introduced. The system of equations in the matrix-vector form was used.Results. The spectra of natural frequencies, damping coefficients, and eigenforms of beam vibrations are determined. It is shown that the coefficient matrix has a banded and pentadiagonal form. The matrix elements are functions of the characteristic index. The damping coefficient and the frequency of free vibrations are determined from a system of two nonlinear equations. The solution of the system of equations is found using the method of coordinate descent. An example of calculation of a welded I-beam is considered. Five elements of the spectra of damping coefficients and natural frequencies are calculated.Conclusions. State-of-the-art MATLAB solvers allows numerical and graphical methods to be combined. In the solved examples, the advantages of these methods were successfully applied to determine the eigenvalues of matrices and eigenfunctions. The high validity and accuracy of the results obtained confirm the simplicity and versatility of the methodology for determining the characteristics of free vibrations of beams of variable cross-section.

Randomized progressive iterative approximation for B-spline curve and surface fittings

For large-scale data fitting, the least-squares progressive iterative approximation is a widely used method in many applied domains because of its intuitive geometric meaning and efficiency. In this work, we present a randomized progressive iterative approximation (RPIA) for the B-spline curve and surface fittings. In each iteration, RPIA locally adjusts the control points according to a random criterion of index selections. The difference for each control point is computed concerning the randomized block coordinate descent method. From geometric and algebraic aspects, the illustrations of RPIA are provided. We prove that RPIA constructs a series of fitting curves (resp., surfaces), whose limit curve (resp., surface) can converge in expectation to the least-squares fitting result of the given data points. Numerical experiments are given to confirm our results and show the benefits of RPIA.

Application of the Parabola Method in Nonconvex Optimization

We consider the Golden Section and Parabola Methods for solving univariate optimization problems. For multivariate problems, we use these methods as line search procedures in combination with well-known zero-order methods such as the coordinate descent method, the Hooke and Jeeves method, and the Rosenbrock method. A comprehensive numerical comparison of the obtained versions of zero-order methods is given in the present work. The set of test problems includes nonconvex functions with a large number of local and global optimum points. Zero-order methods combined with the Parabola method demonstrate high performance and quite frequently find the global optimum even for large problems (up to 100 variables).

Joint trajectory, transmission time and power optimization for multi-UAV data collecting system

Unmanned aerial vehicles (UAVs) have been generally applied in the field of communication due to their small size, flexible mobility, and convenient deployment. As a mobile base station, the UAV node can quickly establish a line-of-sight link with the ground node, thereby improving communication performance. In this paper, we study a multi-UAV assisted data collecting system. Specifically, in the case of limited system energy consumption, UAV flight energy consumption and ground node data transmission energy consumption are considered as an general limitation, and considering the channel interference between nodes, a multi-UAV assisted data collection model is studied. An non-convex problem that maximizes the minimum amount of data collected from ground nodes is further formulated. Since the original optimization problem is non-convex that difficult to solve directly, the problem is first decomposed into four sub-problems, and then the solution of each sub-problem is obtained by using successive convex approximation and block coordinate descent method. Finally, based on the solution of the four subproblems, an iterative algorithm for joint optimization of data transmission planning, transmission power, UAV trajectory and mission time is proposed. Simulation experiments show that the proposed algorithm can obtain more transmission data than the baseline algorithms.

Coordinate descent methods beyond smoothness and separability

Coordinate descent methods beyond smoothness and separability

A randomized sparse Kaczmarz solver for sparse signal recovery via minimax‐concave penalty

The randomized sparse Kaczmarz (RSK) method is an algorithm used to calculate sparse solutions for the basis pursuit problem. In this paper, we propose an algorithm framework for computing sparse solutions of linear systems, which includes the sparse Kaczmarz and sparse block Kaczmarz algorithms. In order to overcome the limitations of the penalty, we design an effective and new randomized sparse Kaczmarz algorithm (RSK‐MCP) based on the non‐convex minimax‐concave penalty (MCP) in sparse signal reconstruction. Additionally, we prove that the RSK‐MCP algorithm is equivalent to the randomized coordinate descent method for the corresponding dual problem. Based on this result, we demonstrate that the RSK‐MCP algorithm exhibits linear convergence, meaning it converges to a sparse solution of the MCP model when the regularization of MCP is a strongly convex function. Numerical experiments indicate that the RSK‐MCP algorithm outperforms RSK‐L1 in terms of both efficiency and accuracy.

Multi-UAV-assisted Internet of Remote Things communication within satellite–aerial–terrestrial integrated network

Due to the limited transmission capabilities of terrestrial intelligent devices within the Internet of Remote Things (IoRT), this paper proposes an optimization scheme aimed at enhancing data transmission rate while ensuring communication reliability. This scheme focuses on multi-unmanned aerial vehicle (UAV)-assisted IoRT data communication within the satellite–aerial–terrestrial integrated network (SATIN), which is one of the key technologies for the sixth generation (6G) networks. To optimize the system’s data transmission rate, we introduce a multi-dimensional coverage and power optimization (CPO) algorithm, rooted in the block coordinate descent (BCD) method. This algorithm concurrently optimizes various parameters, including the number and deployment of UAVs, the correlation between IoRT devices and UAVs, and the transmission power of both devices and UAVs. To ensure comprehensive coverage of a large-scale randomly distributed array of terrestrial devices, combined with machine learning algorithm, we present the Dynamic Deployment based on K-means (DDK) algorithm. Additionally, we address the non-convexity challenge in resource allocation for transmission power through variable substitution and the successive convex approximation technique (SCA). Simulation results substantiate the remarkable efficacy of our CPO algorithm, showcasing a maximum 240% improvement in the uplink transmission rate of IoRT data compared to conventional methods.

Optimal Spatial–Temporal Scheduling for EVs With Stochastic Behaviors and Possible Inauthentic Information

This paper studies the spatial-temporal scheduling for electric vehicles with stochastic charging behaviors. A new spatial-temporal scheduling model is proposed to achieve both the spatial coordination and temporal coordination for EV users and is modeled as a non-convex optimization problem which can be solved by the coordinate descent method. Further, a new billing method of charging cost is proposed based on the Vickrey-Clarke-Groves to avoid possible inauthentic information from users. Specially, the individual rationality and incentive compatibility of the method are theoretically proved. Moreover, a stochastic optimization problem is proposed to describe the stochastic charging behavior of users, which can fit the real application better than the deterministic one. Consequently, an efficient charging strategy is obtained via chance constraint method. Simulation results and comparison studies show the effectiveness of both the model and the billing method.

A Novel Normalized-Cut Solver With Nearest Neighbor Hierarchical Initialization.

Normalized-Cut (N-Cut) is a famous model of spectral clustering. The traditional N-Cut solvers are two-stage: 1) calculating the continuous spectral embedding of normalized Laplacian matrix; 2) discretization via K-means or spectral rotation. However, this paradigm brings two vital problems: 1) two-stage methods solve a relaxed version of the original problem, so they cannot obtain good solutions for the original N-Cut problem; 2) solving the relaxed problem requires eigenvalue decomposition, which has O(n3) time complexity ( n is the number of nodes). To address the problems, we propose a novel N-Cut solver designed based on the famous coordinate descent method. Since the vanilla coordinate descent method also has O(n3) time complexity, we design various accelerating strategies to reduce the time complexity to O(|E|) ( |E| is the number of edges). To avoid reliance on random initialization which brings uncertainties to clustering, we propose an efficient initialization method that gives deterministic outputs. Extensive experiments on several benchmark datasets demonstrate that the proposed solver can obtain larger objective values of N-Cut, meanwhile achieving better clustering performance compared to traditional solvers.

Parameter-Free Multiview K -Means Clustering With Coordinate Descent Method.

Recently, more and more real-world datasets have been composed of heterogeneous but related features from diverse views. Multiview clustering provides a promising attempt at a solution for partitioning such data according to heterogeneous information. However, most existing methods suffer from hyper-parameter tuning trouble and high computational cost. Besides, there is still an opportunity for improvement in clustering performance. To this end, a novel multiview framework, called parameter-free multiview k -means clustering with coordinate descent method (PFMVKM), is presented to address the above problems. Specifically, PFMVKM is completely parameter-free and learns the weights via a self-weighted scheme, which can avoid the intractable process of hyper-parameters tuning. Moreover, our model is capable of directly calculating the cluster indicator matrix, with no need to learn the cluster centroid matrix and the indicator matrix simultaneously as previous multiview methods have to do. What's more, we propose an efficient optimization algorithm utilizing the idea of coordinate descent, which can not only reduce the computational complexity but also improve the clustering performance. Extensive experiments on various types of real datasets illustrate that the proposed method outperforms existing state-of-the-art competitors and conforms well with the actual situation.

Energy efficient resource allocation of IRS-Assisted UAV network

<p>The integration of unmanned aerial vehicle (UAV) networks with intelligent reflecting surface (IRS) technology offers a promising solution to enhance wireless communication by dynamically altering signal propagation. This study addresses the challenge of maximizing system energy efficiency (EE) in IRS-assisted UAV networks. The primary objective is to optimize power allocation and IRS reflection design to achieve this goal. To tackle the optimization problem, we employ a block coordinate descent (BCD) method, decomposing it into three subproblems: phase shift optimization, power allocation, and trajectory planning. These subproblems are iteratively solved using an improved particle swarm optimization (PSO) algorithm. Simulation results demonstrate that the proposed PSO algorithm effectively plans high-quality UAV trajectories in complex environments, significantly enhancing EE. The findings suggest that the IRS-assisted UAV model outperforms traditional UAV models, offering substantial improvements in wireless communication quality and EE.</p>