Block Coordinate Update Research Articles

A Robust Sparse Sensor Placement Strategy Based on Indicators of Noise for Ocean Monitoring

A well-performing data-driven sparse sensor deployment strategy is critical for marine monitoring systems, as it enables the optimal reconstruction of marine physical quantities with fewer sensors. However, ocean data typically contain substantial amounts of noise, including outliers (incomplete data) and inherent measurement noise, which heightens the complexity of sensor deployment. Therefore, this study optimizes the sparse sensor placement model by establishing noise indicators, including small noise weight and large noise weight, which are measured by entropy to minimize the prediction bias. Building on this, a robust sparse sensor placement algorithm is proposed, which utilizes the block coordinate update (BCU) iteration method to solve the function. During the iterative updating process, the proposed algorithm simultaneously updates the selection matrix, reconstruction matrix, and noise matrix. This allows for effective identification and mitigation of noise in the data through evaluation. Consequently, the deployed sensors achieve superior reconstruction performance compared to other deployment methods that do not incorporate noise evaluation. Experiments are also conducted on datasets of sea surface temperature (SST) and global ocean salinity, which demonstrate that our strategy significantly outperforms several other considered methods in terms of reconstruction accuracy while enabling autonomous sensor deployment under noisy conditions.

High-Dimensional Multi-Objective Bayesian Optimization With Block Coordinate Updates: Case Studies in Intelligent Transportation System

Many transportation system problems can be formulated as high-dimensional expensive multi-objective problems. They are challenging for Gaussian process-based Bayesian optimization methods to find the Pareto fronts due to the curse of dimensionality and the boundary issue in the acquisition function optimization. This paper presents a multi-objective Bayesian optimization method with block coordinate updates, Block-MOBO, to solve high-dimensional expensive multi-objective problems. Block-MOBO first partitions the decision variable space into different blocks, each of which includes a low-dimensional multi-objective problem. At each iteration, one block is considered and the decision variables not in this block are approximated by context-vector generation embedded with the Pareto prior knowledge thus promoting convergence. To tackle the boundary issue, we present <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> -greedy acquisition function in a Bayesian and multi-objective fashion, which recommends candidates either from the exploitation-exploration trade-off perspective or with probability <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> from the Pareto dominance relationship perspective. We verify the effectiveness of Block-MOBO by comparing it with other multi-objective Bayesian methods on two real-world optimization problems in transportation system and three multi-objective synthetic test suites. The experimental results show that Block-MOBO can find more evenly distributed and non-dominated solutions in the whole search space with lower complexity compared with other state-of-the-art approaches. Our analyses illustrate that block coordinate updates and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> -greedy acquisition function contribute to computational complexity reduction and convergence-diversity trade-offs, respectively.

Correntropy-Based Low-Rank Matrix Factorization With Constraint Graph Learning for Image Clustering.

This article proposes a novel low-rank matrix factorization model for semisupervised image clustering. In order to alleviate the negative effect of outliers, the maximum correntropy criterion (MCC) is incorporated as a metric to build the model. To utilize the label information to improve the clustering results, a constraint graph learning framework is proposed to adaptively learn the local structure of the data by considering the label information. Furthermore, an iterative algorithm based on Fenchel conjugate (FC) and block coordinate update (BCU) is proposed to solve the model. The convergence properties of the proposed algorithm are analyzed, which shows that the algorithm exhibits both objective sequential convergence and iterate sequential convergence. Experiments are conducted on six real-world image datasets, and the proposed algorithm is compared with eight state-of-the-art methods. The results show that the proposed method can achieve better performance in most situations in terms of clustering accuracy and mutual information.

Novel High-Precision and High-Robustness Localization Algorithm for Underwater-Environment-Monitoring Wireless Sensor Networks

In marine ecological environment monitoring, the acquisition of node location information is crucial, and the absence of location information can render the collected data meaningless. Compared to the rest of the distance-based localization methods, the received signal strength (RSS)-based localization technique has gained significant interest due to its low cost and the absence of time synchronization. However, the acoustic signal propagates in the complex and changeable aqueous medium, and, in addition to the time-varying path loss factor (PLF), there is often a certain absorption loss, which seriously deteriorates the localization accuracy of the RSS-based technique. To address the above challenges, we propose a novel high-precision and high-robustness localization (NHHL) algorithm that introduces an estimation parameter to conjointly estimate the marine node location and the ambient PLF. Firstly, the original non-convex localization problem is converted into an alternating nonnegative constrained least squares (ANCLS) framework with the unknown PLF and absorption loss, and a two-step localization method based on the primitive dual interior point method and block co-ordinate update method is presented to find the optimal solution. In the first step, the penalty function is utilized to reformulate the localization problem and find an approximate solution. Nevertheless, due to inherent errors, it is unable to approximate the constraint boundary and the global optimum solution. Subsequently, in the second step, the original localization problem is further transformed into a generalized trust region sub-problem (GTRS) framework, and the approximate solution of the interior point method is utilized as the initial estimation, and then iteratively solved by block co-ordinate update to obtain the precise location and PLF conjointly. Furthermore, the closed-form expression of the Cramér–Rao lower bound (CRLB) for the case of the unknown path loss factor and absorption loss is derived to evaluate the our NHHL algorithm. Finally, the simulation results demonstrate the superiority of the presented NHHL algorithm compared with the selected benchmark methods in various marine simulation scenarios.

Fast and Interpretable Dynamics for Fisher Markets via Block-Coordinate Updates

We consider the problem of large-scale Fisher market equilibrium computation through scalable first-order optimization methods. It is well-known that market equilibria can be captured using structured convex programs such as the Eisenberg-Gale and Shmyrev convex programs. Highly performant deterministic full-gradient first-order methods have been developed for these programs. In this paper, we develop new block-coordinate first-order methods for computing Fisher market equilibria, and show that these methods have interpretations as tâtonnement-style or proportional response-style dynamics where either buyers or items show up one at a time. We reformulate these convex programs and solve them using proximal block coordinate descent methods, a class of methods that update only a small number of coordinates of the decision variable in each iteration. Leveraging recent advances in the convergence analysis of these methods and structures of the equilibrium-capturing convex programs, we establish fast convergence rates of these methods.

Correntropy based model predictive controller with multi-constraints for robust path trajectory tracking of self-driving vehicle

Self-driving vehicles must be equipped with path tracking capability to enable automatic and accurate identification of the reference path. Model Predictive Controller (MPC) is an optimal control method that has received considerable attention for path tracking, attributed to its ability to handle control problems with multiple constraints. However, if the data acquired for determining the reference path is contaminated by non-Gaussian noise and outliers, the tracking performance of MPC would degrades significantly. To this end, Correntropy-based MPC (CMPC) is proposed in this paper to address the issue. Different from the conventional MPC model, the objective of CMPC is constructed using the robust metric Maximum Correntropy Criterion (MCC) to transform the optimization problem of MPC to a non-concave problem with multiple constraints, which is then solved by the Block Coordinate Update (BCU) framework. To find the solution efficiently, the linear inequality constraints of CMPC are relaxed as a penalty term. Furthermore, an iterative algorithm based on Fenchel Conjugate (FC) and the BCU framework is proposed to solve the relaxed optimization problem. It is shown that both objective sequential convergence and iterate sequence convergence are satisfied by the proposed algorithm. Simulation results generated by CarSim show that the proposed CMPC has better performance than conventional MPC in path tracking when noise and outliers exist.

Joint mmWave Beamforming and Resource Allocation in NOMA-MEC Network for Internet of Things

Mobile edge computing (MEC) technology can reduce the computational burden of Internet of Things (IoT) devices by offloading computationally intensive tasks to physically proximate edge servers. The combination of millimeter-wave (mmWave) massive multiple input multiple output (MIMO) and non-orthogonal multiple access (NOMA) technique has draw significant attention for its high spectral efficiency. This paper merges the computation offloading and mmWave massive MIMO-NOMA communication to seek energy efficient performance for IoT devices in MEC network. First, we propose an IoT device clustering algorithm to schedule all IoT devices into limited beams. Following this, we minimize the total energy consumption of computing and offloading in the NOMA-MEC network by jointly optimizing hybrid beamforming, offloading task assignment (OTA) ratio and power allocation. Then, the non-convex joint optimization problem is solved by proposing a suboptimal solution with two stages. In the first stage, a cyclic block-coordinate update algorithm is proposed for analog beamforming design, and then the digital beamforming vectors are obtained by zero-forcing method. In the second stage, an alternating minimization algorithm is proposed to optimize the OTA ratio and power allocation. Finally, simulation results are provided to validate the convergence of the proposed algorithms and the superiority of our proposed schemes over other benchmark schemes, which demonstrate the outstanding performance leveraging the massive MIMO-NOMA technique during computational offloading.

Semi-Asynchronous Hierarchical Federated Learning Over Mobile Edge Networks

Mobile edge network has been recognized as a promising technology for future wireless communications. However, mobile edge networks usually gathering large amounts of data, which makes it difficult to explore data science efficiently. Currently, federated learning has been proposed as an appealing approach to allow users to cooperatively reap the benefits from trained participants. In this paper, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Semi-Asynchronous Hierarchical Federated Learning</i> (SAHFL) framework for mobile edge networks that enables elastic edge to cloud model aggregation from data sensing. We further formulate a joint edge node association and resource allocation problem under the proposed SAHFL framework to prevent personalities of heterogeneous devices and achieve communication-efficiency. To deal with our proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Mixed integer nonlinear programming</i> (MINLP) problem, we introduce a distributed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Alternating Direction Method of Multipliers</i> (ADMM)- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Block Coordinate Update</i> (BCU) algorithm. With this algorithm, a tradeoff between training accuracy and transmission latency has been derived. Numerical results demonstrate the advantages of the proposed algorithm in terms of training overhead and model performance.

Faster randomized block sparse Kaczmarz by averaging

The standard randomized sparse Kaczmarz (RSK) method is an algorithm to compute sparse solutions of linear systems of equations and uses sequential updates, and thus, does not take advantage of parallel computations. In this work, we introduce a parallel (mini batch) version of RSK based on averaging several Kaczmarz steps. Naturally, this method allows for parallelization and we show that it can also leverage large overrelaxation. We prove linear expected convergence and show that, given that parallel computations can be exploited, the method provably provides faster convergence than the standard method. This method can also be viewed as a variant of the linearized Bregman algorithm, a randomized dual block coordinate descent update, a stochastic mirror descent update, or a relaxed version of RSK and we recover the standard RSK method when the batch size is equal to one. We also provide estimates for inconsistent systems and show that the iterates converges to an error in the order of the noise level. Finally, numerical examples illustrate the benefits of the new algorithm.

Millimeter-Wave NR-U and WiGig Coexistence: Joint User Grouping, Beam Coordination, and Power Control

Millimeter wave (mmWave) communication is a promising New Radio in Unlicensed (NR-U) technology to meet with the ever-increasing data rate and connectivity requirements in future wireless networks. However, the development of NR-U networks should consider the coexistence with the incumbent Wireless Gigabit (WiGig) networks. In this paper, we introduce a novel multiple-input multiple-output non-orthogonal multiple access (MIMO-NOMA) based mmWave NR-U and WiGig coexistence network for uplink transmission. Our aim for the proposed coexistence network is to maximize the spectral efficiency while ensuring the strict NR-U delay requirement and the WiGig transmission performance in real time environments. A joint user grouping, hybrid beam coordination and power control strategy is proposed, which is formulated as a Lyapunov optimization based mixed-integer nonlinear programming (MINLP) with unit-modulus and nonconvex coupling constraints. Hence, we introduce a penalty dual decomposition (PDD) framework, which first transfers the formulated MINLP into a tractable augmented Lagrangian (AL) problem. Thereafter, we integrate both convex-concave procedure (CCCP) and inexact block coordinate update (BCU) methods to approximately decompose the AL problem into multiple nested convex subproblems, which can be iteratively solved under the PDD framework. Numerical results illustrate the performance improvement ability of the proposed strategy, as well as demonstrating the effectiveness to guarantee the NR-U traffic delay and WiGig network performance.

Block-enhanced precision matrix estimation for large-scale datasets

The ℓ1-regularized Gaussian maximum likelihood method is a common approach for sparse precision matrix estimation, but one that poses a computational challenge for high-dimensional datasets. We present a novel ℓ1-regularized maximum likelihood method for performant large-scale sparse precision matrix estimation utilizing the block structures in the underlying computations. We identify the computational bottlenecks and contribute a block coordinate descent update as well as a block approximate matrix inversion routine, which is then parallelized using a shared-memory scheme. We demonstrate the effectiveness, accuracy, and performance of these algorithms. Our numerical examples and comparative results with various modern open-source packages reveal that these precision matrix estimation methods can accelerate the computation of covariance matrices by two to three orders of magnitude, while keeping memory requirements modest. Furthermore, we conduct large-scale case studies for applications from finance and medicine with several thousand random variables to demonstrate applicability for real-world datasets.

Scalable generalized median graph estimation and its manifold use in bioinformatics, clustering, classification, and indexing

In this paper, we present GMG-BCU—a local search algorithm based on block coordinate update for estimating a generalized median graph for a given collection of labeled or unlabeled input graphs. Unlike all competitors, GMG-BCU is designed for both discrete and continuous label spaces and can be configured to run in linear time w.r.t. the size of the graph collection whenever median node and edge labels are computable in linear time. These properties make GMG-BCU usable for applications such as differential microbiome data analysis, graph classification, clustering, and indexing. We also prove theoretical properties of generalized median graphs, namely, that they exist under reasonable assumptions which are met in almost all application scenarios, that they are in general non-unique, that they are NP-hard to compute and APX-hard to approximate, and that no polynomial α-approximation exists for any α unless the graph isomorphism problem is in P. Extensive experiments on six different datasets show that our heuristic GMG-BCU always outperforms the state of the art in terms of runtime or quality (on most datasets, both w.r.t. runtime and quality), that it is the only available heuristic which can cope with collections containing several thousands of graphs, and that it shows very promising potential when used for the aforementioned applications. GMG-BCU is freely available on GitHub: https://github.com/dbblumenthal/gedlib/.

RSS-Based Byzantine Fault-Tolerant Localization Algorithm Under NLOS Environment

Localization is one of the most critical tasks in wireless sensor networks, but achieving a relatively accurate location estimation is challenging when there have Byzantine fault and non-line-of-sight (NLOS) bias simultaneously. In this context, a localization method, based on received signal strength (RSS), is proposed in this letter to mitigate the impact of Byzantine fault and NLOS bias on the localization accuracy of wireless sensor networks. The proposed method relies on a Byzantine fault-tolerant localization algorithm (BFLA), which converts the localization problem into a generalized trust-region subproblem (GTRS) by applying certain approximations. In order to obtain a feasible solution to the GTRS, a block-coordinate update (BCU) function with a regularization term is used to divide the localization problem into two subproblems. An iterative method, whose start-point is obtained by an unconstrained squared-range (USR) algorithm, is then used to obtain a solution. Numerical simulations are carried out to show the effectiveness of the proposed method, compared with the state-of-the-art approaches in different scenarios.

Robust High-Order Manifold Constrained Low Rank Representation for Subspace Clustering

Due to the effectiveness in learning the subspace structures, low-rank representation (LRR) and its variations have been widely applied in various fields, such as computer vision and pattern recognition. However, in real applications, it is a challenge to handle the complex noises. To address this problem, we propose a novel robust LRR method based on kernel risk-sensitive loss (KRSL) with high-order manifold constraint, called RHLRR, in which the KRSL is introduced to deal with the noises and the multiple hypergraph regularization term is used as a high order manifold constraint to effectively capture the locality, similarity and the intrinsic geometric information in data. Besides, an iterative algorithm based on the half-quadratic (HQ) and the accelerated block coordinate update (BCU) is developed. The experimental results demonstrate that the proposed method can outperform other state-of-the-art LRR variants.

Asynchronous Lagrangian scenario decomposition

We present a distributed asynchronous algorithm for solving two-stage stochastic mixed-integer programs (SMIP) using scenario decomposition, aimed at industrial-scale instances of the stochastic unit commitment (SUC) problem. The algorithm is motivated by large differences in run times observed among scenario subproblems of SUC instances, which can result in inefficient use of distributed computing resources by synchronous parallel algorithms. Our algorithm performs dual iterations asynchronously using a block-coordinate subgradient descent method which allows performing block-coordinate updates using delayed information, while candidate primal solutions are recovered from the solutions of scenario subproblems using heuristics. We present a high performance computing implementation of the asynchronous algorithm, detailing the operations performed by each parallel process and the communication mechanisms among them. We conduct numerical experiments using SUC instances of the Western Electricity Coordinating Council system with up to 1000 scenarios and of the Central Western European system with up to 120 scenarios. We also conduct numerical experiments on generic SMIP instances from the SIPLIB library (DCAP and SSLP). The results demonstrate the general applicability of the proposed algorithm and its ability to solve industrial-scale SUC instances within operationally acceptable time frames. Moreover, we find that an equivalent synchronous parallel algorithm would leave cores idle up to 80.4% of the time on our realistic test instances, an observation which underscores the need for designing asynchronous optimization schemes in order to fully exploit distributed computing on real world applications.

An Efficient Proximal Block Coordinate Homotopy Method for Large-Scale Sparse Least Squares Problems

In this paper, an efficient and robust algorithm framework is presented for large-scale sparse least squares problems. This framework decomposes the original sparse least squares problem into a sequence of small-scale $l_1$-minimization subproblems. Every subproblem is solved by an improved $l_1$-homotopy method which differs from the original $l_1$-homotopy method by adopting a warm-start procedure and an $\varepsilon$-precision verification-correction technique. Moreover, based on a carefully designed block coordinate update strategy, the algorithm framework is proved to converge to a $\tilde \tau$-``precise" solution in a finite number of steps and the value of the objective function linearly converges. Numerical comparisons between the presented algorithms and a number of state-of-the-art algorithms on real and randomly generated data sets demonstrate the robustness and high performance of the presented algorithms. As an example, the presented algorithm only needs 13 seconds to solve an $l_1$-minimization problem with tens of millions of samples and features.

Maximum Correntropy Criterion-Based Robust Semisupervised Concept Factorization for Image Representation.

Concept factorization (CF) has shown its great advantage for both clustering and data representation and is particularly useful for image representation. Compared with nonnegative matrix factorization (NMF), CF can be applied to data containing negative values. However, the performance of CF method and its extensions will degenerate a lot due to the negative effects of outliers, and CF is an unsupervised method that cannot incorporate label information. In this article, we propose a novel CF method, with a novel model built based on the maximum correntropy criterion (MCC). In order to capture the local geometry information of data, our method integrates the robust adaptive embedding and CF into a unified framework. The label information is utilized in the adaptive learning process. Furthermore, an iterative strategy based on the accelerated block coordinate update is proposed. The convergence property of the proposed method is analyzed to ensure that the algorithm converges to a reliable solution. The experimental results on four real-world image data sets show that the new method can almost always filter out the negative effects of the outliers and outperform several state-of-the-art image representation methods.

Block Belief Propagation for Parameter Learning in Markov Random Fields

Traditional learning methods for training Markov random fields require doing inference over all variables to compute the likelihood gradient. The iteration complexity for those methods therefore scales with the size of the graphical models. In this paper, we propose block belief propagation learning (BBPL), which uses block-coordinate updates of approximate marginals to compute approximate gradients, removing the need to compute inference on the entire graphical model. Thus, the iteration complexity of BBPL does not scale with the size of the graphs. We prove that the method converges to the same solution as that obtained by using full inference per iteration, despite these approximations, and we empirically demonstrate its scalability improvements over standard training methods.

Robust High-Order Manifold Constrained Sparse Principal Component Analysis for Image Representation

In order to efficiently utilize the information in the data and eliminate the negative effects of outliers in the principal component analysis (PCA) method, in this paper, we propose a novel robust sparse PCA method based on maximum correntropy criterion (MCC) with high-order manifold constraints called the RHSPCA. Compared with the traditional PCA methods, the proposed RHSPCA has the following benefits: 1) the MCC regression term is more robust to outliers than the MSE-based regression term; 2) thanks to the high-order manifold constraints, the low-dimensional representations can preserve the local relations of the data and greatly improve the clustering and classification performance for image processing tasks; and 3) in order to further counteract the adverse effects of outliers, the MCC-based samples’ mean is proposed to better centralize the data. We also propose a new solver based on the half-quadratic technique and accelerated block coordinate update strategy to solve the RHSPCA model. Extensive experimental results show that the proposed method can outperform the state-of-the-art robust PCA methods on a variety of image processing tasks, including reconstruction, clustering, and classification, on outliers contaminated datasets.

Anomaly-Tolerant Network Traffic Estimation via Noise-Immune Temporal Matrix Completion Model

Accurately estimating origin-destination (OD) network traffic is crucial for network management and capacity planning. However, the potential network anomaly and complex noise make this goal difficult to achieve. Existing network traffic estimation methods usually impute network traffic independent of anomaly detection, which ignores the potential relationship between the two tasks to help each other in achieving better performance. Moreover, these approaches can only be suitable for simple Gaussian or outlier noise assumptions, which cannot be applied to more complex noise distributions in practical applications. To address these issues, we propose a novel anomaly-tolerant network traffic estimation approach for simultaneously estimating network traffic and detecting network anomaly. Specifically, by utilizing the inherent low-rank property and temporal characteristic of traffic matrix, we formulate the network traffic estimation problem as a noise-immune temporal matrix completion (NiTMC) model, where the complex noise is fitted by mixture of Gaussian (MoG), and the network anomaly is smoothed by the $L_{2,1}$ -norm regularization. In addition, we also design a convergence-guaranteed optimization algorithm based on the expectation maximization (EM) and block coordinate update (BCU) methods to solve the proposed model. Furthermore, to deal with large-scale network problems, we develop a scalable and memory-efficient algorithm by employing stochastic proximal gradient descent (SPGD) method. Finally, the extensive experiments performed on real datasets demonstrate that our proposed NiTMC model outperforms the previously widely used network traffic estimation methods.