Geometric Median Research Articles

High-dimensional multivariate analysis of variance via geometric median and bootstrapping.

The geometric median, which is applicable to high-dimensional data, can be viewed as a generalization of the univariate median used in 1-dimensional data. It can be used as a robust estimator for identifying the location of multi-dimensional data and has a wide range of applications in real-world scenarios. This paper explores the problem of high-dimensional multivariate analysis of variance (MANOVA) using the geometric median. A maximum-type statistic that relies on the differences between the geometric medians among various groups is introduced. The distribution of the new test statistic is derived under the null hypothesis using Gaussian approximations, and its consistency under the alternative hypothesis is established. To approximate the distribution of the new statistic in high dimensions, a wild bootstrap algorithm is proposed and theoretically justified. Through simulation studies conducted across a variety of dimensions, sample sizes, and data-generating models, we demonstrate the finite-sample performance of our geometric median-based MANOVA method. Additionally, we implement the proposed approach to analyze a breast cancer gene expression dataset.

Improving Sliced Wasserstein Distance with Geometric Median for Knowledge Distillation

Improving Sliced Wasserstein Distance with Geometric Median for Knowledge Distillation

Communication-efficient federated learning via personalized filter pruning

With the popularity of mobile devices and the continuous growth of interactive data, FL (Federated Learning) has gradually become an effective mean to address the problems of privacy leakage and data silos. However, due to the heterogeneity and imbalance of participants, FL faces many challenges, including model accuracy, security, heterogeneous devices and data, privacy preservation, as well as communication overhead and efficiency. To address challenges such as high communication overhead and low model accuracy in FL, we innovatively introduce model pruning into the FL framework and propose a personalized filter pruning-based FL method named PF2 Learning (Personalized Filter Pruning Federal Learning). This method achieves reasonable filter pruning for all local models by performing personalized pruning on each local model. Specifically, on the device side, we use a pruning strategy based on the norm geometric median, which also considers the order dependence between adjacent layers and evaluates the contribution of filters involved in pruning to optimize the pruning for local models at a unified pruning rate. On the server side, we calculate the unified pruning ratio of the model based on the contribution of participants' model filters and training errors to maintain the consistency of the participants' model structures. To validate the effectiveness of our proposed method, we conducted federated image classification tasks on ResNet-18, VGG-11, DenseNet-121, and InceptionNet-V1 models using CIFAR-10, FEMNIST, and ImageNet datasets. The experimental results show that PF2 Learning outperforms most FL pruning methods and exhibits better model performance and accuracy.

Online stochastic Newton methods for estimating the geometric median and applications

In the context of large samples, a small number of individuals might spoil basic statistical indicators like the mean. It is difficult to detect automatically these atypical individuals, and an alternative strategy is using robust approaches. This paper focuses on estimating the geometric median of a random variable, which is a robust indicator of central tendency. In order to deal with large samples of data arriving sequentially, online stochastic Newton algorithms for estimating the geometric median are introduced and we give their rates of convergence. Since estimates of the median and those of the Hessian matrix can be recursively updated, we also determine confidences intervals of the median in any designated direction and perform online statistical tests.

Robust Optimization and Inference on Manifolds

We propose a robust and scalable procedure for general optimization and inference problems on manifolds leveraging the classical idea of `median-of-means' estimation. This is motivated by ubiquitous examples and applications in modern data science in which a statistical learning problem can be cast as an optimization problem over manifolds. Being able to incorporate the underlying geometry for inference while addressing the need for robustness and scalability presents great challenges. We address these challenges by first proving a key lemma that characterizes some crucial properties of geometric medians on manifolds. In turn, this allows us to prove robustness and tighter concentration of our proposed final estimator in a subsequent theorem. This estimator aggregates a collection of subset estimators by taking their geometric median over the manifold. We illustrate bounds on this estimator via calculations in explicit examples. The robustness and scalability of the procedure is illustrated in numerical examples on both simulated and real data sets.

A robust model-based clustering based on the geometric median and the median covariation matrix

A robust model-based clustering based on the geometric median and the median covariation matrix

The Inverse Weber Problem on the Plane and the Sphere

Weber’s inverse problem in the plane is to modify the positive weights associated with n fixed points in the plane at minimum cost, ensuring that a given point a priori becomes the Euclidean weighted geometric median. In this paper, we investigate Weber’s inverse problem in the plane and generalize it to the surface of the sphere. Our study uses a subspace orthogonal to a subspace generated by two vectors X and Y associated with the given points and weights. The main achievement of our work lies in determining a vector perpendicular to the vectors X and Y, in Rn; which is used to determinate a solution of Weber’s inverse problem. In addition, lower bounds are obtained for the minimum of the Weber function, and an upper bound for the difference of the minimal of Weber’s direct and inverse problems. Examples of application at the plane and unit sphere are given.

Hybrid algorithms with alternative embedded local search schemes for the p-median problem

Progress in the development of location theory and clustering methods is mainly aimed at improving the performance of algorithms. We study the continuous p-median problem with the Euclidean metric. Various algorithmic combinations of the ALA procedure embedded into more sophisticated algorithmic combinations are used. Two repeated steps of the ALA algorithm solve the simplest discrete optimization problem, and then a series of central point search problems (Weber problem) for each cluster. A parallel implementation of the ALA algorithm is used. The paper studies the issue of choosing from two strategies: solve the Weber problem until accuracy ε is achieved at each iteration, or perform only one iteration of the Weiszfeld algorithm at each ALA iteration.

Ultra-short-term wind power forecasting based on personalized robust federated learning with spatial collaboration

An ultra-short-term wind power forecasting method based on personalized robust federated learning (PRFL) is proposed to exploit spatio-temporal correlation in a privacy-preserving way to facilitate the collaboration between wind farms. Firstly, standard federated learning (FL) with a bidirectional long short-term memory neural network (Bi-LSTM) as the underlying forecasting model is adopted as the primary framework. Secondly, to mitigate FL's vulnerability to anomalies, a Geometric median based federated aggregation scheme is utilized instead of the traditional FedAvg algorithm, enabling enhancement of the robustness against anomalous updates from individual wind farms. Finally, a personalized federated learning (PFL) strategy incorporating transfer learning is proposed to alleviate the challenge of limited model applicability due to data heterogeneity in conventional FL approaches. The case study involving 30 wind farms demonstrates the superior forecasting accuracy of PRFL compared to traditional FL-based methods. In particular, PRFL yields relative improvements ranging from 2.67 % to 7.49 % compared to local forecasting method without spatial correlation, while ensuring data privacy through the local storage of raw data at each collaborative wind farm. Furthermore, empirical examinations incorporating artificially induced anomalous scenarios affirm PRFL's outstanding resilience in handling anomalies during collaborative training, surpassing the capabilities offered by conventional approaches.

Multiplant Location Involving Resource Allocation

Recently, a multisource, raw material allocation form of Weber's classic single‐facility location problem was rediscovered and recognized for its significance in contemporary planning and decision‐making. This variation of the Weber problem investigates the location of a production plant while permitting the selection of each required raw material source. This article reviews the Weber problem with an emphasis on its extension to incorporate multiple facilities. The only formulated multiplant Weber problem involving resource allocation remains unsolved due to its complexity. An effective approach integrating GIS processing (i.e., the Voronoi diagram and vector‐based overlay) with the classic optimization algorithm (i.e., the Weiszfeld algorithm) is developed to address raw material sourcing in the process of siting facilities. The implementation relies entirely on open‐source Python packages, making the work reproducible, replicable, and expandable. Application findings demonstrate that the utility and computational efficiency of the proposed method to tackle this challenging problem are superior to those of the most advanced commercial optimization software.

Quantifying workload using nonlinear dynamical measures of biomechanical parameters during cycling on a roller trainer.

The aim of the present study was to determine the effectiveness of nonlinear parameters in distinguishing individual workload in cycling by using bike-integrated sensor data. The investigation focused on two nonlinear parameters: The ML1, which analyzes the geometric median in phase space, and the maximum Lyapunov exponent as nonlinear measure of local system stability. We investigated two hypothesis: 1. ML1α, derived from kinematic crank data, is as good as ML1F, derived from force crank data, at distinguishing between individual load levels. 2. Increasing load during cycling leads to decreasing local system stability evidenced by linearly increasing maximal Lyapunov exponents generated from kinematic data. A maximal incremental cycling step test was conducted on an ergometer, generating complete datasets from 10 participants in a laboratory setting. Pedaling torque and kinematic data of the crank were recorded. ML1F, ML1α, and Lyapunov parameters (λst, λlt, ιst, ιlt) were calculated for each participant at comparable load levels. The results showed a significant linear increase in ML1α across three individual load levels, with a lower but still large effect compared to ML1F. The contrast analysis also confirmed a linearly increasing trend for λst across three load levels, but this was not confirmed for λlt. However, the intercepts ιst and ιlt of the short- and longterm divergence showed a statistically significant linear increase across the load levels. In summary, nonlinear parameters seem fundamentally suitable to distinguish individual load levels in cycling. It is concluded that higher load during cycling is associated with decreasing local system stability. These findings may aid in developing improved e-bike propulsion algorithms. Further research is needed to determine the impact of factors occurring in field application.

Estimating Outlier-Immunized Common Harmonic Waves for Brain Network Analyses on the Stiefel Manifold.

Since brain network organization is essentially governed by the harmonic waves derived from the Eigen-system of the underlying Laplacian matrix, discovering the harmonic-based alterations provides a new window to understand the pathogenic mechanism of Alzheimer's disease (AD) in a unified reference space. However, current reference (common harmonic waves) estimation studies over the individual harmonic waves are often sensitive to outliers, which are obtained by averaging the heterogenous individual brain networks. To address this challenge, we propose a novel manifold learning approach to identify a set of outlier-immunized common harmonic waves. The backbone of our framework is calculating the geometric median of all individual harmonic waves on the Stiefel manifold, instead of Fréchet mean, thus improving the robustness of learned common harmonic waves to the outliers. A manifold optimization scheme with theoretically guaranteed convergence is tailored to solve our method. The experimental results on synthetic data and real data demonstrate that the common harmonic waves learned by our approach are not only more robust to the outliers than the state-of-the-art methods, but also provide a putative imaging biomarker to predict the early stage of AD.

Joint think locally and globally: Communication-efficient federated learning with feature-aligned filter selection

Federated learning, as a distributed machine learning framework that can protect data privacy, has recently received increasing attention. In federated learning, a shared global model is obtained through parameter interaction, which leads to frequent parameter communication during the training process. The limited bandwidth of IoT and edge devices that deploy federated learning further affects communication and learning efficiency. In this paper, an enhanced federated learning technique is presented by proposing a feature-aligned filter selection method. Besides, it is believed that the training gap between the global model and the local model on each node should be focused on during the training process. Then, we can select the local contribution parameters to improve the communication efficiency while ensuring the performance of the shared global model. Therefore, the Geometric Median of each layer in the global model is adopted as the criterion to select important filters in the local model, and then the parameters interact with other nodes to achieve efficient communication. The results under a variety of experimental settings demonstrate that our proposed federated learning scheme can effectively enhance communication efficiency and ensure the performance of the global model. Compared with the state-of-the-art methods, a maximum of 6.5× improvements in communication efficiency can be obtained.

WGM-dSAGA: Federated Learning Strategies with Byzantine Robustness Based on Weighted Geometric Median

Federated learning techniques accomplish federated modeling and share global models without sharing data. Federated learning offers a good answer to complex data and privacy security issues. Although there are many ways to target federated learning, Byzantine attacks are the ones we concentrate on. Byzantine attacks primarily impede learning by tampering with the local model parameters provided by a client to the master node throughout the federation learning process, leading to a final global model that diverges from the optimal solution. To address this problem, we combine aggregation rules with Byzantine robustness using a gradient descent optimization algorithm based on variance reduction. We propose a WGM-dSAGA method with Byzantine robustness, called weighted geometric median-based distributed SAGA. We replace the original mean aggregation strategy in the distributed SAGA with a robust aggregation rule based on weighted geometric median. When less than half of the clients experience Byzantine attacks, the experimental results demonstrate that our proposed WGM-dSAGA approach is highly robust to different Byzantine attacks. Our proposed WGM-dSAGA algorithm provides the optimal gap and variance under a Byzantine attack scenario.

Vision transformer attention with multi-reservoir echo state network for anomaly recognition

Anomalous event recognition requires an instant response to reduce the loss of human life and property; however, existing automated systems show limited performance due to considerations related to the temporal domain of the videos and ignore the significant role of spatial information. Furthermore, although current surveillance systems can detect anomalous events, they require human intervention to recognise their nature and to select appropriate countermeasures, as there are no fully automatic surveillance techniques that can simultaneously detect and interpret anomalous events. Therefore, we present a framework called Vision Transformer Anomaly Recognition (ViT-ARN) that can detect and interpret anomalies in smart city surveillance videos. The framework consists of two stages: the first involves online anomaly detection, for which a customised, lightweight, one-class deep neural network is developed to detect anomalies in a surveillance environment, while in the second stage, the detected anomaly is further classified into the corresponding class. The size of our anomaly detection model is compressed using a filter pruning strategy based on a geometric median, with the aim of easy adaptability for resource-constrained devices. Anomaly classification is based on vision transformer features and is followed by a bottleneck attention mechanism to enhance the representation. The refined features are passed to a multi-reservoir echo state network for a detailed analysis of real-world anomalies such as vandalism and road accidents. A total of 858 and 1600 videos from two datasets are used to train the proposed model, and extensive experiments on the LAD-2000 and UCF-Crime datasets comprising 290 and 400 testing videos reveal that our framework can recognise anomalies more effectively, outperforming other state-of-the-art approaches with increases in accuracy of 10.14% and 3% on the LAD-2000 and UCF-Crime datasets, respectively.

Byzantine-Robust Aggregation in Federated Learning Empowered Industrial IoT

Federated learning (FL) is a promising paradigm to empower on-device intelligence in Industrial Internet of Things (IIoT) due to its capability of training machine learning models across multiple IIoT devices while preserving the privacy of their local data. However, the distributed architecture of FL relies on aggregating the parameter list from the remote devices, which poses potential security risks caused by malicious devices. In this article, we propose a flexible and robust aggregation rule, called auto-weighted geometric median ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM</monospace> ), and analyze the robustness against outliers in the inputs. To obtain the value of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM</monospace> , we design an algorithm based on the alternating optimization strategy. Using <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM</monospace> as aggregation rule, we propose two robust FL solutions <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM_FL</monospace> and <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM_PFL</monospace> . <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM_FL</monospace> learns a shared global model using the standard FL paradigm, and <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoGM_PFL</monospace> learns a personalized model for each device. We conduct extensive experiments on the FEMNIST and Bosch IIoT datasets. The experimental results show that our solutions are robust against both model poisoning and data poisoning attacks. In particular, our solutions sustain high performance even when 30% of the nodes perform model or 50% of the nodes perform data poisoning attacks.

Privacy-Preserved Generative Network for Trustworthy Anomaly Detection in Smart Grids: A Federated Semisupervised Approach

The deep integration of industrial internet of things technologies in the industrial smart grid (ISG) brings many privacy and security attacks, threatening the trustworthiness of underlying system infrastructures and associated services. That, in turn, raises the necessity for anomaly detection staged by appropriate authorities. Deep learning can provide a promising solution for anomaly detection, but it remains untrustworthy as it fails to do well with small-size labeled data and class-imbalanced data. To solve these issues, this article introduces a novel privacy-preserving federated semisupervised class-rebalanced (Fed-SCR) framework for the detection of anomalous power data in fog-assisted smart grids. Fed-SCR introduces a semisupervised generative network to enhance the quality of generated minority samples and model the relationships between labeled and unlabeled data. Moreover, the generator and discriminator are block-structured leveraging temporal convolutions to improve the representation power during the training. A novel aggregation scheme [termed federated geometric median aggregation (Fed-GMA)] is also introduced based on selective and periodic geometric-median-based aggregation with the main aim to increase the robustness of the federated semi supervised class-rebalanced (Fed-SRC) against noisy gradients while maintaining communication efficiency. The evaluations of Fed-SCR on public power grid datasets reveal its efficiency in improving the trustworthiness of the ISG platform, outperforming the competing methods in terms of binary classification (accuracy: 97.28) and multiclass classification (accuracy: 96.36–95.04).

An SSD-MobileNet Acceleration Strategy for FPGAs Based on Network Compression and Subgraph Fusion

Over the last decade, various deep neural network models have achieved great success in image recognition and classification tasks. The vast majority of high-performing deep neural network models have a huge number of parameters and often require sacrificing performance and accuracy when they are deployed on mobile devices with limited area and power consumption. To address this problem, we present an SSD-MobileNet-v1 acceleration method based on network compression and subgraph fusion for Field-Programmable Gate Arrays (FPGAs). Firstly, a regularized pruning algorithm based on sensitivity analysis and Filter Pruning via Geometric Median (FPGM) was proposed. Secondly, the Quantize Aware Training (QAT)-based network full quantization algorithm was designed. Finally, a strategy for computing subgraph fusion is proposed for FPGAs to achieve continuous scheduling of Programmable Logic (PL) operators. The experimental results show that using the proposed acceleration strategy can reduce the number of model parameters by a factor of 11 and increase the inference speed on the FPGA platform by a factor of 9–10. The acceleration algorithm is applicable to various mobile edge devices and can be applied to the real-time monitoring of forest fires to improve the intelligence of forest fire detection.

Influence of individual and subpopulation factors on the concentration of major and trace elements in saliva

BACKGROUND: The determination of the major and trace element composition of saliva could be a promising approach for non-invasive health monitoring. Such investigations are constrained, nevertheless, by the potential impact of several variables on the concentration of the elements.&#x0D; AIM: To study the influence of individual and subpopulation factors of donors on the concentration of some major and trace elements in saliva.&#x0D; MATERIALS AND METHODS: The results of the determination of elements concentration (Ag, Al, Ca, Cd, Cr, Cu, Fe, Mg, Mn, P, Pb, Si, Ti, Zn) in the whole saliva of 60 donors by direct (without digestion) analysis by arc atomic emission spectrometry using the dry residue technique are presented.&#x0D; RESULTS: Distribution of concentrations in the sample for Al, Cr, Cu, Fe, Mg, Mn, Si, Ti, and Zn showed a lognormal law; thus, the geometric mean or median is preferable as an indicator reflecting the average value. The distribution of Ca and P concentrations obeys the normal law. The concentration of Si and Ti is lower in the saliva of young men compared with women; the opposite pattern is observed for Mn. The mean Ag concentration in saliva is higher in the group of smokers than in non-smokers. Increase in the number of teeth affected by caries was correlated with an increase in concentrations of Cu, Al, and Zn in the saliva. For several analytes, interelement correlations were revealed, in particular, between the concentrations of Ca and P.&#x0D; CONCLUSION: The study of such factors of influence enables clinicians to determine the normal concentration ranges of major and trace elements in each specific clinical case with greater accuracy.

Immunogenicity and reactogenicity of a third dose of BNT162b2 vaccine for COVID-19 after a primary regimen with BBIBP-CorV or BNT162b2 vaccines in Lima, Peru.

The administration of a third (booster) dose of COVID-19 vaccines in Peru initially employed the BNT162b2 (Pfizer) mRNA vaccine. The national vaccination program started with healthcare workers (HCW) who received BBIBP-CorV (Sinopharm) vaccine as primary regimen and elderly people previously immunized with BNT162b2. This study evaluated the reactogenicity and immunogenicity of the "booster" dose in these two groups in Lima, Peru. We conducted a prospective cohort study, recruiting participants from November to December of 2021 in Lima, Peru. We evaluated immunogenicity and reactogenicity in HCW and elderly patients previously vaccinated with either two doses of BBIBP-CorV (heterologous regimen) or BTN162b2 (homologous regimen). Immunogenicity was measured by anti-SARS-CoV-2 IgG antibody levels immediately before boosting dose and 14 days later. IgG geometric means (GM) and medians were obtained, and modeled using ANCOVA and quantile regressions. The GM of IgG levels increased significantly after boosting: from 28.5±5.0 AU/mL up to 486.6±1.2 AU/mL (p<0.001) which corresponds to a 17-fold increase. The heterologous vaccine regimen produced higher GM of post-booster anti-SARS-CoV-2 IgG levels, eliciting a 13% increase in the geometric mean ratio (95%CI: 1.02-1.27) and a median difference of 92.3 AU/ml (95%CI: 24.9-159.7). Both vaccine regimens were safe and well tolerated. Previous COVID-19 infection was also associated with higher pre and post-booster IgG GM levels. Although both boosting regimens were highly immunogenic, two doses of BBIBP-CorV boosted with BTN162b2 produced a stronger IgG antibody response than the homologous BNT162b2 regimen in the Peruvian population. Additionally, both regimens were mildly reactogenic and well-tolerated.