Lottery Ticket Hypothesis Research Articles

Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks.

Graph neural networks (GNNs) tend to suffer from high computation costs due to the exponentially increasing scale of graph data and a large number of model parameters, which restricts their utility in practical applications. To this end, some recent works focus on sparsifying GNNs (including graph structures and model parameters) with the lottery ticket hypothesis (LTH) to reduce inference costs while maintaining performance levels. However, the LTH-based methods suffer from two major drawbacks: 1) they require exhaustive and iterative training of dense models, resulting in an extremely large training computation cost, and 2) they only trim graph structures and model parameters but ignore the node feature dimension, where vast redundancy exists. To overcome the above limitations, we propose a comprehensive graph gradual pruning framework termed CGP. This is achieved by designing a during-training graph pruning paradigm to dynamically prune GNNs within one training process. Unlike LTH-based methods, the proposed CGP approach requires no retraining, which significantly reduces the computation costs. Furthermore, we design a cosparsifying strategy to comprehensively trim all the three core elements of GNNs: graph structures, node features, and model parameters. Next, to refine the pruning operation, we introduce a regrowth process into our CGP framework, to reestablish the pruned but important connections. The proposed CGP is evaluated over a node classification task across six GNN architectures, including shallow models [graph convolutional network (GCN) and graph attention network (GAT)], shallow-but-deep-propagation models [simple graph convolution (SGC) and approximate personalized propagation of neural predictions (APPNP)], and deep models [GCN via initial residual and identity mapping (GCNII) and residual GCN (ResGCN)], on a total of 14 real-world graph datasets, including large-scale graph datasets from the challenging Open Graph Benchmark (OGB). Experiments reveal that the proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of the existing methods.

Finding core labels for maximizing generalization of graph neural networks

Graph neural networks (GNNs) have become a popular approach for semi-supervised graph representation learning. GNNs research has generally focused on improving methodological details, whereas less attention has been paid to exploring the importance of labeling the data. However, for semi-supervised learning, the quality of training data is vital. In this paper, we first introduce and elaborate on the problem of training data selection for GNNs. More specifically, focusing on node classification, we aim to select representative nodes from a graph used to train GNNs to achieve the best performance. To solve this problem, we are inspired by the popular lottery ticket hypothesis, typically used for sparse architectures, and we propose the following subset hypothesis for graph data: “There exists a core subset when selecting a fixed-size dataset from the dense training dataset, that can represent the properties of the dataset, and GNNs trained on this core subset can achieve a better graph representation”. Equipped with this subset hypothesis, we present an efficient algorithm to identify the core data in the graph for GNNs. Extensive experiments demonstrate that the selected data (as a training set) can obtain performance improvements across various datasets and GNNs architectures.

Why and how to construct an epistemic justification of machine learning?

Consider a set of shuffled observations drawn from a fixed probability distribution over some instance domain. What enables learning of inductive generalizations which proceed from such a set of observations? The scenario is worthwhile because it epistemically characterizes most of machine learning. This kind of learning from observations is also inverse and ill-posed. What reduces the non-uniqueness of its result and, thus, its problematic epistemic justification, which stems from a one-to-many relation between the observations and many learnable generalizations? The paper argues that this role belongs to any complexity regularization which satisfies Norton’s Material Theory of Induction (MTI) by localizing the inductive risk to facts in the given domain. A prime example of the localization is the Lottery Ticket Hypothesis (LTH) about overparameterized neural networks. The explanation of MTI’s role in complexity regularization of neural networks is provided by analyzing the stability of Empirical Risk Minimization (ERM), an inductive rule that controls the learning process and leads to an inductive generalization on the given set of observations. In cases where ERM might become asymptotically unstable, making the justification of the generalization by uniform convergence unavailable, LTH and MTI can be used to define a local stability. A priori, overparameterized neural networks are such cases and the combination of LTH and MTI can block ERM’s trivialization caused by equalizing the strengths of its inductive support for risk minimization. We bring closer the investigation of generalization in artificial neural networks and the study of inductive inference and show the division of labor between MTI and the optimality justifications (developed by Gerhard Schurz) in machine learning.

Structured Pruning for Deep Convolutional Neural Networks: A Survey.

The remarkable performance of deep Convolutional neural networks (CNNs) is generally attributed to their deeper and wider architectures, which can come with significant computational costs. Pruning neural networks has thus gained interest since it effectively lowers storage and computational costs. In contrast to weight pruning, which results in unstructured models, structured pruning provides the benefit of realistic acceleration by producing models that are friendly to hardware implementation. The special requirements of structured pruning have led to the discovery of numerous new challenges and the development of innovative solutions. This article surveys the recent progress towards structured pruning of deep CNNs. We summarize and compare the state-of-the-art structured pruning techniques with respect to filter ranking methods, regularization methods, dynamic execution, neural architecture search, the lottery ticket hypothesis, and the applications of pruning. While discussing structured pruning algorithms, we briefly introduce the unstructured pruning counterpart to emphasize their differences. Furthermore, we provide insights into potential research opportunities in the field of structured pruning. A curated list of neural network pruning papers can be found at: https://github.com/he-y/Awesome-Pruning. A dedicated website offering a more interactive comparison of structured pruning methods can be found at: https://huggingface.co/spaces/he-yang/Structured-Pruning-Survey.

Worst-Case VCG Redistribution Mechanism Design Based on the Lottery Ticket Hypothesis

We study worst-case VCG redistribution mechanism design for the public project problem. The mechanism design task comes down to designing a payment function that maximizes the worst-case allocative efficiency ratio. We use a multilayer perceptron (MLP) with ReLU activation to model the payment function and use mixed integer programming (MIP) to solve for the worst-case type profiles that maximally violate the mechanism design constraints. We collect these worst-case type profiles and use them as training samples to train toward better worst-case mechanisms. In practice, we require a tiny neural network structure for the above approach to scale. The Lottery Ticket Hypothesis states that a large network is likely to contain a "winning ticket" -- a much smaller subnetwork that "won the initialization lottery", which makes its training particularly effective. Motivated by this hypothesis, we train a large network and prune it into a tiny subnetwork. We run MIP-based worst-case training on the drawn subnetwork and evaluate the resulting mechanism's worst-case performance. If the subnetwork does not achieve good worst-case performance, then we record the type profiles that cause the current draw to be bad. To draw again, we restore the large network to its initial weights and prune using recorded type profiles from earlier draws, therefore avoiding drawing the same ticket twice. We expect to eventually encounter a tiny subnetwork that leads to effective training for our worst-case mechanism design task. Lastly, a by-product of multiple ticket draws is an ensemble of mechanisms with different worst cases, which improves the worst-case performance further. Using our approach, we find previously unknown optimal mechanisms for up to 5 agents. Our results confirm the tightness of existing theoretical upper bounds. For up to 20 agents, we derive significantly improved worst-case mechanisms, surpassing a long list of existing manual results.

Co-design of SLC/MLC FeFET-based highly error-tolerant low-power CiM and strong lottery ticket hypothesis-based algorithm

This study co-designs single-level cell (SLC) mask and multilevel cell (MLC) weight twin FeFET devices and a strong lottery ticket hypothesis (SLTH)-based neural network (NN) algorithm to achieve highly error-tolerant low-power Computation-in-Memory (CiM). The SLC mask FeFET masks or transfers the NN weight stored in the MLC weight FeFET, and the masked NN weight reduces the CiM power consumption. The proposed SLC mask FeFETs, which are trained, and MLC weight FeFETs, in which V TH are uniformly randomized, achieve 87% inference accuracy against 10-year data retention and read disturb. The SLC mask FeFETs show 86% inference accuracy even at 2000 endurance cycles. In addition, shared-bottom-select-gate (BSG) SLTH CiM and common-mask SLTH CiM for the NN convolutional layer are proposed to reduce the CiM area by sharing BSG and mask FeFET. Moreover, NN weight mapping schemes for SLTH CiM are proposed. The proposed mapping schemes show a tradeoff between inference accuracy and CiM area. One of the schemes reduces the CiM area by 45% with a 9.1% accuracy loss.

Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks.

The design of sparse neural networks, i.e., of networks with a reduced number of parameters, has been attracting increasing research attention in the last few years. The use of sparse models may significantly reduce the computational and storage footprint in the inference phase. In this context, the lottery ticket hypothesis (LTH) constitutes a breakthrough result, that addresses not only the performance of the inference phase, but also of the training phase. It states that it is possible to extract effective sparse subnetworks, called winning tickets, that can be trained in isolation. The development of effective methods to play the lottery, i.e., to find winning tickets, is still an open problem. In this article, we propose a novel class of methods to play the lottery. The key point is the use of concave regularization to promote the sparsity of a relaxed binary mask, which represents the network topology. We theoretically analyze the effectiveness of the proposed method in the convex framework. Then, we propose extended numerical tests on various datasets and architectures, that show that the proposed method can improve the performance of state-of-the-art algorithms.

A Sparse Sharing Multitask Framework for Building Footprint Extraction From Remote Sensing Imagery Following the Dual Lottery Ticket Hypothesis

A Sparse Sharing Multitask Framework for Building Footprint Extraction From Remote Sensing Imagery Following the Dual Lottery Ticket Hypothesis

SwinWave-SR: Multi-scale lightweight underwater image super-resolution

The resource-limited nature of underwater vision equipment leads to poor, otherwise low-resolution information affecting the downstream underwater robotics and ocean engineering tasks. Underwater Image Enhancement (UIE) methods have emerged, particularly Super-Resolution (SR), to tackle the aforementioned challenge by restoring the corresponding low-resolution image to a high-quality counterpart. Vision Transformers (ViTs) have recently been employed for SR tasks thanks to their superior performance over mainstream convolution neural networks. The success of ViTs is largely due to their self-attention mechanism; however, they may encounter challenges in dealing with severe and unpredictable degradation in underwater imaging. In contrast, Multi-scale ViTs (MViTs) variants such as the Swin transformers have overcome that challenge by preserving long-range dependencies over multi-scale feature hierarchies through evolving channel capacity. MViTs tend to induce spatial efficiency through classical down-sampling, such as average pooling over key/values, which results in an inevitable loss of high-frequency components. To address this lack, in the current work, we propose a novel algorithm, SwinWave-SR, for efficient and accurate multi-scale SR for underwater images. Our proposed algorithm is based on Swin transformer consisting of a wavelet block to restrict the information drop by downsampling in an invertible fashion. Consequently, the key components are preserved to assist self-attention learning while reducing its computational cost simultaneously. To further complement it, we explore a prominent compression regime, namely the Lottery Ticket Hypothesis (LTH), to discover a lightweight sub-network with competitive performance to its original model by reducing computational costs up to 70.44%. Overall, SwinWave-SR improves peak signal-to-noise ratio (PSNR) by 0.95 dB ∼ 2.23 dB compared to the state-of-the-art SwinIR while reducing the number of parameters by 29.56% and the calculation cost by 18.734%. Experimental results show that the proposed SwinWave-SR method outperforms the state-of-the-art SR methods on four benchmark underwater datasets and significantly improves PSNR and structural similarity index (SSIM).

3D-KCPNet: Efficient 3DCNNs based on tensor mapping theory

As the deep neural networks (DNNs) with satisfied expression ability usually require large scale to gain adequate performance, and deploying large-scale DNNs on resource-limited environments is still a challenge, neural network compression becomes a hot topic nowadays. Among the multiple compression methods, tensor decomposition reveals many specific advantages, such as regular data structure comes from linear algebra, convenient approach of training from scratch, ideal compression ratio, etc. Nevertheless, for some compact neural modules such as two-dimensional/three-dimensional (2D/3D) convolutional kernels, traditional tensor decomposition in the way of approximation has encountered intractable obstacles. Fortunately, some works utilize the tensor mapping approach, which just regards the data structure of tensor decomposition as neural layers, to reconstruct the convolutional kernel into a new lightweight module with several thinner convolutional kernels. The only two flies in the ointment are there lack necessary theories of tensor mapping, and there is still no tensor mapping way to compress high-order three-dimensional convolutional neural networks (3DCNNs). In this paper, we first deeply analyze the tensor mapping theory including convergence and precision, which separately establishes the rationality of tensor mapping and its superiority over the traditional tensor approximation, according to the Lottery Ticket Hypothesis. Then we propose an efficient method termed as 3D-KCPNet, to compress 3DCNNs based on the novel Kronecker canonical polyadic (KCP) tensor decomposition. The proposed method can not only compress the 3D convolutional kernels, but also convert a 3D convolution to efficient 1 × 1 × 1 and 2D depthwise convolutions. The experiments on the video recognition datasets VIVA Challenge, UCF11, UCF50, and UCF101 show that the accuracy of 3D-KCPNet can surpass its original baseline model and the corresponding tensor approximation model.

Enhanced 3D Pose Estimation in Multi-Person, Multi-View Scenarios through Unsupervised Domain Adaptation with Dropout Discriminator.

Data-driven pose estimation methods often assume equal distributions between training and test data. However, in reality, this assumption does not always hold true, leading to significant performance degradation due to distribution mismatches. In this study, our objective is to enhance the cross-domain robustness of multi-view, multi-person 3D pose estimation. We tackle the domain shift challenge through three key approaches: (1) A domain adaptation component is introduced to improve estimation accuracy for specific target domains. (2) By incorporating a dropout mechanism, we train a more reliable model tailored to the target domain. (3) Transferable Parameter Learning is employed to retain crucial parameters for learning domain-invariant data. The foundation for these approaches lies in the H-divergence theory and the lottery ticket hypothesis, which are realized through adversarial training by learning domain classifiers. Our proposed methodology is evaluated using three datasets: Panoptic, Shelf, and Campus, allowing us to assess its efficacy in addressing domain shifts in multi-view, multi-person pose estimation. Both qualitative and quantitative experiments demonstrate that our algorithm performs well in two different domain shift scenarios.

Learning sparse reparameterization with layer-wise continuous sparsification

Sparse reparameterization in Deep Neural Networks (DNNs) aims to achieve a better tradeoff between the network parameter count and performance. Recently, the lottery ticket hypothesis suppose that excellent sub-networks (“winning tickets”) exist in dense randomly-initialized networks. These sparse sub-networks trained from scratch are able to reach the performance of their dense counterparts. Compared with Iterative Magnitude Pruning that relies on pruning strategies, the Continuous Sparsification algorithm learns the “winning tickets” with gradient-based methods, achieving better performance. In this paper, we propose Layer-wise Continuous Sparsification (LCS) scheme for finding sparse sub-networks, in which the parameterized relaxation of step functions used to remove network parameters in each layer is integrated into the DNN loss as an optimization objective. LCS utilizes a family of sigmoid functions to asynchronously filter important per-layer weights throughout training, yielding sparser and better sub-networks. Experiments show that our method surpasses state-of-the-art methods for sparse reparameterization. Additionally, the proposed method can be utilized as a regularization technique to further improve the accuracy of dense networks11Our code is publicly available at https://github.com/RiyaoDong/LCS..

Exploring lottery ticket hypothesis in few-shot learning

Lottery Ticket Hypothesis (LTH) [14] has gathered great focus since being proposed. Researchers then succeed in figuring out alternative ways to find the ”winning ticket” and extending the vanilla version to various kinds of situations ranging from image segmentation to language pretraining. However, these works all emphasize fully supervised learning with plenty of training instances, whilst they ignore the important scenario of learning from few examples, i.e., Few-Shot Learning (FSL). Different from classical many-shot learning tasks, the common FSL setting assumes the disjoint of source and target categories. To the best of our knowledge, the lottery ticket hypothesis has for the first time, systematically studied in few-shot learning scenarios. To validate the hypothesis, we conduct extensive experiments on several few-shot learning methods with three widely-used datasets, miniImageNet, CUB, CIFARFS. Results reveal that we can even find “winning tickets” for some high-performance methods. In addition, our experiments on Cross-Domain FSL further validates the transferability of the found “winning tickets”. Furthermore, the process of finding LTH can be costly. So we study the early-stage LTH for FSL via exploring the Inverse Scale Space(ISS). Empirical results validate the efficacy of early-stage LTH.

On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective.

This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of the form (Affine, ReLU, Affine). Our main finding is that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of these zonotopes are functions of the network parameters. This geometric characterization provides new perspectives to three tasks. (i) We propose a new tropical perspective to the lottery ticket hypothesis, where we view the effect of different initializations on the tropical geometric representation of a network's decision boundaries. (ii) Moreover, we propose new tropical based optimization reformulations that directly influence the decision boundaries of the network for the task of network pruning. (iii) At last, we discuss the reformulation of the generation of adversarial attacks in a tropical sense. We demonstrate that one can construct adversaries in a new tropical setting by perturbing a specific set of decision boundaries by perturbing a set of parameters in the network.

Your lottery ticket is damaged: Towards all-alive pruning for extremely sparse networks

Network pruning has been widely adopted for reducing computational costs and memory consumption in low-resource devices. Recent studies show that the lottery ticket hypothesis achieves high accuracy under high compression ratios (i.e., 80–90% of the parameters in original networks are removed). Nevertheless, finding well-trainable networks with sparse parameters (i.e., < 10% of the parameters remaining) is still a challenging task, commonly believed by the lack of model capacity. This paper revisits the training process of existing pruning methods and observe that dead connections, which do not contribute to model capacity, occur in existing pruning methods. To this end, we propose a novel pruning method, namely all-alive pruning (AAP), which produces the pruned networks with only trainable weights with no dead connections. Notably, AAP is broadly applicable to various pruning methods and model architectures. We demonstrate that AAP equipped with the existing pruning methods (e.g., iterative pruning, one-shot pruning, and dynamic pruning) consistently improves the accuracy of the original methods at high compression ratios on the various image- and language-based tasks.

Sunflower seeds classification based on sparse convolutional neural networks in multi-objective scene

Generally, sunflower seeds are classified by machine vision-based methods in production, which include using photoelectric sensors to identify light-sensitive signals through traditional algorithms for which the equipment cost is relatively high and using neural network image recognition methods to identify images through cameras for which the computational cost is high. To address these problems, a multi-objective sunflower seed classification method based on sparse convolutional neural networks is proposed. Sunflower seeds were obtained from the video recorded using the YOLOv5 Object detection algorithm, and a ResNet-based classification model was used to classify the seeds according to differences in appearance. The ResNet has the disadvantages of having numerous parameters and high storage requirements; therefore, this study referred to the Lottery Ticket Hypothesis and used the Iterative Magnitude Pruning algorithm to compress the sunflower seed classification model, aiming to ascertain the optimal sparse sub-network from the classification model. Experiments were conducted to compare the effects on model performance before and after pruning, pruning degree, and different pruning methods. The results showed that the performance of the ResNet-based sunflower seed classification model using global pruning was the least affected by pruning, with a 92% reduction in the number of parameters, the best accuracy is 0.56% better than non-pruned and 9.17% better than layer-wise pruning. These findings demonstrate that using the Iterative Magnitude Pruning algorithm can render the sunflower seed classification model lightweight with less performance loss. The reduction in computational resources through model compression reduces the cost of sunflower seed classification, making it more applicable to practical production, and this model can be used as a cost-effective alternative to key sunflower seed classification techniques in practical production.

Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding

Aspect detection systems for online reviews, especially based on unsupervised models, are considered better strategically to process online reviews, generally a very large collection of unstructured data. Aspect embedding-based deep learning models are designed for this problem however they still rely on redundant word embedding and they are sensitive to initialization which may have a significant impact on model performance. In this research, a pruning approach is used to reduce the redundancy of deep learning model connections and is expected to produce a model with similar or better performance. This research includes several experiments and comparisons of the results of pruning the model network weights based on the general neural network pruning strategy and the lottery ticket hypothesis. The result of this research is that pruning of the unsupervised aspect detection model, in general, can produce smaller submodels with similar performance even with a significant amount of weights pruned. Our sparse model with 80% of its total weight pruned has a similar performance to the original model. Our current pruning implementation, however, has not been able to produce sparse models with better performance.

LTH-ECG: Lottery Ticket Hypothesis-based Deep Learning Model Compression for Atrial Fibrillation Detection from Single Lead ECG On Wearable and Implantable Devices.

Atrial Fibrillation (AF) is a kind of arrhythmia, which is a major morbidity factor, and AF can lead to stroke, heart failure and other cardiovascular complications. Electrocardiogram (ECG) is the basic marker to test the condition of heart and it can effectively detect AF condition. Single lead ECG has the practical advantage for being small form factor and it is easy to deploy. With the sophistication of the current deep learning (DL) models, researchers have been able to construct cardiologist-level models to detect different arrhythmias including AF condition detection from single lead short-time ECG signals. However, such models are computationally expensive and require huge memory size for deployment (more than 100 MB to deploy state-of-the-art 34-layer convolutional neural network-based ECG classification model). Such models need to be significantly trimmed with insignificantly loss of its classification performance for deployment in practical applications like single lead ECG classification in wearable and implantable devices. We have found that classical deep learning model compression techniques like pruning, quantization are not capable of substantial model size reduction without compromising on the model performance. In this paper, we propose LTH-ECG, which is our novel goal-driven winning lottery ticket discovery method, where lottery ticket hypothesis (LTH)-based iterative model pruning is used with the aim of over-pruning avoidance. LTH-ECG reduces the model size by 142x times with insignificant loss of classification performance (less than 1 % test F1-score penalty). Clinical Relevance- LTH-ECG will enable practical deployment for remote screening of AF condition using single lead short-time ECG recordings such that patients can on-demand monitor AF condition remotely through wearable ECG sensing devices and report cardiological abnormality to the concerned physician. LTH-ECG acts as an early warning system for effective AF condition screening.

Playing Lottery Tickets with Vision and Language

Large-scale pre-training has recently revolutionized vision-and-language (VL) research. Models such as LXMERT and UNITER have significantly lifted the state of the art over a wide range of VL tasks. However, the large number of parameters in such models hinders their application in practice. In parallel, work on the lottery ticket hypothesis (LTH) has shown that deep neural networks contain small matching subnetworks that can achieve on par or even better performance than the dense networks when trained in isolation. In this work, we perform the first empirical study to assess whether such trainable subnetworks also exist in pre-trained VL models. We use UNITER as the main testbed (also test on LXMERT and ViLT), and consolidate 7 representative VL tasks for experiments, including visual question answering, visual commonsense reasoning, visual entailment, referring expression comprehension, image-text retrieval, GQA, and NLVR2. Through comprehensive analysis, we summarize our main findings as follows. (i) It is difficult to find subnetworks that strictly match the performance of the full model. However, we can find relaxed winning tickets at 50%-70% sparsity that maintain 99% of the full accuracy. (ii) Subnetworks found by task-specific pruning transfer reasonably well to the other tasks, while those found on the pre-training tasks at 60%/70% sparsity transfer universally, matching 98%/96% of the full accuracy on average over all the tasks. (iii) Besides UNITER, other models such as LXMERT and ViLT can also play lottery tickets. However, the highest sparsity we can achieve for ViLT is far lower than LXMERT and UNITER (30% vs. 70%). (iv) LTH also remains relevant when using other training methods (e.g., adversarial training).

Federated Anomaly Detection on System Logs for the Internet of Things: A Customizable and Communication-Efficient Approach

Runtime log-based anomaly detection is one of several key building blocks in ensuring system security, as well as post-incident forensic investigations. However, existing log-based anomaly detection approaches that are implemented on large-scale Internet of Things (IoT) systems generally upload local data from edge devices to a centralized (cloud) server for processing and analysis. Such a workflow incurs significant communication and computation overheads, with potential privacy implications. Hence, in this paper, we propose a customizable and communication-efficient federated anomaly detection scheme (hereafter referred to as FedLog), designed to facilitate the identification of abnormal log patterns in large-scale IoT systems. Specifically, we first craft a Temporal Convolutional Network-Attention Mechanism-based Convolutional Neural Network (TCN-ACNN) model, to effectively extract fine-grained features from system logs. Second, we develop a new federated learning framework to support IoT devices in establishing a comprehensive anomaly detection model in a collaborative and privacy-preserving manner. Third, a lottery ticket hypothesis based masking strategy is designed to achieve customizable and communication-efficient federated learning in handling non-Independent and Identically Distributed (non-IID) log datasets. We then evaluate the performance of our proposed scheme with those of DeepLog (published in CCS, 2017) and Loganomaly (published in IJCAI, 2019) in both centralized learning and federated learning settings, using two publicly available and widely used real-world datasets (i.e., HDFS and BGL). The findings demonstrate the utility of the proposed FedLog scheme, in terms of log-based anomaly detection.