FedPref: Federated Learning Across Heterogeneous Multi-objective Preferences

Cloud computing has established a convenient approach for computing offloading, where the data produced by edge devices is gathered and processed in a centralized server. However, it results in critical issues related to latency. Recently, a neural network-based on-device learning approach is proposed, which offers a solution to the latency problem by relocating processing data to edge devices; even so, a single edge device may face insufficient training data to train a high-quality model, because of its limited available processing capabilities and energy resources. To address this issue, we extend the work to a federated learning system which enables the edge devices to exchange their trained parameters and update local models. However, in federated learning for anomaly detection, the reliability of local models would be different. For example, a number of trained models are likely to contain the features of anomalous data because of noise corruption or anomaly detection failure. Besides, as the communication protocol amongst edges could be exploited by attackers, the training data or model weights may have potential risks of being poisoned. Therefore, when we design a federated training algorithm, we should carefully select the local models that participate in model aggregation. In this work, we leverage an observed dataset to compute prediction errors, so that the unsatisfying local models can be excluded from federated training. Experimental results show that the federated learning approach improves anomaly detection accuracy. Besides, the proposed model aggregation solution achieves obvious improvement compared with the popular Federated Averaging method.

Recently there has been significant interest in using machine learning to improve the accuracy of cardinality estimation. This work has focused on improving average estimation error, but not all estimates matter equally for downstream tasks like query optimization. Since learned models inevitably make mistakes, the goal should be to improve the estimates that make the biggest difference to an optimizer. We introduce a new loss function, Flow-Loss, for learning cardinality estimation models. Flow-Loss approximates the optimizer's cost model and search algorithm with analytical functions, which it uses to optimize explicitly for better query plans. At the heart of Flow-Loss is a reduction of query optimization to a flow routing problem on a certain "plan graph", in which different paths correspond to different query plans. To evaluate our approach, we introduce the Cardinality Estimation Benchmark (CEB) which contains the ground truth cardinalities for sub-plans of over 16 K queries from 21 templates with up to 15 joins. We show that across different architectures and databases, a model trained with Flow-Loss improves the plan costs and query runtimes despite having worse estimation accuracy than a model trained with Q-Error. When the test set queries closely match the training queries, models trained with both loss functions perform well. However, the Q-Error-trained model degrades significantly when evaluated on slightly different queries (e.g., similar but unseen query templates), while the Flow-Loss-trained model generalizes better to such situations, achieving 4 -- 8× better 99th percentile runtimes on unseen templates with the same model architecture and training data.

Video streaming on mobile devices such as PDA's, laptop PCs, pocket PCs and cell phones is becoming increasingly popular. These mobile devices are typically constrained by their battery capacity, bandwidth, screen resolution and video decoding and rendering capabilities. Consequently, video personalization strategies are used to provide these resource-constrained mobile devices with personalized video content that is most relevant to the client's request while simultaneously satisfying the client's resource constraints. Proxy-based caching of video content is a proven strategy to reduce both client latencies and server loads. In this paper, we propose novel video personalization server and caching mechanisms, the combination of which can efficiently disseminate personalized videos to multiple resource-constrained clients. The video personalization servers use an automatic video segmentation and video indexing scheme based on semantic video content. The caching proxies implement a novel cache replacement and multi-stage client request aggregation algorithm, specifically suited for caching personalized video files generated by the personalization servers. The cache design also implements a personalized video segment calculation algorithm based on client's content preferences and resource constraints. The paper reports series of experiments that demonstrate the efficacy of the proposed techniques in scalably disseminating personalized video content to resource constrained client-devices.

Decision letter: Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study

Editor's evaluation: Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study

Federated learning (FL) is a novel approach to privacy‐preserving machine learning, enabling remote devices to collaborate on model training without exchanging data among clients. However, it faces several challenges, including limited client‐side processing capabilities and non‐IID data distributions. To address these challenges, we propose a partitioned FL architecture that a large CNN is divided into smaller networks, which train concurrently with other clients. Within a cluster, multiple clients concurrently train the ensemble model. The Jensen–Shannon divergence quantifies the similarity of predictions across submodels. To address discrepancies in model parameters between local and global models caused by data distribution, we propose an ensemble learning method that integrates a penalty term into the local model’s loss calculation, thereby ensuring synchronization. This method amalgamates predictions and losses across multiple submodels, effectively mitigating accuracy loss during the integration process. Extensive experiments with various Dirichlet parameters demonstrate that our system achieves accelerated convergence and enhanced performance on the CIFAR‐10 and CIFAR‐100 image classification tasks while remaining robust to partial participation, diverse datasets, and numerous clients. On the CIFAR‐10 dataset, our method outperforms FedAvg, FedProx, and SplitFed by 6%–8%; in contrast, it outperforms them by 12%–18% on CIFAR‐100.

Data generated at the network edge can be processed locally by leveraging the paradigm of edge computing. To fully utilize the widely distributed data, we concentrate on a wireless edge computing system that conducts model training using decentralized peer-to-peer (P2P) methods. However, there are two major challenges on the way towards efficient P2P model training: limited resources (e.g., network bandwidth and battery life of mobile edge devices) and time-varying network connectivity due to device mobility or wireless channel dynamics, which have received less attention in recent years. To address these two challenges, this paper adaptively constructs a dynamic and efficient P2P topology, where model aggregation occurs at the edge devices. In a nutshell, we first formulate the topology construction for P2P learning (TCPL) problem with resource constraints as an integer programming problem. Then a learning-driven method is proposed to adaptively construct a topology at each training epoch. We further give the convergence analysis on training machine learning models even with non-convex loss functions. Extensive simulation results show that our proposed method can improve the model training efficiency by about 11% with resource constraints and reduce the communication cost by about 30% under the same accuracy requirement compared to the benchmarks.

Worldwide Internet of Medical Things (IoMT) sector has been experiencing a vertiginous rate of evolution in the past few years, going from a little wristwatch to a large aeroplane. Smart Health Care (SHC) systems utilize innovation technologies like IoMT, cloud edge computing and Artificial Intelligence (AI). With connected wearable devices and quick replies, SHC improves healthcare management by making it more efficient, convenient, and personalized. Deep Learners (DL) in the cloud are trained using the data collected from these devices. These servers have a lot of memory and a lot of processing expenditures. By utilizing a decentralized architecture known as Federated Learning (FL), several edge clients can work together to build a unified DL model effectively protecting the privacy of their own data. When a model loses all memory of its prior training data after receiving fresh input is referred as Catastrophic Forgetting (CF) problem. When the data distribution on each device changes over time, this can happen in a FL environment. As a Federated Increment Learning (FIL) system, Re-Fed can reduce CF by letting all clients each client remembers past samples based on how important. However, discrepant arrival times of the new task and data from the malfunctioning clients are not handled by Re-Fed FIL.This paper propose a Federated Improved Re-Fed Incremental Learning (FIRFIL) which handle the above issue through temporally weighted aggregation. In this research, a Time-Invariant Stochastic Spiking Long Short Term Memory (TISSLSTM) is used in a FIRFIL scenario. Internet of Things (IoT) devices sent the data acquired from various wearable sensors including those for blood sugar, heart rate, and chest readings to edge devices equipped with TISSLSTM for training. In FIRFIL, every edge device uses its own private data set to train a local model. A centralized server receives the local models and merges them into one global model. Next, the edge devices are updated with trained global model once again. This loop is continued until either the global model converges, or specific amounts of training rounds have passed. Next, we use the trained model to forecast client-specific diseases based on incoming data. A temporal weighted aggregation model in the server handle temporally variants data from clients. The proposed model is simply known as FIRFIL-TISSLSTM. At last, the test result demonstrate that the proposed model achieves 95.09%, 95.25% and 94.28% of accuracy on Comprehensive Heart Disease Dataset, UCI Heart Disease Dataset and Kaggle Heart Disease Dataset respectively outperforming traditional models. Also, the proposed model records lower energy consumption values 89.7J, 80.3J, 86.1J of energy consumption and reduced latency values of 173.8ms, 162.5ms, 168.4ms of latency on same datasets highlight its efficiency compared to other standard models.

Amid data privacy concerns, Federated Learning (FL) has emerged as a promising machine learning paradigm that enables privacy-preserving collaborative model training. However, there exists a need for a platform that matches data owners (supply) with model requesters (demand). In this paper, we present CrowdFL, a platform to facilitate the crowdsourcing of FL model training. It coordinates client selection, model training, and reputation management, which are essential steps for the FL crowdsourcing operations. By implementing model training on actual mobile devices, we demonstrate that the platform improves model performance and training efficiency. To the best of our knowledge, it is the first platform to support crowdsourcing-based FL on edge devices.

A federated learning (FL) scheme (denoted as Fed-KSVM) is designed to train kernel support vector machines (SVMs) over multiple edge devices with low memory consumption. To decompose the training process of kernel SVM, each edge device first constructs high-dimensional random feature vectors of its local data, and then trains a local SVM model over the random feature vectors. To reduce the memory consumption on each edge device, the optimization problem of the local model is divided into several subproblems. Each subproblem only optimizes a subset of the model parameters over a block of random feature vectors with a low dimension. To achieve the same optimal solution to the original optimization problem, an incremental learning algorithm called block boosting is designed to solve these subproblems sequentially. After training of the local models, the central server constructs a global SVM model by averaging the model parameters of these local models. Fed-KSVM only increases the iterations of training the local SVM models to save the memory consumption, while the communication rounds between the edge devices and the central server are not affected. Theoretical analysis shows that the kernel SVM model trained by Fed-KSVM converges to the optimal model with a linear convergence rate. Because of such a fast convergence rate, Fed-KSVM reduces the communication cost during training by up to 99% compared with the centralized training method. The experimental results also show that Fed-KSVM reduces the memory consumption on the edge devices by nearly 90% while achieving the highest test accuracy, compared with the state-of-the-art schemes.

Federated learning (FL) is a distributed machine learning method in which client nodes train deep neural network models locally using their own training data and then send that trained model to a server, which then aggregates all of the trained models into a globally trained model. This protects personal information while enabling machine learning with vast amounts of data through parallel learning. Nodes that train local models are typically mobile or edge devices from which data can be easily obtained. These devices typically run on batteries and use wireless communication, which limits their power, making their computing performance and reliability significantly lower than that of high-performance computing servers. Therefore, training takes a long time, and if something goes wrong, the client may have to start training again from the beginning. If this happens frequently, the training of the global model may slow down and the final performance may deteriorate. In a general computing system, a checkpointing method can be used to solve this problem, but applying an existing checkpointing method to FL may result in excessive overheads. This paper proposes a new FL method for situations with many fault-prone nodes that efficiently utilizes checkpoints.

Downhole fluid sampling is ubiquitous during exploration and appraisal because formation fluid properties have a strong impact on field development decisions. Efficient planning of sampling operations and interpretation of obtained data require a model-based approach. We present a framework for forward and inverse modeling of filtrate contamination cleanup during fluid sampling. The framework consists of a deep learning (DL) proxy forward model coupled with a Markov Chain Monte Carlo (MCMC) approach for the inverse model. The DL forward model is trained using precomputed numerical simulations of immiscible filtrate cleanup over a wide range of in situ conditions. The forward model consists of a multilayer neural network with both recurrent and linear layers, where inputs are defined by a combination of reservoir and fluid properties. A model training and selection process is presented, including network depth and layer size impact assessment. The inverse framework consists of an MCMC algorithm that stochastically explores the solution space using the likelihood of the observed data computed as the mismatch between the observations and the model predictions. The developed DL forward model achieved up to 50% increased accuracy compared with prior proxy models based on Gaussian process regression. Additionally, the new approach reduced the memory footprint by a factor of ten. The same model architecture and training process proved applicable to multiple sampling probe geometries without compromising performance. These attributes, combined with the speed of the model, enabled its use in real-time inversion applications. Furthermore, the DL forward model is amendable to incremental improvements if new training data becomes available. Flowline measurements acquired during cleanup and sampling hold valuable information about formation and fluid properties that may be uncovered through an inversion process. Using measurements of water cut and pressure, the MCMC inverse model achieved 93% less calls to the forward model compared to conventional gradient-based optimization along with comparable quality of history matches. Moreover, by obtaining estimates of the full posterior parameter distributions, the presented model enables more robust uncertainty quantification.

ObjectivesDecisions about health often involve risk, and different decision makers interpret and value risk information differently. Furthermore, an individual’s attitude toward health-specific risks can contribute to variation in health preferences and behavior. This study aimed to determine whether and how health-risk attitude and heterogeneity of health preferences are related. MethodsTo study the association between health-risk attitude and preference heterogeneity, we selected 3 discrete choice experiment case studies in the health domain that included risk attributes and accounted for preference heterogeneity. Health-risk attitude was measured using the 13-item Health-Risk Attitude Scale (HRAS-13). We analyzed 2 types of heterogeneity via panel latent class analyses, namely, how health-risk attitude relates to (1) stochastic class allocation and (2) systematic preference heterogeneity. ResultsOur study did not find evidence that health-risk attitude as measured by the HRAS-13 distinguishes people between classes. Nevertheless, we did find evidence that the HRAS-13 can distinguish people’s preferences for risk attributes within classes. This phenomenon was more pronounced in the patient samples than in the general population sample. Moreover, we found that numeracy and health literacy did distinguish people between classes. ConclusionsModeling health-risk attitude as an individual characteristic underlying preference heterogeneity has the potential to improve model fit and model interpretations. Nevertheless, the results of this study highlight the need for further research into the association between health-risk attitude and preference heterogeneity beyond class membership, a different measure of health-risk attitude, and the communication of risks.

Supervised machine learning (ML) models rely on labels in the training data to learn the patterns of interest. In Earth science applications, these labels are usually collected by humans either as labels annotated on imagery (such as land cover class) or as in situ measurements (such as soil moisture). Both annotations and in situ measurements contain uncertainties resulting from factors such as class misinterpretation and device error. These training data uncertainties propagate through the ML model training and result in uncertainties in the model outputs. Therefore, it is essential to quantify these uncertainties and incorporate them in the model [1].In this research, we will present results of inputting semantic segmentation label uncertainties into the model training and show that it improves model performance. The experiment is run using the LandCoverNet training dataset which contains global land cover labels based on time-series of Sentinel-2 multispectral imagery [2]. These labels are human annotations derived using a consensus algorithm based on the input labels from three independent annotators. The training dataset contains the consensus label and consensus score, and we treat the latter as a measure of uncertainty for each labeled pixel in the data. Our model architecture is a Convolutional Neural Network (CNN) trained on a subset of LandCoverNet with the rest of the dataset used for validation. We compare the results of this experiment with the same model trained on the dataset without the uncertainty information and show the improvement in the accuracy of the model. [1] Elmes, A., Alemohammad, H., Avery, R., Caylor, K., Eastman, J., Fishgold, L., Friedl, M., Jain, M., Kohli, D., Laso Bayas, J., Lunga, D., McCarty, J., Pontius, R., Reinmann, A., Rogan, J., Song, L., Stoynova, H., Ye, S., Yi, Z.-F., Estes, L. (2020). Accounting for Training Data Error in Machine Learning Applied to Earth Observations. Remote Sensing, 12(6), 1034. https://doi.org/10.3390/rs12061034[2] Alemohammad, H., Booth, K. (2020). LandCoverNet: A global benchmark land cover classification training dataset. NeurIPS 2020 Workshop on AI for Earth Sciences. http://arxiv.org/abs/2012.03

Microservice architectures are increasingly used to modularize IoT applications and deploy them in distributed and heterogeneous edge computing environments. Over time, these microservice-based IoT applications are susceptible to performance anomalies caused by resource hogging (e.g., CPU or memory), resource contention, etc., which can negatively impact their Quality of Service and violate their Service Level Agreements. Existing research on performance anomaly detection for edge computing environments focuses on model training approaches that either achieve high accuracy at the expense of a time-consuming and resource-intensive training process or prioritize training efficiency at the cost of lower accuracy. To address this gap, while considering the resource constraints and the large number of devices in modern edge platforms, we propose two clustering-based model training approaches: (1) intra-cluster parameter transfer learning (ICPTL)-based model training and (2) cluster-level model (CM) training. These approaches aim to find a tradeoff between the training efficiency of anomaly detection models and their accuracy. We compared the models trained under ICPTL and CM to models trained for specific devices (most accurate, least efficient) and a single general model trained for all devices (least accurate, most efficient). Our findings show that ICPTL’s model accuracy is comparable to that of the model per device approach while requiring only 40% of the training time. In addition, CM further improves training efficiency by requiring 23% less training time and reducing the number of trained models by approximately 66% compared to ICPTL, yet achieving a higher accuracy than a single general model.