Centralized Machine Learning Research Articles

Movie Poster Classification Using Federated Learning

Privacy is an important aspect in the movie industry, with production companies going the extra mile to ensure that their data isn't leaked before the intended date of release. With machine learning branching out in almost every industrial field, the movie industry is no exception. However, at present, most project implementations require collaborators, such as the production firms, to make their data available to the development team in order to train the intended model. We investigate the replacement of traditional centralized machine learning models with a decentralized approach of Federated Learning, wherein models can be trained on client devices without the need for each client to send their personal data to the central server. The research has been carried out on the problem statement of classifying genres of movie posters by using neural networks and optimizing the use of color theory. This preserves a protected environment for movie productions' design teams to determine whether their poster designs align with the genre of the movie which would potentially lead to the reach of the movie to the intended audience.

FedSC: A federated learning algorithm based on client-side clustering

<abstract><p>In traditional centralized machine learning frameworks, the consolidation of all data in a central data center for processing poses significant concerns related to data privacy breaches and data sharing complexities. In contrast, federated learning presents a privacy-preserving paradigm by training models on local devices, thus circumventing the need for data transfer. However, in the case of non-IID (non-independent and identically distributed) data distribution, the performance of federated learning will drop. Addressing this predicament, this study introduces the FedSC algorithm as a remedy. The FedSC algorithm initially partitions clients into clusters based on the distribution of their data types. Within each cluster, clients exhibit comparable local optimal solutions, thus facilitating the aggregation of a superior global model. Moreover, the global model trained by the previous cluster serves as the initial model parameter for subsequent clusters, enabling the incorporation of data contributions from each cluster to foster the development of an enhanced global model. Experimental results corroborate the superiority of the FedSC algorithm over alternative federated learning approaches, particularly in non-IID data distributions, thereby establishing its capacity to achieve heightened accuracy.</p></abstract>

UbiNN: A Communication Efficient Framework for Distributed Machine Learning in Edge Computing

Deployment of distributed machine learning at the edge is conducive to reducing latency and protecting privacy associated with transmitting data back to the cloud. Nonetheless, as machine learning models scale, bandwidth resources are limited and heterogeneity in data collection from heterogeneous edge devices is visible, resulting in low model accuracy and high communication overhead. In this paper, a novel distributed deep learning framework called ubiquitous neural network (UbiNN) is proposed to improve communication efficiency without affecting the accuracy of the local neural network model at the edge or the global neural network model in the cloud. As for the accuracy of the neural network model, a common dataset with a small portion of insensitive data is constructed for training the neural network model, and its accuracy is enhanced by a new algorithm based on knowledge distillation and covariance computation (KDCC). Experimental results demonstrate that the test accuracy of UbiNN is extremely close to the centralized machine learning and non-federated learning scheme DDNN and up to 18.74% better than that of other classic federated learning schemes when using public datasets such as MNIST, CIFAR-10, and REUTERS-21578. Meanwhile, communication overheads are substantially reduced in terms of data transmission volume and latency.

Federated System for Transport Mode Detection

Data on transport usage is important in a wide range of areas. These data are often obtained manually through costly and inaccurate interviews. In the last decade, several researchers explored the use of smartphone sensors for the automatic detection of transport modes. However, such works have focused on developing centralized machine learning mechanisms. This centralized approach requires user data to be transferred to a central server and, therefore, does not satisfy a transport mode detection mechanism’s practical response time and privacy needs. This research presents the Federated System for Transport Mode Detection (FedTM). The main contribution of FedTM is exploring Federated Learning on transport mode detection using smartphone sensors. In FedTM, both the training and inference process is moved to the client side (smartphones), reducing response time and increasing privacy. The FedTM was designed using a Neural Network for the classification task and obtained an average accuracy of 80.6% in three transport classes (cars, buses and motorcycles). Other contributions of this work are: (i) The use of data collected only on the curves of the route. Such reduction in data collection is important, given that the system is decentralized and the training and inference phases take place on smartphones with less computational capacity. (ii) FedTM and centralized classifiers are compared with regard to execution time and detection performance. Such a comparison is important for measuring the pros and cons of using Federated Learning in the transport mode detection task.

Efficient Asynchronous Federated Learning for AUV Swarm.

The development of automatic underwater vehicles (AUVs) has brought about unprecedented profits and opportunities. In order to discover the hidden valuable data detected by an AUV swarm, it is necessary to aggregate the data detected by AUV swarm to generate a powerful machine learning model. Traditional centralized machine learning generates a large number of data exchanges and faces problems of enormous training data, large-scale models, and communication. In underwater environments, radio waves are strongly absorbed, and acoustic communication is the only feasible technology. Unlike electromagnetic wave communication on land, the bandwidth of underwater acoustic communication is extremely limited, with the transmission rate being only 1/105 of the electromagnetic wave. Therefore, traditional centralized machine learning cannot support underwater AUV swarm training. In recent years, federated learning could only interact with model parameters without interacting with data, which greatly reduced communication costs. Therefore, this paper introduces federated learning into the collaboration of an AUV swarm. In order to further reduce the constraints of underwater scarce communication resources on federated learning and alleviate the straggler effect, in this work, we designed an asynchronous federated learning method. Finally, we constructed the optimization problem of minimizing the weighted sum of delay and energy consumption, relying on jointly optimizing the AUV CPU frequency and signal transmission power. In order to solve this complex optimization problem of high-dimensional non-convex time series accumulation, we transformed the problem into a Markov decision process (MDP) and use the proximal policy optimization 2 (PPO2) algorithm to solve this problem. The simulation results demonstrate the effectiveness and superiority of our method.

A Federated Learning Framework Based on Incremental Weighting and Diversity Selection for Internet of Vehicles

With the rapid increase of data, centralized machine learning can no longer meet the application requirements of the Internet of Vehicles (IoV). On the one hand, both car owners and regulators pay more attention to data privacy and are unwilling to share data, which forms the isolated data island challenge. On the other hand, the incremental data generated in IoV are massive and diverse. All these issues have brought challenges of data increment and data diversity. The current common federated learning or incremental learning frameworks cannot effectively integrate incremental data with existing machine learning (ML) models. Therefore, this paper proposes a Federated Learning Framework Based on Incremental Weighting and Diversity Selection for IoV (Fed-IW&DS). In Fed-IW&DS, a vehicle diversity selection algorithm was proposed, which uses a variety of performance indicators to calculate diversity scores, effectively reducing homogeneous computing. Also, it proposes a vehicle federated incremental algorithm that uses an improved arctangent curve as the decay function, to realize the rapid fusion of incremental data with existing ML models. Moreover, we have carried out several sets of experiments to test the validity of the proposed Fed-IW&DS framework’s performance. The experimental results show that, under the same global communication round and similar computing time, the Fed-IW&DS framework has significantly improved performance in all aspects compared to the frameworks FED-AVG, FED-SGD, FED-prox & the decay functions linear, square curve and arc tangent. Specifically, the Fed-IW&DS framework improves the Acc (accuracy), loss (loss), and Matthews correlation coefficient (MCC) by approximately 32%, 83%, and 66%, respectively. This result shows that Fed-IW&DS is a more reliable solution than the common frameworks of federated learning, and it can effectively deal with the dynamic incremental data in the IoV scenario. Our findings should make a significant contribution to the field of federated learning.

Multichannel ALOHA Optimization for Federated Learning With Multiple Models

Large-scale wireless sensor networks are instrumental for several Internet of Things (IoT) applications involving data analytics and machine learning. The huge data volume generated by such networks imposes a change of paradigm from centralized machine learning to decentralized. Federated Learning (FL) is a well-known type of decentralized machine learning, whose efficiency heavily depends on the use of wireless communication resources. Random Access (RA) protocols, such as ALOHA, despite their simplicity, can improve the convergence time of FL systems if multiple orthogonal channels are used. This letter considers that devices are involved in the optimization of more than one model in a FL system, and then proposes an optimum method to allocate wireless resources in a multi-channel ALOHA setup. The proposed method outperforms uniform and fully-shared channel allocations in terms of convergence time.

FedOComp: Two-Timescale Online Gradient Compression for Over-the-Air Federated Learning

Federated learning (FL) is a machine learning framework, where multiple distributed edge Internet of Things (IoT) devices collaboratively train a model under the orchestration of a central server while keeping the training data distributed on the IoT devices. FL can mitigate the privacy risks and costs from data collection in traditional centralized machine learning. However, the deployment of standard FL is hindered by the expense of the communication of the gradients from the devices to the server. Hence, many gradient compression methods have been proposed to reduce the communication cost. However, the existing methods ignore the structural correlations of the gradients and, therefore, lead to a large compression loss which will decelerate the training convergence. Moreover, many of the existing compression schemes do not enable over-the-air aggregation and, hence, require huge communication resources. In this work, we propose a gradient compression scheme, named FedOComp, which leverages the correlations of the stochastic gradients in FL systems for efficient compression of the high-dimension gradients with over-the-air aggregation. The proposed design can achieve a smaller deceleration of the training convergence compared to other gradient compression methods since the compression kernel exploits the structural correlations of the gradients. It also directly enables over-the-air aggregation to save communication resources. The derived convergence analysis and simulation results further illustrate that under the same power cost, the proposed scheme has a much faster convergence rate and higher test accuracy compared to existing baselines.

Federated Learning: Collaborative Machine Learning without Centralized Training Data

Federated learning (also known as collaborative learning) is a machine learning technique that trains an algorithm without transferring data samples across numerous decentralized edge devices or servers. This strategy differs from standard centralized machine learning techniques in which all local datasets are uploaded to a single server, as well as more traditional decentralized alternatives, which frequently presume that local data samples are uniformly distributed. Federated learning allows several actors to collaborate on the development of a single, robust machine learning model without sharing data, allowing crucial issues such as data privacy, data security, data access rights, and access to heterogeneous data to be addressed. Defence, telecommunications, internet of things, and pharmaceutical industries are just a few of the sectors where it has applications.

Open problems in medical federated learning

PurposeThis study aims to summarize the critical issues in medical federated learning and applicable solutions. Also, detailed explanations of how federated learning techniques can be applied to the medical field are presented. About 80 reference studies described in the field were reviewed, and the federated learning framework currently being developed by the research team is provided. This paper will help researchers to build an actual medical federated learning environment.Design/methodology/approachSince machine learning techniques emerged, more efficient analysis was possible with a large amount of data. However, data regulations have been tightened worldwide, and the usage of centralized machine learning methods has become almost infeasible. Federated learning techniques have been introduced as a solution. Even with its powerful structural advantages, there still exist unsolved challenges in federated learning in a real medical data environment. This paper aims to summarize those by category and presents possible solutions.FindingsThis paper provides four critical categorized issues to be aware of when applying the federated learning technique to the actual medical data environment, then provides general guidelines for building a federated learning environment as a solution.Originality/valueExisting studies have dealt with issues such as heterogeneity problems in the federated learning environment itself, but those were lacking on how these issues incur problems in actual working tasks. Therefore, this paper helps researchers understand the federated learning issues through examples of actual medical machine learning environments.

Federated machine learning for a facilitated implementation of Artificial Intelligence in healthcare - a proof of concept study for the prediction of coronary artery calcification scores.

The implementation of Artificial Intelligence (AI) still faces significant hurdles and one key factor is the access to data. One approach that could support that is federated machine learning (FL) since it allows for privacy preserving data access. For this proof of concept, a prediction model for coronary artery calcification scores (CACS) has been applied. The FL was trained based on the data in the different institutions, while the centralized machine learning model was trained on one allocation of data. Both algorithms predict patients with risk scores ≥5 based on age, biological sex, waist circumference, dyslipidemia and HbA1c. The centralized model yields a sensitivity of c. 66% and a specificity of c. 70%. The FL slightly outperforms that with a sensitivity of 67% while slightly underperforming it with a specificity of 69%. It could be demonstrated that CACS prediction is feasible via both, a centralized and an FL approach, and that both show very comparable accuracy. In order to increase accuracy, additional and a higher volume of patient data is required and for that FL is utterly necessary. The developed "CACulator" serves as proof of concept, is available as research tool and shall support future research to facilitate AI implementation.

Providing Location Information at Edge Networks: A Federated Learning-Based Approach

Recently, the development of mobile edge computing has enabled exhilarating edge artificial intelligence (AI) with fast response and low communication costs. The location information of edge devices is essential to support the edge AI in many scenarios, like smart home, intelligent transportation systems, and integrated health care. Taking advantage of deep learning intelligence, the centralized machine learning (ML)-based positioning technique has received heated attention from both academia and industry. However, some potential issues, such as location information leakage and huge data traffic, limit its application. Fortunately, a newly emerging privacy-preserving distributed ML mechanism, named federated learning (FL), is expected to alleviate these concerns. In this article, we illustrate a framework of FL-based localization systems as well as the involved entities at edge networks. Moreover, the advantages of such a system are elaborated. On the practical implementation of it, we investigate the field-specific issues associated with system-level solutions, which are further demonstrated over a real-word database. Moreover, future challenging open problems in this field are outlined.

Privacy-Preserving Federated Depression Detection From Multisource Mobile Health Data

Depression is one of the most common mental illnesses, and the symptoms shown by patients are different, making it difficult to diagnose in the process of clinical practice and pathological research. Although researchers hope that artificial intelligence can contribute to the diagnosis and treatment of depression, the traditional centralized machine learning methods need to aggregate patient data, and the data privacy of patients with mental illness needs to be strictly confidential, which hinders machine learning algorithms’ clinical application. To solve the problem of medical data privacy with depression, in this article, we implement a study of federated learning to analyze and diagnose depression. First, we propose a general multiview federated learning framework using multisource data, which can extend any traditional machine learning model to support federated learning across different institutions or parties. Second, we employ later fusion methods to solve the problem of inconsistent time series of multiview data. Finally, we compare the federated framework with other cooperative learning frameworks in performance and discuss the related results. The experimental results show that in the case of participating in federated learning with enough participants, the prediction accuracy of depression score can reach 85.13%, which is about 15% higher than local training. When the number of participants is small and the amount of data is sufficient, the prediction accuracy of depression score can also reach 84.32%, and the improvement rate is about 9%.

Federated Learning: A signal processing perspective

The dramatic success of deep learning is largely due to the availability of data. Data samples are often acquired on edge devices, such as smartphones, vehicles, and sensors, and in some cases cannot be shared due to privacy considerations. Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local data sets, without explicitly exchanging the data. Learning in a federated manner differs from conventional centralized machine learning and poses several core unique challenges and requirements, which are closely related to classical problems studied in the areas of signal processing and communications. Consequently, dedicated schemes derived from these areas are expected to play an important role in the success of federated learning and the transition of deep learning from the domain of centralized servers to mobile edge devices.

BC-EdgeFL: A Defensive Transmission Model Based on Blockchain-Assisted Reinforced Federated Learning in IIoT Environment

Under the times of the Industrial Internet of Things, the traditional centralized machine learning management method cannot deal with such huge data streams, and the problem of data privacy has aroused widespread concern. In view of these difficulties, in this article, we use the advantages of edge computing and federated learning, combined with the outstanding characteristics of the blockchain, to propose a secure data transmission method. First, we separate the local model updating process from the mobile device independent process; second, we add an edge server so that most of the computation is carried out on the server, which improves the learning efficiency; and finally, we use a distributed architecture of the blockchain to protect data security and privacy. Extensive simulation experiments show that the accuracy of our model can reach 98 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> . In addition, BC-EdgeFLs interception rate of illegal information can reach 0.8, which has good defensive capabilities. Therefore, the security of data transmission can be strongly guaranteed.

Editorial: Introduction to the Issue on Distributed Machine Learning for Wireless Communication

The papers in this special section focus on the use of distributed machine learning for wireless communications. With the emergence of new application scenarios (e.g., real-time and interactive services and Internet of Things) and the fast development of smart terminals, wireless data traffic has increased drastically, and the existing wireless networks cannot completely meet the technical requirements of the next generation mobile communication networks, e.g., 6G. In recent years, machine learning-based methods have been considered as potential technologies for 6G, because in wireless communication systems, key issues behind synchronization, channel estimation, signal detection, and iterative decoding can be solved by well-designed machine learning algorithms. Currently, most wireless network machine learning solutions require the training data and learning process to be centralized in one or more data centers. However, these centralized machine learning methods expose disadvantages, e.g., privacy security, significant signaling overhead, increased implementation complexity, and high latency, which limit their practicality. The wireless networks of the future must make quicker and more reliable decisions at the network edge.

Federated Learning for Intrusion Detection in the Critical Infrastructures: Vertically Partitioned Data Use Case

One of the challenges in the Internet of Things systems is the security of the critical data, for example, data used for intrusion detection. The paper research construction of an intrusion detection system that ensures the confidentiality of critical data at a given level of intrusion detection accuracy. For this goal, federated learning is used to train an intrusion detection model. Federated learning is a computational model for distributed machine learning that allows different collaborating entities to train one global model without sharing data. This paper considers the case when entities have data that are different in attributes. Authors believe that it is a common situation for the critical systems constructed using Internet of Things (IoT) technology, when industrial objects are monitored by different sets of sensors. To evaluate the applicability of the federated learning for this case, the authors developed an approach and an architecture of the intrusion detection system for vertically partitioned data that consider the principles of federated learning and conducted the series of experiments. To model vertically partitioned data, the authors used the Secure Water Treatment (SWaT) data set that describes the functioning of the water treatment facility. The conducted experiments demonstrate that the accuracy of the intrusion detection model trained using federated learning is compared with the accuracy of the intrusion detection model trained using the centralized machine learning model. However, the computational efficiency of the learning and inference process is currently extremely low. It is explained by the application of homomorphic encryption for input data protection from different data owners or data sources. This defines the necessity to elaborate techniques for generating attributes that could model horizontally partitioned data even for the cases when the collaborating entities share datasets that differ in their attributes.

A federated learning approach for thermal comfort management

Existing thermal comfort prediction approaches by machine learning models have been achieving great success based on large datasets in sustainable Industry 4.0 environment. However, the industrial Internet of Things (IoT) environment generates small-scale datasets where each dataset may contain lots of worker’s private data. The latter is challenging the current prediction approaches as small datasets running a large number of iterations can result in overfitting. Moreover, worker’s privacy has been a public concern throughout recent years. Therefore, there must be a trade-off between developing accurate thermal comfort prediction models and worker’s privacy-preserving. To tackle this challenge, we present a privacy-preserving machine learning technique, federated learning (FL), where an FL-based neural network algorithm (Fed-NN) is proposed for thermal comfort prediction. Fed-NN departs from current centralized machine learning approaches where a universal learning model is updated through a secured parameter aggregation process in place of sharing raw data among different industrial IoT environments. Besides, we designed a branch selection protocol to solve the problem of communication overhead in federating learning. Experimental studies on a real dataset reveal the robustness, accuracy, and stability of our algorithm in comparison to other machine learning algorithms while taking privacy into consideration.

Toward Federated-Learning-Enabled Visible Light Communication in 6G Systems

Visible light communication (VLC) technology was introduced as a key enabler for the next generation of wireless networks, mainly thanks to its simple and low-cost implementation. However, several challenges prohibit the realization of the full potential of VLC, namely, limited modulation bandwidth, ambient light interference, optical diffuse reflection effects, devices&#x0027; nonlinearity, and random receiver orientation. On the contrary, centralized machine learning (ML) techniques have demonstrated significant potential in handling different challenges related to wireless communication systems. Specifically, it has been shown that ML algorithms exhibit superior capabilities in handling complicated network tasks, such as channel equalization, estimation and modeling, resources allocation, and opportunistic spectrum access control, to name a few. Nevertheless, concerns pertaining to privacy and communication overhead when sharing raw data of the involved clients with a server constitute major bottlenecks in the implementation of centralized ML techniques. This has led to the emergence of a new distributed ML paradigm, namely federated learning (FL), which can reduce the cost associated with transferring raw data, and preserve privacy by training ML models locally and collaboratively at the clients&#x0027; side. Hence, it becomes evident that integrating FL into VLC networks can provide ubiquitous and reliable implementation of VLC systems. With this motivation, this is the first in-depth review in the literature on the application of FL in VLC networks. To that end, besides the different architectures and related characteristics of FL, we provide a thorough overview on the main design aspects of FL-based VLC systems. Finally, we also highlight some potential future research directions of FL that are envisioned to substantially enhance the performance and robustness of VLC systems.

Collaborative deep learning framework on IoT data with bidirectional NLSTM neural networks for energy consumption forecasting

Energy consumption forecasting based on IoT data and deep learning algorithm inheriting distributed and collaborative learning is a widely studied topic both in engineering and computer science fields. For different households with drastically different energy consumption patterns, the traditional centralized machine learning (ML) and deep learning (DL) methods suffer problems including inaccuracy, inefficiency and laggings of the prediction performance. In this study, we propose a sophisticated multi-channel bidirectional nested LSTM framework (MC-BiNLSTM) combined with discrete stationary wavelet transform (SWT) for highly accurate and efficient energy consumption forecasting. The main contributions of this study include the decomposition using SWT for accuracy improvement and the collaborative BiNLSTM structure for efficiency improvement. A real-world IoT energy consumption dataset, named UK-DALE, is adopted for the comparative study. The experimental results showed the outperformance of the proposed method from various perspectives over the cutting-edge methods existed in the literature.