Unbalanced Data Distribution Research Articles

Defense against backdoor attack in federated learning

As a new distributed machine learning framework, Federated Learning (FL) effectively solves the problems of data silo and privacy protection in the field of artificial intelligence. However, for its independent devices, heterogeneous data and unbalanced data distribution, it is more vulnerable to adversarial attack, especially backdoor attack. In this paper, we investigate typical backdoor attacks in FL, containing model replacement attack and adaptive backdoor attack. Based on attack initiating round, we divide backdoor attack into convergence-round attack and early-round attack. In addition, we respectively design a defense scheme with model pre-aggregation and similarity measurement to detect and remove backdoor model under convergence-round attack and a defense scheme with backdoor neuron activation to remove backdoor under early-round attack. Experiments and performance analysis show that compared to benchmark schemes, our defense scheme with similarity measurement obtains the highest backdoor detection accuracy under model replacement attack (25% increase) and adaptive backdoor attack (67% increase) at the convergence round. Moreover, detection effect is the most stable. Compared to defense of participant-level differential privacy and adversarial training, our defense scheme with backdoor neuron activation can rapidly remove malicious effects of backdoor without reducing the main task accuracy under early-round attack. Thus, the robustness of FL can be improved greatly with our defense schemes. We make our key codes public at Github https://github.com/lsw3130104597/Backdoor_detection.

Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation

Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model while the training data remains decentralized. Existing FL methods typically share model parameters or employ co-distillation to address the issue of unbalanced data distribution. However, they suffer from communication bottlenecks. More importantly, they risk privacy leakage risk. In this work, we develop a privacy preserving and communication efficient method in a FL framework with one-shot offline knowledge distillation using unlabeled, cross-domain, non-sensitive public data. We propose a quantized and noisy ensemble of local predictions from completely trained local models for stronger privacy guarantees without sacrificing accuracy. Based on extensive experiments on image classification and text classification tasks, we show that our method outperforms baseline FL algorithms with superior performance in both accuracy and data privacy preservation.

A hierarchical learning model for inferring the labels of points of interest with unbalanced data distribution

The point of interest (POI) is a critical part of spatial database, which has been widely used in many fields, such as navigation, mapping and urban planning. The data quality of POIs (e.g., missing and incorrect labels) has a large impact on the effectiveness of geospatial applications, especially regarding the non-professional collection characteristics of OpenStreetMap (OSM) data. The conventional neural network model predicted multi-category data labels directly from a single level, and did not consider the uneven distribution of data among POI categories. The predicted labels tended to be the types with larger data volume, and it is difficult to generalize the small-scale samples. Taking into account the large difference in the data volume among POI categories, this paper proposes a neural network prediction method based on multi-level POI category organization. Through the hierarchical aggregation of small sample categories, we established a POI category tree structure, which achieved a relatively balanced division of data volume at different levels of the tree structure. Specifically, the proposed method first roughly classified the POIs at an abstraction level and then inferred the detailed labels according to the hierarchy of the tree structure. We conducted extensive experiments on two datasets and the results demonstrated that our method outperformed traditional methods by a large margin.

Two-Stage Solar Flare Forecasting Based on Convolutional Neural Networks

Solar flares are solar storm events driven by the magnetic field in the solar activity area. Solar flare, often associated with solar proton event or CME, has a negative impact on ratio communication, aviation, and aerospace. Therefore, its forecasting has attracted much attention from the academic community. Due to the limitation of the unbalanced distribution of the observation data, most techniques failed to effectively learn complex magnetic field characteristics, leading to poor forecasting performance. Through the statistical analysis of solar flare magnetic map data observed by SDO/HMI from 2010 to 2019, we find that unsupervised clustering algorithms have high accuracy in identifying the sunspot group in which the positive samples account for the majority. Furthermore, for these identified sunspot groups, the ensemble model that integrates the capability of boosting and convolutional neural network (CNN) achieves high-precision prediction of whether the solar flares will occur in the next 48 hours. Based on the above findings, a two-stage solar flare early warning system is established in this paper. The F1 score of our method is 0.5639, which shows that it is superior to the traditional methods such as logistic regression and support vector machine (SVM).

Dynamic Scheduling for Over-the-Air Federated Edge Learning With Energy Constraints

Machine learning and wireless communication technologies are jointly facilitating an intelligent edge, where federated edge learning (FEEL) is emerging as a promising training framework. As wireless devices involved in FEEL are resource limited in terms of communication bandwidth, computing power and battery capacity, it is important to carefully schedule them to optimize the training performance. In this work, we consider an over-the-air FEEL system with analog gradient aggregation, and propose an energy-aware dynamic device scheduling algorithm to optimize the training performance within the energy constraints of devices, where both communication energy for gradient aggregation and computation energy for local training are considered. The consideration of computation energy makes dynamic scheduling challenging, as devices are scheduled before local training, but the communication energy for over-the-air aggregation depends on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{2}$ </tex-math></inline-formula> -norm of local gradient, which is known only after local training. We thus incorporate estimation methods into scheduling to predict the gradient norm. Taking the estimation error into account, we characterize the performance gap between the proposed algorithm and its offline counterpart. Experimental results show that, under a highly unbalanced local data distribution, the proposed algorithm can increase the accuracy by 4.9% on CIFAR-10 dataset compared with the myopic benchmark, while satisfying the energy constraints.

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

Synthetic aperture radar automatic target recognition (SAR ATR) uses computer processing capabilities to infer the classes of the targets without human intervention. For SAR ATR, deep learning gradually emerges as a powerful tool and achieves promising performance. However, it faces serious challenges of how to deal with incremental recognition scenarios. The existing deep learning-based SAR ATR methods usually predefine the total number of recognition classes. In realistic applications, the new tasks/classes will be added continuously. If all old data are stored and mixed with newly added data to update the model, the storage pressure and time consumption make the application infeasible. In this article, the high plastic error correction incremental learning (HPecIL) is proposed to address the model degradation and plasticity decline in the incremental scenario. Multiple optimal models trained on old tasks are used to correct accumulative errors and alleviate model degradation. Moreover, the sharp data distribution shift due to newly added data can also result in the model underperforming. A class-balanced training batch is constructed to deal with the issue of unbalanced data distribution. To make a tradeoff between model stability and model plasticity, low-effect nodes in the model are removed to boost the efficiency of model update. The proposed HPecIL outperforms the other state-of-the-art methods in incremental recognition scenarios. The experimental results demonstrate the effectiveness of the proposed method.

Garbage Classification Using Ensemble DenseNet169

Garbage is a big problem for the sustainability of the environment, economy, and society, where the demand for waste increases along with the growth of society and its needs. Where in 2019 Indonesia was able to produce 66-67 million tons of waste, which is an increase from the previous year of 2 to 3 million tons of waste. Waste management efforts have been carried out by the government, including by making waste sorting regulations. This sorting is known as 3R (reduce, reuse, recycle), but most people do not sort their waste properly. In this study, a model was developed that can sort out 6 types of waste including: cardboard, glass, metal, paper, plastic, trash. The model was built using the transfer learning method with a pretrained model DenseNet169. Where the optimal results are shown for the classes that have been oversampling previously with an accuracy of 91%, an increase of 1% compared to the model that has an unbalanced data distribution. The next model optimization is done by applying the ensemble method to the four models that have been oversampled on the training dataset with the same architecture. This method shows an increase of 3% to 5% while the final accuracy on the test of dataset is 96%.

Intrusion detection system in the Smart Distribution Network: A feature engineering based AE-LightGBM approach

Bi-directional communication network is the foundation of Smart Distribution Network(SDN), but it also exposes SDN to more serious communication risks. Most of the current researches​ solve this problem by Intrusion Detection Systems(IDSs), yet they focus more on the detection performance, while ignoring the real-time requirements, redundant network traffic features, and unbalanced data distribution in SDN communication network. To address these problems, this paper proposes a feature engineering based AutoEncoder(AE)-LightGBM intrusion detection system for SDN. The proposed system uses Borderline-SMOTE to optimize the data distribution firstly, after that, AE is used for feature engineering to extract the main features. Finally LightGBM is trained to recognize the intrusion using the extracted features. Experimental results on the KDDCup99 and NSL-KDD datasets show that the accuracy, precision, and F1-score performance of the proposed model are better than those of traditional models and related works, and have significant advantages in real-time performance.

Age Group Classification using Convolutional Neural Network (CNN)

Age group classification is a complex task that is used to classify facial images or videos into predetermined age categories. It is an important task due to its numerous applications such as health, security, authentication system, recruitment, and also in intelligent social robots. Convolutional Neural Network (CNN) has recently shown excellent performance in analysing human face images and videos. This paper proposed an age group classification task using CNN that trained and tested with an All-Age Face (AAF) dataset. FaceNet deep learning model that uses CNN was applied in this study to compute a 128-d embedding that quantifies the face of the age group. The experiment included two age groups: Adolescence and Mature Adulthood. The proposed age group classification model achieved 84.90% accuracy for the training images and 85.12% accuracy for the test images. The experimental results showed that CNN is capable of achieving competitive classification accuracy throughout two age groups in the AAF dataset with unbalanced data distribution.

Performance Analysis of Weather Forecasting using Machine Learning Algorithms (Analisis Performansi Prakiraan Cuaca Menggunakan Algoritma Machine Learning)

Weather forecasting are very important in various fields of human life, including in big cities. The need for accurate weather forecasts will be effective and efficient in managing the quality of civilization flexibly. In many cases it is found that the results of weather forecasts in the same city differ depending on the radius. This of course requires a precise and accurate algorithm to determine it. The algorithm used is based on machine learning type of artificial neural network which compares backpropagation and bayessian regularization. The results obtained show that bayessian regularization outperforms backpropagation with the smallest MSE and the highest accuracy and the shortest computation time to determine sunny, cloudy, light rain and heavy rain forecasts. The unbalanced distribution of data causes fluctuations in the MSE calculation and accuracy. The addition of training will improve system performance which is indicated by a significant increase in accuracy. Likewise, decreasing the MSE can increase the accuracy of the system to reach the point of convergence. This is an indicator that the performance of Bayessian regularization is the recommended algorithm for forecasting weather in cities and their surroundings, even between provinces or between countries.

A Novel Transfer Dictionary Learning Strategy for Rolling Bearing Fault Identification With a Mixed Noise Model

Rolling bearings are an essential core component in the power transmission systems of rotating machinery. Many previous dictionary learning (DL) methods have proven to be powerful for rolling bearing fault diagnosis. Existing diagnostic methods based on DL usually involve analyzing training and test data drawn from the same distribution. However, an unbalanced data distribution is a common phenomenon in practice. In addition, the signals in the source and target domains are often mixed with noise and irrelevant information in complex environments, which greatly reduces the performance of DL. Therefore, a new DL method based on a mixed noise model for transfer DL (TDL-MN) is presented. Specifically, the TDL-MN model is constructed to overcome noise and irrelevant signal interference in the DL process under variable conditions. Moreover, rolling bearing health status is diagnosed through sparse representation (SR) classification by calculating the target domain samples corresponding to the redundancy error in the SR vector. The results for two cases verify the ability of the TDL-MN method to accurately identify bearing fault types under complex and variable working conditions. Comparisons with other methods verify the applicability and superiority of the proposed method.

A Multimode Anomaly Detection Method Based on OC-ELM for Aircraft Engine System

The practical industrial processes possess the characteristics of multimode, unbalanced data distribution, and complex types of abnormalities, which are challenging to the anomaly detection task of complex industrial systems. In this paper, a novel anomaly detection framework based on one-class extreme learning machine (OC-ELM) for the multimode system is presented. To tackle the multiple operation modes, a clustering algorithm is first applied to distinguish the operation modes of the system. The corresponding detection models are built under different operation modes resulting in the multiple models operated in parallel. In addition, the proposed method constructs the reasonable boundary of the complex data distribution, reflecting the equipment running in the healthy or the normal state. The anomaly detection index is obtained according to the deviation degree between the testing sample and the normal model. As a result, a global monitoring index reflecting the degradation state is obtained by combining the anomaly monitoring indices of the equipment under multiple operation modes. The proposed method is verified on a public dataset of aircraft engines, and the advantages are demonstrated by comparing with the implemented detection model without handling the information of operation modes, and the multiple principal component analysis method.

A Model-Bias Matrix Factorization Approach for Course Score Prediction

Recommender algorithms are widely used in e-commercial platforms to recommend users suitable items according to users’ preferences. In recent years, an increasing amount of attention has been paid to the application of recommender system in education. There are many online learning systems that can recommend students suitable courses according to students’ learning performances. However, there are few universities using recommender system to recommend students suitable elective courses. It is generally known that students in higher grade take the courses earlier than those in lower grade. Therefore, the elective course scores of sophomores can be predicted by using the course score information from students of higher grades. However, the unbalanced distribution of course-enrollment data makes it hard to predict the score of the courses that are in a low selection rate. Therefore, we propose a model-bias matrix factorization algorithm to predict sophomores’ elective course scores, which takes into account the score prediction deviation caused by the course selection rate so as to make more accurate prediction than the traditional matrix factorization approaches. The experimental results show that our proposed model outperforms the state-of-the-art methods in the task of university students’ course score prediction.

Effect Improved for High-Dimensional and Unbalanced Data Anomaly Detection Model Based on KNN-SMOTE-LSTM

High-dimensional and unbalanced data anomaly detection is common. Effective anomaly detection is essential for problem or disaster early warning and maintaining system reliability. A significant research issue related to the data analysis of the sensor is the detection of anomalies. The anomaly detection is essentially an unbalanced sequence binary classification. The data of this type contains characteristics of large scale, high complex computation, unbalanced data distribution, and sequence relationship among data. This paper uses long short-term memory networks (LSTMs) combined with historical sequence data; also, it integrates the synthetic minority oversampling technique (SMOTE) algorithm and K-nearest neighbors (kNN), and it designs and constructs an anomaly detection network model based on kNN-SMOTE-LSTM in accordance with the data characteristic of being unbalanced. This model can continuously filter out and securely generate samples to improve the performance of the model through kNN discriminant classifier and avoid the blindness and limitations of the SMOTE algorithm in generating new samples. The experiments demonstrated that the structured kNN-SMOTE-LSTM model can significantly improve the performance of the unbalanced sequence binary classification.

PD45-04 CLINICOPATHOLOGIC, LABORATORY, AND RADIOGRAPHIC INTEGRATED MACHINE LEARNING ALGORITHM FOR MALIGNANT RENAL MASSES

You have accessJournal of UrologyKidney Cancer: Epidemiology & Evaluation/Staging/Surveillance II (PD45)1 Apr 2020PD45-04 CLINICOPATHOLOGIC, LABORATORY, AND RADIOGRAPHIC INTEGRATED MACHINE LEARNING ALGORITHM FOR MALIGNANT RENAL MASSES Sagar Patel*, Caitlin Hensel, Jiaxian He, Rupali Bose, Charles Ellis, and Stephen Riggs Sagar Patel*Sagar Patel* More articles by this author , Caitlin HenselCaitlin Hensel More articles by this author , Jiaxian HeJiaxian He More articles by this author , Rupali BoseRupali Bose More articles by this author , Charles EllisCharles Ellis More articles by this author , and Stephen RiggsStephen Riggs More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000000932.04AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: With pervasive abdominal imaging modalities, detection rates of incidental renal tumors have dramatic increased. The minority of clinicians utilize a holistic approach to work-up and educate patients about malignancy risk of newly diagnosed renal masses, subsequently limiting shared decision making, impairing transparency of recent clinical data, and ultimately, increasing nephrectomy utilization. To individualize renal mass diagnosis, we have evaluated clinicopathologic, laboratory, and radiographic parameters to develop a comprehensive machine learning approach to quantitate the likelihood of renal malignancy. METHODS: We retrospectively analyzed all nephrectomy patients from 2014-2019 using our prospectively maintained RedCap database from the electronic medical records. Demographic, clinical, and surgical pathology results were utilized. Individual components of nephrometry score was calculated at time of presentation using high-resolution abdominal imaging. Logistic regression models were developed to predict likelihood of malignant mass at presentation to our tertiary center, Levine Cancer Institute. For our unbalanced data distribution, ROC curves were used to determine cutoff values while balancing false positive (FPR) and false negative rates (FNR). For variables with less than 10% missing data, R package MICE was implemented. RESULTS: 1020 renal masses (99 benign, 921 malignant) were utilized to develop a machine learning algorithm. Benign nephrectomy specimen rate was 9.7%. Cohort was split into training (N = 765) and testing (N = 255) sets. Table 1 depicts input variables for predictive model. Based on the prediction calculated by logistic regression and the ROC metrics generated from the training data, an optimal cutoff of 0.89 was determined to maintain a 73% TPR and a 60% TNR in the test data. CONCLUSIONS: Using machine learning, we developed an integrated tool to quantify the preoperative likelihood of malignancy for new renal masses. Through optimization of our machine learning algorithm, 60% of true benign tumors (59 patients) would have evaded nephrectomies in our cohort. Model precision computations can improve understanding of renal cancer diagnosis, reduce healthcare cost, and decrease resource utilization. Source of Funding: NA © 2020 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 203Issue Supplement 4April 2020Page: e914-e914 Advertisement Copyright & Permissions© 2020 by American Urological Association Education and Research, Inc.MetricsAuthor Information Sagar Patel* More articles by this author Caitlin Hensel More articles by this author Jiaxian He More articles by this author Rupali Bose More articles by this author Charles Ellis More articles by this author Stephen Riggs More articles by this author Expand All Advertisement PDF downloadLoading ...

Adaptive Indoor Area Localization for Perpetual Crowdsourced Data Collection.

The accuracy of fingerprinting-based indoor localization correlates with the quality and up-to-dateness of collected training data. Perpetual crowdsourced data collection reduces manual labeling effort and provides a fresh data base. However, the decentralized collection comes with the cost of heterogeneous data that causes performance degradation. In settings with imperfect data, area localization can provide higher positioning guarantees than exact position estimation. Existing area localization solutions employ a static segmentation into areas that is independent of the available training data. This approach is not applicable for crowdsoucred data collection, which features an unbalanced spatial training data distribution that evolves over time. A segmentation is required that utilizes the existing training data distribution and adapts once new data is accumulated. We propose an algorithm for data-aware floor plan segmentation and a selection metric that balances expressiveness (information gain) and performance (correctly classified examples) of area classifiers. We utilize supervised machine learning, in particular, deep learning, to train the area classifiers. We demonstrate how to regularly provide an area localization model that adapts its prediction space to the accumulating training data. The resulting models are shown to provide higher reliability compared to models that pinpoint the exact position.

Detection of Social Network Spam Based on Improved Extreme Learning Machine

With the rapid advancement of the online social network, social media like Twitter has been increasingly critical to real life and become the prime objective of spammers. Twitter spam detection refers to a complex task for the involvement of a range of characteristics, and spam and non-spam have caused unbalanced data distribution in Twitter. To solve the mentioned problems, Twitter spam characteristics are analyzed as the user attribute, content, activity and relationship in this study, and a novel spam detection algorithm is designed based on regularized extreme learning machine, called the Improved Incremental Fuzzy-kernel-regularized Extreme Learning Machine (I2FELM), which is used to detect the Twitter spam accurately. As revealed from the experience validation results, the proposed I2FELM can efficiently identify the balanced and unbalanced dataset. Moreover, with few characteristics taken, the I2FELM can more effectively detect spam, which proves the effectiveness of the algorithm.

Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection

Collaborative learning allows multiple clients to train a joint model without sharing their data with each other. Each client performs training locally and then submits the model updates to a central server for aggregation. Since the server has no visibility into the process of generating the updates, collaborative learning is vulnerable to poisoning attacks where a malicious client can generate a poisoned update to introduce backdoor functionality to the joint model. The existing solutions for detecting poisoned updates, however, fail to defend against the recently proposed attacks, especially in the non-IID (independent and identically distributed) setting. In this article, we present a novel defense scheme to detect anomalous updates in both IID and non-IID settings. Our key idea is to realize client-side cross-validation, where each update is evaluated over other clients’ local data. The server will adjust the weights of the updates based on the evaluation results when performing aggregation. To adapt to the unbalanced distribution of data in the non-IID setting, a dynamic client allocation mechanism is designed to assign detection tasks to the most suitable clients. During the detection process, we also protect the client-level privacy to prevent malicious clients from knowing the participations of other clients, by integrating differential privacy with our design without degrading the detection performance. Our experimental evaluations on three real-world datasets show that our scheme is significantly robust to two representative poisoning attacks.

FedSCR: Structure-based Communication Reduction for Federated Learning

Federated Learning allows edge devices to collaboratively train a shared model on their local data without leaking user privacy. The non-independent-and-identically-distributed (Non-IID) property of data distribution, which leads to severe accuracy degradation, and enormous communication overhead for aggregating parameters should be tackled in federated learning. In this article, we conduct a detailed analysis of parameter updates on the Non-IID datasets and compare the difference with the IID setting. Experimental results exhibit that parameter update matrices are structure-sparse and show that more gradients could be identified as negligible updates on the Non-IID data. As a result, we propose a structure-based communication reduction algorithm, called FedSCR, that reduces the number of parameters transported through the network while maintaining the model accuracy. FedSCR aggregates the parameter updates over channels and filters, identifies and removes the redundant updates by comparing the aggregated values with a threshold. Unlike the traditional structured pruning methods, FedSCR retains the complete model that does not require to be retrained and fine-tuned. The local loss and weight divergence on each device vary a lot because of the unbalanced data distribution. We further propose an adaptive FedSCR, that dynamically changes the bounded threshold, to enhance the model robustness on the Non-IID data. Evaluation results show that our proposed strategies achieve almost 50 percent upstream communication reduction without loss of accuracy. FedSCR can be integrated into state-of-the-art federated learning algorithms to dramatically reduce the number of parameters pushed to the global server with a tolerable accuracy reduction.

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

In the domain of speech emotion recognition (SER), generally there is an unbalanced data distribution of emotional samples in existing emotional datasets. Moreover, different fragment areas in an utterance contribute diversely to SER. To address these two issues, this paper proposes a new SER method by combining a unified first-order attention network with data balance. The proposed method firstly utilizes the strategy of data balance to augment and balance the training data. Then, a pre-trained convolutional neural network (CNN) model (i.e., VGGish) is fine-tuned on target emotional datasets to learn segment-level speech features from the extracted Log Mel-spectrograms. Next, the unified first-order attention mechanism, including different feature-pooling strategies such as sum, min, max, mean, and standard deviation (std), is embedded into the output of a bi-directional long short-term memory (Bi-LSTM) network. This is used for learning high-level discriminative segment-level features, and simultaneously aggregating the learned segment-level features into fixed-length utterance-level features for SER. Finally, based on utterance-level features, the softmax layer in a Bi-LSTM network is adopted to conduct final emotion classification task. Extensive experiments are implemented on three public datasets such as BAUM-1s, AFEW5.0, and CHEAVD2.0, demonstrate the advantage of the proposed method.