Distribution Of Training Data Research Articles

VV-YOLO: A Vehicle View Object Detection Model Based on Improved YOLOv4

Vehicle view object detection technology is the key to the environment perception modules of autonomous vehicles, which is crucial for driving safety. In view of the characteristics of complex scenes, such as dim light, occlusion, and long distance, an improved YOLOv4-based vehicle view object detection model, VV-YOLO, is proposed in this paper. The VV-YOLO model adopts the implementation mode based on anchor frames. In the anchor frame clustering, the improved K-means++ algorithm is used to reduce the possibility of instability in anchor frame clustering results caused by the random selection of a cluster center, so that the model can obtain a reasonable original anchor frame. Firstly, the CA-PAN network was designed by adding a coordinate attention mechanism, which was used in the neck network of the VV-YOLO model; the multidimensional modeling of image feature channel relationships was realized; and the extraction effect of complex image features was improved. Secondly, in order to ensure the sufficiency of model training, the loss function of the VV-YOLO model was reconstructed based on the focus function, which alleviated the problem of training imbalance caused by the unbalanced distribution of training data. Finally, the KITTI dataset was selected as the test set to conduct the index quantification experiment. The results showed that the precision and average precision of the VV-YOLO model were 90.68% and 80.01%, respectively, which were 6.88% and 3.44% higher than those of the YOLOv4 model, and the model's calculation time on the same hardware platform did not increase significantly. In addition to testing on the KITTI dataset, we also selected the BDD100K dataset and typical complex traffic scene data collected in the field to conduct a visual comparison test of the results, and then the validity and robustness of the VV-YOLO model were verified.

IMPLEMENTATION OF ARTIFICIAL NEURAL NETWORK WITH BACKPROPAGATION ALGORITHM FOR RATING CLASSIFICATION ON SALES OF BLACKMORES IN TOKOPEDIA

The rating assessment classification contains feedback from consumers, which is given in the form of stars which aims to assess a product. However, the amount of data in the classification process often have differences in each class or is called an imbalanced dataset. These problems can affect the classification results. An imbalanced dataset can be overcome by applying random oversampling. To classify the rating assessment, this study proposes the Neural network method, which has a good accuracy level with the backpropagation algorithm and applies random oversampling to overcome the unbalanced amount of data. The results indicate that the neural network method with the backpropagation algorithm can classify the available data with an accuracy level of 85%. The application of resampling data using random oversampling and determining the amount of distribution of training data, testing data, number of epochs and the correct number of batch sizes affect the results obtained.

GraphCS: Graph-based client selection for heterogeneity in federated learning

Federated Learning coordinates many mobile devices to train an artificial intelligence model while preserving data privacy collaboratively. Mobile devices are usually equipped with totally different hardware configurations, leading to various training capabilities. At the same time, the distribution of the local training data is highly heterogeneous across different clients. Randomly selecting the clients to participate in the training process results in poor model performance and low system efficiency.In this paper, we propose GraphCS, a graph-based client selection framework for heterogeneity in Federated Learning. GraphCS first measures the distribution coupling across the clients via the model gradients. After that, it divides the clients into different groups according to the diversity of the local datasets. At the same time, it well estimates the runtime training capability of each client by jointly considering the hardware configuration and resource contention caused by the concurrently running apps. With the distribution coupling information and runtime training capability, GraphCS selects the best clients in order to well balance the model accuracy and overall training progress. We evaluate the performance of GraphCS with mobile devices with different hardware configurations on various datasets. The experiment results show that our approach improves model accuracy up to 45.69%. Meanwhile, it reduces communication and computation overhead 87.35% and 89.48% at best, respectively. Furthermore, GraphCS accelerates the overall training process up to 35×.

Residual neural network with mixed loss based on batch training technique for identification of EGFR mutation status in lung cancer.

Epidermal growth factor receptor (EGFR) is the key to targeted therapy with tyrosine kinase inhibitors in lung cancer. Traditional identification of EGFR mutation status requires biopsy and sequence testing, which may not be suitable for certain groups who cannot perform biopsy. In this paper, using easily accessible and non-invasive CT images, the residual neural network (ResNet) with mixed loss based on batch training technique is proposed for identification of EGFR mutation status in lung cancer. In this model, the ResNet is regarded as the baseline for feature extraction to avoid the gradient disappearance. Besides, a new mixed loss based on the batch similarity and the cross entropy is proposed to guide the network to better learn the model parameters. The proposed mixed loss utilizes the similarity among batch samples to evaluate the distribution of training data, which can reduce the similarity of different classes and the difference of the same classes. In the experiments, VGG16Net, DenseNet, ResNet18, ResNet34 and ResNet50 models with the mixed loss are trained on the public CT dataset with 155 patients including EGFR mutation status from TCIA. The trained networks are employed to the collected preoperative CT dataset with 56 patients from the cooperative hospital for validating the efficiency of the proposed models. Experimental results show that the proposed models are more appropriate and effective on the lung cancer dataset for identifying the EGFR mutation status. In these models, the ResNet34 with mixed loss is optimal (accuracy = 81.58%, AUC = 0.8861, sensitivity = 80.02%, specificity = 82.90%).

Out-of-Distribution (OOD) Detection and Generalization Improved by Augmenting Adversarial Mixup Samples

Deep neural network (DNN) models are usually built based on the i.i.d. (independent and identically distributed), also known as in-distribution (ID), assumption on the training samples and test data. However, when models are deployed in a real-world scenario with some distributional shifts, test data can be out-of-distribution (OOD) and both OOD detection and OOD generalization should be simultaneously addressed to ensure the reliability and safety of applied AI systems. Most existing OOD detectors pursue these two goals separately, and therefore, are sensitive to covariate shift rather than semantic shift. To alleviate this problem, this paper proposes a novel adversarial mixup (AM) training method which simply executes OOD data augmentation to synthesize differently distributed data and designs a new AM loss function to learn how to handle OOD data. The proposed AM generates OOD samples being significantly diverged from the support of training data distribution but not completely disjoint to increase the generalization capability of the OOD detector. In addition, the AM is combined with a distributional-distance-aware OOD detector at inference to detect semantic OOD samples more efficiently while being robust to covariate shift due to data tampering. Experimental evaluation validates that the designed AM is effective on both OOD detection and OOD generalization tasks compared to previous OOD detectors and data mixup methods.

The Effects of Daubechies Wavelet Basis Function (DWBF) and Decomposition Level on the Performance of Artificial Intelligence-Based Atrial Fibrillation (AF) Detection Based on Electrocardiogram (ECG) Signals

This research studies the effects of both Daubechies wavelet basis function (DWBF) and decomposition level (DL) on the performance of detecting atrial fibrillation (AF) based on electrocardiograms (ECGs). ECG signals (consisting of 23 AF data and 18 normal data from MIT-BIH) were decomposed at various levels using several types of DWBF to obtain four wavelet coefficient features (WCFs), namely, minimum (min), maximum (max), mean, and standard deviation (stdev). These features were then classified to detect the presence of AF using a support vector machine (SVM) classifier. Distribution of training and testing data for the SVM uses the 5-fold cross-validation (CV) principle to produce optimum detection performance. In this study, AF detection performance is measured and analyzed based on accuracy, sensitivity, and specificity metrics. The results of the analysis show that accuracy tends to decrease with increases in the decomposition level. In addition, it becomes stable in various types of DWBF. For both sensitivity and specificity, the results of the analysis show that increasing the decomposition level also causes a decrease in both sensitivity and specificity. However, unlike the accuracy, changing the DWBF type causes both two metrics to fluctuate over a wider range. The statistical results also indicate that the highest AF accuracy detection (i.e., 94.17%) is obtained at the Daubechies 2 (DB2) function with a decomposition level of 4, whereas the highest sensitivity, 97.57%, occurs when the AF detection uses DB6 with a decomposition level of 2. Finally, DB2 with decomposition level 4 results in 96.750% for specificity. The finding of this study is that selecting the appropriate DL has a more significant effect than DWBF on AF detection using WCF.

Analisis Pengaruh Komposisi Data Training dan Data Testing pada Penggunaan PCA dan Algoritma Decision Tree untuk Klasifikasi Penderita Penyakit Liver

Liver disease is one of the diseases that is difficult to detect and becomes the largest contributor to deaths because it is considered a silent killer without symptoms. Liver disease can be detected based on abnormalities in the number of contents in the human body. The Indian Liver Patient Dataset (ILPD) dataset has many variables related to content in the body of liver patient data which are used as parameters in the classification of liver disease patients. Previous studies have shown that only two variables influence the ILPD dataset. The purpose of this study is to examine the use of the Principal Component Analysis (PCA) method to determine the optimal number of features in the context of classification of liver disease and examine the percentage distribution of data training and data testing which produces the best accuracy. The ILPD dataset was obtained from the UCI Machine Learning website with a total of 583 rows of data and 11 features. The percentage of training data and testing data used is 50%:50%, 60%:40%, 70%:30%, 73%:27%, 75%:25%, 80%:20%, 83%:17%, 85%:15% and 90%:10%. The use of different training and testing data percentages serves to find the best accuracy. The PCA result feature is used as input for the Decision Tree C4.5 classification algorithm. The experimental results show that using the training and testing data distribution percentage of 90%:10% and after the application of PCA produces the highest accuracy, namely 78.40% which is obtained for the number of PCA components n = 8.

Meta Multi-Instance Multi-Label learning by heterogeneous network fusion

Multi-Instance Multi-Label Learning (MIML) models complex objects (bags), each of which is composed with a set of instances and associated with a set of labels. Current MIML solutions still focus on a single-type of bags and assume an independent and identically distribution (IID) of training data. But these bags are linked with objects of other types, which also encode the semantics of bags. In addition, they generally need abundant labeled data for training. To effectively mine MIML objects linked with objects of other types, we propose a heterogeneous network embedding and meta learning based approach (MetaMIML). MetaMIML introduces the context learner with network embedding to learn context representations of bags for structure information extraction, the task learner to extract the meta knowledge for fast adapting to new tasks with scarce training objects, and finally fuses the structural and attribute information to predict the labels of bags/instances. In this way, MetaMIML can not only naturally deal with MIML objects at data level improving, but also exploit the power of meta-learning at the model enhancing. Experiments on benchmark datasets demonstrate that MetaMIML achieves a significantly better performance than state-of-the-art algorithms.

Investigation of inverse design of multilayer thin-films with conditional invertible neural networks

In this work, we apply conditional invertible neural networks (cINN) to inversely design multilayer thin-films given an optical target in order to overcome limitations of state-of-the-art optimization approaches. Usually, state-of-the-art algorithms depend on a set of carefully chosen initial thin-film parameters or employ neural networks which must be retrained for every new application. We aim to overcome those limitations by training the cINN to learn the loss landscape of all thin-film configurations within a training dataset. We show that cINNs can generate a stochastic ensemble of proposals for thin-film configurations that are reasonably close to the desired target depending only on random variables. By refining the proposed configurations further by a local optimization, we show that the generated thin-films reach the target with significantly greater precision than comparable state-of-the-art approaches. Furthermore, we tested the generative capabilities on samples which are outside of the training data distribution and found that the cINN was able to predict thin-films for out-of-distribution targets, too. The results suggest that in order to improve the generative design of thin-films, it is instructive to use established and new machine learning methods in conjunction in order to obtain the most favorable results.

Fairness-aware training of face attribute classifiers via adversarial robustness

Developing fair deep learning models for identity-sensitive applications (e.g., face attribute recognition) has gained increasing attention from the research community. Indeed, it has been observed that deep models can easily overfit to the bias of the training set, resulting in discriminative performance against certain demographic groups during test time. Motivated by the observation that a biased classifier could result in different adversarial robustness among training samples in different demographic groups (robustness bias), we argue that such adversarial robustness information of individual training samples could imply whether the training data distribution is fair among different demographic groups. In other words, under a fair classifier, the training samples from different demographic groups are expected to show similar or comparable adversarial robustness. Therefore, in this work, we propose to re-weight the training loss of individual training samples using their adversarial robustness, and provide the fairness-awareness in the training process. Extensive experimental results on CelebA dataset show that the face attribute classifiers could learn significantly greater demographic fairness under our proposed training objective and outperform other state-of-the-art re-weighting fairness algorithms on different face recognition applications. Moreover, our proposed method also reduces the non-trivial robustness bias among different demographic groups, preventing the under-represented demographic groups from higher adversarial threats.

Metric to evaluate distribution shift from behavioral cloning for fuel-optimal landing policies

The development of robust fuel-optimal feedback controllers for the pinpoint landing problem has application to a variety of aerospace vehicles including lunar and planetary landers. Imitation learning has been used for a variety of low-fidelity lander formulations to train a policy based on numerically generated open-loop optimal trajectories. The simplest technique, Behavioral Cloning, can suffer from distribution shift, where the distribution of training data is different than the distribution of states seen in rollout. This paper presents a new metric to measure this shift, and a comprehensive study is performed on a variety of lander formulations and policy classes. It is found that shift increases with dimensionality of and the level of coupling in equations of motion, and that the non-parametric policy yields more shift than the parametric one.

A novel online degradation model for proton exchange membrane fuel cell based on online transfer learning

A novel online degradation prediction model is proposed to prognosticate the future degradation trend (FDT) of proton exchange membrane fuel cell (PEMFC) stack in this paper. In order to overcome the fact that existing FDT prediction methods of PEMFC stack based on data-driven model rely on the assumption that the operating conditions of the training data and testing data need to be consistent, an end-to-end prediction algorithm based on the combination of transfer learning and transformer neural network, referred to as TLTNN, is proposed to predict the FDT of PEMFC stack. Besides, in order to demonstrate the effectiveness and superiority of the proposed method in prognostics tasks of PEMFC stack FDT, the prediction performance is validated on the PEMFC test system. The results show that the RMSE, MAE and MAPE values of the predicted degradation voltage are 0.00598 V, 0.004842 V and 0.1518%, respectively, which indicates that the proposed TLTNN strategy based on transfer online learning can be used to predict the degradation voltage of PEMFC stack and the superiority of the proposed method is better, thus solving the problem that the distribution of training and test data must be the same in traditional machine learning models.

Open Set Domain Adaptation via Instance Affinity Metric and Fine-Grained Alignment for Remote Sensing Scene Classification

In the practical application of remote sensing scene classification (RSSC), domain adaptation is introduced to handle the situation where the distribution of training data (source) and test data (target) is different. Compared to general domain adaptation, open set domain adaptation (OSDA) is suitable for more realistic situations where there are additional unknown classes in the target domain. The key to solving this problem is to separate unknown samples from target data to avoid negative transfer caused by mismatching unknown/known samples. In this letter, we propose a novel and effective method, named Instance Affinity metric-based Fine-grained Adaptation Network (IAFAN), for OSDA in RSSC. Concretely, an Unknown Sample Separation (USS) mechanism based on the instance affinity-aware matrix is pioneeringly proposed to endow the model with the ability to distinguish unknown samples. In addition, considering the high inter-class similarity and rich intra-class diversity of remote sensing images, we introduce the Sample Discriminability Enhancement (SDE) loss to further increase the inter-class distance and narrow the intra-class difference, thereby alleviating the negative transfer caused by sample misclassification and mismatching. In the feature confusion stage, we specially design Mask-mmd for OSDA as an adaptation metric to conduct semantic fine-grained cross-domain alignment of known samples while keeping unknown samples out of alignment, which avoids negative transfer during the adaptation process. Finally, we evaluate IAFAN on transfer tasks between different public remote sensing datasets, and the results verify that our method significantly outperforms previous methods in RSSC.

An Uncertainty-Aware Health Monitoring Model for Wind Turbine Drivetrains Based On Bayesian Neural Network

Vibration-based fault diagnostics in rotating machinery when combined with deep learning approaches have yielded promising results. Nevertheless, standard deep learning methods can be unreliable when faced with new data from previously unseen faults or unusual operating conditions. To overcome this challenge, a Bayesian convolutional neural network is explored in this study to diagnose faults in wind turbine gearboxes. This Bayesian statistics framework provides accurate results with low uncertainty by evaluating test data with the same training data distribution. When presented with data that it has not seen before, the network signals its uncertainty and recommends human intervention. This helps to reduce the likelihood of incorrect diagnoses resulting from wrong overconfident classifications. The study compares the performance of Bayesian and standard neural networks using a simulation-based database of acceleration signals. These signals are represented as 2D cyclic spectral coherence maps generated from a multi-body dynamic model of a five-Megawatt wind turbine. By incorporating uncertainty into the prediction results, this approach has the potential to significantly reduce false positives and improve maintenance operations’ efficiency and effectiveness.

Better Modeling Out-of-Distribution Regression on Distributed Acoustic Sensor Data Using Anchored Hidden State Mixup

Generalizing the application of machine learning models to situations where the statistical distribution of training and test data are different has been a complex problem. Our contributions in this article are threefold: 1) we introduce an anchored-based out-of-distribution (OOD) Regression Mixup algorithm, leveraging manifold hidden state mixup and observation similarities to form a novel regularization penalty; 2) we provide a first of its kind high-resolution distributed acoustic sensor dataset that is suitable for testing OOD regression modeling, allowing other researchers to benchmark progress in this area; and 3) we demonstrate with an extensive evaluation the generalization performance of the proposed method against existing approaches and then show that our method achieves state-of-the-art performance. We also demonstrate a wider applicability of the proposed method by exhibiting improved generalization performances on other types of regression datasets, including Udacity and Rotation-MNIST datasets.

Semi-supervised joint adaptation transfer network with conditional adversarial learning for rotary machine fault diagnosis

At present, artificial intelligence is booming and has made major breakthroughs in fault diagnosis scenarios. However, the high diagnostic accuracy of most mainstream fault diagnosis methods must rely on sufficient data to train the diagnostic models. In addition, there is another assumption that needs to be satisfied: the consistency of training and test data distribution. When these prerequisites are not available, the effectiveness of the diagnosis model declines dramatically. To address this problem, we propose a semi-supervised joint adaptation transfer network with conditional adversarial learning for rotary machine fault diagnosis. To fully utilize the fault features implied in unlabeled data, pseudo-labels are generated through threshold filtering to obtain an initial pre-trained model. Then, a joint domain adaptation transfer network module based on conditional adversarial learning and distance metric is introduced to ensure the consistency of the distribution in two different domains. Lastly, in three groups of experiments with different settings: a single fault with variable load, a single fault with variable speed, and a mixed fault with variable speed and load, it was confirmed that our method can obtain competitive diagnostic performance.

Molecular potential energy computation via graph edge aggregate attention neural network

Accurate potential energy surface (PES) calculation is the basis of molecular dynamics research. Using deep learning (DL) methods can improve the speed of PES calculation while achieving competitive accuracy to ab initio methods. However, the performance of DL model is extremely sensitive to the distribution of training data. Without sufficient training data, the DL model suffers from overfitting issues that lead to catastrophic performance degradation on unseen samples. To solve this problem, based on the message passing paradigm of graph neural networks, we innovatively propose an edge-aggregate-attention mechanism, which specifies the weight based on node and edge information. Experiments on MDI7 and QM9 datasets show that our model not only achieves higher PES calculation accuracy but also has better generalization ability compared with Schnet, which demonstrates that edge-aggregate-attention can better capture the inherent features of equilibrium and non-equilibrium molecular conformations.

Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalances.

Deep neural networks (DNNs) have rapidly become a de facto choice for medical image understanding tasks. However, DNNs are notoriously fragile to the class imbalance in image classification. We further point out that such imbalance fragility can be amplified when it comes to more sophisticated tasks such as pathology localization, as imbalances in such problems can have highly complex and often implicit forms of presence. For example, different pathology can have different sizes or colors (w.r.t.the background), different underlying demographic distributions, and in general different difficulty levels to recognize, even in a meticulously curated balanced distribution of training data. In this paper, we propose to use pruning to automatically and adaptively identify hard-to-learn (HTL) training samples, and improve pathology localization by attending them explicitly, during training in supervised, semi-supervised, and weakly-supervised settings. Our main inspiration is drawn from the recent finding that deep classification models have difficult-to-memorize samples and those may be effectively exposed through network pruning [15] - and we extend such observation beyond classification for the first time. We also present an interesting demographic analysis which illustrates HTLs ability to capture complex demographic imbalances. Our extensive experiments on the Skin Lesion Localization task in multiple training settings by paying additional attention to HTLs show significant improvement of localization performance by ~2-3%.

Multi-Genre Symbolic Music Generation using Deep Convolutional Generative Adversarial Network

Music is an art that uses sound to convey emotions and ideas. It is a universal language that transcends cultural boundaries and can move and inspire individuals of all ages and cultures. As with every art form, the generation of music is a complex and challenging task. Despite the challenges, music generation has made considerable strides in recent years, owing to the application of artificial intelligence and machine learning. However, most research was focused on the generation of only one genre of music, i.e., classical, jazz, etc., while there are more than 40 genres of music, each with sub-genres. This paper proposes a model for multi-genre music generation using Generative Adversarial Networks (GAN). Considering symbolic music, MIDI tracks were converted into piano-roll form after the extraction of musical information. Subsequently, a GAN based model was trained to learn the distribution of training data, and it generates new data using the learned parameters. Generated music was evaluated based on a survey of musicians and professionals. The survey results validate the GAN’s ability to generate music of multiple genres.

Knowledge Conditioned Variational Learning for One-Class Facial Expression Recognition.

The openness of application scenarios and the difficulties of data collection make it impossible to prepare all kinds of expressions for training. Hence, detecting expression absent during the training (called alien expression) is important to enhance the robustness of the recognition system. So in this paper, we propose a facial expression recognition (FER) model, named OneExpressNet, to quantify the probability that a test expression sample belongs to the distribution of training data. The proposed model is based on variational auto-encoder and enjoys several merits. First, different from conventional one class classification protocol, OneExpressNet transfers the useful knowledge from the related domain as a constraint condition of the target distribution. By doing so, OneExpressNet will pay more attention to the descriptive region for FER. Second, features from both source and target tasks will aggregate after constructing a skip connection between the encoder and decoder. Finally, to further separate alien expression from training expression, empirical compact variation loss is jointly optimized, so that training expression will concentrate on the compact manifold of feature space. The experimental results show that our method can achieve state-of-the-art results in one class facial expression recognition on small-scale lab-controlled datasets including CFEE and KDEF, and large-scale in-the-wild datasets including RAF-DB and ExpW.