Representation Of Training Data Research Articles

PCAlign: a general data augmentation framework for point clouds

With the advancement of 3D scanning technologies and deep learning theories, point cloud-based deep learning networks have gained considerable attention in the fields of 3D vision and computer graphics. Leveraging the rich geometric information present in 3D point clouds, these networks facilitate more accurate feature learning tasks. However, existing networks often suffer from generalization defects caused by variations in pose and inconsistent representations of training data. In this paper, we propose a novel data augmentation framework to overcome these limitations. Our approach utilizes principal component analysis (PCA) to generate four aligned copies of a point cloud. These copies are then input into a multi-channel structure, which is compatible with popular backbones of point cloud-based deep networks. Finally, the outputs of the multi-channel structure are merged to generate rotation-invariant feature learning results. Experimental evaluations demonstrate the efficacy of our framework, showcasing significant improvements in various existing point cloud-based deep learning methods. Notably, our method exhibits enhanced robustness in classification tasks, particularly when dealing with point clouds containing random pose variations and non-uniform densities. Project link: https://github.com/LAB123-tech/PCAlign.

Artificial Intelligence in Military Medicine.

Artificial intelligence (AI) has garnered significant attention for its pivotal role in the national security and health care sectors. However, its utilization in military medicine remains relatively unexplored despite its immense potential. AI operates through evolving algorithms that process extensive datasets, continuously improving accuracy and emulating human learning processes. Generative AI, a type of machine learning, uses algorithms to generate new content, such as images, text, videos, audio, and computer code. These models employ deep learning to encode simplified representations of training data and generate new work resembling the original without being identical. Although many AI applications in military medicine are theoretical, the U.S. Military has implemented several initiatives, often without widespread awareness among its personnel. This article aims to shed light on two resilience initiatives spearheaded by the Joint Artificial Intelligence Center, which is now the Chief Digital and Artificial Intelligence Office. These initiatives aim to enhance commanders' dashboards for predicting troop behaviors and develop models to forecast troop suicidality. Additionally, it outlines 5 key AI applications within military medicine, including (1) clinical efficiency and routine decision-making support, (2) triage and clinical care algorithms for large-scale combat operations, (3) patient and resource movements in the medical common operating picture, (4) health monitoring and biosurveillance, and (5) medical product development. Even with its promising potential, AI brings forth inherent risks and limitations that require careful consideration and discussion. The article also advocates for a forward-thinking approach for the U.S. Military to effectively leverage AI in advancing military health and overall operational readiness.

Backdozer : A Backdoor Detection Methodology for DRL-based Traffic Controllers

While the advent of Deep Reinforcement Learning (DRL) has substantially improved the efficiency of Autonomous Vehicles (AVs), it makes them vulnerable to backdoor attacks that can potentially cause traffic congestion or even collisions. Backdoor functionality is typically implanted by poisoning training datasets with stealthy malicious data, designed to preserve high accuracy on legitimate inputs while inducing desired misclassifications for specific adversary-selected inputs. Existing countermeasures against backdoors predominantly concentrate on image classification, utilizing image-based properties, rendering these methods inapplicable to the regression tasks of DRL-based AV controllers that rely on continuous sensor data as inputs. In this paper, we introduce the first-ever defense against backdoors on regression tasks of DRL-based models, called Backdozer . Our method systematically extracts more abstract features from representations of training data by projecting them into a specific latent subspace and segregating them into several disjoint groups based on the distribution of legitimate outputs. The key observation of Backdozer is that authentic representations for each group reside in one latent subspace, whereas the incorporation of malicious data impacts that subspace. Backdozer optimizes a sample-wise weight vector for the representations capturing the disparities in projections originating from different groups. We experimentally demonstrate that Backdozer can attain \(100\% \) accuracy in detecting backdoors. We also evaluate its effectiveness against three closely related state-of-the-art defenses.

Scheduling and Aggregation Design for Asynchronous Federated Learning Over Wireless Networks

Federated Learning (FL) is a collaborative machine learning (ML) framework that combines on-device training and server-based aggregation to train a common ML model among distributed agents. In this work, we propose an asynchronous FL design with periodic aggregation to tackle the straggler issue in FL systems. Considering limited wireless communication resources, we investigate the effect of different scheduling policies and aggregation designs on the convergence performance. Driven by the importance of reducing the bias and variance of the aggregated model updates, we propose a scheduling policy that jointly considers the channel quality and training data representation of user devices. The effectiveness of our channel-aware data-importance-based scheduling policy, compared with state-of-the-art methods proposed for synchronous FL, is validated through simulations. Moreover, we show that an ``age-aware'' aggregation weighting design can significantly improve the learning performance in an asynchronous FL setting.

Classification of Spatial Objects with the Use of Graph Neural Networks

Classification is one of the most-common machine learning tasks. In the field of GIS, deep-neural-network-based classification algorithms are mainly used in the field of remote sensing, for example for image classification. In the case of spatial data in the form of polygons or lines, the representation of the data in the form of a graph enables the use of graph neural networks (GNNs) to classify spatial objects, taking into account their topology. In this article, a method for multi-class classification of spatial objects using GNNs is proposed. The method was compared to two others that are based solely on text classification or text classification and an adjacency matrix. The use case for the developed method was the classification of planning zones in local spatial development plans. The experiments indicated that information about the topology of objects has a significant impact on improving the classification results using GNNs. It is also important to take into account different input parameters, such as the document length, the form of the training data representation, or the network architecture used, in order to optimize the model.

An IoMT enabled deep learning framework for automatic detection of fetal QRS: A solution to remote prenatal care

The amalgamation of the Internet of medical things with artificial intelligence shows tremendous benefits in health care. Accurate detection of the fetal QRS complex is highly demanded in fetal heart rate monitoring. Detecting fetal heart rate using electrophysiological signals obtained from abdominal electrodes seems a promising alternative approach. The challenges in determining fetal heart rate from abdominal ECG (AECG) require eliminating maternal components and other noises from the signal at higher accuracy. We propose a novel approach using an IoT-based deep learning architecture to detect fetal QRS complex without eliminating the maternal components in the abdominal ECG. The novelty of the proposed algorithm is twofold: (1) The method uses the time–frequency image (TFI) of abdominal signals as input to the deep neural network and hence promotes the availability of rich features and improves the accurate detection of the fetal QRS complex. (2) The algorithm adapts pre-trained models based on transfer learning for the classification task and thus improves the fetal QRS detection. Two time–frequency approaches, namely Hilbert Huang Transform (HHT) and Stockwell transform (ST), are used to represent input AECG signals as two-dimensional images. The 2013 challenge database is used to evaluate the performance of the proposed approach. The TFI representations of training data using HHT and ST are independently used to train the pre-trained models MobileNet and ResNet18. A comparative analysis is provided in the results between the TFI and deep network architecture. The proposed solution can be suitable for an IoT environment enabling remote fetal monitoring.

Device fingerprinting using deep convolutional neural networks

Device fingerprinting is a problem of identifying a network device using network traffic data to secure against cyber-attacks. Automated device classification from a large set of network traffic features space is challenging for the devices connected in the cyberspace. In this work, the idea is to define a device-specific unique fingerprint by analysing solely inter-arrival time of packets as a feature to identify a device. Neural networks are the universal function approximation which learn abstract, highlevel, nonlinear representation of training data. Deep convolution neural network is used on images of inter-arrival time signature for device fingerprinting of 58 non-IoT devices of 5-11 types. To evaluate the performance, we compared ResNet-50 layer and basic CNN-5 layer architectures. We observed that device type identification models perform better than device identification. We also found that when deep learning models are attacked over device signature, the models identify the change in signature, and classify the device in the wrong class thereby the classification performance of the models degrades. The performance of the models to detect the attacks are significantly different from each other though both models indicate the system under attack.

Barriers and dynamical paths in alternating Gibbs sampling of restricted Boltzmann machines

Restricted Boltzmann machines (RBM) are bilayer neural networks used for the unsupervised learning of model distributions from data. The bipartite architecture of RBM naturally defines an elegant sampling procedure, called alternating Gibbs sampling (AGS), where the configurations of the latent-variable layer are sampled conditional to the data-variable layer and vice versa. We study here the performance of AGS on several analytically tractable models borrowed from statistical mechanics. We show that standard AGS is not more efficient than classical Metropolis-Hastings (MH) sampling of the effective energy landscape defined on the data layer. However, RBM can identify meaningful representations of training data in their latent space. Furthermore, using these representations and combining Gibbs sampling with the MH algorithm in the latent space can enhance the sampling performance of the RBM when the hidden units encode weakly dependent features of the data. We illustrate our findings on three datasets: Bars and Stripes and MNIST, well known in machine learning, and the so-called lattice proteins dataset, introduced in theoretical biology to study the sequence-to-structure mapping in proteins.

Classification of Similar Sports Images Using Convolutional Neural Network with Hyper-Parameter Optimization

With the exponential growth of the presence of sport in the media, the need for effective classification of sports images has become crucial. The traditional approaches require carefully hand-crafted features, which make them impractical for massive-scale data and less accurate in distinguishing images that are very similar in appearance. As the deep learning methods can automatically extract deep representation of training data and have achieved impressive performance in image classification, our goal was to apply them to automatic classification of very similar sports disciplines. For this purpose, we developed a CNN-TL-DE method for image classification using the fine-tuning of transfer learning for training a convolutional neural network model with the use of hyper-parameter optimization based on differential evolution. Through the automatic optimization of neural network topology and essential training parameters, we significantly improved the classification performance evaluated on a dataset composed from images of four similar sports—American football, rugby, soccer, and field hockey. The analysis of interpretable representation of the trained model additionally revealed interesting insights into how our model perceives images which contributed to a greater confidence in the model prediction. The performed experiments showed our proposed method to be a very competitive image classification method for distinguishing very similar sports and sport situations.

Target recognition in SAR images via sparse representation in the frequency domain

Classical sparse modeling requires accurate alignment between the query and the training data. This precondition is disadvantageous for target recognition tasks, where, although the target is present in the images, it is infeasible to perfectly register it during training. In addition, the classical approach is less powerful under unconstrained operating conditions. To solve these problems, this paper presents a new sparse signal modeling strategy in the frequency domain. Because signal energy is mainly concentrated on a small portion of low-frequency components, this set of spectrum carries vital information that can be used to discriminates the class of a target. We generated representations by aggregating low-frequency components. They were then used to build sparse signal models. More specifically, the spectral representation of training data were concatenated to form an over-complete dictionary to encode the counterpart of the query as a linear combination of themselves. Sparsity was harnessed to generate an optimal solution, from which an inference can be made. Multiple comparative analyses were made to demonstrate the advantages of the proposed strategy, especially in unconstrained environments.

Modelling of a post-combustion CO2 capture process using deep belief network

This paper presents a study on using deep learning for the modelling of a post-combustion CO2 capture process. Deep learning has emerged as a very powerful tool in machine learning. Deep learning technique includes two phases: an unsupervised pre-training phase and a supervised back-propagation phase. In the unsupervised pre-training phase, a deep belief network (DBN) is pre-trained to obtain initial weights of the subsequent supervised phase. In the supervised back-propagation phase, the network weights are fine-tuned in a supervised manner. DBN with many layers of Restricted Boltzmann Machine (RBM) can extract a deep hierarchical representation of training data. In terms of the CO2 capture process, the DBN model predicts CO2 production rate and CO2 capture level using the following variables as model inputs: inlet flue gas flow rate, CO2 concentration in inlet flue gas, pressure of flue gas, temperature of flue gas, lean solvent flow rate, MEA concentration and temperature of lean solvent. A greedy layer-wise unsupervised learning algorithm is introduced to optimize DBN, which can bring better generalization than a single hidden layer neural network. The developed deep architecture network models can then be used in the optimisation of the CO2 capture process.

Deep Haar scattering networks

An orthogonal Haar scattering transform is a deep network computed with a hierarchy of additions, subtractions and absolute values over pairs of coefficients. Unsupervised learning optimizes Haar pairs to obtain sparse representations of training data with an algorithm of polynomial complexity. For signals defined on a graph, a Haar scattering is computed by cascading orthogonal Haar wavelet transforms on the graph, with Haar wavelets having connected supports. It defines a representation which is invariant to local displacements of signal values on the graph. When the graph connectivity is unknown, unsupervised Haar learning can provide a consistent estimation of connected wavelet supports. Classification results are given on image data bases, defined on regular grids or graphs, with a connectivity which may be known or unknown.

Multi-Label Rules Algorithm Based Associative Classification

Current associative classification (AC) algorithms generate only the most obvious class linked with a rule in the training data set and ignore all other classes. We handle this problem by proposing a learning algorithm based on AC called Multi-label Classifiers based Associative Classification (MCAC) that learns rules associated with multiple classes from single label data. MCAC algorithm extracts classifiers from the whole training data set discovering all possible classes connected with a rule as long as they have sufficient training data representation. Another distinguishing feature of the MCAC algorithm is the classifier building method that cuts down the number of rules treating one known problem in AC mining which is the exponential growth of rules. Experimentations using real application data related to a complex scheduling problem known as the trainer timetabling problem reveal that MCAC's predictive accuracy is highly competitive if contrasted with known AC algorithms.

Spatial associative classification: propositional vs structural approach

In Spatial Data Mining, spatial dimension adds a substantial complexity to the data mining task. First, spatial objects are characterized by a geometrical representation and relative positioning with respect to a reference system, which implicitly define both spatial relationships and properties. Second, spatial phenomena are characterized by autocorrelation, i.e., observations of spatially distributed random variables are not location-independent. Third, spatial objects can be considered at different levels of abstraction (or granularity). The recently proposed SPADA algorithm deals with all these sources of complexity, but it offers a solution for the task of spatial association rules discovery. In this paper the problem of mining spatial classifiers is faced by building an associative classification framework on SPADA. We consider two alternative solutions for associative classification: a propositional and a structural method. In the former, SPADA obtains a propositional representation of training data even in spatial domains which are inherently non-propositional, thus allowing the application of traditional data mining algorithms. In the latter, the Bayesian framework is extended following a multi-relational data mining approach in order to cope with spatial classification tasks. Both methods are evaluated and compared on two real-world spatial datasets and results provide several empirical insights on them.