Traditional Machine Learning Approaches Research Articles

Physics-Informed Machine Learning Improves Detection of Head Impacts.

In this work we present a new physics-informed machine learning model that can be used to analyze kinematic data from an instrumented mouthguard and detect impacts to the head. Monitoring player impacts is vitally important to understanding and protecting from injuries like concussion. Typically, to analyze this data, a combination of video analysis and sensor data is used to ascertain the recorded events are true impacts and not false positives. In fact, due to the nature of using wearable devices in sports, false positives vastly outnumber the true positives. Yet, manual video analysis is time-consuming. This imbalance leads traditional machine learning approaches to exhibit poor performance in both detecting true positives and preventing false negatives. Here, we show that by simulating head impacts numerically using a standard Finite Element head-neck model, a large dataset of synthetic impacts can be created to augment the gathered, verified, impact data from mouthguards. This combined physics-informed machine learning impact detector reported improved performance on test datasets compared to traditional impact detectors with negative predictive value and positive predictive values of 88 and 87% respectively. Consequently, this model reported the best results to date for an impact detection algorithm for American football, achieving an F1 score of 0.95. In addition, this physics-informed machine learning impact detector was able to accurately detect true and false impacts from a test dataset at a rate of 90% and 100% relative to a purely manual video analysis workflow. Saving over 12 h of manual video analysis for a modest dataset, at an overall accuracy of 92%, these results indicate that this model could be used in place of, or alongside, traditional video analysis to allow for larger scale and more efficient impact detection in sports such as American Football.

Monitoring Driving in a Monotonous Environment: Classification and Recognition of Driving Fatigue Based on Long Short-Term Memory Network

The driver is one of the most important factors in road traffic. Monitoring the driver’s driving status can greatly improve the safety and road operation efficiency of urban road traffic in the case of multiple traffic modes. Fatigue has a significant impact on drivers’ safety on the road, particularly while driving in a monotonous environment for a long time. In this study, the eye movement parameters of 36 drivers were collected through the simulation experiment of a driving simulator. The pupil area and percentage of eye closure (PERCLOS) in driving scenes of the expressway and low-grade rural road were combined with the Stanford Sleepiness Scale (SSS) to determine the threshold of fatigue degree in different monotonous driving scenarios. A recognition model of different fatigue degrees of drivers is built based on the deep learning method of a long short-term memory network (LSTM) to detect the varied fatigue degrees of drivers. The result shows that the fatigue degree of drivers increases as driving time increases on both expressways and low-grade rural roads. In the same driving time, the driver felt tired faster on the expressway, and the fatigue degree was significantly higher than that on the country road. The recognition rate of the established fatigue degree recognition model for driver’s awake state, mild fatigue, moderate fatigue, and severe fatigue is 100%, 93.1%, 98.4%, and 100% respectively, and the total recognition rate can reach 97.8%, which is higher than the recognition accuracy of the traditional machine learning approach.

Ensemble learning of deep learning and traditional machine learning approaches for skin lesion segmentation and classification

SummaryMelanoma is a type of a skin cancer or lesion which has the detrimental ramifications on the human health but with early diagnosis it can be cured easily. The actual identification of skin lesion is very challenging because of factors like a very minute difference between lesion and skin and it is very difficult to differentiate among skin cancer types due to visual comparability. Hence an autonomous system for the diagnosis of true skin cancer type is very useful. In this article, we took the leverage of ensemble learning by combining the features of deep learning architectures with traditional features extraction approaches. For segmentation, we have two pipelines for the feature extraction. We extract the features through traditional split and merge approach as well as from deep learning algorithms of contextual encoding along with the attention mechanism. Later we combine the features of both architectures and predict the segmented region through intersection over union mechanism. After that segmented region is classified into three types of skin lesion using hybrid features of Alex‐Net and VGG‐16 through the transfer learning approach. The evaluation has been performed using the ISIC and PH2 datasets for which achieved segmentation accuracy is 97.8% and 96.7%, respectively. Moreover, hybrid classification network able to attain the 98.2% accuracy.

A Deep Learning-Based Approach for Accurate Diagnosis of Alcohol Usage Severity Using EEG Signals

The majority of conventional alcohol detection approaches are based on traditional machine learning approaches and are unable to retrieve the deeply hidden attributes of alcohol-based electroencephalography (EEG) signals. The intention of this research is to evolve a deep learning-based method for detecting alcohol-related EEG signals automatically. It also investigates whether a deep learning method for alcoholism classification is effective without using any explicit feature extraction and selection steps. To assess this, the paper implements and compares deep learning-based strategies to classify alcohol-related EEG signals. A framework that uses prerecorded raw EEG data as input to a deep learning algorithm called convolutional neural network (CNN), long short-term memory (LSTM), and a cascade model called CNN + LSTM is also implemented for detecting alcoholism on an openly accessible UCI Alcoholic EEG dataset. Some conventional techniques are also implemented on prerecorded raw EEG signals, and the classification results are compared with the proposed methods. The results of the experiments show that the proposed models are capable of achieving average classification accuracies of 99.60%, 98.12%, and 95.95% on the training dataset and 92.77%, 89%, and 91% on the testing dataset along with error rates of 7.5%, 11.90%, and 8.7% from CNN, LSTM, and CNN + LSTM, respectively.

Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from Different Chemical Spaces.

In this work, we benchmark a variety of single- and multi-task graph neural network (GNN) models against lower-bar and higher-bar traditional machine learning approaches employing human engineered molecular features. We consider four GNN variants - Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN), and Attentive Fingerprint (AttentiveFP). So far deep learning models have been primarily benchmarked using lower-bar traditional models solely based on fingerprints, while more realistic benchmarks employing fingerprints, whole-molecule descriptors and predictions from other related endpoints (e. g., LogD7.4) appear to be scarce for industrial ADME datasets. In addition to time-split test sets based on Genentech data, this study benefits from the availability of measurements from an external chemical space (Roche data). We identify GAT as a promising approach to implementing deep learning models. While all the deep learning models significantly outperform lower-bar benchmark traditional models solely based on fingerprints, only GATs seem to offer a small but consistent improvement over higher-bar benchmark traditional models. Finally, the accuracy of in vitro assays from different laboratories predicting the same experimental endpoints appears to be comparable with the accuracy of GAT single-task models, suggesting that most of the observed error from the models is a function of the experimental error propagation.

Deep learning-based identification of genetic variants: application to Alzheimer's disease classification.

Deep learning is a promising tool that uses nonlinear transformations to extract features from high-dimensional data. Deep learning is challenging in genome-wide association studies (GWAS) with high-dimensional genomic data. Here we propose a novel three-step approach (SWAT-CNN) for identification of genetic variants using deep learning to identify phenotype-related single nucleotide polymorphisms (SNPs) that can be applied to develop accurate disease classification models. In the first step, we divided the whole genome into nonoverlapping fragments of an optimal size and then ran convolutional neural network (CNN) on each fragment to select phenotype-associated fragments. In the second step, using a Sliding Window Association Test (SWAT), we ran CNN on the selected fragments to calculate phenotype influence scores (PIS) and identify phenotype-associated SNPs based on PIS. In the third step, we ran CNN on all identified SNPs to develop a classification model. We tested our approach using GWAS data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) including (N= 981; cognitively normal older adults (CN) = 650 and AD = 331). Our approach identified the well-known APOE region as the most significant genetic locus for AD. Our classification model achieved an area under the curve (AUC) of 0.82, which was compatible with traditional machine learning approaches, random forest and XGBoost. SWAT-CNN, a novel deep learning-based genome-wide approach, identified AD-associated SNPs and a classification model for AD and may hold promise for a range of biomedical applications.

Accurate and rapid prediction of tuberculosis drug resistance from genome sequence data using traditional machine learning algorithms and CNN

Effective and timely antibiotic treatment depends on accurate and rapid in silico antimicrobial-resistant (AMR) predictions. Existing statistical rule-based Mycobacterium tuberculosis (MTB) drug resistance prediction methods using bacterial genomic sequencing data often achieve varying results: high accuracy on some antibiotics but relatively low accuracy on others. Traditional machine learning (ML) approaches have been applied to classify drug resistance for MTB and have shown more stable performance. However, there is no study that uses deep learning architecture like Convolutional Neural Network (CNN) on a large and diverse cohort of MTB samples for AMR prediction. We developed 24 binary classifiers of MTB drug resistance status across eight anti-MTB drugs and three different ML algorithms: logistic regression, random forest and 1D CNN using a training dataset of 10,575 MTB isolates collected from 16 countries across six continents, where an extended pan-genome reference was used for detecting genetic features. Our 1D CNN architecture was designed to integrate both sequential and non-sequential features. In terms of F1-scores, 1D CNN models are our best classifiers that are also more accurate and stable than the state-of-the-art rule-based tool Mykrobe predictor (81.1 to 93.8%, 93.7 to 96.2%, 93.1 to 94.8%, 95.9 to 97.2% and 97.1 to 98.2% for ethambutol, rifampicin, pyrazinamide, isoniazid and ofloxacin respectively). We applied filter-based feature selection to find AMR relevant features. All selected variant features are AMR-related ones in CARD database. 78.8% of them are also in the catalogue of MTB mutations that were recently identified as drug resistance-associated ones by WHO. To facilitate ML model development for AMR prediction, we packaged every step into an automated pipeline and shared the source code at https://github.com/KuangXY3/MTB-AMR-classification-CNN.

Multi-tissue DNA methylation microarray signature is predictive of gene function

ABSTRACT Background Transcriptional correlation networks derived from publicly available gene expression microarrays have been previously shown to be predictive of known gene functions, but less is known about the predictive capacity of correlated DNA methylation at CpG sites. Guilt-by-association co-expression methods can adapted for use with DNA methylation when a representative methylation value is created for each gene. We examine how methylation compares to expression in predicting Gene Ontology terms using both co-methylation and traditional machine learning approaches across different types of representative methylation values per gene. Methods We perform guilt-by-association gene function prediction with a suite of models called Methylation Array Network Analysis, using a network of correlated methylation values derived from over 24,000 samples. In generating the correlation matrix, the performance of different methods of collapsing probe-level data effect on the resulting gene function predictions was compared, along with the use of different regions surrounding the gene of interest. Results Using mean comethylation of a given gene to its annotated term had an overall highest prediction macro-AUC of 0.60 using mean gene body methylation, across all Gene Ontology terms. This was increased using the logistic regression approach with the highest macro-AUC of 0.82 using mean gene body methylation, compared to the naive predictor of 0.72. Conclusion Genes correlated in their methylation state are functionally related. Genes clustered in co-methylation space were enriched for chromatin state, PRC2, immune response, and development-related terms.

Housing Price Prediction with Machine Learning

For socioeconomic development and the well-being of citizens, developing a precise model for predicting housing prices is always required. So that, a real estate broker or a house seller/buyer can get an intuition in making well-knowledgeable decisions from the model. In this work, a various set of machine learning algorithms such as Linear Regression, Decision Tree, Random Forest are being implemented to predict the housing prices using available datasets. The housing datasets of 506 samples and 13 feature variables from January 2015 to November 2019 were taken from the StatLib library which is maintained at Carnegie Mellon University. Since housing price is emphatically connected to different factors like location, area, the number of rooms; it requires all of this information to predict individual housing prices. This paper will apply both traditional and advanced machine learning approaches to investigate the difference among several advanced models to explore various impacts of features on prediction methods. This paper will also provide an optimistic result for housing price prediction by comprehensively validating multiple techniques in model execution on regression.

Ontology-aware neural network: a general framework for pattern mining from microbiome data.

With the rapid accumulation of microbiome data around the world, numerous computational bioinformatics methods have been developed for pattern mining from such paramount microbiome data. Current microbiome data mining methods, such as gene and species mining, rely heavily on sequence comparison. Most of these methods, however, have a clear trade-off, particularly, when it comes to big-data analytical efficiency and accuracy. Microbiome entities are usually organized in ontology structures, and pattern mining methods that have considered ontology structures could offer advantages in mining efficiency and accuracy. Here, we have summarized the ontology-aware neural network (ONN) as a novel framework for microbiome data mining. We have discussed the applications of ONN in multiple contexts, including gene mining, species mining and microbial community dynamic pattern mining. We have then highlighted one of the most important characteristics of ONN, namely, novel knowledge discovery, which makes ONN a standout among all microbiome data mining methods. Finally, we have provided several applications to showcase the advantage of ONN over other methods in microbiome data mining. In summary, ONN represents a paradigm shift for pattern mining from microbiome data: from traditional machine learning approach to ontology-aware and model-based approach, which has found its broad application scenarios in microbiome data mining.

Sentiment analysis and spam filtering using the YAC2 clustering algorithm with transferability

Two notable applications of text classification are sentiment analysis and spam filtering. Traditional machine learning approaches to text classification are often complex, non-transferrable, and require supervision. This paper introduces an unsupervised approach to text classification which is relatively simple and transfers between problem domains, while providing accuracy comparable or better than established alternatives. We present an integrated solution which combines a new clustering algorithm, Yet Another Clustering Algorithm (YAC2), with a domain transferrable feature engineering approach for Twitter sentiment analysis and spam filtering of YouTube comments. We evaluate the effectiveness of this integrated solution for Twitter sentiment analysis using three datasets: Starbucks, Verizon, and Southwest Airlines. YouTube spam filtering is evaluated using four datasets: Psy, LMFAO, Shakira,and Katy Perry. We compare the results with established clusteringsolutions: KNN, Spectral, and DBSCAN. Our integrated solution performs better than all the alternatives for sentiment analysis. For spam filtering, YAC2 and KNN perform within 1% of each other and far better than Spectral and DBSCAN for all datasets. Additionally, our feature engineering approach improves accuracy compared to using a traditional method, while significantly reducing model dimensionality, matrix sparsity and providing transferability across the datasets tested.

TBSM: A traffic burst-sensitive model for short-term prediction under special events

Traffic prediction is an important management tool for traffic guidance and control and an effective decision-making tool to help travelers plan routes and avoid congested road sections. However, due to the transient and sudden nature of traffic bursts caused by events and data limitations, mainstream methods do not perform well in short-term traffic prediction for special events (SEs). To address this challenge, we propose a traffic burst-sensitive model (TBSM) for short-term traffic prediction. Specifically, we first define a new state unit with the short-term trend and observed state to represent both the burst case and usual case. Second, a state-and-trend unit similarity degree (SD) measurement method and increment-based prediction model are proposed. The key parameter of this model balances the weight of the short-term trend with the observed state. Finally, we use a deep deterministic policy gradient (DDPG) framework containing long short-term memory (LSTM) networks to realize the self-learning and adjustment of weights to ensure the generality and burst sensitivity of the model. The TBSM is implemented in the district of Beijing Workers’ Stadium, where SEs occur frequently. The results demonstrate that the proposed model performs significantly better than other traditional machine learning approaches and deep learning approaches for SEs. Our TensorFlow implementation of the TBSM is available at https://github.com/buaajh/TBSM-Traffic-burst-sensitive-model-.

PART 1 - Traditional machine learning approaches

PART 1 - Traditional machine learning approaches

Comparing Cross-Subject Performance on Human Activities Recognition Using Learning Models

Human activities recognition (HAR) plays a vital role in fields like ambient assisted living and health monitoring, in which cross-subject recognition is one of the main challenges coming from the diversity of various users. Although recent studies have achieved satisfactory results in a non-cross-subject condition, the recognition performance has significant degradation under the cross-subject criterion. In this paper, we evaluate three traditional machine learning methods and five deep neural network architectures under the same metrics on three popular HAR datasets: mHealth, PAMAP2, and UCIDSADS. The experimental results show that traditional machine learning approaches are generally more robust to the new subject scenarios under strict leave-one-subject-out cross-validation. Extra analysis indicates that hand-crafted features are one major reason for the better performance of traditional machine learning on cross-subject HAR, while deep learning is more prone to learning subject-dependent features under an end-to-end training process. A novel training strategy for decision-tree-based methods is also proposed in this paper, resulting in an improvement on the random forest model which achieves competitive performance at an average F1-score (accuracy) of 94.49% (95.09%), 91.64% (92.21%), and 92.70% (93.29%) on the three datasets, compared with state-of-the-art solutions for cross-subject HAR.

Classification of Motor Imagery EEG Signals Based on Deep Autoencoder and Convolutional Neural Network Approach

The technology of the brain-computer interface (BCI) employs electroencephalogram (EEG) signals to establish direct interaction between the human body and its surroundings with promising applications in medical rehabilitative services and cognitive science. Deep learning approaches, particularly the detection and analysis of motor imagery signals using convolutional neural network (CNN) frameworks have produced outstanding results in the BCI system in recent years. The complex process of data representation, on the other hand, limits practical applications, and the end-to-end approach reduces the accuracy of recognition. Moreover, since noise and other signal sources can interfere with brain electrical capacitance, EEG classifiers are difficult to improve and have limited generalisation ability. To address these issues, this paper proposes a new approach for EEG motor imagery signal classification by using a variational autoencoder to remove noise from the signals, followed by a combination of deep autoencoder (DAE) and a CNN architecture to classify EEG motor imagery signals which is capable of training a deep neural network to replicate its input to output using encoding and decoding operations. Experimental results show that the proposed approach for motor imagery EEG signal classification is feasible and that it outperforms current CNN-based approaches and several traditional machine learning approaches.

An Optimized Convolutional Neural Network with Combination Blocks for Chinese Sign Language Identification

(Aim) Chinese sign language is an essential tool for hearing-impaired to live, learn and communicate in deaf communities. Moreover, Chinese sign language plays a significant role in speech therapy and rehabilitation. Chinese sign language identification can provide convenience for those hearing impaired people and eliminate the communication barrier between the deaf community and the rest of society. Similar to the research of many biomedical image processing (such as automatic chest radiograph processing, diagnosis of chest radiological images, etc.), with the rapid development of artificial intelligence, especially deep learning technologies and algorithms, sign language image recognition ushered in the spring. This study aims to propose a novel sign language image recognition method based on an optimized convolutional neural network. (Method) Three different combinations of blocks: Conv-BN-ReLU-Pooling, Conv-BN-ReLU, Conv-BN-ReLU-BN were employed, including some advanced technologies such as batch normalization, dropout, and Leaky ReLU. We proposed an optimized convolutional neural network to identify 1320 sign language images, which was called as CNN-CB method. Totally ten runs were implemented with the hold-out randomly set for each run. (Results) The results indicate that our CNN-CB method gained an overall accuracy of 94.88 ± 0.99%. (Conclusion) Our CNN-CB method is superior to thirteen state-ofthe-art methods: eight traditional machine learning approaches and five modern convolutional neural network approaches

ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition

The self-regulated recognition of human activities from time-series smartphone sensor data is a growing research area in smart and intelligent health care. Deep learning (DL) approaches have exhibited improvements over traditional machine learning (ML) models in various domains, including human activity recognition (HAR). Several issues are involved with traditional ML approaches; these include handcrafted feature extraction, which is a tedious and complex task involving expert domain knowledge, and the use of a separate dimensionality reduction module to overcome overfitting problems and hence provide model generalization. In this article, we propose a DL-based approach for activity recognition with smartphone sensor data, i.e., accelerometer and gyroscope data. Convolutional neural networks (CNNs), autoencoders (AEs), and long short-term memory (LSTM) possess complementary modeling capabilities, as CNNs are good at automatic feature extraction, AEs are used for dimensionality reduction and LSTMs are adept at temporal modeling. In this study, we take advantage of the complementarity of CNNs, AEs, and LSTMs by combining them into a unified architecture. We explore the proposed architecture, namely, “ConvAE-LSTM”, on four different standard public datasets (WISDM, UCI, PAMAP2, and OPPORTUNITY). The experimental results indicate that our novel approach is practical and provides relative smartphone-based HAR solution performance improvements in terms of computational time, accuracy, F1-score, precision, and recall over existing state-of-the-art methods.

Deep Mining from Omics Data.

Since the advent of high-throughput omics technologies, various molecular data such as genes, transcripts, proteins, and metabolites have been made widely available to researchers. This has afforded clinicians, bioinformaticians, statisticians, and data scientists the opportunity to apply their innovations in feature mining and predictive modeling to a rich data resource to develop a wide range of generalizable prediction models. What has become apparent over the last 10years is that researchers have adopted deep neural networks (or "deep nets") as their preferred paradigm of choice for complex data modeling due to the superiority of performance over more traditional statistical machine learning approaches, such as support vector machines. A key stumbling block, however, is that deep nets inherently lack transparency and are considered to be a "black box" approach. This naturally makes it very difficult for clinicians and other stakeholders to trust their deep learning models even though the model predictions appear to be highly accurate. In this chapter, we therefore provide a detailed summary of the deep net architectures typically used in omics research, together with a comprehensive summary of the notable "deep feature mining" techniques researchers have applied to open up this black box and provide some insights into the salient input features and why these models behave as they do. We group these techniques into the following three categories: (a) hidden layer visualization and interpretation; (b) input feature importance and impact evaluation; and (c) output layer gradient analysis. While we find that omics researchers have made some considerable gains in opening up the black box through interpretation of the hidden layer weights and node activations to identify salient input features, we highlight other approaches for omics researchers, such as employing deconvolutional network-based approaches and development of bespoke attribute impact measures to enable researchers to better understand the relationships between the input data and hidden layer representations formed and thus the output behavior of their deep nets.

Payload Assisted Unmanned Aerial Vehicle Structural Health Monitoring (UAVSHM) for Active Damage Detection

Structural Health Monitoring (SHM) incorporates techniques for implementing a damage detection and characterization for engineering structures. Visual inspection remains the primary technique to inspect the health of civil structures such as bridges, roads, dams and buildings despite being time-consuming and posing a significant risk to human life in inspecting specific structures. Unmanned Aerial Vehicle Assisted Structural Health Monitoring (UAVSHM) has emerged as a viable and promising option for SHM. However, UAVs are equipped with limited battery supply and onboard computational power. They are unsuitable for active damage detection, i.e., detecting damages to civil structures in real time. This paper firstly advocates using an edge device as a UAV payload to conduct active damage detection. Secondly, it compares traditional machine learning and transfer learning approaches to build Convolution Neural Network (CNN) models for damage detection using an edge device. The results show that edge devices have an acceptable latency to facilitate real-time damage detection for UAVSHM.

Integrating Machine Learning Algorithms With Quantum Annealing Solvers for Online Fraud Detection

Machine learning has been increasingly applied in identification of fraudulent transactions. However, most application systems can only detect the incidents after they have already occurred, not at or near real time. As spurious transactions are far fewer than the normal ones, the highly imbalanced data makes fraud detection even more challenging. This study has proposed a detection framework and implemented it using Support Vector Machine (SVM) enhanced with quantum annealing solvers. To evaluate its detection performance and examine the impact of feature selection, we have applied this quantum machine learning (QML) system along with systems built with twelve traditional machine learning methods on two datasets: Israel credit card transactions (non-time series) which is moderately imbalanced and a bank loan dataset (time series) that is highly imbalanced. The result shows that, the quantum enhanced SVM has categorically outperformed the rest in both speed and accuracy with the bank loan dataset. However, it’s detection accuracy is similar to others with Israel credit card transactions data. Furthermore, for both datasets, feature selection has been shown to significantly improve the detection speed, although the improvement on accuracy is marginal. These findings have demonstrated the potential of QML applications on time series based, highly imbalanced data, and the merit of traditional machine learning approaches in non-time series data. This study provides insight on selecting appropriate approach with different types of datasets while taking the tradeoffs of speed, accuracy, and cost into consideration.