Shallow Learning Approaches Research Articles

Multi-source machine learning and spaceborne remote sensing data accurately predict three-dimensional soil moisture in an in-service uranium disposal cell

One reason arid and semi-arid environments have been used to store waste is due to low groundwater recharge, presumably limiting the potential for meteoric water to mobilize and transport contaminants into groundwater. The U.S. Department of Energy Office of Legacy Management (LM) is evaluating selected uranium mill tailings disposal cell covers to be managed as evapotranspiration (ET) covers, where vegetation is used to naturally remove water from the cover profile via transpiration, further reducing deep percolation. An important parameter in monitoring the performance of ET covers is soil moisture (SM). If SM is too high, water may drain into tailings material, potentially transporting contaminants into groundwater; if SM is too low, radon flux may increase through the cover. However, monitoring SM via traditional instrumentation is invasive, expensive, and may fail to account for spatial heterogeneity, especially over vegetated disposal cells. Here we investigated the potential for non-invasive SM monitoring using radar remote sensing and other geospatial data to see if this approach could provide a practical, accurate, and spatially comprehensive tool to monitor SM. We used theoretical simulations to analyze the sensitivity of multi-frequency radar backscatter to SM at different depths of a field-scale (3 ha) drainage lysimeter embedded within an in-service LM disposal cell. We then evaluated a shallow and deep form of machine learning (ML) using Google Earth Engine to integrate multi-source observations and estimate the SM profile across six soil layers from depths of 0–2 m. The ML models were trained using in situ SM measurements from 2019 and validated using data from 2014 to 2018 and 2020–2021. Model predictors included backscatter observations from satellite synthetic aperture radar, vegetation, temperature products from optical infrared sensors, and accumulated, gridded rainfall data. The radar simulations confirmed that the lower frequencies (L- and P-band) and smaller incidence angles show better sensitivity to deeper soil layers and an overall larger SM dynamic range relative to the higher frequencies (C- and X-band). The ML models produced accurate SM estimates throughout the soil profile (r values from 0.75 to 0.94; RMSE = 0.003–0.017 cm3/cm3; bias = 0.00 cm3/cm3), with the simpler shallow-learning approach outperforming a selected deep-learning model. The ML models we developed provide an accurate, cost-effective tool for monitoring SM within ET covers that could be applied to other vegetated disposal cell covers, potentially including those with rock-armored covers.

Time-Series Embeddings from Language Models: A Tool for Wind Direction Nowcasting

Wind direction nowcasting is crucial in various sectors, particularly for ensuring aviation operations and safety. In this context, the TELMo (Time-series Embeddings from Language Models) model, a sophisticated deep learning architecture, has been introduced in this work for enhanced wind-direction nowcasting. Developed by using three years of data from multiple stations in the complex terrain of an international airport, TELMo incorporates the horizontal u (east–west) and v (north–south) wind components to significantly reduce forecasting errors. On a day with high wind direction variability, TELMo achieved mean absolute error values of 5.66 for 2-min, 10.59 for 10-min, and 14.79 for 20-min forecasts, processed within a swift 9-ms/step timeframe. Standard degree-based analysis, in comparison, yielded lower performance, emphasizing the effectiveness of the u and v components. In contrast, a Vanilla neural network, representing a shallow-learning approach, underperformed in all analyses, highlighting the superiority of deep learning methodologies in wind direction nowcasting. TELMo is an efficient model, capable of accurately forecasting wind direction for air traffic operations, with an error less than 20° in 97.49% of the predictions, aligning with recommended international thresholds. This model design enables its applicability across various geographical locations, making it a versatile tool in global aviation meteorology.

VAE-Based Interpretable Latent Variable Model for Process Monitoring.

Latent variable-based process monitoring (PM) models have been generously developed by shallow learning approaches, such as multivariate statistical analysis and kernel techniques. Owing to their explicit projection objectives, the extracted latent variables are usually meaningful and easily interpretable in mathematical terms. Recently, deep learning (DL) has been introduced to PM and has exhibited excellent performance because of its powerful presentation capability. However, its complex nonlinearity prevents it from being interpreted as human-friendly. It is a mystery how to design a proper network structure to achieve satisfactory PM performance for DL-based latent variable models (LVMs). In this article, a variational autoencoder-based interpretable LVM (VAE-ILVM) is developed for PM. Based on Taylor expansions, two propositions are proposed to guide the design of appropriate activation functions for VAE-ILVM, allowing nondisappearing fault impact terms contained in the generated monitoring metrics (MMs). During threshold learning, the sequence of counting that test statistics exceed the threshold is considered a martingale, a representative of weakly dependent stochastic processes. A de la Peña inequality is then adopted to learn a suitable threshold. Finally, two chemical examples verify the effectiveness of the proposed method. The use of de la Peña inequality significantly reduces the minimum required sample size for modeling.

IoT driven joint compressed sensing and shallow learning approach for ECG signal-reconstruction

Because biological signal transmission in real time might be very demanding, cloud and internet of things (IoT) infrastructure are required. To do this, the main component of the signal serves as the focal point of a reconstruction strategy that has been developed. The input is transferred to the intended destination once it has been encoded. Security is an important consideration that must not be disregarded. For long-term healthcare monitoring via lightweight wireless networks, electrocardiogram (ECG) compression is a major difficulty. Reducing energy consumption in wireless data transmission and precisely calculating error rates for data reconstruction are two essential components of compressed sensing. The application of effective encoding methods is crucial for these considerations. We present multi-task compressed sensing (MT-CS), a unique method for compressing ECG data. When used to wireless network systems with several embedded sensors, this technique is quite effective. From the ECG data, the model learns the fundamental adaptive properties needed for correlation. We use the multiparameter intelligent monitoring in intensive care (MIMIC-II) dataset to investigate the performance of the suggested MT-CS reconstruction technique in order to assess its strength and application. In comparison to current compressed sensing methods, the simulation results demonstrate that the suggested reconstruction methodology utilizing MT-CS generates high-quality reconstruction signals with fewer observations.

Synthetic Graphic Well Log Generation Using an Enhanced Deep Learning Workflow: Imbalanced Multiclass Data, Sample Size, and Scalability Challenges

Summary The present study introduces an enhanced deep learning (DL) workflow based on transfer learning (TL) for producing high-resolution synthetic graphic well logs (SGWLs). To examine the scalability of the proposed workflow, a carbonate reservoir with a high geological heterogeneity has been chosen as the case study, and the developed workflow is evaluated on unseen data (i.e., blind well). Data sources include conventional well logs and graphical well logs (GWLs) from neighboring wells. During drilling operations, GWLs are standard practice for collecting data. GWL provides a rapid visual representation of subsurface lithofacies to establish geological correlations. This investigation examines five wells in a southwest Iranian oil field. Due to subsurface geological heterogeneities, the primary challenge of this research lies in addressing the imbalanced facies distribution. The traditional artificial intelligence strategies that manage imbalanced data [e.g., the modified synthetic minority oversampling technique (M-SMOTE) and Tomek link (TKL)] are mainly designed to solve binary problems. However, to adapt these methods to the upcoming imbalanced multiclass situation, one-vs.-one (OVO) and one-vs.-all (OVA) decomposition strategies and ad-hoc techniques are used. Well-known VGG16-1D and ResNet18-1D are used as adaptive very-deep algorithms. Additionally, to highlight the robustness and efficiency of these algorithms, shallow learning approaches of support vector machine (SVM) and random forest (RF) as conventional facies classification methods are also used. The other main challenge is the need for enough data points to train the very deep algorithms, resolved through TL. After identifying a blind well, the other four wells’ data are entered for model training. The average kappa statistic and F-measure, as appropriate imbalance data evaluation metrics, are implemented to assess the designed workflows’ performance. The numerical and visual comparison analysis shows that the VGG16-1D TL model performs better on the blind well data set when combined with the OVA scheme as a decomposition technique and TKL as a binary imbalance data combat tactic. An average kappa statistic of 86.33% and a mean F-measure of 92.09% demonstrate designed workflow superiority. Considering the prevalence of different imbalanced facies distributions, the developed scalable workflow can be efficient and productive for generating SGWL.

Deep learning criminal networks

Recent advances in deep learning methods have enabled researchers to develop and apply algorithms for the analysis and modeling of complex networks. These advances have sparked a surge of interest at the interface between network science and machine learning. Despite this, the use of machine learning methods to investigate criminal networks remains surprisingly scarce. Here, we explore the potential of graph convolutional networks to learn patterns among networked criminals and to predict various properties of criminal networks. Using empirical data from political corruption, criminal police intelligence, and criminal financial networks, we develop a series of deep learning models based on the GraphSAGE framework that are able to recover missing criminal partnerships, distinguish among types of associations, predict the amount of money exchanged among criminal agents, and even anticipate partnerships and recidivism of criminals during the growth dynamics of corruption networks, all with impressive accuracy. Our deep learning models significantly outperform previous shallow learning approaches and produce high-quality embeddings for node and edge properties. Moreover, these models inherit all the advantages of the GraphSAGE framework, including the generalization to unseen nodes and scaling up to large graph structures.

A scoping review on deep learning for next-generation RNA-Seq. data analysis.

In the last decade, transcriptome research adopting next-generation sequencing (NGS) technologies has gathered incredible momentum amongst functional genomics scientists, particularly amongst clinical/biomedical research groups. The progressive enfoldment/adoption of NGS technologies has incited an abundance of next-generation transcriptomic data harbouring an opulence of new knowledge in public databases. Nevertheless, knowledge discovery from these next-generation RNA-Seq. data analysis necessitates extensive bioinformatics know-how besides elaborate data analysis software packages consistent with the type and context of data analysis. Several reliability and reproducibility concerns continue to impede RNA-Seq. data analysis. Characteristic challenges comprise of data quality, hardware and networking provisions, selection and prioritisation of data analysis tools, and yet significantly implementing of robust machine learning algorithms for maximised exploitation of these experimental transcriptomic data. Over the years, numerous machine learning algorithms have been implemented for improved transcriptomic data analysis executing predominantly shallow learning approaches. More recently, deep learning algorithms are becoming more mainstream, and enactment for next-generation RNA-Seq. data analysis could be revolutionary in the coming years in the biomedical domain. In this scoping review, we attempt to determine the existing literature's size and potential nature in deep learning and NGS RNA-Seq. data analysis. An analysis of the contemporary topics of next-generation RNA-Seq. data analysis based on deep learning algorithms is critically reviewed, emphasising open-source resources.

Shallow and deep learning of an artificial neural network model describing a hot flow stress Evolution: A comparative study

In recent years, the utilization of artificial neural networks (ANNs) as regression models to solve the issue of hot flow stress forecasting has become a standard approach. In a connection with this kind of regression issue, employed ANNs are usually learned via a shallow learning technique while only limited attention has been paid to a deep learning method. In the frame of the submitted research, the shallow learning approach is thoroughly compared to the deep learning techniques which are based on the use of a Restricted Boltzmann Machine (RBM) and an Auto-Encoder (AE). To do so, these learning techniques are applied on a feed-forward multi-layer ANN describing the experimental hot flow curve dataset of micro-alloyed medium carbon steel. In comparison with the shallow learning method, both deep learning approaches provided higher accuracy in the network response – especially in the case of a higher number of hidden layers. The results have also shown that neither the RBM-based deep learning method nor the AE-based method had a significant effect on the duration of the necessary calculations. However, it turned out that the RBM-based method can, under certain conditions, lead to a more reliable network performance.

Automatic recognition of Arabic alphabets sign language using deep learning

<span>Technological advancements are helping people with special needs overcome many communications’ obstacles. Deep learning and computer vision models are innovative leaps nowadays in facilitating unprecedented tasks in human interactions. The Arabic language is always a rich research area. In this paper, different deep learning models were applied to test the accuracy and efficiency obtained in automatic Arabic sign language recognition. In this paper, we provide a novel framework for the automatic detection of Arabic sign language, based on transfer learning applied on popular deep learning models for image processing. Specifically, by training AlexNet, VGGNet and GoogleNet/Inception models, along with testing the efficiency of shallow learning approaches based on support vector machine (SVM) and nearest neighbors algorithms as baselines. As a result, we propose a novel approach for the automatic recognition of Arabic alphabets in sign language based on VGGNet architecture which outperformed the other trained models. The proposed model is set to present promising results in recognizing Arabic sign language with an accuracy score of 97%. The suggested models are tested against a recent fully-labeled dataset of Arabic sign language images. The dataset contains 54,049 images, which is considered the first large and comprehensive real dataset of Arabic sign language to the furthest we know.</span>

Weed Classification from Natural Corn Field-Multi-Plant Images Based on Shallow and Deep Learning.

Crop and weed discrimination in natural field environments is still challenging for implementing automatic agricultural practices, such as weed control. Some weed control methods have been proposed. However, these methods are still restricted as they are implemented under controlled conditions. The development of a sound weed control system begins by recognizing the crop and the different weed plants presented in the field. In this work, a classification approach of Zea mays L. (Crop), narrow-leaf weeds (NLW), and broadleaf weeds (BLW) from multi-plant images are presented. Moreover, a large image dataset was generated. Images were captured in natural field conditions, in different locations, and growing stages of the plants. The extraction of regions of interest (ROI) is carried out employing connected component analysis (CCA), whereas the classification of ROIs is based on Convolutional Neural Networks (CNN) and compared with a shallow learning approach. To measure the classification performance of both methods, accuracy, precision, recall, and F1-score metrics were used. The best alternative for the weed classification task at early stages of growth and in natural corn field environments was the CNN-based approach, as indicated by the 97% accuracy value obtained.

Comparing Deep Learning and Shallow Learning Techniques for API Calls Malware Prediction: A Study

Recognition of malware is critical in cybersecurity as it allows for avoiding execution and the downloading of malware. One of the possible approaches is to analyze the executable’s Application Programming Interface (API) calls, which can be done using tools that work in sandboxes, such as Cuckoo or CAPEv2. This chain of calls can then be used to classify if the considered file is benign or malware. This work aims to compare six modern shallow learning and deep learning techniques based on tabular data, using two datasets of API calls containing malware and goodware, where the corresponding chain of API calls is expressed for each instance. The results show the quality of shallow learning approaches based on tree ensembles, such as CatBoost, both in terms of F1-macro score and Area Under the ROC curve (AUC ROC), and training time, making them optimal for making inferences on Edge AI solutions. The results are then analyzed with the explainable AI SHAP technique, identifying the API calls that most influence the process, i.e., those that are particularly afferent to malware and goodware.

A multilayer shallow learning approach to variation prediction and variation source identification in multistage machining processes

Variation propagation modelling in multistage machining processes through use of analytical approaches has been widely investigated for the purposes of dimension prediction and variation source identification. Yet the variation prediction of complex features is non-trivial task to model mathematically. Moreover, the application of the variation propagation approaches and associated variation source identification techniques using Skin Model Shapes is unclear. This paper proposes a multilayer shallow neural network regression approach to predict geometrical deviations of parts given manufacturing errors. The neural network is trained on a simulated data, generated from machining simulation of a point cloud of a part. Further, given a point cloud data of a machined feature, the source of variation can be identified by optimally matching the deviation patterns of the actual surface with that of shallow neural network generated surface. To demonstrate the method, a two-stage machining process and a virtual part that has planar, cylindrical and torus features was considered. The geometric characteristics of machined features and the sources variation could be predicted at an error of 1% and 4.25%, respectively. This work extends the application of Skin Model Shapes in variation propagation analysis in multistage manufacturing.

Gait Activity Classification on Unbalanced Data from Inertial Sensors Using Shallow and Deep Learning.

Activity recognition is one of the most active areas of research in ubiquitous computing. In particular, gait activity recognition is useful to identify various risk factors in people’s health that are directly related to their physical activity. One of the issues in activity recognition, and gait in particular, is that often datasets are unbalanced (i.e., the distribution of classes is not uniform), and due to this disparity, the models tend to categorize into the class with more instances. In the present study, two methods for classifying gait activities using accelerometer and gyroscope data from a large-scale public dataset were evaluated and compared. The gait activities in this dataset are: (i) going down an incline, (ii) going up an incline, (iii) walking on level ground, (iv) going down stairs, and (v) going up stairs. The proposed methods are based on conventional (shallow) and deep learning techniques. In addition, data were evaluated from three data treatments: original unbalanced data, sampled data, and augmented data. The latter was based on the generation of synthetic data according to segmented gait data. The best results were obtained with classifiers built with augmented data, with F-measure results of 0.812 ( = 0.078) for the shallow learning approach, and of 0.927 ( = 0.033) for the deep learning approach. In addition, the data augmentation strategy proposed to deal with the unbalanced problem resulted in increased classification performance using both techniques.

Time Coherent Full-Body Poses Estimated Using Only Five Inertial Sensors: Deep versus Shallow Learning

Full-body motion capture typically requires sensors/markers to be placed on each rigid body segment, which results in long setup times and is obtrusive. The number of sensors/markers can be reduced using deep learning or offline methods. However, this requires large training datasets and/or sufficient computational resources. Therefore, we investigate the following research question: “What is the performance of a shallow approach, compared to a deep learning one, for estimating time coherent full-body poses using only five inertial sensors?”. We propose to incorporate past/future inertial sensor information into a stacked input vector, which is fed to a shallow neural network for estimating full-body poses. Shallow and deep learning approaches are compared using the same input vector configurations. Additionally, the inclusion of acceleration input is evaluated. The results show that a shallow learning approach can estimate full-body poses with a similar accuracy (~6 cm) to that of a deep learning approach (~7 cm). However, the jerk errors are smaller using the deep learning approach, which can be the effect of explicit recurrent modelling. Furthermore, it is shown that the delay using a shallow learning approach (72 ms) is smaller than that of a deep learning approach (117 ms).

A Method for Guaranteeing Wireless Communication Based on a Combination of Deep and Shallow Learning

Wireless communication has changed and improved people's lives and society, especially with the arrival of the Internet of Things (IoT) era. Despite the maturity of wireless communication, the security issue of communication remains the most stubborn and troublesome problem due to the increasingly complex and large amounts of data. An intrusion detection system is the guarantee of secure communication. However, variable protocols and drastic growth in data volume make intrusion detection a difficult task. In this paper, we proposed a framework of anomaly-based network intrusion detection system to finish the detection job. First, UNSW-NB15 is selected as the research object. Based on this new dataset, we built a detection model combining a deep learning method and a shallow learning approach. The former one is a deep auto-encoder used for feature learning, which can discover important representations of data and accelerate detection. The latter one is a powerful support vector machine (SVM), where the artificial bee colony (ABC) algorithm is used to find optimal parameters for SVM with five-fold cross validation (5FCV). Various experiments are conducted and the simulation results prove that the proposed method performs quite better than some of state-of-the-art intrusion detection approaches, including the method based on the principal component analysis (PCA) and some other machine learning strategies.

GestureVLAD: Combining Unsupervised Features Representation and Spatio-Temporal Aggregation for Doppler-Radar Gesture Recognition

In this paper we propose a novel framework to process Doppler-radar signals for hand gesture recognition. Doppler-radar sensors provide many advantages over other emerging sensing modalities, including low development costs and high sensitivity to capture subtle gestures with precision. Furthermore, they have attractive properties for ubiquitous deployment and can be conveniently embedded into different devices. In this scope, current recognition methods still rely in deep CNN-LSTM and $3D$ CNN-LSTM structures that require sufficient labelled data to optimize millions of parameters and significant amount of computational resources for inference; which limits their deployment. Indeed, subtle gestures recognition is a challenging task due to the high variability of gestures among different subjects. To overcome the challenges in the recognition task and the limitations of the current methods, we propose a shallow learning approach for gesture recognition, that is based on unsupervised range-Doppler features representation, along with a learnable pooling aggregation via NetVLAD. The proposed framework can encode extremely valuable information across time, and results in features that are highly discriminative for hand gesture recognition. Experimentation on publicly available Doppler-radar data shows that the proposed framework outperforms state-of-the-art approaches in terms of recognition accuracy and speed for sequence-level hand gesture classification.

A Hierarchical Fused Fuzzy Deep Neural Network for Data Classification

Deep learning (DL) is an emerging and powerful paradigm that allows large-scale task-driven feature learning from big data. However, typical DL is a fully deterministic model that sheds no light on data uncertainty reductions. In this paper, we show how to introduce the concepts of fuzzy learning into DL to overcome the shortcomings of fixed representation. The bulk of the proposed fuzzy system is a hierarchical deep neural network that derives information from both fuzzy and neural representations. Then, the knowledge learnt from these two respective views are fused altogether forming the final data representation to be classified. The effectiveness of the model is verified on three practical tasks of image categorization, high-frequency financial data prediction and brain MRI segmentation that all contain high level of uncertainties in the raw data. The fuzzy dDL paradigm greatly outperforms other nonfuzzy and shallow learning approaches on these tasks.