Deep Learning Neural Network Research Articles

Pupillometry as a biomarker of postural control: Deep-learning models reveal side-specific pupillary responses to increased intensity of balance tasks.

Pupillometry has been used in the studies of postural control to assess cognitive load during dual tasks, but its response to increased balance task intensity has not been investigated. Furthermore, it is unknown whether side-specific changes in pupil diameter occur with more demanding balance tasks providing additional insights into postural control. The two aims of this study were to analyze differences in steady-state pupil diameter between balance tasks with increased intensity and to determine whether there are side-specific changes. Forty-eight healthy subjects performed parallel and left and right one-legged stances on a force plate with and without foam with right and left pupil diameters measured with a mobile infrared eye-tracker. Differences between balance tasks in parameters (average pupil diameter of each eye, average of both pupil diameters and the difference between the left and right pupil diameter) were analyzed using a two-way repeated measures analysis of variance, and deep learning neural network models were used to investigate how pupillometry predicted each balance task. The pupil diameter of the left eye, the average pupil diameter of both eyes and the difference in pupil diameters increased statistically significantly from simpler to more demanding balance tasks, with this being more pronounced for the left eye. The deep learning neural network models revealed side-specific changes in pupil diameter with more demanding balance tasks. This study confirms pupillary responses to increased intensity of balance task and indicates side-specific pupil responses that could be related to task-specific involvement of higher levels of postural control.

Foam detection in a stirred tank using deep learning neural networks

A deep learning method based on the Convolutional Neural Network (CNN) U-Net architecture has been explored as an image analysis tool for the detection and automated quantification of foam generated in a stirred tank. Multiple datasets of different sizes and with different types of foam were obtained experimentally and processed with data augmentation techniques in order to train the deep learning networks. The impact of adding attention and residual gates to the U-Net model to improve its performance, as well as the size of the dataset used to train the model have been assessed. The main factor influencing the accuracy of the models to detect the foam in the stirred tank is the size and quality of the dataset use to train the U-Net models; it must be sufficiently large and varied in terms of foam type/structure. The addition of attention gates and residual blocks to the U-Net model slightly improves the accuracy of foam detection, particularly for images that contain additional objects or obstructions, however the training time is 30 % longer than the standard U-Net model. In all cases, this image analysis method based on the U-Net model largely out performs conventional image analysis, which was not able to automatically differentiate between foam, additional objects in the tank and bubbles in the liquid. Two methods for the measurement of foam quantity (maximum foam height and foam volume) have also been developed based on the foam regions detected by the models. This is important for the application of the tool, which is to be used to understand the impact of operating conditions on foam formation in lab-/pilot-scale stirred tanks and then develop scale-up guidelines for foam control in industrial tanks.

A new method for predicting precipitation δ18O distribution based on deep learning and spatio-temporal clustering

ABSTRACT Predicting precipitation δ18O accurately is crucial for understanding water cycles, paleoclimates, and hydrological applications. Yet forecasting its spatio-temporal distribution remains challenging due to complex climate interactions and extreme events. We developed a method combining spatio-temporal clustering and deep learning neural networks to improve multi-site, multi-year precipitation δ18O predictions. Using a comprehensive dataset from 33 German sites (1978–2012), our model considers precipitation δ18O and its controlling factors, including precipitation and temperature distribution. We applied the K-means ++ method for classification and divided data into training and prediction sets. The convolutional neural network (CNN) model extracted spatial features, while the bi-directional long short-term memory (Bi-LSTM) model focused on temporal features. Spatio-temporal clustering using K-means ++ improved forecast accuracy and reduced errors. This study highlights the potential of deep learning and clustering techniques for forecasting complex spatio-temporal data and offers insights for future research on isotope distributions.

A Graph Reinforcement Learning Framework for Neural Adaptive Large Neighbourhood Search

Adaptive Large Neighbourhood Search (ALNS) is a popular metaheuristic with renowned efficiency in solving combinatorial optimisation problems. However, despite 18 years of intensive research into ALNS, the design of an effective adaptive layer for selecting operators to improve the solution remains an open question. In this work, we isolate this problem by formulating it as a Markov Decision Process, in which an agent is rewarded proportionally to the improvement of the incumbent. We propose Graph Reinforcement Learning for Operator Selection (GRLOS), a method based on Deep Reinforcement Learning and Graph Neural Networks, as well as Learned Roulette Wheel (LRW), a lightweight approach inspired by the classic Roulette Wheel adaptive layer. The methods, which are broadly applicable to optimisation problems that can be represented as graphs, are comprehensively evaluated on 5 routing problems using a large portfolio of 28 destroy and 7 repair operators. Results show that both GRLOS and LRW outperform the classic selection mechanism in ALNS, owing to the operator choices being learned in a prior training phase. GRLOS is also shown to consistently achieve better performance than a recent Deep Reinforcement Learning method due to its substantially more flexible state representation. The evaluation further examines the impact of the operator budget and type of initial solution, and is applied to problem instances with up to 1000 customers. The findings arising from our extensive benchmarking bear relevance to the wider literature of hybrid methods combining metaheuristics and machine learning.

Decoding dynamic visual scenes across the brain hierarchy.

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Communication—Serialization of Electrochemical Impedance Spectra to Forecast Low-Frequency Impedances via Deep Learning Neural Networks

Electrochemical impedance spectroscopy offers valuable insights into interfacial properties but acquiring low-frequency (LF) data can be time-consuming. This work explores time series analysis for forecasting LF impedance based on readily available high-frequency (HF) data. The time series methods like long-short term memory, convolutional neural network, and transformer were studied. Suitable neural network (NN) structure was proposed. The impact of forecasting window size was investigated to determine how much information was necessary for NN to establish useful connection. This approach holds promise for faster, more efficient, and insightful analysis of electrochemical systems.

Deep Learning and Hybrid Deep Neural Network to Quantify Agglutination in Dual-Phase Lab-in-Tubing Microfluidics

Deep Learning and Hybrid Deep Neural Network to Quantify Agglutination in Dual-Phase Lab-in-Tubing Microfluidics

AB067. New drug development for the use of PARP1-E2F1 transcriptional inhibitors in the treatment of glioblastoma.

Glioblastoma (GBM) is the most malignant brain tumor and ranks among the most lethal of all human cancers, without improvements in survival over the last 30 years. Data obtained in our group suggest that PARP1, a well-known DNA-repairing protein, could also play a key role in the regulation of cell cycle through its interaction with the transcription factor E2F1. Therefore, considering that most oncogenic processes are associated with cell cycle deregulation, we hypothesized that disruption of PARP1-E2F1 interaction would provide a novel therapeutic approach to different types of cancer. The identification of novel compounds disrupting PARP1-E2F1 interaction was carried out by combining in silico and in vitro screening, using a rational drug design. The virtual screen was performed using a molecular library of several million compounds at the selected target site, using AtomNet® (Atomwise, San Francisco, CA, USA), the first deep learning neural network for structure-based drug design and discovery. Since there is no complete structural information of the PARP1-E2F1 protein-protein interaction, a homologous structure of the BRCT domain of BRCA1 complex with the phospho-peptide (PDBID: 1T2V) was used to identify the potential binding interface of BRCT domain of PARP-1 (PDBID: 2COK) and the E2F1 protein. Top scoring compounds were clustered and filtered to obtain a final subset of 83 compounds that were incorporated to our in vitro screening, which included both transcriptional E2F1 activity and survival studies. Complete culture medium supplemented with the compounds selected in the in silico screening (10 μM) were added and incubated for 24 hours. E2F1 activity was observed by measuring luminescence. For the viability assay, the fluorescence reading was performed (excitation 544 nm and emission 590 nm). The in silico and in vitro screening resulted in 12 compounds that inhibited E2F1 transcriptional activity and significantly reduced cell number. The highest inhibition of both E2F1 transcriptional activity and cell growth was observed with compound 3797, which was selected for further studies. Both in silico and in vitro results indicate that inhibition of PARP1-E2F1 transcriptional activity may provide a new rationale for designing novel therapeutic approaches for the treatment of GBM.

Soil erosion susceptibility prediction using ensemble hybrid models with multicriteria decision-making analysis: Case study of the Medjerda basin, northern Africa

Soil erosion is considered one of the most prevalent natural hazards in semiarid regions, leading to the instability of ecosystems and human life. The main purpose of this research was to investigate and analyze soil erosion susceptibility maps in the Medjerda basin in northern Africa. This study utilizes four ensemble models based on the analytical hierarchy process (AHP) multicriteria decision-making analysis, namely, deep learning neural network AHP (DLNN-AHP), frequency ratio AHP (FR-AHP), Monte Carlo AHP (MC-AHP), and fuzzy AHP (F-AHP). Eight predictor variables were considered as inputs to the model, namely, the slope degree, digital elevation model (DEM), topographic wetness index (TWI), distance to river (DFR), distance to road (DFRD), normalized difference vegetation index (NDVI), rainfall erosivity (R), factor and soil erodibility factor (K). Soil erosion inventory maps were developed from field surveys and satellite images. The dataset was randomly divided into 70% for training and 30% for testing. The performances of the utilized models were compared using a receiver operating characteristic (ROC) curve. The results highlighted that all the models utilized exhibited good performance, with DLNN-AHP (93.1%) exhibiting slight superiority, followed by FR-AHP (90.9%), F-AHP (88.9%), and MC-AHP (88.5%). Among the influencing factors, the distance to the river and rainfall erosivity had the most significant impacts on the incidence of soil erosion. Moreover, the current findings revealed that 38.3% of the study area is extremely highly susceptible to soil erosion. The results of this study can aid in developing decision-support tools for planners and managers aiming to mitigate the adverse effects of soil erosion.

Estimating three-dimensional foot bone kinematics from skin markers using a deep learning neural network model

The human foot is a complex structure comprising 26 bones, whose coordinated movements facilitate proper deformation of the foot, ensuring stable and efficient locomotion. Despite their critical role, the kinematics of foot bones during movement remain largely unexplored, primarily due to the absence of non-invasive methods for measuring foot bone kinematics. This study addresses this gap by proposing a neural network model for estimating foot bone movements using surface markers. To establish a mapping between the positions and orientations of the foot bones and 41 skin markers attached on the human foot, computed tomography scans of the foot with the markers were obtained with eleven healthy adults and thirteen cadaver specimens in different foot postures. The neural network architecture comprises four layers, with input and output layers containing the 41 marker positions and the positions and orientations of the nine foot bones, respectively. The mean errors between estimated and true foot bone position and orientation were 0.5 mm and 0.6 degrees, respectively, indicating that the neural network can provide 3D kinematics of the foot bones with sufficient accuracy in a non-invasive manner, thereby contributing to a better understanding of foot function and the pathogenetic mechanisms underlying foot disorders.

An improved autoencoder for denoising acoustic emission signals in rock fracturing

ABSTRACT Rock fracture acoustic emission (AE) signals are commonly used non-destructive testing data in geological exploration, resource exploitation, and engineering fields. However, these signals are often accompanied by noise interference caused by environmental factors. In this study, we propose an enhanced model for denoising rock fracture AE signals, called simplified fully convolutional denoising autoencoder (SFCDAE). This model is based on the denoising autoencoder principle in the field of deep learning neural networks. The SFCDAE model consists of only seven layers, with minimal preprocessing of data input. By comparing denoising performance evaluation indicators, higher peak signal-to-noise ratio (PSNR) and lower root mean square error (RMSE) were achieved. On average, PSNR increased by 5.575% and RMSE decreased by 22.225%. Using simulated environmental noise to validate the model, it was found that the model has good robustness and can remove artefacts from sudden noise. The practical application value of the LSTM classification model was validated using data containing real experimental noise, resulting in a higher classification accuracy of 80.083%. These results indicate that the proposed model has better denoising performance compared to existing intelligent models and has certain practical value.

Addressing performance improvement of a neural network model for Reynolds-averaged Navier–Stokes solutions with high wake formation

PurposeDeep-learning techniques are recently gaining a lot of importance in the field of turbulence. This study focuses on addressing the problem of data imbalance to improve the performance of an existing deep learning neural network to infer the Reynolds-averaged Navier–Stokes solution, proposed by Thuerey et al. (2020), in the cases of airfoils with high wake formation behind them. The model is based on a U-Net architecture, which calculates pressure and velocity solutions for fluid flow around an airfoil.Design/methodology/approachIn this work, we propose various methods for training the model on selectively generated data with different distributions, which would be representative of the under-performing test samples. The property we chose for selectively generating data was the fraction of negative x-velocity in the domain. We have used Grad-CAM to compare the layer activations of different models trained using the proposed methods.FindingsWe observed that using our methods, the average performance on the samples with high wake formation (i.e. flow over airfoils at high angle of attack) has improved. Using one of the proposed methods, an average performance improvement of 15.65% was observed for samples of unknown airfoils compared to a similar model trained using the original method.Originality/valueThis work demonstrates the use of imbalanced learning in the field of fluid mechanics. The performance of the model is improved by giving significance to the distribution of the training data without changes to the model architecture.

Deep recurrent-convolutional neural network learning and physics Kalman filtering comparison in dynamic load identification

Deep recurrent-convolutional neural network learning and physics Kalman filtering comparison in dynamic load identification

Using Deep Learning Neural Networks to Predict Violent vs. Nonviolent Extremist Behaviors

ABSTRACT Recent analyses of radicalization processes have shown that extremist attitudes and violent behavior may be related in some cases, but are rarely collinear. It therefore benefits analysts of political violence to leverage tools that assist in the distinction of characteristics that might move an individual towards violence (vs. nonviolence) in support of their beliefs. To this end, the current study explores the efficacy of deep learning neural networks for classifying extremists as potentially violent or nonviolent based on dozens of common predictors derived from various perspectives on radicalization. Specifically, this study uses 337 predictors from the Profiles of Individual Radicalization in the U.S. dataset to populate a neural network with two hidden layers composed of four processing nodes. The model correctly predicted whether an individual engaged in violence (or not) in 94.2 percent of cases, on average. Analyses further identified several predictors that were most important in classifying violent and nonviolent cases. These analyses demonstrate neural networks may be effective tools in the study of radicalization and extremism, particularly regarding the disaggregation of salient outcomes.

IoT Computing Collaboration and Data-Aware Routing Algorithm for Edge Computing and DL

With the development of mobile edge computing and neural network Deep Learning (DL), more and more scholars are studying the combination of the two. This paper mainly studies the application of mobile edge computing and neural network DL in IoT computing collaboration and data-aware routing algorithms. Therefore, this paper proposes the deployment options of MEC technology and ETSIMEC in mobile edge computing, combining mobile edge computing with DL, designs an optimization algorithm based on Markov decision process and feature expression learning, and then analyzes and optimizes IoT computing and VANET routing algorithm. In order to have a clearer direction for the optimization algorithm, this paper also designs the edge computing model training and experiment comparison, the DL algorithm comparison experiment, and the routing algorithm simulation experiment and performance analysis. Combined with the experimental results, it is optimized and compared with traditional IoT computing and routing algorithms. Finally, it is concluded that the computing efficiency of IoT computing based on edge computing and DL designed in this paper is 21.33% higher than that of traditional IoT computing. The efficiency of the routing algorithm based on edge computing and DL designed in this paper is 9.29% higher than that of the traditional routing algorithm.

Compressed Deep Learning Models for Wearable Atrial Fibrillation Detection through Attention.

Deep learning (DL) models have shown promise for the accurate detection of atrial fibrillation (AF) from electrocardiogram/photoplethysmography (ECG/PPG) data, yet deploying these on resource-constrained wearable devices remains challenging. This study proposes integrating a customized channel attention mechanism to compress DL neural networks for AF detection, allowing the model to focus only on the most salient time-series features. The results demonstrate that applying compression through channel attention significantly reduces the total number of model parameters and file size while minimizing loss in detection accuracy. Notably, after compression, performance increases for certain model variants in key AF databases (ADB and C2017DB). Moreover, analyzing the learned channel attention distributions after training enhances the explainability of the AF detection models by highlighting the salient temporal ECG/PPG features most important for its diagnosis. Overall, this research establishes that integrating attention mechanisms is an effective strategy for compressing large DL models, making them deployable on low-power wearable devices. We show that this approach yields compressed, accurate, and explainable AF detectors ideal for wearables. Incorporating channel attention enables simpler yet more accurate algorithms that have the potential to provide clinicians with valuable insights into the salient temporal biomarkers of AF. Our findings highlight that the use of attention is an important direction for the future development of efficient, high-performing, and interpretable AF screening tools for wearable technology.

Identifying and training deep learning neural networks on biomedical-related datasets.

This manuscript describes the development of a resources module that is part of a learning platform named 'NIGMS Sandbox for Cloud-based Learning' https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox at the beginning of this Supplement. This module delivers learning materials on implementing deep learning algorithms for biomedical image data in an interactive format that uses appropriate cloud resources for data access and analyses. Biomedical-related datasets are widely used in both research and clinical settings, but the ability for professionally trained clinicians and researchers to interpret datasets becomes difficult as the size and breadth of these datasets increases. Artificial intelligence, and specifically deep learning neural networks, have recently become an important tool in novel biomedical research. However, use is limited due to their computational requirements and confusion regarding different neural network architectures. The goal of this learning module is to introduce types of deep learning neural networks and cover practices that are commonly used in biomedical research. This module is subdivided into four submodules that cover classification, augmentation, segmentation and regression. Each complementary submodule was written on the Google Cloud Platform and contains detailed code and explanations, as well as quizzes and challenges to facilitate user training. Overall, the goal of this learning module is to enable users to identify and integrate the correct type of neural network with their data while highlighting the ease-of-use of cloud computing for implementing neural networks. This manuscript describes the development of a resource module that is part of a learning platform named ``NIGMS Sandbox for Cloud-based Learning'' https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox [1] at the beginning of this Supplement. This module delivers learning materials on the analysis of bulk and single-cell ATAC-seq data in an interactive format that uses appropriate cloud resources for data access and analyses.

Resolution Enhancement of Metabolomic J-Res NMR Spectra Using Deep Learning.

J-Resolved (J-Res) nuclear magnetic resonance (NMR) spectroscopy is pivotal in NMR-based metabolomics, but practitioners face a choice between time-consuming high-resolution (HR) experiments or shorter low-resolution (LR) experiments which exhibit significant peak overlap. Deep learning neural networks have been successfully used in many fields to enhance quality of natural images, especially with regard to resolution, and therefore offer the prospect of improving two-dimensional (2D) NMR data. Here, we introduce the J-RESRGAN, an adapted and modified generative adversarial network (GAN) for image super-resolution (SR), which we trained specifically for metabolomic J-Res spectra to enhance peak resolution. A novel symmetric loss function was introduced, exploiting the inherent vertical symmetry of J-Res NMR spectra. Model training used simulated high-resolution J-Res spectra of complex mixtures, with corresponding low-resolution spectra generated via blurring and down-sampling. Evaluation of peak pair resolvability on J-RESRGAN demonstrated remarkable improvement in resolution across a variety of samples. In simulated plasma data, 100% of peak pairs exhibited enhanced resolution in super-resolution spectra compared to their low-resolution counterparts. Similarly, enhanced resolution was observed in 80.8-100% of peak pairs in experimental plasma, 85.0-96.7% in urine, 94.4-98.9% in full fat milk, and 82.6-91.7% in orange juice. J-RESRGAN is not sample type, spectrometer or field strength dependent and improvements on previously acquired data can be seen in seconds on a standard desktop computer. We believe this demonstrates the promise of deep learning methods to enhance NMR metabolomic data, and in particular, the power of J-RESRGAN to elucidate overlapping peaks, advancing precision in a wide variety of NMR-based metabolomics studies. The model, J-RESRGAN, is openly accessible for download on GitHub at https://github.com/yanyan5420/J-RESRGAN.

The street space planning and design of artificial intelligence-assisted deep learning neural network in the Internet of Things

This study begins by discussing Internet of Things (IoT) technology and analyzing the classification of street space into four types, along with the Green Looking Ratio (GLR). Following this, the Fully Convolutional Network (FCN)-8s framework is employed to construct a street view image semantic segmentation model based on FCN principles. Subsequently, IoT technology is utilized to analyze the proportion of GLR and satisfaction in the street space within the historical urban area of T City. The findings reveal a significant positive correlation (significance level p < 0.05, R2 = 0.919) between the GLR satisfaction score of street view images and the average GLR of the area. Among the four types of street space—life leisure, historical streets, traffic areas, and landscape-style streets—the dissatisfaction rates with GLR are 35 %, 33 %, 20 %, and 18 %, respectively, correlating with varying GLR satisfaction levels. To enhance street space greening, planting ponds and boxes are proposed for "blind spots” and "dead corners,” thereby completing greenery in these areas. These initiatives aim to improve street greening policies, integrate street function zones, and advance the scientific greening of urban streets. The analysis of GLR and satisfaction in street spaces provides valuable insights for refining urban street space greening efforts.

A reconfigurable multi-precision quantization-aware nonlinear activation function hardware module for DNNs

In recent years, the increasing variety of nonlinear activation functions (NAFs) in deep neural networks (DNNs) has led to higher computational demands. However, hardware implementation faces challenges such as lack of flexibility, high hardware cost, and limited accuracy. This paper proposes a highly flexible and low-cost hardware solution for implementing activation functions to overcome these issues. Based on the piecewise linear (PWL) approximation method, our method supports NAFs with different accuracy configurations through a customized implementation strategy to meet the requirements in different scenario applications. In this paper, the symmetry of the activation function is investigated, and incorporate curve translation preprocessing and data quantization to significantly reduce hardware storage costs. The modular hardware architecture proposed in this study supports NAFs of multiple accuracies, which is suitable for designing deep learning neural network accelerators in various scenarios, avoiding the need to design dedicated hardware circuits for the activation function layer and enhances circuit design efficiency. The proposed hardware architecture is validated on the Xilinx XC7Z010 development board. The experimental results show that the average absolute error (AAE) is reduced by about 35.6 % at a clock frequency of 312.5 MHz. Additionally, the accuracy loss of the model is maximized to −0.684 % after replacing the activation layer function of DNNs under the PyTorch framework.