Neural Network Research Articles

Deep weighted survival neural networks to survival risk prediction

Deep weighted survival neural networks to survival risk prediction

Algorithm for simulating calligrapher’s stroke features using neural networks

In the field of calligraphy art, simulating the stroke features of calligraphers has always been a challenging task. Traditional methods often rely on manually designed rules or feature extraction algorithms, which are difficult to accurately capture calligraphy details and time-consuming. This article aimed to explore a more effective method to simulate the stroke features of calligraphers using neural network technology. This article mainly explored the simulation algorithm of calligrapher stroke features based on neural networks, including calligrapher stroke feature statistics, calligrapher stroke feature abstraction, and calligrapher stroke feature extraction. By using Convolutional Neural Network (CNN) to learn and analyze calligraphy works, a better performance neural network model was established to achieve automatic recognition and classification of calligrapher stroke features. By continuously optimizing model parameters, the accuracy of calligrapher stroke feature simulation can be improved. The average similarity rates for imitating stroke features of five calligraphers (regular script, cursive script, clerical script, seal script, and running script) using Artificial Neural Network (ANN) were 0.72, 0.62, 0.40, 0.33, and 0.53, respectively. The average similarity rates for imitating stroke features of 5 calligraphers (regular script, cursive script, clerical script, seal script, and running script) using CNN were 0.93, 0.78, 0.87, 0.67, and 0.80, respectively. The research results of this article promoted the inheritance and innovation of calligraphy art, and expanded the expression forms and application fields of calligraphy art.

Physics-informed two-tier neural network for non-linear model order reduction

AbstractIn recent years, machine learning (ML) has had a great impact in the area of non-intrusive, non-linear model order reduction (MOR). However, the offline training phase often still entails high computational costs since it requires numerous, expensive, full-order solutions as the training data. Furthermore, in state-of-the-art methods, neural networks trained by a small amount of training data cannot be expected to generalize sufficiently well, and the training phase often ignores the underlying physical information when it is applied with MOR. Moreover, state-of-the-art MOR techniques that ensure an efficient online stage, such as hyper reduction techniques, are either intrusive or entail high offline computational costs. To resolve these challenges, inspired by recent developments in physics-informed and physics-reinforced neural networks, we propose a non-intrusive, physics-informed, two-tier deep network (TTDN) method. The proposed network, in which the first tier achieves the regression of the unknown quantity of interest and the second tier rebuilds the physical constitutive law between the unknown quantities of interest and derived quantities, is trained using pretraining and semi-supervised learning strategies. To illustrate the efficiency of the proposed approach, we perform numerical experiments on challenging non-linear and non-affine problems, including multi-scale mechanics problems.

Elucidating Thermodynamically Driven Structure–Property Relations for Zeolite Adsorption Using Neural Networks

Elucidating Thermodynamically Driven Structure–Property Relations for Zeolite Adsorption Using Neural Networks

Human-like dissociations between confidence and accuracy in convolutional neural networks.

Prior research has shown that manipulating stimulus energy by changing both stimulus contrast and variability results in confidence-accuracy dissociations in humans. Specifically, even when performance is matched, higher stimulus energy leads to higher confidence. The most common explanation for this effect, derived from cognitive modeling, is the positive evidence heuristic where confidence neglects evidence that disconfirms the choice. However, an alternative explanation is the signal-and-variance-increase hypothesis, according to which these dissociations arise from changes in the separation and variance of perceptual representations. Because artificial neural networks lack built-in confidence heuristics, they can serve as a test for the necessity of confidence heuristics in explaining confidence-accuracy dissociations. Therefore, we tested whether confidence-accuracy dissociations induced by stimulus energy manipulations emerge naturally in convolutional neural networks (CNNs). We found that, across three different energy manipulations, CNNs produced confidence-accuracy dissociations similar to those found in humans. This effect was present for a range of CNN architectures from shallow 4-layer networks to very deep ones, such as VGG-19 and ResNet-50 pretrained on ImageNet. Further, we traced back the reason for the confidence-accuracy dissociations in all CNNs to the same signal-and-variance increase that has been proposed for humans: higher stimulus energy increased the separation and variance of evidence distributions in the CNNs' output layer leading to higher confidence even for matched accuracy. These findings cast doubt on the necessity of the positive evidence heuristic to explain human confidence and establish CNNs as promising models for testing cognitive theories of human behavior.

Parameter estimation for allometric trophic network models: A variational Bayesian inverse problem approach

Abstract Differential equation models are powerful tools for predicting biological systems, capable of projecting far into the future and incorporating data recorded at arbitrary times. However, estimating these models' parameters from observations can be challenging because numerical methods are often required to approximate their solution. An example of such a model is the allometric trophic network model, for which studies considering its inverse problem are limited, particularly in the Bayesian framework. Here we develop a variational Bayesian method for parameter inference of the allometric trophic network model and explore how accurately we can recover its parameter values. We represent the model as a Bayesian neural network, which combines an artificial neural network with Bayesian inference, using a surrogate for the posterior distribution of model parameters, and train this model by evolutionary optimization to avoid potentially costly computation of the gradient with respect to the model parameters. Using synthetic data, we compare the accuracy of this variational inference to ordinary least squares estimation. To reduce the number of estimated parameters, we focus on the inference of functional response parameters. Our variational Bayesian method yields parameter estimates that are comparable to the ordinary least squares results in terms of accuracy. The method provides a promising approach for including uncertainty quantification in parameter estimation, which the simple ordinary least squares approach as it is does not address. Regardless of the method, potential multimodality of the inference problem is nonetheless important to keep in mind. The present study suggests a technique for parameter inference of ordinary differential equation models in the Bayesian context. We propose the method especially for validation of the allometric trophic network model against empirical data.

Classification of Ship Type from Combination of HMM–DNN–CNN Models Based on Ship Trajectory Features

This study proposes an enhanced ship-type classification model that employs a sequential processing methodology integrating hidden Markov model (HMM), deep neural network (DNN), and convolutional neural network (CNN) techniques. Four different ship types—fishing boat, passenger, container, and other ship—were classified using multiple ship trajectory features extracted from the automatic identification system (AIS) and small fishing vessel tracking system. For model optimization, both ship datasets were transformed into various formats corresponding to multiple models, incorporating data enhancement and augmentation approaches. Speed over ground, course over ground, rate of turn, rate of turn in speed, berth distance, latitude/longitude, and heading were used as input parameters. The HMM–DNN–CNN combination was obtained as the optimal model (average F-1 score: 97.54%), achieving individual classification performances of 99.03%, 97.46%, and 95.83% for fishing boats, passenger ships, and container ships, respectively. The proposed approach outperformed previous approaches in prediction accuracy, with further improvements anticipated when implemented on a large-scale real-time data collection system.

Data discretization impact on deep learning for missing value imputation of continuous data

In various fields of information examination, for example, AI, profoundly getting the hang of missing information is a typical issue. Missing qualities should be tended to since they can adversely affect the exactness and adequacy of prescient models. This research investigates how data discretization affects deep learning methods for filling the missing values in datasets with continuous features. They provide a unique method for imputing missing values using deep neural networks (DNNs) called extravagant expectation maximization-deep neural network (EEM-DNN). This approach discretizes continuous features into separate intervals initially. This is justified by treating the issue of missing value imputation as a classification work, with the missing values being considered a distinct class. A DNN, designed explicitly for imputation, is then trained using the discretized data. The expectation maximization concepts are incorporated into the network architecture, and as a result, the network iteratively improves its imputation predictions. They run comprehensive experiments on several datasets from different fields to gauge the efficacy of the suggested strategy. The effectiveness of EEM-DNN is compared to that of other imputation approaches, such as traditional imputation techniques and deep learning methods without data discretization. Our findings show that data discretization significantly enhances imputation accuracy. In terms of imputation accuracy and prediction performance on downstream tasks, the EEM-DNN method regularly performs better than alternative methods. It also examines if various discretization techniques affect the overall imputation process. They find that the trade-off between bias and variance in imputed data depends on the discretization method selected. This highlights the significance of choosing a suitable discretization approach depending on the unique properties of the dataset.

Artificial Intelligence Applied in Early Prediction of Lower Limb Fracture Complications

Background: Artificial intelligence has become a valuable tool for diagnosing and detecting postoperative complications early. Through imaging and biochemical markers, clinicians can anticipate the clinical progression of patients and the risk of long-term complications that could impact the quality of life or even be life-threatening. In this context, artificial intelligence is crucial for identifying early signs of complications and enabling clinicians to take preventive measures before problems worsen. Materials and methods: This observational study analyzed medical charts from the electronic archive of the Clinical Emergency Hospital in Galați, Romania, covering a four-year period from 2018 to 2022. A neural network model was developed to analyze various socio-demographic and paraclinical data. Key features included patient demographics, laboratory investigations, and clinical outcomes. Statistical analyses were performed to identify significant risk factors associated with deep venous thrombosis (DVT). Results: The analysis revealed a higher prevalence of female patients (60.78%) compared to male patients, indicating a potential gender-related risk factor for DVT. The incidence of DVT was highest among patients aged 71 to 90 years, affecting 56.86% of individuals in this age group, suggesting that advanced age significantly contributes to the risk of developing DVT. Additionally, among the DVT patients, 15.69% had a body mass index (BMI) greater than 30, categorizing them as obese, which is known to increase the risk of thrombotic events. Furthermore, this study highlighted that the highest frequency of DVT was associated with femur fractures, occurring in 52% of patients with this type of injury. The neural network analysis indicated that elevated levels of direct bilirubin (≥1.5 mg/dL) and prothrombin activity (≤60%) were strong predictors of fracture-related complications, with sensitivity and specificity rates of 78% and 82%, respectively. These findings underscore the importance of monitoring these laboratory markers in at-risk populations for early intervention. Conclusions: This study identified critical risk factors for developing DVT, including advanced age, high BMI, and femur fractures, which necessitate longer recovery periods. Additionally, the findings indicate that elevated direct bilirubin and prothrombin activity play a significant role in predicting DVT development. These results suggest that AI can effectively enhance the anticipation of clinical evolution in patients, aiding in early intervention and management strategies.

How Sophisticated Are Neural Networks Needed to Predict Long-Term Nonadiabatic Dynamics?

Nonadiabatic dynamics is key for understanding solar energy conversion and photochemical processes in condensed phases. This often involves the non-Markovian dynamics of the reduced density matrix in open quantum systems, where knowledge of the system's prior states is necessary to predict its future behavior. In this study, we explore time-series machine learning methods for predicting long-time nonadiabatic dynamics based on short-time input data, comparing these methods with the physics-based transfer tensor method (TTM). To understand the impact of memory time on these approaches, we demonstrate that non-Markovian dynamics can be represented as a linear map within the Nakajima-Zwanzig generalized quantum master equation framework. We further propose a practical method to estimate the effective memory time, within a given tolerance, for reduced density matrix propagation. Our predictive models are applied to various physical systems, including spin-boson models, multistate harmonic (MSH) models with Ohmic spectral densities and for a realistic organic photovoltaic system composed of a carotenoid-porphyrin-fullerene triad dissolved in tetrahydrofuran. Results indicate that the simple linear-mapping fully connected neural network (FCN) outperforms the more complicated nonlinear-mapping networks including the gated recurrent unit (GRU) and the convolutional neural network/long short-term memory (CNN-LSTM) in systems with short memory times, such as spin-boson and MSH models. Conversely, the nonlinear CNN-LSTM and GRU models yield higher accuracy in the triad MSH systems characterized by long memory times. These findings offer valuable insights into the role of effective memory time in non-Markovian quantum dynamics, providing practical guidance for the application of time-series machine learning models to complex chemical systems.

A dynamic neural network model for the identification of asbestos roofings in hyperspectral images covering a large regional area

AbstractAsbestos has been used extensively in several applications. Once it is known as a dangerous mineral, its usage has been prohibited and its identification and remediation play a very important role from the health safety point of view. Nowadays, deep learning techniques are used in many applications, especially for image analysis. They can be used to significantly reduce the time and cost of traditional detection methods. In this paper, taking advantage of asbestos spectral signature, a deep neural network is introduced in order to implement a complete methodology to identify asbestos roofings starting from hyperspectral images in a regional context. The novelty of the proposed approach is a dynamic mixing of models with different features, in order to accommodate classifications on widespread areas of both urban and rural territories. Indeed, the dataset used during the experiments described in this paper is a large one, consisting of many wide hyperspectral images with a geometric resolution of 1 m and with 186 bands, covering an entire region of approximately 8,000 . This is in contrast to other works in the literature where the analyzed areas are limited in size and uniform for physical features.

Effects of Biological Adhesion on the Hydrodynamic Characteristics of Different Panel Net Materials: A BP Neural Network Approach

Effects of Biological Adhesion on the Hydrodynamic Characteristics of Different Panel Net Materials: A BP Neural Network Approach

Data-driven weight initialization strategy for convolutional neural networks

Data-driven weight initialization strategy for convolutional neural networks

Synchronisation of quaternion-valued neural networks with neutral delay and discrete delay via aperiodic intermittent control

This paper studies the synchronisation of quaternion-valued neural networks with neutral delay and discrete delay by aperiodic intermittent control. By using inequality techniques and constructing appropriate Lyapunov–Krasovskii functional, a main criterion expressed as quaternion-valued linear matrix inequalities is derived to pledge the synchronisation of the considered quaternion-valued neural networks. The theoretical result is demonstrated by an illustrated example as well as simulations.

Bayesian-Neural-Network-Based Approach for Probabilistic Prediction of Building-Energy Demands

Reliable prediction of building-level energy demand is crucial for the building managers to optimize and regulate energy consumption. Conventional prediction models omit the uncertainties associated with demand over time; hence, they are mostly inaccurate and unreliable. In this study, a Bayesian neural network (BNN)-based probabilistic prediction model is proposed to tackle this challenge. By quantifying the uncertainty, BNNs provide probabilistic predictions that capture the variations in the energy demand. The proposed model is trained and evaluated on a subset of the building operations dataset of Lawrence Berkeley National Laboratory (LBNL), Berkeley, California, which includes diverse attributes related to climate and key building-performance indicators. We have performed thorough hyperparameter tuning and used fixed-horizon validation to evaluate trained models on various test data to assess generalization ability. To validate the results, quantile random forest (QRF) was used as a benchmark. This study compared BNN with LSTM, showing that BNN outperformed LSTM in uncertainty quantification.

Leveraging Convolutional Neural Networks for Enhanced Educational Question Classification

Leveraging Convolutional Neural Networks for Enhanced Educational Question Classification

Analysis of impact of limb segment length variations during reinforcement learning in four-legged robot.

Crawling robots are becoming increasingly prevalent in both industrial and private applications. Despite their many advantages over other robot types, they have complex movement mechanics. Artificial intelligence can simplify this by reinforcement learning. This process requires configuring the training environment and defining input parameters, including a robot model for movement training. To translate the virtual results into real-world scenarios, a 3D model with appropriate mechanical parameters must be developed.These parameters can vary significantly between multiple mechanical configurations, which will further impact the reinforcement learning process of such a robot. For this reason, it was decided to test which limb configurations would work best in this process. Initially, various kinematic types of walking robots were analysed, drawing on the anatomy of mammals, reptiles, and insects for the biological model. The reptilian model was chosen for its balance of stability, dynamics, and energy efficiency. The article reviews the preparation of robot models and the configuration of the Unity3D development environment using the ML-Agents toolkit. The experiment examined how different limb lengths affect training, resulting in movement algorithms for various quadruped robot configurations using artificial neural networks. Based on the numerical results, the best configuration was the default, with the same length of the tibia as the thigh, achieving a reward function value of 883.9 and an episode length of 245.5. Taking into account the same criteria, the least efficient configuration was definitely the one characterised by the shortest thigh and the longest tibia among those considered. In its case, the reward function reached a value of only 526.2 with an episode lasting 999.0, which means that it never achieved the intended goal.

Application of the Standardized Precipitation Index for Forecasting Seasonal Drought in Vietnam

Drought is a natural disaster that causes significant damages to the economy, society and environment. Forecasting droughts is crucial for disaster risk management and water security. This study introduces a new method for forecasting seasonal droughts in Vietnam using the Standardized Precipitation Index (SPI) which is based on a combination of dynamic modeling and statistical methods. The precipitation forecast products from regional climate models (RegCM, clWRF) with inputs from Climate Forecast System (CFSv2) were used to calculate the SPI index after statistical calibration. Results show that calibrating model precipitation with statistical methods such as Artificial Neural Networks (ANN) and Multivariate Linear Regression (MLR) significantly improves the precipitation forecast accuracy. Evaluation indices indicate that the SPI calculated with the post-calibration model precipitation has a good reproducibility for drought events, particularly mild droughts. However, there remains some limitations on accurate forecast of the intensity and spatial distribution of moderate drought events in certain regions such as Central Vietnam.

Retina-Inspired X-Ray Optoelectronic Synapse Using Amorphous Ga2O3 Thin Film.

Machine vision techniques are widely applied for object identification in daily life and industrial production, where images are captured and processed by sensors, memories, and processing units sequentially. Neuromorphic optoelectronic synapses, as a preferable option to promote the efficiency of image recognition, are hotly pursued in non-ionizing radiation range, but rarely in ionizing radiation including X-rays. Here, the study proposes an X-ray optoelectronic synapse using amorphous Ga2O3 (a-Ga2O3) thin film. Boosted by the interfacial VO 2+ defects and its slow neutralization rate, the enhanced electron tunneling process at metal/a-Ga2O3 interface produces remarkable X-ray-induced post-synaptic current, contributing to a sensitivity of 20.5, 64.3, 164.1 µC mGy-1 cm-2 for the 1st, 5th, and 10th excitation periods, respectively. Further, a 64×64 imaging sensor is constructed on a commercial amorphous Si (a-Si) thin film transistor (TFT) array. The image contrast can be apparently improved under a series of X-ray pulses due to an outstanding long-term plasticity of the single pixel, which is beneficial to the subsequent image recognition and classification based on artificial neural network. The merits of large-scale production ability and good compatibility with modern microelectronic techniques belonging to amorphous oxide semiconductors may promote the development of neuro-inspired X-ray imagers and corresponding machine vision systems.

Neural network-based predictions of antimicrobial resistance phenotypes in multidrug-resistant Acinetobacter baumannii from whole genome sequencing and gene expression.

Whole genome sequencing (WGS) potentially represents a rapid approach for antimicrobial resistance genotype-to-phenotype prediction. However, the challenge still exists to predict fully minimum inhibitory concentrations (MICs) and antimicrobial susceptibility phenotypes based on WGS data. This study aimed to establish an artificial intelligence-based computational approach in predicting antimicrobial susceptibilities of multidrug-resistant Acinetobacter baumannii from WGS and gene expression data. Antimicrobial susceptibility testing (AST) was performed using the broth microdilution method for 10 antimicrobial agents. In silico multilocus sequence typing (MLST), antimicrobial resistance genes, and phylogeny based on cgSNP and cgMLST strategies were analyzed. High-throughput qPCR was performed to measure the expression level of antimicrobial resistance (AMR) genes. Most isolates exhibited a high level of resistance to most of the tested antimicrobial agents, with the majority belonging to the IC2/CC92 lineage. Phylogenetic analysis revealed undetected transmission events or local outbreaks. The percentage agreements between AMR phenotype and genotype ranged from 70.08% to 89.96%, with the coefficient of agreement (κ) extending from 0.025 and 0.881. The prediction of AST employed by deep neural network models achieved an accuracy of up to 98.64% on the testing data set. Additionally, several linear regression models demonstrated high prediction accuracy, reaching up to 86.15% within an error range of one gradient, indicating a linear relationship between certain gene expressions and the corresponding antimicrobial MICs. In conclusion, neural network-based predictions could be used as a tool for the surveillance of antimicrobial resistance in multidrug-resistant A. baumannii.