Neural Model Research Articles

Artificial neural network-based modeling of Malachite green adsorption onto baru fruit endocarp: insights into equilibrium, kinetic, and thermodynamic behavior.

In this study, artificial neural network (ANN) tools were employed to forecast the adsorption capacity of Malachite green (MG) by baru fruit endocarp waste (B@FE) under diverse conditions, including pH, adsorbent dosage, initial dye concentration, contact time, and temperature. Enhanced adsorption efficiency was notably observed under alkaline pH conditions (pH 10). Kinetic analysis indicated that the adsorption process closely followed a pseudo-second-order model, while equilibrium studies revealed the Langmuir isotherm as the most suitable model, estimating a maximum adsorption capacity of 57.85 mg g-1. Furthermore, the chemical adsorption of MG by B@FE was confirmed using the Dubinin-Radushkevich isotherm. Thermodynamic analysis suggested that the adsorption is spontaneous and endothermic. Various ANN architectures were explored, employing different activation functions such as identity, logistic, tanh, and exponential. Based on evaluation metrics like the coefficient of determination (R2) and root mean square error (RMSE), the optimal network configuration was identified as a 5-11-1 architecture, consisting of five input neurons, eleven hidden neurons, and one output neuron. Notably, the logistic activation function was applied in both the hidden and output layers for this configuration. This study highlights the efficacy of B@FE as an efficient adsorbent for MG removal from aqueous solutions and demonstrates the potential of ANN models in predicting adsorption behavior across varying environmental conditions, emphasizing their utility in this field.

Artificial Neural Network-Based Modelling for Yield Strength Prediction of Austenitic Stainless-Steel Welds

Artificial Neural Network-Based Modelling for Yield Strength Prediction of Austenitic Stainless-Steel Welds

Development of an artificial neural network based-prediction model for bond strength of FRP bars in concrete

Fiber-reinforced polymer (FRP) bars have garnered increasing attention in recent years due to their superior corrosion resistance, offering a potential solution to the significant drawback of steel corrosion in concrete. For the widespread utilization of FRP bars in concrete structures, determining the bond strength between FRP bars and concrete is a crucial topic. This study seeks to develop a prediction model to estimate the bond strength of FRP bars in concrete, utilizing an extended dataset from 1010 pull-out tests. Initially, the study evaluates the applicability of several bond strength formulas from existing codes. Subsequently, two prediction models, namely a multivariate linear regression model and an artificial neural network (ANN) model, are introduced for estimating the bond strength of FRP bars in concrete. The results indicate that the correlation between the evaluation values of existing formulas and the experimental value is very low. This is because these formulas have not yet been updated to encompass the expanded usage scopes of FRP bars with various surface processing methods and types of concrete. While the multivariate linear regression model outperforms these formulas, its accuracy is still relatively low; in contrast, the ANN demonstrates superior performance, achieving an R^2 value for both the validation and test set of more than 0.92. The findings highlight that, when considering a broader range of applications, the ANN serves as a robust tool for accurately predicting the bond strength of FRP bars in concrete, in comparison to traditional formulas and linear regression models. This assessment approach provides engineers with a convenient, high-precision tool for designs utilizing various forms of FRP bars and diverse types of concrete in practical design scenarios

A Hierarchical Architecture for Neural Materials

AbstractNeural reflectance models are capable of reproducing the spatially‐varying appearance of many real‐world materials at different scales. Unfortunately, existing techniques such as NeuMIP have difficulties handling materials with strong shadowing effects or detailed specular highlights. In this paper, we introduce a neural appearance model that offers a new level of accuracy. Central to our model is an inception‐based core network structure that captures material appearances at multiple scales using parallel‐operating kernels and ensures multi‐stage features through specialized convolution layers. Furthermore, we encode the inputs into frequency space, introduce a gradient‐based loss, and employ it adaptive to the progress of the learning phase. We demonstrate the effectiveness of our method using a variety of synthetic and real examples.

Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data, and Symptomatic Assessments.

In this study, we propose a novel framework that utilizes deep learning and attention mechanisms to predict the radiographic progression of patellofemoral osteoarthritis (PFOA) over a period of 7 years. This study included subjects (1,832 subjects, 3,276 knees) from the baseline of the Multicenter Osteoarthritis Study (MOST). Patellofemoral joint regions of interest were identified using an automated landmark detection tool (BoneFinder) on lateral knee X-rays. An end-to-end deep learning method was developed for predicting PFOA progression based on imaging data in a five-fold cross-validation setting. To evaluate the performance of the models, a set of baselines based on known risk factors were developed and analyzed using gradient boosting machine (GBM). Risk factors included age, sex, body mass index, and Western Ontario and McMaster Universities Arthritis Index score, and the radiographic osteoarthritis stage of the tibiofemoral joint (Kellgren and Lawrence [KL] score). Finally, to increase predictive power, we trained an ensemble model using both imaging and clinical data. Among the individual models, the performance of our deep convolutional neural network attention model achieved the best performance with an area under the receiver operating characteristic curve (AUC) of 0.856 and average precision (AP) of 0.431, slightly outperforming the deep learning approach without attention (AUC = 0.832, AP = 0.4) and the best performing reference GBM model (AUC = 0.767, AP = 0.334). The inclusion of imaging data and clinical variables in an ensemble model allowed statistically more powerful prediction of PFOA progression (AUC = 0.865, AP = 0.447), although the clinical significance of this minor performance gain remains unknown. The spatial attention module improved the predictive performance of the backbone model, and the visual interpretation of attention maps focused on the joint space and the regions where osteophytes typically occur. This study demonstrated the potential of machine learning models to predict the progression of PFOA using imaging and clinical variables. These models could be used to identify patients who are at high risk of progression and prioritize them for new treatments. However, even though the accuracy of the models were excellent in this study using the MOST dataset, they should be still validated using external patient cohorts in the future.

Deep learning with plasma plume image sequences for anomaly detection and prediction of growth kinetics during pulsed laser deposition

Materials synthesis platforms that are designed for autonomous experimentation are capable of collecting multimodal diagnostic data that can be utilized for feedback to optimize material properties. Pulsed laser deposition (PLD) is emerging as a viable autonomous synthesis tool, and so the need arises to develop machine learning (ML) techniques that are capable of extracting information from in situ diagnostics. Here, we demonstrate that intensified-CCD image sequences of the plasma plume generated during PLD can be used for anomaly detection and the prediction of thin film growth kinetics. We develop multi-output (2 + 1)D convolutional neural network regression models that extract deep features from plume dynamics that not only correlate with the measured chamber pressure and incident laser energy, but more importantly, predict parameters of an auto-catalytic film growth model derived from in situ laser reflectivity experiments. Our results demonstrate how ML with in situ plume diagnostics data in PLD can be utilized to maintain deposition conditions in an optimal regime. Further, the predictive capabilities of plume dynamics on the kinetics of film growth or other film properties prior to deposition provides a means for rapid pre-screening of growth conditions for the non-expert, which promises to accelerate materials optimization with PLD.

Effects of local reduction of endogenous α-synuclein using antisense oligonucleotides on the fibril-induced propagation of pathology through the neural network in wild-type mice

In Parkinson's disease and other synucleinopathies, fibrillar forms of α-synuclein (aSyn) are hypothesized to structurally convert and pathologize endogenous aSyn, which then propagates through the neural connections, forming Lewy pathologies and ultimately causing neurodegeneration. Inoculation of mouse-derived aSyn preformed fibrils (PFFs) into the unilateral striatum of wild-type mice causes widespread aSyn pathologies in the brain through the neural network. Here, we used the local injection of antisense oligonucleotides (ASOs) against Snca mRNA to confine the area of endogenous aSyn protein reduction and not to affect the PFFs properties in this model. We then varied the timing and location of ASOs injection to examine their impact on the initiation and propagation of aSyn pathologies in the whole brain and the therapeutic effect using abnormally-phosphorylated aSyn (pSyn) as an indicator. By injecting ASOs before or 0–14 days after the PFFs were inoculated into the same site in the left striatum, the reduction in endogenous aSyn in the striatum leads to the prevention and inhibition of the regional spread of pSyn pathologies to the whole brain including the contralateral right hemisphere. ASO post-injection inhibited extension from neuritic pathologies to somatic ones. Moreover, injection of ASOs into the right striatum prevented the remote regional spread of pSyn pathologies from the left striatum where PFFs were inoculated and no ASO treatment was conducted. This indicated that the reduction in endogenous aSyn protein levels at the propagation destination site can attenuate pSyn pathologies, even if those at the propagation initiation site are not inhibited, which is consistent with the original concept of prion-like propagation that endogenous aSyn is indispensable for this regional spread. Our results demonstrate the importance of recruiting endogenous aSyn in this neural network propagation model and indicate a possible potential for ASO treatment in synucleinopathies.

Interpolation of Environmental Data Using Deep Learning and Model Inference

Interpolation of Environmental Data Using Deep Learning and Model Inference

CNSMolGen: A Bidirectional Recurrent Neural Network-Based Generative Model for De Novo Central Nervous System Drug Design.

Central nervous system (CNS) drugs have had a significant impact on treating a wide range of neurodegenerative and psychiatric disorders. In recent years, deep learning-based generative models have shown great potential for accelerating drug discovery and improving efficacy. However, specific applications of these techniques in CNS drug discovery have not been widely reported. In this study, we developed the CNSMolGen model, which uses a framework of bidirectional recurrent neural networks (Bi-RNNs) for de novo molecular design of CNS drugs. Results showed that the pretrained model was able to generate more than 90% of completely new molecular structures, which possessed the properties of CNS drug molecules and were synthesizable. In addition, transfer learning was performed on small data sets with specific biological activities to evaluate the potential application of the model for CNS drug optimization. Here, we used drugs against the classical CNS disease target serotonin transporter (SERT) as a fine-tuned data set and generated a focused database against the target protein. The potential biological activities of the generated molecules were verified by using the physics-based induced-fit docking study. The success of this model demonstrates its potential in CNS drug design and optimization, which provides a new impetus for future CNS drug development.

Recognizing Sign Language using Machine Learning and Deep Learning Models

Individuals with hearing impairments communicate mostly through sign language. Our goal was to create an American Sign Language recognition dataset and utilize it in a neural network-based machine learning model that can interpret hand gestures and positions into natural language. In our study, we incorporated the SVM, CNN and Resnet-18 models to enhance predictability when interpreting ASL signs through this new dataset, which includes provisions such as lighting and distance limitations. Our research also features comparison results between all the other models implemented under invariant conditions versus those using our proposed CNN model. As demonstrated by its high levels of precision at 95.10% despite changes encountered during testing procedures like varying data sets or scene configurations where losses are minimal (0.545), there exists great potential for future applications in image recognition systems requiring deep learning techniques. Furthermore, these advancements may lead to significant improvements within various fields related explicitly to speech-language therapy sessions designed specifically around helping people overcome challenges associated with deafness while building bridges towards improved social integration opportunities.

ОДНОВИМІРНА ЗГОРТКОВА МОДЕЛЬ НЕЙРОННОЇ МЕРЕЖІ ДЛЯ ОБРОБЛЕННЯ СИГНАЛІВ З АМПЛІТУДНОЮ МОДУЛЯЦІЄЮ БАГАТЬОХ СКЛАДОВИХ

The processing of radio signals using artificial neural networks (ANNs) has great potential for research, which can be explained by the adaptability of ANNs to various transmission conditions and the ability to detect abstract patterns of changes in signal parameters. The article reviews the works of other authors devoted to different ways of using ANNs for processing radio signals. Taking into account the information in the reviewed works, the research task was formed, which consists in developing an optimized ANN model for radio signal processing. Signals with amplitude modulation of many components (AMMC) were chosen to form training samples for ANN. The choice of modulation type is justified by greater energy efficiency compared to other widely used digital modulation types, such as quadrature amplitude modulation. Mathematic basis of AMMC signal generation is described. The process of finding the coordinates of three component 8-AMMC signal constellation is explained, the formation of signals in the time plane based on the found coordinates is explained as well as their discretization and the addition of white noise. An iterative algorithm for generating initial data for ANN based on the described ratios is proposed. The general structure of one-dimensional convolutional neural network is considered. Functions of individual neurons, connections between them, the formation of layers and the convolution operation are described mathematically. On the basis of the previously given ratios, a final display of the network was formed. Specific dimensions and activation functions for layers are selected. The use of convolutional layers is justified by time invariance. Based on the reviewed mathematical models, selected activation functions and dimensions, a neural model was formed. The process of validating the effectiveness of the formed neural model is described, which is based on comparing the symbolic error probabilities of the proposed and reference models at different signal-to-noise ratios. The validation results are presented. The advantages of the obtained model over the previously proposed purely recurrent model and the AMMC reference receiver are explained.

Integrating Graph Neural Network-Based Surrogate Modeling with Inverse Design for Granular Flows

Integrating Graph Neural Network-Based Surrogate Modeling with Inverse Design for Granular Flows

A digital twin model of the axial temperature field of a DC cable for millisecond calculations

The most important features of digital twin technology for transmission and transformation equipment are symbiotic evolution and state prediction. In constructing the digital twin model for DC cables, the transient and steady-state temperature fields need to be considered comprehensively in the modeling process to consider the multi-physical field coupling, mathematical model, operational data, and environmental conditions in specific cases, which makes it difficult to satisfy the real-time requirements of the digital twin. To solve this problem, in this paper, a digital twin technology architecture for DC cables is proposed, and a neural network learning model is constructed to minimize the latency due to the complexity of the model simulation. Moreover, an efficient digital twin model that is based on the model-driven approach is developed for the fast computation of the axial temperature field of DC cables. The use of the Sequence-to-Sequence residual neural network algorithm for the reduced-order multi-physics field coupling modeling reduces the computational time and ensures as much accuracy as possible up to the finite element model simulation. The simulation results show that the use of the digital twin model reduces the simulation time from 275 s to 0.56 s, and the relative error between the accuracy of the digital twin model and the finite element model for the axial temperature field of the DC cable is 0.46 %. Finally, verification of the reasonableness and feasibility of the proposed method is carried out via an experimental platform according to the standard of CIGRE TB496. In particular, the maximum value of the DC cable temperature is verified to have a small error compared with the result of the Sequence-to-Sequence calculation under specific working conditions.

A CNN Model for Physical Activity Recognition and Energy Expenditure Estimation from an Eyeglass-Mounted Wearable Sensor

Metabolic syndrome poses a significant health challenge worldwide, prompting the need for comprehensive strategies integrating physical activity monitoring and energy expenditure. Wearable sensor devices have been used both for energy intake and energy expenditure (EE) estimation. Traditionally, sensors are attached to the hip or wrist. The primary aim of this research is to investigate the use of an eyeglass-mounted wearable energy intake sensor (Automatic Ingestion Monitor v2, AIM-2) for simultaneous recognition of physical activity (PAR) and estimation of steady-state EE as compared to a traditional hip-worn device. Study data were collected from six participants performing six structured activities, with the reference EE measured using indirect calorimetry (COSMED K5) and reported as metabolic equivalents of tasks (METs). Next, a novel deep convolutional neural network-based multitasking model (Multitasking-CNN) was developed for PAR and EE estimation. The Multitasking-CNN was trained with a two-step progressive training approach for higher accuracy, where in the first step the model for PAR was trained, and in the second step the model was fine-tuned for EE estimation. Finally, the performance of Multitasking-CNN on AIM-2 attached to eyeglasses was compared to the ActiGraph GT9X (AG) attached to the right hip. On the AIM-2 data, Multitasking-CNN achieved a maximum of 95% testing accuracy of PAR, a minimum of 0.59 METs mean square error (MSE), and 11% mean absolute percentage error (MAPE) in EE estimation. Conversely, on AG data, the Multitasking-CNN model achieved a maximum of 82% testing accuracy in PAR, a minimum of 0.73 METs MSE, and 13% MAPE in EE estimation. These results suggest the feasibility of using an eyeglass-mounted sensor for both PAR and EE estimation.

Multi-Task Learning in Natural Language Processing: An Overview

Deep learning approaches have achieved great success in the field of Natural Language Processing (NLP). However, directly training deep neural models often suffer from overfitting and data scarcity problems that are pervasive in NLP tasks. In recent years, Multi-Task Learning (MTL), which can leverage useful information of related tasks to achieve simultaneous performance improvement on these tasks, has been used to handle these problems. In this paper, we give an overview of the use of MTL in NLP tasks. We first review MTL architectures used in NLP tasks and categorize them into four classes, including parallel architecture, hierarchical architecture, modular architecture, and generative adversarial architecture. Then we present optimization techniques on loss construction, gradient regularization, data sampling, and task scheduling to properly train a multi-task model. After presenting applications of MTL in a variety of NLP tasks, we introduce some benchmark datasets. Finally, we make a conclusion and discuss several possible research directions in this field.

Automatically Temporal Labeled Data Generation Using Positional Lexicon Expansion for Focus Time Estimation of News Articles

Many facts change over time, which is a fundamental aspect of our physical environment. In the case of pandemic articles, the user is not interested in the creation date of the document but in the facts and the cause of the last pandemic. Fake news can be better combated by having a document with a temporal focus. Currently, neither the sequence of events nor the temporal focus is considered when obtaining news documents. Despite the limited number of temporal aspects in the available datasets, it is difficult to test and evaluate the temporal conclusions of the model. The goal of this work is to develop a temporal focus news article retrieval model based on co-training to advance research in semi-supervised learning. A mapping of the dataset is performed using (1) the evolving focus time of news articles and (2) the semi-supervised method based on coincidence contexts for learning low-dimensional continuous vectors for learning neural contrast embedding models generating focus time-based query in sequential news articles to facilitate temporal understanding by learning low-dimensional continuous vectors. A diverse dataset of news articles is used to evaluate the effectiveness of the proposed method. With semi-supervised learning and lexicon expansion, the result of the developed model can achieve 89%. The method performed better than previous baselines and traditional machine learning models with improvements of 12.65% and 4.7%, respectively.

Fault diagnosis of drone motors driven by current signal data with few samples

Multi rotor unmanned aerial vehicles (UAVs) are extensively utilized across various domains, and the motor constitutes a pivotal element in the UAV power system. The majority of UAV failures and crashes stem from motor malfunctions, underscoring the imperative need for comprehensive research on fault diagnosis in UAV motors to ensure the stable and reliable execution of flight tasks. This study focuses on quadrotor UAVs as the research subject and devises targeted fault simulation experiments based on the structural features and operational characteristics of the DC brushless motor used in quadrotor UAVs, specifically examining the stator, rotor, and bearings. To address challenges related to the UAV’s own loads, limited space for redundant parts, and the high cost and difficulty associated with installing sensors for traditional fault diagnostic signals such as vibration and temperature, this study opts to use current signals as a substitute. This approach resolves the issue of challenging data collection for UAVs and investigates a current signal based fault diagnosis method for UAV motors. Lastly, in response to the limited training samples available for fault data due to the UAV’s highly sensitive characteristics regarding the health status of its components and flight stability, traditional machine learning and deep learning methods encounter difficulties in identifying representative features with a small number of training samples, leading to the risk of overfitting and reduced model accuracy in fault diagnosis. To overcome this challenge, we propose a hybrid neural network fault diagnosis model that incorporates a width learning system and a convolutional neural network (CNN). The width learning system eliminates temporal characteristics from the original current signal, capturing more comprehensive and representative sample features in the width feature space. Subsequently, the CNN is employed for feature extraction and classification tasks. In empirical small sample fault diagnosis experiments using current signal data for UAV motors, our proposed model outperforms other models used for comparison.

Data–Physics Fusion-Driven Defect Predictions for Titanium Alloy Casing Using Neural Network

The quality of Ti alloy casing is crucial for the safe and stable operation of aero engines. However, the fluctuation of key process parameters during the investment casting process of titanium alloy casings has a significant influence on the volume and number of porosity defects, and this influence cannot be effectively suppressed at present. Therefore, this paper proposes a strategy to control the influence of process parameters on shrinkage volume and number. This study constructed multiple regression prediction models and neural network prediction models of porosity volume and number for a ZTC4 casing by simulating the gravity investment casting process. The results show that the multiple regression prediction model and neural network prediction model of shrinkage cavity total volume have an accuracy of over 99%. The accuracy of the neural network prediction model is higher than that of the multiple regression model, and the neural network model realizes the accurate prediction of shrinkage defect volume and defect number through pouring temperature, pouring time, and mold shell temperature. The sensitivity degree of casing defects to key process parameters, from high to low, is as follows: pouring temperature, pouring time, and mold temperature. Further optimizing the key process parameter window reduces the influence of process parameter fluctuation on the volume and number of porosity defects in casing castings. This study provides a reference for actual production control process parameters to reduce shrinkage cavity and loose defects.

A cross layer graphical neural network based convolutional neural network framework for image dehazing

The current version of imaging equipment cannot quickly and effectively make up for the reduction of visibility triggered by bad weather. Traditional strategies minimize hazy impacts by employing an image depth model and a physical model. Following experts, erroneous depth data reduces the efficacy of the dehazing algorithm. Dehazing methods based on CNN are imperfect to handle region which is bright or similar to atmospheric light and thus leads to oversaturation of pixels. These challenges can be addressed by proposing a novel model that incorporates the idea of a Graphical Neural Network. The amount of light coming from the atmosphere is estimated using normalization where the contrast of the image gets adjusted using Bias Contrast stretch Histogram Equalization. An enhanced Transmission map estimator is used to render the hazy scene. Finally, the cross-layer graphical neural network-based CNN model is applied to produce a haze-free image and eliminate the over-saturation of pixels. Extensive evaluation findings show that the proposed approach can significantly recuperate misty imagery, even if the images have a substantial amount of haze.

Effects of the spatial resolution of the Virtual Epileptic Patient on the identification of epileptogenic networks

Abstract Digital twins play an increasing role in clinical decision making. This study evaluates a digital brain twin approach in presurgical evaluation, the Virtual Epileptic Patient (VEP), which estimates the epileptogenic zone in patients with drug-resistant epilepsy. We built the personalized digital brain twins of 14 patients and a series of synthetic dataset by considering different spatial configurations of the epileptogenic and/or propagation zone networks (EZN and PZN, respectively). Brain source signals were simulated with a high spatial resolution neural field model (NFM) composed of 81942 nodes, embedding both long-range (between brain regions) and short-range (within brain regions) coupling. Brain signals were then projected to stereotactic electroencephalographic (SEEG) contacts with an accurate forward solution. An inversion procedure based on a low spatial resolution neural mass model (NMM) composed of 162 nodes was applied to estimate the excitability of each region in each simulation. The ensuing estimated EZN/PZN was compared to the simulated ground truth by means of classification metrics. Overall, we observed correct but degraded performance when using an NMM to estimate the EZN from data simulated with an NFM, which was significant for the simplest spatial configurations. We quantified the reduced performance and demonstrated that the oversimplification of the forward problem is its principal cause. We showed that the absence of local coupling in the NMM affects the inversion process by an overestimation of the excitability, representing a significant clinical impact when using this procedure in the context of presurgical planning. In conclusion, this study highlighted the importance to shift from an NMM towards a full NFM modeling approach for the estimation of EZN, with a particularly relevant need when considering the most complex clinical cases.