Neural Model Research Articles

Interpolation of Environmental Data Using Deep Learning and Model Inference

Interpolation of Environmental Data Using Deep Learning and Model Inference

CNSMolGen: A Bidirectional Recurrent Neural Network-Based Generative Model for De Novo Central Nervous System Drug Design.

Central nervous system (CNS) drugs have had a significant impact on treating a wide range of neurodegenerative and psychiatric disorders. In recent years, deep learning-based generative models have shown great potential for accelerating drug discovery and improving efficacy. However, specific applications of these techniques in CNS drug discovery have not been widely reported. In this study, we developed the CNSMolGen model, which uses a framework of bidirectional recurrent neural networks (Bi-RNNs) for de novo molecular design of CNS drugs. Results showed that the pretrained model was able to generate more than 90% of completely new molecular structures, which possessed the properties of CNS drug molecules and were synthesizable. In addition, transfer learning was performed on small data sets with specific biological activities to evaluate the potential application of the model for CNS drug optimization. Here, we used drugs against the classical CNS disease target serotonin transporter (SERT) as a fine-tuned data set and generated a focused database against the target protein. The potential biological activities of the generated molecules were verified by using the physics-based induced-fit docking study. The success of this model demonstrates its potential in CNS drug design and optimization, which provides a new impetus for future CNS drug development.

Recognizing Sign Language using Machine Learning and Deep Learning Models

Individuals with hearing impairments communicate mostly through sign language. Our goal was to create an American Sign Language recognition dataset and utilize it in a neural network-based machine learning model that can interpret hand gestures and positions into natural language. In our study, we incorporated the SVM, CNN and Resnet-18 models to enhance predictability when interpreting ASL signs through this new dataset, which includes provisions such as lighting and distance limitations. Our research also features comparison results between all the other models implemented under invariant conditions versus those using our proposed CNN model. As demonstrated by its high levels of precision at 95.10% despite changes encountered during testing procedures like varying data sets or scene configurations where losses are minimal (0.545), there exists great potential for future applications in image recognition systems requiring deep learning techniques. Furthermore, these advancements may lead to significant improvements within various fields related explicitly to speech-language therapy sessions designed specifically around helping people overcome challenges associated with deafness while building bridges towards improved social integration opportunities.

ОДНОВИМІРНА ЗГОРТКОВА МОДЕЛЬ НЕЙРОННОЇ МЕРЕЖІ ДЛЯ ОБРОБЛЕННЯ СИГНАЛІВ З АМПЛІТУДНОЮ МОДУЛЯЦІЄЮ БАГАТЬОХ СКЛАДОВИХ

The processing of radio signals using artificial neural networks (ANNs) has great potential for research, which can be explained by the adaptability of ANNs to various transmission conditions and the ability to detect abstract patterns of changes in signal parameters. The article reviews the works of other authors devoted to different ways of using ANNs for processing radio signals. Taking into account the information in the reviewed works, the research task was formed, which consists in developing an optimized ANN model for radio signal processing. Signals with amplitude modulation of many components (AMMC) were chosen to form training samples for ANN. The choice of modulation type is justified by greater energy efficiency compared to other widely used digital modulation types, such as quadrature amplitude modulation. Mathematic basis of AMMC signal generation is described. The process of finding the coordinates of three component 8-AMMC signal constellation is explained, the formation of signals in the time plane based on the found coordinates is explained as well as their discretization and the addition of white noise. An iterative algorithm for generating initial data for ANN based on the described ratios is proposed. The general structure of one-dimensional convolutional neural network is considered. Functions of individual neurons, connections between them, the formation of layers and the convolution operation are described mathematically. On the basis of the previously given ratios, a final display of the network was formed. Specific dimensions and activation functions for layers are selected. The use of convolutional layers is justified by time invariance. Based on the reviewed mathematical models, selected activation functions and dimensions, a neural model was formed. The process of validating the effectiveness of the formed neural model is described, which is based on comparing the symbolic error probabilities of the proposed and reference models at different signal-to-noise ratios. The validation results are presented. The advantages of the obtained model over the previously proposed purely recurrent model and the AMMC reference receiver are explained.

Integrating Graph Neural Network-Based Surrogate Modeling with Inverse Design for Granular Flows

Integrating Graph Neural Network-Based Surrogate Modeling with Inverse Design for Granular Flows

A digital twin model of the axial temperature field of a DC cable for millisecond calculations

The most important features of digital twin technology for transmission and transformation equipment are symbiotic evolution and state prediction. In constructing the digital twin model for DC cables, the transient and steady-state temperature fields need to be considered comprehensively in the modeling process to consider the multi-physical field coupling, mathematical model, operational data, and environmental conditions in specific cases, which makes it difficult to satisfy the real-time requirements of the digital twin. To solve this problem, in this paper, a digital twin technology architecture for DC cables is proposed, and a neural network learning model is constructed to minimize the latency due to the complexity of the model simulation. Moreover, an efficient digital twin model that is based on the model-driven approach is developed for the fast computation of the axial temperature field of DC cables. The use of the Sequence-to-Sequence residual neural network algorithm for the reduced-order multi-physics field coupling modeling reduces the computational time and ensures as much accuracy as possible up to the finite element model simulation. The simulation results show that the use of the digital twin model reduces the simulation time from 275 s to 0.56 s, and the relative error between the accuracy of the digital twin model and the finite element model for the axial temperature field of the DC cable is 0.46 %. Finally, verification of the reasonableness and feasibility of the proposed method is carried out via an experimental platform according to the standard of CIGRE TB496. In particular, the maximum value of the DC cable temperature is verified to have a small error compared with the result of the Sequence-to-Sequence calculation under specific working conditions.

A CNN Model for Physical Activity Recognition and Energy Expenditure Estimation from an Eyeglass-Mounted Wearable Sensor

Metabolic syndrome poses a significant health challenge worldwide, prompting the need for comprehensive strategies integrating physical activity monitoring and energy expenditure. Wearable sensor devices have been used both for energy intake and energy expenditure (EE) estimation. Traditionally, sensors are attached to the hip or wrist. The primary aim of this research is to investigate the use of an eyeglass-mounted wearable energy intake sensor (Automatic Ingestion Monitor v2, AIM-2) for simultaneous recognition of physical activity (PAR) and estimation of steady-state EE as compared to a traditional hip-worn device. Study data were collected from six participants performing six structured activities, with the reference EE measured using indirect calorimetry (COSMED K5) and reported as metabolic equivalents of tasks (METs). Next, a novel deep convolutional neural network-based multitasking model (Multitasking-CNN) was developed for PAR and EE estimation. The Multitasking-CNN was trained with a two-step progressive training approach for higher accuracy, where in the first step the model for PAR was trained, and in the second step the model was fine-tuned for EE estimation. Finally, the performance of Multitasking-CNN on AIM-2 attached to eyeglasses was compared to the ActiGraph GT9X (AG) attached to the right hip. On the AIM-2 data, Multitasking-CNN achieved a maximum of 95% testing accuracy of PAR, a minimum of 0.59 METs mean square error (MSE), and 11% mean absolute percentage error (MAPE) in EE estimation. Conversely, on AG data, the Multitasking-CNN model achieved a maximum of 82% testing accuracy in PAR, a minimum of 0.73 METs MSE, and 13% MAPE in EE estimation. These results suggest the feasibility of using an eyeglass-mounted sensor for both PAR and EE estimation.

Multi-Task Learning in Natural Language Processing: An Overview

Deep learning approaches have achieved great success in the field of Natural Language Processing (NLP). However, directly training deep neural models often suffer from overfitting and data scarcity problems that are pervasive in NLP tasks. In recent years, Multi-Task Learning (MTL), which can leverage useful information of related tasks to achieve simultaneous performance improvement on these tasks, has been used to handle these problems. In this paper, we give an overview of the use of MTL in NLP tasks. We first review MTL architectures used in NLP tasks and categorize them into four classes, including parallel architecture, hierarchical architecture, modular architecture, and generative adversarial architecture. Then we present optimization techniques on loss construction, gradient regularization, data sampling, and task scheduling to properly train a multi-task model. After presenting applications of MTL in a variety of NLP tasks, we introduce some benchmark datasets. Finally, we make a conclusion and discuss several possible research directions in this field.

Automatically Temporal Labeled Data Generation Using Positional Lexicon Expansion for Focus Time Estimation of News Articles

Many facts change over time, which is a fundamental aspect of our physical environment. In the case of pandemic articles, the user is not interested in the creation date of the document but in the facts and the cause of the last pandemic. Fake news can be better combated by having a document with a temporal focus. Currently, neither the sequence of events nor the temporal focus is considered when obtaining news documents. Despite the limited number of temporal aspects in the available datasets, it is difficult to test and evaluate the temporal conclusions of the model. The goal of this work is to develop a temporal focus news article retrieval model based on co-training to advance research in semi-supervised learning. A mapping of the dataset is performed using (1) the evolving focus time of news articles and (2) the semi-supervised method based on coincidence contexts for learning low-dimensional continuous vectors for learning neural contrast embedding models generating focus time-based query in sequential news articles to facilitate temporal understanding by learning low-dimensional continuous vectors. A diverse dataset of news articles is used to evaluate the effectiveness of the proposed method. With semi-supervised learning and lexicon expansion, the result of the developed model can achieve 89%. The method performed better than previous baselines and traditional machine learning models with improvements of 12.65% and 4.7%, respectively.

Fault diagnosis of drone motors driven by current signal data with few samples

Multi rotor unmanned aerial vehicles (UAVs) are extensively utilized across various domains, and the motor constitutes a pivotal element in the UAV power system. The majority of UAV failures and crashes stem from motor malfunctions, underscoring the imperative need for comprehensive research on fault diagnosis in UAV motors to ensure the stable and reliable execution of flight tasks. This study focuses on quadrotor UAVs as the research subject and devises targeted fault simulation experiments based on the structural features and operational characteristics of the DC brushless motor used in quadrotor UAVs, specifically examining the stator, rotor, and bearings. To address challenges related to the UAV’s own loads, limited space for redundant parts, and the high cost and difficulty associated with installing sensors for traditional fault diagnostic signals such as vibration and temperature, this study opts to use current signals as a substitute. This approach resolves the issue of challenging data collection for UAVs and investigates a current signal based fault diagnosis method for UAV motors. Lastly, in response to the limited training samples available for fault data due to the UAV’s highly sensitive characteristics regarding the health status of its components and flight stability, traditional machine learning and deep learning methods encounter difficulties in identifying representative features with a small number of training samples, leading to the risk of overfitting and reduced model accuracy in fault diagnosis. To overcome this challenge, we propose a hybrid neural network fault diagnosis model that incorporates a width learning system and a convolutional neural network (CNN). The width learning system eliminates temporal characteristics from the original current signal, capturing more comprehensive and representative sample features in the width feature space. Subsequently, the CNN is employed for feature extraction and classification tasks. In empirical small sample fault diagnosis experiments using current signal data for UAV motors, our proposed model outperforms other models used for comparison.

Attention-Based Two-Branch Hybrid Fusion Network for Medical Image Segmentation

Accurate segmentation of medical images is vital for disease detection and treatment. Convolutional Neural Networks (CNN) and Transformer models are widely used in medical image segmentation due to their exceptional capabilities in image recognition and segmentation. However, CNNs often lack an understanding of the global context and may lose spatial details of the target, while Transformers struggle with local information processing, leading to reduced geometric detail of the target. To address these issues, this research presents a Global-Local Fusion network model (GLFUnet) based on the U-Net framework and attention mechanisms. The model employs a dual-branch network that utilizes ConvNeXt and Swin Transformer to simultaneously extract multi-level features from pathological images. It enhances ConvNeXt’s local feature extraction with spatial and global attention up-sampling modules, while improving Swin Transformer’s global context dependency with channel attention. The Attention Feature Fusion module and skip connections efficiently merge local detailed and global coarse features from CNN and Transformer branches at various scales. The fused features are then progressively restored to the original image resolution for pixel-level prediction. Comprehensive experiments on datasets of stomach and liver cancer demonstrate GLFUnet’s superior performance and adaptability in medical image segmentation, holding promise for clinical analysis and disease diagnosis.

Data–Physics Fusion-Driven Defect Predictions for Titanium Alloy Casing Using Neural Network

The quality of Ti alloy casing is crucial for the safe and stable operation of aero engines. However, the fluctuation of key process parameters during the investment casting process of titanium alloy casings has a significant influence on the volume and number of porosity defects, and this influence cannot be effectively suppressed at present. Therefore, this paper proposes a strategy to control the influence of process parameters on shrinkage volume and number. This study constructed multiple regression prediction models and neural network prediction models of porosity volume and number for a ZTC4 casing by simulating the gravity investment casting process. The results show that the multiple regression prediction model and neural network prediction model of shrinkage cavity total volume have an accuracy of over 99%. The accuracy of the neural network prediction model is higher than that of the multiple regression model, and the neural network model realizes the accurate prediction of shrinkage defect volume and defect number through pouring temperature, pouring time, and mold shell temperature. The sensitivity degree of casing defects to key process parameters, from high to low, is as follows: pouring temperature, pouring time, and mold temperature. Further optimizing the key process parameter window reduces the influence of process parameter fluctuation on the volume and number of porosity defects in casing castings. This study provides a reference for actual production control process parameters to reduce shrinkage cavity and loose defects.

A cross layer graphical neural network based convolutional neural network framework for image dehazing

The current version of imaging equipment cannot quickly and effectively make up for the reduction of visibility triggered by bad weather. Traditional strategies minimize hazy impacts by employing an image depth model and a physical model. Following experts, erroneous depth data reduces the efficacy of the dehazing algorithm. Dehazing methods based on CNN are imperfect to handle region which is bright or similar to atmospheric light and thus leads to oversaturation of pixels. These challenges can be addressed by proposing a novel model that incorporates the idea of a Graphical Neural Network. The amount of light coming from the atmosphere is estimated using normalization where the contrast of the image gets adjusted using Bias Contrast stretch Histogram Equalization. An enhanced Transmission map estimator is used to render the hazy scene. Finally, the cross-layer graphical neural network-based CNN model is applied to produce a haze-free image and eliminate the over-saturation of pixels. Extensive evaluation findings show that the proposed approach can significantly recuperate misty imagery, even if the images have a substantial amount of haze.

Effects of the spatial resolution of the Virtual Epileptic Patient on the identification of epileptogenic networks

Abstract Digital twins play an increasing role in clinical decision making. This study evaluates a digital brain twin approach in presurgical evaluation, the Virtual Epileptic Patient (VEP), which estimates the epileptogenic zone in patients with drug-resistant epilepsy. We built the personalized digital brain twins of 14 patients and a series of synthetic dataset by considering different spatial configurations of the epileptogenic and/or propagation zone networks (EZN and PZN, respectively). Brain source signals were simulated with a high spatial resolution neural field model (NFM) composed of 81942 nodes, embedding both long-range (between brain regions) and short-range (within brain regions) coupling. Brain signals were then projected to stereotactic electroencephalographic (SEEG) contacts with an accurate forward solution. An inversion procedure based on a low spatial resolution neural mass model (NMM) composed of 162 nodes was applied to estimate the excitability of each region in each simulation. The ensuing estimated EZN/PZN was compared to the simulated ground truth by means of classification metrics. Overall, we observed correct but degraded performance when using an NMM to estimate the EZN from data simulated with an NFM, which was significant for the simplest spatial configurations. We quantified the reduced performance and demonstrated that the oversimplification of the forward problem is its principal cause. We showed that the absence of local coupling in the NMM affects the inversion process by an overestimation of the excitability, representing a significant clinical impact when using this procedure in the context of presurgical planning. In conclusion, this study highlighted the importance to shift from an NMM towards a full NFM modeling approach for the estimation of EZN, with a particularly relevant need when considering the most complex clinical cases.

Hyper-Flophet: A neural Prophet-based model for traffic flow forecasting in transportation systems

Nowadays, an accurate and reliable traffic forecast is meaningful in making the right decisions for traffic management systems in vehicular environments. Nevertheless, traffic flow prediction is a significant challenge in Vehicular Ad Hoc Networks (VANETs) that has taken much attention. Therefore, in this paper, we propose a hybrid traffic prediction model based on Prophet model and Long Short-Term Memory neural network (LSTM), called Hyper-Flophet, to predict next traffic flow. Hyper-Flophet model adopts the traditional neural prophet model but with major parameter tuning. First, we propose an efficient algorithm for predicting the traffic flow trend then, we develop an interactive LSTM (I-LSTM) model for auto-regression components. After that, we implement a new future regressor component called network mobility and finally, we enhance the event and holiday component by introducing exponential growth term. Through simulations with real VANET data, we show that the proposed hybrid approach can achieve superior forecasting performance over other models.

Physics‐constrained symbolic model discovery for polyconvex incompressible hyperelastic materials

AbstractWe present a machine learning framework capable of consistently inferring mathematical expressions of hyperelastic energy functionals for incompressible materials from sparse experimental data and physical laws. To achieve this goal, we propose a polyconvex neural additive model (PNAM) that enables us to express the hyperelastic model in a learnable feature space while enforcing polyconvexity. An upshot of this feature space obtained via the PNAM is that (1) it is spanned by a set of univariate basis functions that can be re‐parametrized with a more complex mathematical form, and (2) the resultant elasticity model is guaranteed to fulfill the polyconvexity, which ensures that the acoustic tensor remains elliptic for any deformation. To further improve the interpretability, we use genetic programming to convert each univariate basis into a compact mathematical expression. The resultant multi‐variable mathematical models obtained from this proposed framework are not only more interpretable but are also proven to fulfill physical laws. By controlling the compactness of the learned symbolic form, the machine learning‐generated mathematical model also requires fewer arithmetic operations than its deep neural network counterparts during deployment. This latter attribute is crucial for scaling large‐scale simulations where the constitutive responses of every integration point must be updated within each incremental time step. We compare our proposed model discovery framework against other state‐of‐the‐art alternatives to assess the robustness and efficiency of the training algorithms and examine the trade‐off between interpretability, accuracy, and precision of the learned symbolic hyperelastic models obtained from different approaches. Our numerical results suggest that our approach extrapolates well outside the training data regime due to the precise incorporation of physics‐based knowledge.

Ketosis regulates K+ ion channels, strengthening brain-wide signaling disrupted by age

Abstract Aging is associated with impaired signaling between brain regions when measured using resting-state functional magnetic resonance imaging (fMRI). This age-related destabilization and desynchronization of brain networks reverses itself when the brain switches from metabolizing glucose to ketones. Here, we probe the mechanistic basis for these effects. First, we confirmed their robustness across measurement modalities using two datasets acquired from resting-state EEG (Lifespan: standard diet, 20–80 years, N = 201; Metabolic: individually weight-dosed and calorically-matched glucose and ketone ester challenge, μage = 26.9 ±11.2 years, N = 36). Then, using a multiscale conductance-based neural mass model, we identified the unique set of mechanistic parameters consistent with our clinical data. Together, our results implicate potassium (K+) gradient dysregulation as a mechanism for age-related neural desynchronization and its reversal with ketosis, the latter finding of which is consistent with direct measurement of ion channels. As such, the approach facilitates the connection between macroscopic brain activity and cellular-level mechanisms.

Distributed and dynamical communication: a mechanism for flexible cortico-cortical interactions and its functional roles in visual attention

Perceptual and cognitive processing relies on flexible communication among cortical areas; however, the underlying neural mechanism remains unclear. Here we report a mechanism based on the realistic spatiotemporal dynamics of propagating wave patterns in neural population activity. Using a biophysically plausible, multiarea spiking neural circuit model, we demonstrate that these wave patterns, characterized by their rich and complex dynamics, can account for a wide variety of empirically observed neural processes. The coordinated interactions of these wave patterns give rise to distributed and dynamic communication (DDC) that enables flexible and rapid routing of neural activity across cortical areas. We elucidate how DDC unifies the previously proposed oscillation synchronization-based and subspace-based views of interareal communication, offering experimentally testable predictions that we validate through the analysis of Allen Institute Neuropixels data. Furthermore, we demonstrate that DDC can be effectively modulated during attention tasks through the interplay of neuromodulators and cortical feedback loops. This modulation process explains many neural effects of attention, underscoring the fundamental functional role of DDC in cognition.

Cyclic behavior of laminated bio-based connections with slotted-in steel plates: Genetic algorithm, deterministic neural network-based model parameter identification, and uncertainty quantification

To support more sustainable construction, this paper experimentally investigates the cyclic behavior of laminated timber (Laminated Veneer Lumber (LVL)) and glubam (Glue Laminated Bamboo) connections with slotted-in steel plates in terms of experimental test, numerical simulations and parameter identification. Experimental tests included eight different configurations: two materials (LVL and glubam), two bolt diameters (8 and 10 mm), and one or two bolts. Two different cyclic-loading protocols were applied for each type of connection: only tension and tension/compression. The observed behavior is then compared to a finite element model developed in OpenSeesPy, which takes into account factors such as sliding, contact, pinching, cyclic stiffness, and strength degradation. To identify the best set of parameters for the model, three different approaches are considered: genetic algorithm, fast deterministic neural network, and probabilistic Bayesian method. First, the model identification is carried out by means of a genetic algorithm-based optimization. The parameter-identification results are evaluated in terms of elastic stiffness, yielding point, and ductility. Next, a sensitivity analysis is performed to determine the significance of the parameters, and an innovative approach combining neural network and sensitivity analysis is proposed for fast and preliminary parameter identification. Then, probabilistic Bayesian identification is employed to calculate the posterior distribution of the model parameters identified and the confidence bounds of the estimated response. Finally, different model identification parameters are compared and suggestions for algorithm selection are provided.

Novel robust Elman neural network-based predictive models for bubble point oil formation volume factor and solution gas–oil ratio using experimental data

Novel robust Elman neural network-based predictive models for bubble point oil formation volume factor and solution gas–oil ratio using experimental data