Static Neural Networks Research Articles

Alzheimer's disease stage recognition from MRI and PET imaging data using Pareto-optimal quantum dynamic optimization

The threat posed by Alzheimer's disease (AD) to human health has grown significantly. However, the precise diagnosis and classification of AD stages remain a challenge. Neuroimaging methods such as structural magnetic resonance imaging (sMRI) and fluorodeoxyglucose positron emission tomography (FDG-PET) have been used to diagnose and categorize AD. However, feature selection approaches that are frequently used to extract additional data from multimodal imaging are prone to errors. This paper suggests using a static pulse-coupled neural network and a Laplacian pyramid to combine sMRI and FDG-PET data. After that, the fused images are used to train the Mobile Vision Transformer (MViT), optimized with Pareto-Optimal Quantum Dynamic Optimization for Neural Architecture Search, while the fused images are augmented to avoid overfitting and then classify unfused MRI and FDG-PET images obtained from the AD Neuroimaging Initiative (ADNI) and Open Access Series of Imaging Studies (OASIS) datasets into various stages of AD. The architectural hyperparameters of MViT are optimized using Quantum Dynamic Optimization, which ensures a Pareto-optimal solution. The Peak Signal-to-Noise Ratio (PSNR), the Mean Squared Error (MSE), and the Structured Similarity Indexing Method (SSIM) are used to measure the quality of the fused image. We found that the fused image was consistent in all metrics, having 0.64 SIMM, 35.60 PSNR, and 0.21 MSE for the FDG-PET image. In the classification of AD vs. cognitive normal (CN), AD vs. mild cognitive impairment (MCI), and CN vs. MCI, the precision of the proposed method is 94.73%, 92.98% and 89.36%, respectively. The sensitivity is 90. 70%, 90. 70%, and 90. 91% while the specificity is 100%, 100%, and 85. 71%, respectively, in the ADNI MRI test data.

Neural network adaptive control for turbo‐generator of power systems with prescribed performance and unknown asymmetric actuator dead zone

AbstractThis paper investigates a neural network adaptive controller design method for turbo‐generator of power systems with external disturbance, unknown system dynamic, prescribed performance, and unknown actuator dead zone. First, the unknown system dynamic is estimated and overcome through a neural network. Using the implicit function theorem, the unknown asymmetric dead‐zone behavior of the actuator is overcome by another static neural network. Second, the external disturbance and the reconstruction error of neural networks are handled by a robust term updated online. Moreover, there is no requirement to pre‐know or off‐line estimate the reconstruction error of neural networks and the upper bound of the system external disturbance. Third, based on Lyapunov theory, the smooth control law is proposed. In the meantime, the uniform ultimate boundedness of the networks weights is strictly proved. Furthermore, the system state error satisfies the prescribed transient performance and can converge to a small neighborhood around zero. Finally, a numerical simulation shows the effectiveness of the proposed method.

Enhanced Performance of Dynamic Neural Network Model using Wavelet Activation Functions

Both static and dynamic adaptive neural networks have been broadly utilized in mathematical modeling and numerical analysis. This study aimed to enhance the accomplishment of Dynamic Neural Networks (DNN) models by applying wavelet functions as activation functions. Research that models and forecasts the intensity of solar radiation in Mataram City shows that combining B-Spline and Morlet wavelet activation functions can significantly increase the DNN model performance. Wavelet-DNN (W-DNN) was modeled with an identical architecture; the best showed the increase in the model achievement (0.7596 points for in-sample and 0.8502 points for out-sample data). Mainly for out-sample data, the model's performance using the W-DNN+ intervention model increased by 4.0492 points.

Artificial intelligence-based model for physical-mechanical surface properties of nanostructured coatings

This article presents a computational numerical model for the simulation and analysis of quantum chemistry and Gibbs free energy theory using static (ANNS), dynamic (DANN), and chaotic neural networks (CANN). The model calculates the physical-surface mechanics of hardness, adhesion, and strength. They resulted in nanostructured metal coatings with electrodeposited chromium nanoparticles on low-carbon steel. The ANNS, DANN, and CANN simulations showed that model-obtained values for analyzed properties presented an approximation of 99 % concerning theoretical matters taken as base. Likewise, model accuracy was verified by comparison with reference data (datasheet). The proposed model is not limited to the analyzed case and provides consistent results for predicting surface physical-mechanical properties of electrodeposited coating-substrate arrangements, with a minimum error percentage of 1–1.5 % over learning.

Data driven modeling advancements for air temperature predictions in data centers

While data center cooling energy usage optimization studies have been performed through computational fluid dynamics/heat transfer (CFD/HT), and heuristic methods, data driven modeling techniques are now also being used for these applications. This paper investigates the air temperature prediction capabilities of static artificial neural network (ANN), Gaussian progress regression (GPR), support vector regression (SVR), relevance vector machine (RVM), linear regression, and regression trees; and transient long-short term memory (LSTM), and nonlinear autoregressive neural network with external input (NARX)) data driven modeling frameworks. The static study compared various models and found that GPR provided the best results (average error of 0.56 °C), closely followed by the ANN and SVR (average error of 0.60 °C and 0.68 °C respectively) methods. The transient study compared models based on an experimental data set and found that NARX outperforms LSTM for normal operations (0.83 °C and 1.07 °C average error respectively), and that data driven models are able to provide relatively good predictions, even if the input variables are slightly outside the training domain.

Development of Mathematical Models for Industrial Processes Using Dynamic Neural Networks

Dynamic neural networks (DNNs) are a type of artificial neural network (ANN) designed to work with sequential data where context in time is important. Unlike traditional static neural networks that process data in a fixed order, dynamic neural networks use information about past inputs, which is important if the dynamic of a certain process is emphasized. They are commonly used in natural language processing, speech recognition, and time series prediction. In industrial processes, their use is interesting for the prediction of difficult-to-measure process variables. In an industrial isomerization process, it is crucial to measure the quality attributes that affect the octane number of gasoline. Process analyzers commonly used for this purpose are expensive and subject to failure. Therefore, to achieve continuous production in the event of a malfunction, mathematical models for estimating product quality attributes are imposed as a solution. In this paper, mathematical models were developed using dynamic recurrent neural networks (RNNs), i.e., their subtype of a long short-term memory (LSTM) architecture. The results of the developed models were compared with the results of several types of other data-driven models developed for an isomerization process, such as multilayer perceptron (MLP) artificial neural networks, support vector machines (SVM), and dynamic polynomial models. The obtained results are satisfactory, suggesting a good possibility of application.

Evolving Hebbian Learning Rules in Voxel-Based Soft Robots

According to Hebbian theory, synaptic plasticity is the ability of neurons to strengthen or weaken the synapses among them in response to stimuli. It plays a fundamental role in the processes of learning and memory of biological neural networks. With plasticity, biological agents can adapt on multiple timescales and outclass artificial agents, the majority of which still rely on static Artificial Neural Network (ANN) controllers. In this work, we focus on Voxel-based Soft Robots (VSRs), a class of simulated artificial agents, composed as aggregations of elastic cubic blocks. We propose a Hebbian ANN controller where every synapse is associated with a Hebbian rule that controls the way the weight is adapted during the VSR lifetime. For a given task and morphology, we optimize the controller for the task of locomotion by evolving, rather than the weights, the parameters of the Hebbian rules. Our results show that the Hebbian controller is comparable, often better than a non-Hebbian baseline and that it is more adaptable to damages. We also provide novel insights into the inner workings of plasticity and demonstrate that “true” learning does take place, as the evolved controllers improve over the lifetime and generalize well.

A Study of Sandy Land Changes in the Chifeng Region from 1990 to 2020 Based on Dynamic Convolution

Desertification is the process of land degradation and the reduction or destruction of biological potential in arid, semi-arid, and semi-humid areas, and its impact on agricultural development and the ecological environment cannot be ignored. Therefore, many researchers have aimed to achieve the acquisition of large-scale sandy land areas using sandy land extraction algorithms based on remote sensing images. However, the sandy land extraction accuracy needs to be improved because of the variable contour features in the remote sensing images and the easy confusion with targets such as the Gobi and bare ground areas. In this study, we combine the dynamic convolution with a U-Net model and propose the DU-Net sandy land extraction model. The method is based on dynamic convolution, which can adaptively learn the complex features of the target and build a dynamic convolutional neural network to achieve high-accuracy extraction of complex targets. DU-Net achieved 86.32% in IoU, 93.22% in precision, 94.5% in recall, and 92.66% in F1-score in sandy land extraction accuracy, which are 4.68%, 2.33%, 3.09%, and 2.76% improvements, respectively, compared with the U-Net static neural network. Based on this, we obtained the spatial and temporal evolution trends of sandy land areas based on Landsat images in the Chifeng region in the Inner Mongolia Autonomous Region, China. Meanwhile, in order to investigate the mechanism of spatial and temporal changes in the sandy land area in the study region over the past 30 years, the direct and indirect effects of seven climatic and human socioeconomic activity factors on the changes in the sandy land area in the study region were evaluated based on a structural equation model. The results show that the sandy area in the Chifeng region tended to first increase and then decrease over the study period, with the sandy land area reaching its maximum around the year 2000. In addition, the main driving factor for the change in the sandy land area in the Chifeng region has been human socioeconomic activities, with climatic conditions as the secondary driving factor. The method proposed in this paper realizes the rapid extraction of sandy land areas with high accuracy at a large scale and with a long time series and provides a basis for assessing the effectiveness of ecosystem restoration projects.

Dynamic inference techniques for deep neural networks

With the development of deep neural network, dynamic inference techniques have attracted extensive attention. When the traditional static neural network is faced with complex samples, it will generate a lot of computational redundancy, resulting in a waste of computing resources. In order to solve this problem, on the basis of static network, experts use dynamic reasoning technology to improve the network, so that the network can calculate the samples. Therefore, the network structure has been fundamentally improved by performing operations such as layer skipping and leaving early. Compared with the traditional static networks, the improved model has greatly improved the speed and scale. At the same time, the accuracy of operation has also been greatly improved.

Evolving a Pipeline Approach for Abstract Meaning Representation Parsing Towards Dynamic Neural Networks.

Meaning Representation parsing aims to represent a sentence as a structured, Directed, Acyclic Graph (DAG), in an attempt to extract meaning from text. This paper extends an existing 2-stage pipeline AMR parser with state-of-the-art techniques in dependency parsing. First, Pointer-Generator Networks are used for out-of-vocabulary words in the concept identification stage, with an improved initialization via the use of word-and character-level embeddings. Second, the performance of the Relation Identification module is improved by jointly training the Heads Selection and the Arcs Labeling components. Last, we underline the difficulty of end-to-end training with recurrent modules in a static deep neural network construction approach and explore a dynamic construction implementation, which continuously adapts the computation graph, thus potentially enabling end-to-end training in the proposed pipeline solution.

Output feedback passification of a class of fractional-order static neural networks

In this paper, a linear matrix inequality (LMI)-based design is proposed for the passivity-based output feedback control problem of fractional-order static neural networks (FOSNNs). Different from the current works, the studied model is the static fractional-order neural network. The novel convex Lyapunov function is constructed to obtain new conditions for the existence of output feedback controllers which guarantee the closed-loop systems are passive. In addition, we also extend the obtained results to generalized fractional-order neural networks. A numerical example is presented to verify the proposed results.

A cascade neural network methodology for fault detection and diagnosis in solar thermal plants

Detecting and isolating faults in collector fields of solar thermal power plants is a crucial and challenging task. The system variables in the collector area are highly coupled, which can lead to a high misclassification rate. For this reason, it becomes necessary to combine knowledge of systems engineering with machine learning techniques that unravel the complex dynamics that govern the systems using historical data. Furthermore, the performance of a solar thermal plant is highly dependent on solar irradiance which changes during the day and is subject to perturbations caused by clouds and other atmospheric conditions. Detecting the fault requires using techniques that cope with the disturbances in solar irradiance.In this work, real irradiance profiles with many types of clouds are used. First, a model-based fault detector is applied, obtaining an accuracy of over 89% for all test irradiances. Then, different machine learning techniques are compared: static neural networks with and without decoupling strategy, dynamic neural networks, dynamic neural networks in cascade, classification trees, random forests, radial basis function networks, and self-organizing maps. The combination of neural networks was the only method that obtained a total accuracy of over 73% and F1-scores over 50% for all the test irradiance profiles.

Sensory Adaptation in Biomolecular Memristors Improves Reservoir Computing Performance

Despite its prevalence in neurosensory systems for pattern recognition, event detection, and learning, the effects of sensory adaptation (SA) are not explored in reservoir computing (RC). Monazomycin‐based biomolecular synapse (MzBS) devices that exhibit volatile memristance and short‐term plasticity with two strength‐dependent modes of response are studied: facilitation and facilitation‐then‐depression (i.e., SA). Their ability to perform RC tasks including digit recognition, nonlinear function learning, and aerodynamic gust classification via combination of model‐based device simulations and physical experiments where SA presence is controlled is studied. Simulations exhibiting moderate SA achieve significantly higher accuracy classifying a custom 5 × 5 binary digit set, with experimental validation achieving maximum testing accuracies of 90%. Classifications of the Modified National Institute of Standards and Technology (MNIST) handwritten digit dataset achieve a maximum testing accuracy of 94.34% in devices with SA. Fitting error of the Mackey–Glass time series is also significantly reduced by SA. Experimentally obtained pressure distributions representing gusts on an airfoil in a wind tunnel are classified by MzBS reservoirs. Reservoirs exhibiting SA achieve 100% accuracy, unlike MzBS reservoirs without SA and comparable static neural networks.

Seeking innovation: The research protocol for SMEs' networking

The paper aims to state the research protocol for the innovation-seeking behavior of Small- to Medium-sized Enterprises (SMEs), related to the classification of knowledge needs expressed in the networking databases. The dataset of 9301 networking offers as the outcome of proactive attitudes represents the content of the Enterprise Europe Network (EEN) database. The data set has been semi-automatically obtained using the rvest R package, and then analyzed using static word embedding neural network architecture: Continuous Bag-of-Words (CBoW), predictive model Skip-Gram, and Global Vectors for Word Representation (GloVe) considered the state-of-the-art models, to create topic-specific lexicons. The proportion of offers labeled as Exploitative innovation to Explorative innovation is balanced with a 51%–49% proportion. The prediction rates show good performance with an AUC score of 0.887, and the prediction rates for exploratory innovation 0.878 and explorative innovation 0.857. The performance of predictions with the frequency-inverse document frequency (TF-IDF) technique shows that the research protocol is sufficient to categorize the innovation-seeking behavior of SMEs using static word embedding based on the description of knowledge needs and text classification, but it is not perfect due to the general entropy related to the outcome of networking. In the context of networking, SMEs place a greater emphasis on explorative innovation in their innovation-seeking behavior. They prioritize smart technologies and global business cooperation, whereas current information technologies and software are more of interest to SMEs that adopt an exploitative innovation approach.

Reconfigurable 2D-ferroelectric platform for neuromorphic computing

To meet the requirement of data-intensive computing in the data-explosive era, brain-inspired neuromorphic computing have been widely investigated for the last decade. However, incompatible preparation processes severely hinder the cointegration of synaptic and neuronal devices in a single chip, which limited the energy-efficiency and scalability. Therefore, developing a reconfigurable device including synaptic and neuronal functions in a single chip with same homotypic materials and structures is highly desired. Based on the room-temperature out-of-plane and in-plane intercorrelated polarization effect of 2D α-In2Se3, we designed a reconfigurable hardware platform, which can switch from continuously modulated conductance for emulating synapse to spiking behavior for mimicking neuron. More crucially, we demonstrate the application of such proof-of-concept reconfigurable 2D ferroelectric devices on a spiking neural network with an accuracy of 95.8% and self-adaptive grow-when required network with an accuracy of 85% by dynamically shrinking its nodes by 72%, which exhibits more powerful learning ability and efficiency than the static neural network.

State estimation of static neural networks with mixed delays: A new parameterized delay interval approach

This work concerns with the issue of H∞ state estimation for static neural networks with both leakage delay and time-varying delay. First, a new parameterized delay interval approach is proposed, where two parameters are introduced to divide the delay interval into four segments. Second, an appropriate Lyapunov–Krasovskii functional (LKF) including non-integral quadratic delay-product-type functional and augmented terms is constructed by combining four delay intervals. Meanwhile, for the sake of cooperating with the constructed LKF effectively, generalized free-weighting-matrix integral inequality (GFWMII) is utilized to derive much less conservative criteria. Finally, a novel criterion is derived which ensures that the considered system is asymptotically stable with prescribed H∞ performance. The effectiveness of the achieved approach is demonstrated by three commonly used numerical examples.

Comparative Analysis of ANN-Based Mobility Prediction Performance in an Ad-Hoc Network

The facility of arbitrary node movement one side has advantages on application on the other side very difficult to manage the network because random node mobility directly effects on network connectivity and interrupt on the performance, obtained challenges like routing overhead, packet losses, increases energy consumption, wasted bandwidth for reconnection, decreases throughput etc. Thus an accurate mobility prediction of a node before leaving one position to another or subsequence position can be improve network performance which is effects by node mobility. Now day’s artificial neural networks (ANNs) is very common and trending for approximation and prediction application and also popular for node trajectory prediction. In this paper we explore the architectures of some static (like MLP and RBF) and dynamic (like FTDNN, DTDNN, NARX and LSTM) neural network and search best ANN model by obtaining optimal model parameters to predict node mobility and compared the performance using mobility model (Gauss Markov model) dataset as well as real-world dataset collected from Crawdad to highlight generalization capabilities. Mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and average coordinates distance error (DE) between observed and estimated positions are used to evaluate their performance. The empirical results show that LSTM is the best artificial neural network (ANN) model for mobility prediction in both model based and real-world dataset(testing sets).

Multi-well dynamic liquid-level prediction method of pumping well based on dynamic and static information feature fusion neural network

In a rod pumping system, accurate prediction of dynamic liquid level is a key to rational optimization of pumping system parameters. At present, it is difficult to use a uniform prediction model for dynamic liquid-level prediction in different rod pumping systems. To this end, a method for predicting the dynamic liquid level of multiple wells in a pumping well based on dynamic and static information feature fusion (DSIFF) neural network is proposed in this paper. According to the principle of dynamic liquid-level calculation, suspension displacement, suspension load, pumping rod parameters, formation crude oil density, surface crude oil density, oil pressure, and casing pressure are used as inputs. The subnetwork feature extraction method is proposed for the problem of large network structure caused by the high dimension of the important dynamic data of suspension displacement and suspension load. The Huber loss function is used as the loss function of the prediction model for the case of abnormal data in the large amount of data required for model training. Finally, the results of comparative analysis show that the proposed method solves the problem of unified establishment of dynamic liquid-level prediction model for multiple oil wells and has better robustness for abnormal data.

Adaptive probabilistic load forecasting for individual buildings

Building-level load forecasting has become essential with the support of fine-grained data collected by widely deployed smart meters. It acts as a basis for arranging distributed energy resources, implementing demand response, etc. Compared to aggregated-level load, the electric load of an individual building is more stochastic and thus spawns many probabilistic forecasting methods. Many of them resort to artificial neural networks (ANN) to build forecasting models. However, a well-designed forecasting model for one building may not be suitable for others, and manually designing and tuning optimal forecasting models for various buildings are tedious and time-consuming. This paper proposes an adaptive probabilistic load forecasting model to automatically generate high-performance NN structures for different buildings and produce quantile forecasts for future loads. Specifically, we cascade the long short term memory (LSTM) layer with the adjusted Differential ArchiTecture Search (DARTS) cell and use the pinball loss function to guide the model during the improved model fitting process. A case study on an open dataset shows that our proposed model has superior performance and adaptivity over the state-of-the-art static neural network model. Besides, the improved fitting process of DARTS is proved to be more time-efficient than the original one.

Parameterized Luenberger-Type H∞ State Estimator for Delayed Static Neural Networks.

This article proposes a new Luenberger-type state estimator that has parameterized observer gains dependent on the activation function, to improve the H∞ state estimation performance of the static neural networks with time-varying delay. The nonlinearity of the activation function has a significant impact on stability analysis and robustness/performance. In the proposed state estimator, a parameter-dependent estimator gain is reconstructed by using the properties of the sector nonlinearity of the activation functions that are represented as linear combinations of weighting parameters. In the reformulated form, the constraints of the parameters for the activation function are considered in terms of linear matrix inequalities. Based on the Lyapunov-Krasovskii function and the improved reciprocally convex inequality, enhanced conditions for designing a new state estimator that guarantees H∞ performance are derived through a parameterization technique. The compared results with recent studies demonstrate the superiority and effectiveness of the presented method.