Neural Prediction Research Articles

Binocular imbalance measured by SSVEP predicts impaired stereoacuity in amblyopia

PurposeThe current study aims to implement steady-state visual evoked potentials (SSVEPs) in quantifying the binocular imbalance of amblyopia and to assess the predictive value of SSVEP-derived indices for amblyopic stereoacuity. MethodsWe measure frequency-tagged SSVEP responses elicited by each eye (F1 = 6 Hz through the fellow eye; F2 = 7.5 Hz through the amblyopic eye) within a binocular rivalry paradigm among a cohort of anisometropic amblyopic observers (n = 29, mean age: 12 years). Binocular suppression was quantified by assessing the disparity in SSVEP amplitudes between the eyes, while the strength of interocular interaction was evaluated through the intermodulation response at F1+F2 = 13.5 Hz. Subsequent analyses explored the associations between these neural indices and relevant behavioral metrics in amblyopia. ResultsResults reveal a significant difference in SSVEP amplitudes elicited from the fellow eye and the amblyopic eye, with the former exhibiting notably higher responses. Moreover, the fellow eye demonstrated prolonged dominance duration compared to its amblyopic counterpart. Furthermore, a negative correlation between binocular suppression and interocular interaction was observed, with stereoacuity showing a significant correlation with binocular suppression. Utilizing stepwise mulptiple linear regression analysis, we established that a predictive model combining binocular suppression and visual acuity of the amblyopic eye provided the best prediction of stereoacuity. ConclusionsThese results highlight the potential of binocular suppression, as assessed by SSVEPs within a binocular rivalry paradigm, as a promising neural predictor of stereopsis in amblyopia.

Quantum Architecture Search with Neural Predictor Based on Graph Measures

AbstractQuantum architecture search (QAS) has attracted increasing attention owing to its remarkable ability to automate the design of quantum circuits for variational quantum algorithms (VQAs). However, evaluating the performance of numerous quantum circuits is essential to provide feedback for the search strategy, which inevitably renders QAS computationally expensive. Performance predictors have emerged as highly efficient evaluation methods to mitigate this challenge. However, the performance predictor faces a critical challenge in reducing the required number of circuit‐performance pairs for training. This study encodes circuit architecture by representing a quantum circuit as a relational graph that emphasizes message exchange. Subsequently, valuable information about circuit architecture is extracted through three types of graph measures, including distance‐based, degree‐based, and cluster‐based measures. The graph measures define a smooth space related to circuit performance, facilitating the training of the performance predictor. The effectiveness of the proposed method is assessed across three tasks within variational quantum eigensolvers (VQE): identifying the ground states of the Transverse Field Ising Model (TFIM), the Heisenberg model, and the molecule. The simulation results demonstrate notable enhancements in predictive accuracy achieved by our method, coupled with a substantial reduction in the required number of training samples for the predictor.

Tunnel Prescribed Performance Control for Distributed Path Maneuvering of Multi-UAV Swarms via Distributed Neural Predictor

Tunnel Prescribed Performance Control for Distributed Path Maneuvering of Multi-UAV Swarms via Distributed Neural Predictor

Charging Station Power System Data Prediction Model Based on Deep Learning

The increase growth and adoption of electric vehicles (EVs) are playing a crucial role in advanced transportation system, helping to minimizing the emissions of harmful greenhouse gases and enhancing environmental sustainability. The need for EVs to be charged immediately has gained significant attention due to the rise in EVs sales over the last years. As a result of this, need for electric car charging is important to reducing the impact of electric network and providing minimum charging fares. In order to calculate the demand for charging EVs, a novel Deep Learning (DL)-based Long-Short Term Memory (LSTM) recurrent neural network predictor model is attempted to be developed in this research study. The Modified Aquilla Optimizer Algorithm (MAOA) is used to optimizing the parameter of the new Deep LSTM (DLSTM) neural predictor models and Independent Component Analysis (ICA) is utilized to solve the input time series data while conserving its properties. In this research, a novel ICA—AOA—DLSTM neural predictor model was developed to addressed the challenges of vanishing and exploding gradient in basic recurrent neural learnings. The predictor model was tested on the EV charging dataset from Georgia Tech, Atlanta, USA, indicating its performance. During simulation, the predictor achieved a prediction accuracies of 96.24% with a mean absolute errors of 0.1083 and a RMSE errors of 3.0629 × 10^-5, outperforming previous techniques. Additionally, the mean absolute error was found to be 0.2083, with a mean square error of 3.25516 × 10^-10. These results highlight the effectiveness of the novel deep learning LSTM neural predictor for this dataset compared to existing techniques.

DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning

Neural predictors have shown great potential in the evaluation process of neural architecture search (NAS). However, current predictor-based approaches overlook the fact that training a predictor necessitates a considerable number of trained neural networks as the labeled training set, which is costly to obtain. Therefore, the critical issue in utilizing predictors for NAS is to train a high-performance predictor using as few trained neural networks as possible. Although some methods attempt to address this problem through unsupervised learning, they often result in inaccurate predictions. We argue that the unsupervised tasks intended for the common graph data are too challenging for neural networks, causing unsupervised training to be susceptible to performance crashes in NAS. To address this issue, we propose a CurricuLum-guided Contrastive Learning framework for neural Predictor (DCLP). Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution during contrastive training. Specifically, we propose a scheduler that ranks the training data according to the contrastive difficulty of each data and then inputs them to the contrastive learner in order. This approach concentrates the training data distribution and makes contrastive training more efficient. By using our method, the contrastive learner incrementally learns feature representations via unsupervised data on a smooth learning curve, avoiding performance crashes that may occur with excessively variable training data distributions. We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors, and shows promising potential to discover superior architectures in various search spaces when combined with search strategies. Our code is available at: https://github.com/Zhengsh123/DCLP.

Robust adaptive fault-tolerant control for path maneuvering of autonomous surface vehicles with actuator faults based on the noncooperative game strategy

This paper investigates a parameterized path-guided following/maneuvering control problem of an autonomous surface vehicle (ASV) subject to actuator faults via a noncooperative game strategy. Unlike the existing path maneuvering control methods, the noncooperative game of this paper includes two sub-games. On one hand, the path update is considered to be against the effort of kinematic control. On the other hand, the kinetic control is considered to be against the total disturbances consisting of actuator faults and external disturbances. A robust adaptive fault-tolerant path maneuvering controller is designed based on a noncooperative game approach. Specifically, we design a kinematic control law using an improved dynamic surface control approach to achieve the geometric objective. An improved neural predictor is constructed, in which a concurrent learning-based adaptation is designed to identify unknown fault coefficients. A path update law and a kinetic control law are calculated to cover noncooperative games by using an adaptive dynamic programming approach. Theoretical analysis shows that the closed-loop system is input-to-state stable, and the proposed method can meet the geometric objective, the dynamic objective, and the fault-tolerant objective. Finally, simulation results demonstrate the efficacy of the proposed robust adaptive fault-tolerant control method for path maneuvering.

Convolutional Fuzzy Neural Predictor for Blood Pressure Estimation from Electrocardiography and Photoplethysmography Signals

Convolutional Fuzzy Neural Predictor for Blood Pressure Estimation from Electrocardiography and Photoplethysmography Signals

NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search.

Neural architecture search (NAS) adopts a search strategy to explore the predefined search space to find superior architecture with the minimum searching costs. Bayesian optimization (BO) and evolutionary algorithms (EA) are two commonly used search strategies, but they suffer from being computationally expensive, challenging to implement, and exhibiting inefficient exploration ability. In this article, we propose a neural predictor guided EA to enhance the exploration ability of EA for NAS (NPENAS) and design two kinds of neural predictors. The first predictor is a BO acquisition function for which we design a graph-based uncertainty estimation network as the surrogate model. The second predictor is a graph-based neural network that directly predicts the performance of the input neural architecture. The NPENAS using the two neural predictors are denoted as NPENAS-BO and NPENAS-NP, respectively. In addition, we introduce a new random architecture sampling method to overcome the drawbacks of the existing sampling method. Experimental results on five NAS search spaces indicate that NPENAS-BO and NPENAS-NP outperform most existing NAS algorithms, with NPENAS-NP achieving state-of-the-art performance on four of the five search spaces.

Neural Predictor-Based Dynamic Surface Parallel Control for MIMO Uncertain Nonlinear Strict-Feedback Systems

We propose a neural predictor-based dynamic surface parallel control method for a class of uncertain nonlinear systems in this brief. The dynamics of the physical system is in the strict-feedback form and subject to multi-input, multi-output, and uncertain nonlinearities. The parallel control method is developed based on an ACP methodology, including three steps. Firstly, an artificial system to the physical system is developed using an echo state network-based neural predictor structure. Next, a high-order tuner-based computational experiment is developed to achieve online adaptive training of the echo state network. Finally, parallel execution is developed by using a second-order linear tracking differentiator-based dynamic surface control approach. The total closed-loop system can be proved to be input-to-state stable. The effectiveness of the proposed theoretical results is demonstrated by a simulation of trajectory tracking of an autonomous surface vehicle.

NHITS: Neural Hierarchical Interpolation for Time Series Forecasting

Recent progress in neural forecasting accelerated improvements in the performance of large-scale forecasting systems. Yet, long-horizon forecasting remains a very difficult task. Two common challenges afflicting the task are the volatility of the predictions and their computational complexity. We introduce NHITS, a model which addresses both challenges by incorporating novel hierarchical interpolation and multi-rate data sampling techniques. These techniques enable the proposed method to assemble its predictions sequentially, emphasizing components with different frequencies and scales while decomposing the input signal and synthesizing the forecast. We prove that the hierarchical interpolation technique can efficiently approximate arbitrarily long horizons in the presence of smoothness. Additionally, we conduct extensive large-scale dataset experiments from the long-horizon forecasting literature, demonstrating the advantages of our method over the state-of-the-art methods, where NHITS provides an average accuracy improvement of almost 20% over the latest Transformer architectures while reducing the computation time by an order of magnitude (50 times). Our code is available at https://github.com/Nixtla/neuralforecast.

EINNs: Epidemiologically-Informed Neural Networks

We introduce EINNs, a framework crafted for epidemic forecasting that builds upon the theoretical grounds provided by mechanistic models as well as the data-driven expressibility afforded by AI models, and their capabilities to ingest heterogeneous information. Although neural forecasting models have been successful in multiple tasks, predictions well-correlated with epidemic trends and long-term predictions remain open challenges. Epidemiological ODE models contain mechanisms that can guide us in these two tasks; however, they have limited capability of ingesting data sources and modeling composite signals. Thus, we propose to leverage work in physics-informed neural networks to learn latent epidemic dynamics and transfer relevant knowledge to another neural network which ingests multiple data sources and has more appropriate inductive bias. In contrast with previous work, we do not assume the observability of complete dynamics and do not need to numerically solve the ODE equations during training. Our thorough experiments on all US states and HHS regions for COVID-19 and influenza forecasting showcase the clear benefits of our approach in both short-term and long-term forecasting as well as in learning the mechanistic dynamics over other non-trivial alternatives.

Scalable Spatiotemporal Graph Neural Networks

Neural forecasting of spatiotemporal time series drives both research and industrial innovation in several relevant application domains. Graph neural networks (GNNs) are often the core component of the forecasting architecture. However, in most spatiotemporal GNNs, the computational complexity scales up to a quadratic factor with the length of the sequence times the number of links in the graph, hence hindering the application of these models to large graphs and long temporal sequences. While methods to improve scalability have been proposed in the context of static graphs, few research efforts have been devoted to the spatiotemporal case. To fill this gap, we propose a scalable architecture that exploits an efficient encoding of both temporal and spatial dynamics. In particular, we use a randomized recurrent neural network to embed the history of the input time series into high-dimensional state representations encompassing multi-scale temporal dynamics. Such representations are then propagated along the spatial dimension using different powers of the graph adjacency matrix to generate node embeddings characterized by a rich pool of spatiotemporal features. The resulting node embeddings can be efficiently pre-computed in an unsupervised manner, before being fed to a feed-forward decoder that learns to map the multi-scale spatiotemporal representations to predictions. The training procedure can then be parallelized node-wise by sampling the node embeddings without breaking any dependency, thus enabling scalability to large networks. Empirical results on relevant datasets show that our approach achieves results competitive with the state of the art, while dramatically reducing the computational burden.

Co-evolution of neural architectures and features for stock market forecasting: A multi-objective decision perspective

In a multi-objective setting, a portfolio manager’s highly consequential decisions can benefit from assessing alternative forecasting models of stock index movement. The present investigation proposes a new approach to identify a set of non-dominated neural network models for further selection by the decision-maker. A new co-evolution approach is proposed to simultaneously select the features and topology of neural networks (collectively referred to as neural architecture), where the features are viewed from a topological perspective as input neurons. Further, the co-evolution is posed as a multi-criteria problem to evolve sparse and efficacious neural architectures. The well-known dominance and decomposition based multi-objective evolutionary algorithms are augmented with a non-geometric crossover operator to diversify and balance the search for neural architectures across conflicting criteria. Moreover, the co-evolution is augmented to accommodate the data-based implications of distinct market behaviors prior to and during the ongoing COVID-19 pandemic. A detailed comparative evaluation is carried out with the conventional sequential approach of feature selection followed by neural topology design, as well as a scalarized co-evolution approach. The results on three market indices (NASDAQ, NYSE, and S&P500) in pre- and peri-COVID time windows convincingly demonstrate that the proposed co-evolution approach can evolve a set of non-dominated neural forecasting models with better generalization capabilities.

Neural correlates of sleep-induced benefits on traumatic memory processing.

Recent findings indicate that sleep after trauma compared to sleep loss inhibits intrusive memory development, possibly by promoting adequate memory consolidation and integration. However, the underlying neural mechanisms are still unknown. Here, we examined the neural correlates underlying the effects of sleep on traumatic memory development in 110 healthy participants using a trauma film paradigm and an implicit memory task with fMRI recordings in a between-subjects design. To further facilitate memory integration, we used targeted memory reactivation (TMR) to reactivate traumatic memories during sleep. We found that sleep (i.e., nap) compared to wakefulness reduced the number of intrusive traumatic memories for the experimental trauma groups. TMR during sleep only descriptively reduced the intrusions further. On the level of brain activity, increased activity in the anterior and posterior cingulate cortex, retrosplenial cortex and precuneus was found in the experimental trauma group compared to the control group after wakefulness. After sleep, on the other hand, these findings could not be found in the experimental trauma groups compared to the control group. Sleep compared to wakefulness increased activity in the cerebellum, fusiform gyrus, inferior temporal lobe, hippocampus, and amygdala during implicit retrieval of trauma memories in the experimental trauma groups. Activity in the hippocampus and the amygdala predicted subsequent intrusions. Results demonstrate the beneficial behavioral and neural effects of sleep after experimental trauma and provide indications for early neural predictor factors. This study has implications for understanding the important role of sleep for personalized treatment and prevention in posttraumatic stress disorder.

Deep multi-task learning for early warnings of dust events implemented for the Middle East

Events of high dust loading are extreme meteorological phenomena with important climate and health implications. Therefore, early forecasting is critical for mitigating their adverse effects. Dust modeling is a long-standing challenge due to the multiscale nature of the governing meteorological dynamics and the complex coupling between atmospheric particles and the underlying atmospheric flow patterns. While physics-based numerical modeling is commonly being used, we propose a meteorological-based deep multi-task learning approach for forecasting dust events. Our approach consists of forecasting the local PM10 (primary task) measured in situ, and simultaneously to predict the satellite-based regional PM10 (auxiliary task); thus, leveraging valuable information from a correlated task. We use 18 years of regional meteorological data to train a neural forecast model for dust events in Israel. Twenty-four hours before the dust event, the model can detect 76% of the events with even higher predictability of winter and spring events. Further analysis shows that local dynamics drive most misclassified events, meaning that the coherent driving meteorology in the region holds a predictive skill. Further, we use machine-learning interpretability methods to reveal the meteorological patterns the model has learned, thus highlighting the important features that govern dust events in the Middle East, being primarily lower-tropospheric winds, and Aerosol Optical Depth.

Barrier-Certified Distributed Model Predictive Control of Under-Actuated Autonomous Surface Vehicles via Neurodynamic Optimization

This article addresses the distributed formation control of multiple under-actuated autonomous surface vehicles (ASVs) in a receding-horizon setting. The ASVs are subject to physical constraints, in addition to stationary and moving obstacles. A barrier-certified distributed model predictive control method is proposed with the capability of avoiding collision with stationary and moving obstacles and neighboring ASVs. Specifically, a data-driven neural predictor is used to learn unknown functions in ASV kinetics. A nominal distributed receding-horizon position control law is developed based on the learned unknown function to achieve the desired formation within physical constraints. To ensure the safety requirement, a barrier-certified control law is designed based on control barrier functions to generate the signals of optimal surge force and heading angle within the safety constraints. A receding-horizon heading control law is designed based on the data-driven neural predictor to track the desired heading signals. Constrained quadratic programming problems are formulated based on barrier functions for barrier-certified distributed formation control and solved via neurodynamic optimization using one-layer recurrent neural networks. Thus, the proposed control method is able to ensure obstacle avoidance in the formation control of multiple ASVs in the presence of stationary and moving obstacles. Simulation results are elaborated to validate the efficacy of the proposed barrier-certified distributed model predictive control method for ASV formation.

Neural Predictor-Based Dynamic Surface Predictive Control for Power Converters

In this letter, a neural predictor-based dynamic surface predictive control framework, endowed with the merits of adaptive dynamic surface control and finite control-set model predictive control, is proposed where the estimation of neural predictor is incorporated to identify the system dynamics and lumped unknown uncertainties. The key features of the proposal are that, first, the issue of “explosion of complexity” inherent in the classical back-stepping control is avoided, second, the model uncertainties and disturbances are explicitly dealt with, and, third, the tedious determination procedure of weighting factors is removed. These features lead to a much simpler adaptive predictive control solution, which is convenient to implement in applications. Furthermore, a Lyapunov function is constructed, and the stability analysis is given. It demonstrates that all signals in the closed-loop system are uniformly ultimately bounded. Finally, this proposal is experimentally assessed, where the performance evaluation of steady-state and transient-state confirms the availability of the proposed solution for modular multilevel converter.

Data-driven finite control set model predictive speed control of an autonomous surface vehicle subject to fully unknown kinetics and propulsion dynamics

This article investigates the speed control problem of an autonomous surface vehicle (ASV) suffering from fully unknown kinetics and propulsion dynamics. A data-driven finite control set model predictive control (FCS-MPC) strategy is presented for tracking a desired surge velocity. In the learning loop, a data-driven neural predictor is present to learn the unknown input gains, model nonlinearities, and environmental disturbances simultaneously. Filtered data of vehicle states are recorded and used to update the neural network parameters. In the control loop, a pulsewidth-modulation-driven speed controller is developed based on the FCS-MPC technique. The optimal control actions are selected based on the receding horizon optimization with a prediction model. The learning error system of the data-driven neural predictor is proved to be input-to-state stable via Lyapunov analysis. A major feature of the proposed method is that optimal performance can be provided for speed tracking without using any a priori knowledge of model parameters or external disturbances. Simulation results substantiate the effectiveness of the proposed data-driven FCS-MPC strategy for an ASV to track a reference surge velocity.

Deep Learning LSTM Recurrent Neural Network Model for Prediction of Electric Vehicle Charging Demand

The immense growth and penetration of electric vehicles has become a major component of smart transport systems; thereby decreasing the greenhouse gas emissions that pollute the environment. With the increased volumes of electric vehicles (EV) in the past few years, the charging demand of these vehicles has also become an immediate requirement. Due to which, the prediction of the demand of electric vehicle charging is of key importance so that it minimizes the burden on the electric grids and also offers reduced costs of charging. In this research study, an attempt is made to develop a novel deep learning (DL)-based long-short term memory (LSTM) recurrent neural network predictor model to carry out the forecasting of electric vehicle charging demand. The parameters of the new deep long-short term memory (DLSTM) neural predictor model are tuned for its optimal values using the classic arithmetic optimization algorithm (AOA) and the input time series data are decomposed so as to maintain their features using the empirical mode decomposition (EMD). The novel EMD—AOA—DLSTM neural predictor modeled in this study overcomes the vanishing and exploding gradients of basic recurrent neural learning and is tested for its superiority on the EV charging dataset of Georgia Tech, Atlanta, USA. At the time of simulation, the best results of 97.14% prediction accuracy with a mean absolute error of 0.1083 and a root mean square error of 2.0628 × 10−5 are attained. Furthermore, the mean absolute error was evaluated to be 0.1083 and the mean square error pertaining to 4.25516 × 10−10. The results prove the efficacy of the prediction metrics computed with the novel deep learning LSTM neural predictor for the considered dataset in comparison with the previous techniques from existing works.

Sequence-based drug-target affinity prediction using weighted graph neural networks

BackgroundAffinity prediction between molecule and protein is an important step of virtual screening, which is usually called drug-target affinity (DTA) prediction. Its accuracy directly influences the progress of drug development. Sequence-based drug-target affinity prediction can predict the affinity according to protein sequence, which is fast and can be applied to large datasets. However, due to the lack of protein structure information, the accuracy needs to be improved.ResultsThe proposed model which is called WGNN-DTA can be competent in drug-target affinity (DTA) and compound-protein interaction (CPI) prediction tasks. Various experiments are designed to verify the performance of the proposed method in different scenarios, which proves that WGNN-DTA has the advantages of simplicity and high accuracy. Moreover, because it does not need complex steps such as multiple sequence alignment (MSA), it has fast execution speed, and can be suitable for the screening of large databases.ConclusionWe construct protein and molecular graphs through sequence and SMILES that can effectively reflect their structures. To utilize the detail contact information of protein, graph neural network is used to extract features and predict the binding affinity based on the graphs, which is called weighted graph neural networks drug-target affinity predictor (WGNN-DTA). The proposed method has the advantages of simplicity and high accuracy.