Nonlinear Neural Network Research Articles

Multi-task learning using non-linear autoregressive models and recurrent neural networks for tide level forecasting

Tide level forecasting plays an important role in environmental management and development. Current tide level forecasting methods are usually implemented for solving single task problems, that is, a model built based on the tide level data at an individual location is only used to forecast tide level of the same location but is not used for tide forecasting at another location. This study proposes a new method for tide level prediction at multiple locations simultaneously. The method combines nonlinear autoregressive moving average with exogenous inputs (NARMAX) model and recurrent neural networks (RNNs), and incorporates them into a multi-task learning (MTL) framework. Experiments are designed and performed to compare single task learning (STL) and MTL with and without using non-linear autoregressive models. Three different RNN variants, namely, long short-term memory (LSTM), gated recurrent unit (GRU) and bidirectional LSTM (BiLSTM) are employed together with non-linear autoregressive models. A case study on tide level forecasting at many different geographical locations (5 to 11 locations) is conducted. Experimental results demonstrate that the proposed architectures outperform the classical single-task prediction methods.

A novel comparative study of NNAR approach with linear stochastic time series models in predicting tennis player's performance

BackgroundPrediction models have gained immense importance in various fields for decision-making purposes. In the context of tennis, relying solely on the probability of winning a single match may not be sufficient for predicting a player's future performance or ranking. The performance of a tennis player is influenced by the timing of their matches throughout the year, necessitating the incorporation of time as a crucial factor. This study aims to focus on prediction models for performance indicators that can assist both tennis players and sports analysts in forecasting player standings in future matches.MethodologyTo predict player performance, this study employs a dynamic technique that analyzes the structure of performance using both linear and nonlinear time series models. A novel approach has been taken, comparing the performance of the non-linear Neural Network Auto-Regressive (NNAR) model with conventional stochastic linear and nonlinear models such as Auto-Regressive Integrated Moving Average (ARIMA), Exponential Smoothing (ETS), and TBATS (Trigonometric Seasonal Decomposition Time Series).ResultsThe study finds that the NNAR model outperforms all other competing models based on lower values of Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). This superiority in performance metrics suggests that the NNAR model is the most appropriate approach for predicting player performance in tennis. Additionally, the prediction results obtained from the NNAR model demonstrate narrow 95% Confidence Intervals, indicating higher accuracy and reliability in the forecasts.ConclusionIn conclusion, this study highlights the significance of incorporating time as a factor when predicting player performance in tennis. It emphasizes the potential benefits of using the NNAR model for forecasting future player standings in matches. The findings suggest that the NNAR model is a recommended approach compared to conventional models like ARIMA, ETS, and TBATS. By considering time as a crucial factor and employing the NNAR model, both tennis players and sports analysts can make more accurate predictions about player performance.

Deep evolutionary fusion neural network: a new prediction standard for infectious disease incidence rates

BackgroundPreviously, many methods have been used to predict the incidence trends of infectious diseases. There are numerous methods for predicting the incidence trends of infectious diseases, and they have exhibited varying degrees of success. However, there are a lack of prediction benchmarks that integrate linear and nonlinear methods and effectively use internet data. The aim of this paper is to develop a prediction model of the incidence rate of infectious diseases that integrates multiple methods and multisource data, realizing ground-breaking research.ResultsThe infectious disease dataset is from an official release and includes four national and three regional datasets. The Baidu index platform provides internet data. We choose a single model (seasonal autoregressive integrated moving average (SARIMA), nonlinear autoregressive neural network (NAR), and long short-term memory (LSTM)) and a deep evolutionary fusion neural network (DEFNN). The DEFNN is built using the idea of neural evolution and fusion, and the DEFNN + is built using multisource data. We compare the model accuracy on reference group data and validate the model generalizability on external data. (1) The loss of SA-LSTM in the reference group dataset is 0.4919, which is significantly better than that of other single models. (2) The loss values of SA-LSTM on the national and regional external datasets are 0.9666, 1.2437, 0.2472, 0.7239, 1.4026, and 0.6868. (3) When multisource indices are added to the national dataset, the loss of the DEFNN + increases to 0.4212, 0.8218, 1.0331, and 0.8575.ConclusionsWe propose an SA-LSTM optimization model with good accuracy and generalizability based on the concept of multiple methods and multiple data fusion. DEFNN enriches and supplements infectious disease prediction methodologies, can serve as a new benchmark for future infectious disease predictions and provides a reference for the prediction of the incidence rates of various infectious diseases.

Ultra-short-term wind power probabilistic forecasting based on an evolutionary non-crossing multi-output quantile regression deep neural network

Ultra-short-term wind power probabilistic forecasting is of significance for stable power grid operation; however, it is still challenging due to the inherent nonlinearity and uncertainty. Most state-of-the-art methods have focused on achieving quantile prediction using a combination of linear quantile regression and nonlinear complex deep neural networks. Yet, these methods struggle with several dilemmas. Quantile regression deep neural networks require a complete training once for each quantile. The multi-training mode and complex structure of quantile regression deep neural network can lead to extremely high computational complexity. Most of the training of quantile regression deep neural networks are guided by the loss of each quantile, and the weights are adjusted by gradient descent in which the gradient explosion and quantile crossover may be encountered. Therefore, this paper proposes a non-crossing multi-output quantile regression deep neural network optimized by chaotic particle swarm optimization. It designs a multi-output deep neural network to output all quantile estimations simultaneously through one training, effectively solving the structural complexity problem of traditional quantile regression deep neural networks. Since quantile regression produces a non-differentiable loss function which significantly hinders model training, the proposed neural network is trained by chaotic particle swarm optimization. It not only achieves the effect of optimizing all quantile losses simultaneously, but also can significantly alleviate the dilemma of training in traditional neural network weight optimization. In addition, several non-crossing constraints are designed for avoiding quantile crossover. The proposed model is trained and tested on two real-world wind power case studies. The experiment results show that the proposed model shows superiority in performance criteria, training speed, and avoiding quantile crossover.

Power sharing enhancement through a decentralized droop-based control strategy in an islanded microgrid

A decentralized intelligent droop-based control strategy is proposed in this paper for the enhancement of equal active and reactive power sharing in an islanded inverter-based microgrid. Droop control is mostly preferred because it does not need communication facilities for its implementation. However, the presence of different feeder impedances for the different distributed generators (DGs) in a microgrid make reactive power sharing inaccurate. Furthermore, as a result of considerable load changes and different droop characteristics for the DGs, the inverter's output voltage and frequency are altered and these in turn alter the reactive and active power sharing respectively. With these conditions, the generalized droop control (GDC) strategy fails to effectively share load reactive and active power among the DGs. In order to extenuate these challenges, this paper presents a nonlinear autoregressive exogenous neural network droop-based control (NARX-NN DBC) strategy which does not depend on the varying line impedances, droop gains for the different DGs and is less affected by fluctuating loads in the grid. A microgrid, made up of two DGs and a load is modelled in MATLAB/Simulink environment and validation of the proposed control strategy is done through many simulations. Within the simulation runtime, NARX-NN DBC yielded a maximum frequency percentage deviation of 0.46% from the nominal value of 50 Hz whereas GDC yielded 0.62%. Regarding the voltage, NARX-NN DBC gave maximum deviation of 0.026% meanwhile GDC gave 0.079% from the nominal value for 380 V. In addition, during 0.57–0.64 s with load active power demand of 4.6 kW, NARX-NN DBC registered 0.43% power sharing error whereas GDC registered 6.5%. On the other hand, during 0.57–0.64 s, with 5kVAr load power demand, NARX-NN DBC registered 0.2% whereas, GDC registered 2%. These obtained results clearly show that NARX-NN DBC strategy has a better performance compared to GDC strategy with respect to power sharing in an autonomous microgrid.

Stabilization of nonlinear hybrid stochastic time-delay neural networks with Lévy noise using discrete-time feedback control

<p>This paper aims to formulate a class of nonlinear hybrid stochastic time-delay neural networks (STDNNs) with Lévy noise. Specifically, the coefficients of networks grow polynomially instead of linearly, and the time delay of given neural networks is non-differentiable. In many practical situations, nonlinear hybrid STDNNs with Lévy noise are unstable. Hence, this paper uses feedback control based on discrete-time state and mode observations to stabilize the considered nonlinear hybrid STDNNs with Lévy noise. Then, we establish stabilization criteria of $ H_{\infty} $ stability, asymptotic stability, and exponential stability for the controlled nonlinear hybrid STDNNs with Lévy noise. Finally, a numerical example illustrating the usefulness of theoretical results is provided.</p>

Coexistence of Cyclic Sequential Pattern Recognition and Associative Memory in Neural Networks by Attractor Mechanisms.

Neural networks are developed to model the behavior of the brain. One crucial question in this field pertains to when and how a neural network can memorize a given set of patterns. There are two mechanisms to store information: associative memory and sequential pattern recognition. In the case of associative memory, the neural network operates with dynamical attractors that are point attractors, each corresponding to one of the patterns to be stored within the network. In contrast, sequential pattern recognition involves the network memorizing a set of patterns and subsequently retrieving them in a specific order over time. From a dynamical perspective, this corresponds to the presence of a continuous attractor or a cyclic attractor composed of the sequence of patterns stored within the network in a given order. Evidence suggests that the brain is capable of simultaneously performing both associative memory and sequential pattern recognition. Therefore, these types of attractors coexist within the neural network, signifying that some patterns are stored as point attractors, while others are stored as continuous or cyclic attractors. This article investigates the coexistence of cyclic attractors and continuous or point attractors in certain nonlinear neural networks, enabling the simultaneous emergence of various memory mechanisms. By selectively grouping neurons, conditions are established for the existence of cyclic attractors, continuous attractors, and point attractors, respectively. Furthermore, each attractor is explicitly represented, and a competitive dynamic emerges among these coexisting attractors, primarily regulated by adjustments to external inputs.

Validity and Impact of Methods for Collecting Training Data for Myoelectric Prosthetic Control Algorithms.

Intuitive regression control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time regression performance, but accurately labeling intended hand kinematics after hand amputation is challenging. In this study, we quantified the accuracy and precision of labeling hand kinematics using two common training paradigms: 1) mimic training, where participants mimic predetermined motions of a prosthesis, and 2) mirror training, where participants mirror their contralateral intact hand during synchronized bilateral movements. We first explored this question in healthy non-amputee individuals where the ground-truth kinematics could be readily determined using motion capture. Kinematic data showed that mimic training fails to account for biomechanical coupling and temporal changes in hand posture. Additionally, mirror training exhibited significantly higher accuracy and precision in labeling hand kinematics. These findings suggest that the mirror training approach generates a more faithful, albeit more complex, dataset. Accordingly, mirror training resulted in significantly better offline regression performance when using a large amount of training data and a non-linear neural network. Next, we explored these different training paradigms online, with a cohort of unilateral transradial amputees actively controlling a prosthesis in real-time to complete a functional task. Overall, we found that mirror training resulted in significantly faster task completion speeds and similar subjective workload. These results demonstrate that mirror training can potentially provide more dexterous control through the utilization of task-specific, user-selected training data. Consequently, these findings serve as a valuable guide for the next generation of myoelectric and neuroprostheses leveraging machine learning to provide more dexterous and intuitive control.

Robust Prediction of Healthcare Inflation Rate With Statistical and AI Methods in Iran

The expected healthcare (HC) inflation rate (IR) (HCIR) is an important variable for all economic agents within HC systems. In recent years, during the COVID‐19 pandemic, Iran has experienced a high HCIR in its health system. In this context, a robust approximation of HCIR will be a helpful tool for health authorities and other decision makers. Using monthly time series data of HCIR in Iran, we developed various forecasting techniques based on classical smoothing methods, decomposition ETS (error, trend, and seasonality) approaches, autoregressive (AR) integrated moving average (ARIMA), seasonal ARIMA (SARIMA), and a multilayer nonlinear AR artificial neural network (NARANN) with several training algorithms including Bayesian regularization (BR), Levenberg–Marquardt (LM), scaled conjugate gradient (SCG), Broyden–Fletcher–Goldfarb–Shanno (BFGS) quasi‐Newton, conjugate gradient with Powell–Beale restarts (CGB), conjugate gradient with Fletcher–Reeves updates (CGF), and resilient propagation (RPROP) algorithms. Initially, based upon various criteria and possible combinations, we selected the superior model for each method separately. After that, the best model in each category is involved in 6‐ and 12‐multi‐step‐ahead prediction. In this stage, several error criteria are calculated. According to our findings, in a six‐step forecasting window, the Holt–Winters with a multiplicative seasonal pattern and SARIMA showed less bias, though compared to other alternatives like NARANN‐lm/br, the difference was relatively small. In the next process, by doubling the forecasting window, it is observed that artificial neural network (ANN) (i.e., Bayesian NARANN) strictly outperformed other models. As a result, in shorter steps, the Holt–Winters method can provide a better prediction, while in longer windows, Bayesian NARANN can be implemented vigorously for the prediction. Finally, we used 10 models to predict the future trend of HCIR in Iran till the end of July 2024.

Wetland Mapping by Fusion of Deep learning and Ensemble Model for Enhancing Prediction Outcomes

Constraints perceived in different socioeconomic situations reinforce land use patterns and land cover (LULC) at different levels. However, the statistical information regarding the LULC variations encounters enormous significance for the execution and modelling of appropriate environmental variations and resource management with the available remote sensed data from diverse satellite images and advanced computing technologies; information is generally retrieved from the image classification approaches. However, a broader quantitative analysis of various classification approaches is crucial to choosing an effectual classifier model to acquire appropriate land use regions. We concentrate on the Karavetti region and its related fields in this study. We use a Non-Linear Recurrent Convolutional Neural Network (NLR-CNN) to analyze the data statistically. Well-known techniques such as Support Vector Machine (SVM), Random Forest (RF), and Decision Tree (DT), among others are used to evaluate the model performance. High-resolution images and the data points supplied are also used to assess the accuracy of the categorization and prediction. A confusion matrix is generated where the land cover regions show superior classification accuracy with the fusion model. Also, the NDVI facts and additional metrics like loss, error rate and kappa coefficients are analyzed. Therefore, the outcomes show that the anticipated is considered more robust with better performance to enhance the classification accuracy with the specific land cover regions.

Adaptive Intermittent Pinning Control for Synchronization of Delayed Nonlinear Memristive Neural Networks With Reaction-Diffusion Items.

In this article, the global exponential synchronization problem is investigated for a class of delayed nonlinear memristive neural networks (MNNs) with reaction-diffusion items. First, using the Green formula, Lyapunov theory, and proposing a new fuzzy adaptive pinning control scheme, some novel algebraic criteria are obtained to ensure the exponential synchronization of the concerned networks. Furthermore, the corresponding control gains can be promptly adjusted based on the current states of partial nodes of the networks. Besides, a fuzzy adaptive aperiodically intermittent pinning control law is also designed to synchronize the fuzzy MNNs (FMNNs). The controller with intermittent mechanism can obtain appropriate rest time and save energy consumption. Finally, some numerical examples are provided to confirm the effectiveness of the results in this article.

Diversity-induced trivialization and resilience of neural dynamics.

Heterogeneity is omnipresent across all living systems. Diversity enriches the dynamical repertoire of these systems but remains challenging to reconcile with their manifest robustness and dynamical persistence over time, a fundamental feature called resilience. To better understand the mechanism underlying resilience in neural circuits, we considered a nonlinear network model, extracting the relationship between excitability heterogeneity and resilience. To measure resilience, we quantified the number of stationary states of this network, and how they are affected by various control parameters. We analyzed both analytically and numerically gradient and non-gradient systems modeled as non-linear sparse neural networks evolving over long time scales. Our analysis shows that neuronal heterogeneity quenches the number of stationary states while decreasing the susceptibility to bifurcations: a phenomenon known as trivialization. Heterogeneity was found to implement a homeostatic control mechanism enhancing network resilience to changes in network size and connection probability by quenching the system's dynamic volatility.

Bioengineering, waste processing and fermentation process control for biogas production

The article discusses the analysis of the state of control of the processes of biogas production from animal waste by methane digestion. The article discusses the problems of synthesis of automated control systems of biotechnological processes under conditions of information uncertainty. The analysis of the current state of control of fermentation stage processes shows that insufficient attention is paid to the problem of synthesis of second-tier banks under conditions of information uncertainty. Construction of mathematical modeling of biosynthetic processes is a kinetic model, where experimental and analytical methods are used due to the difficulty of identifying patterns in microbiological processes. The article discusses the application of methods and algorithms for intellectualization of problem solving in ACS for the synthesis of complex biotechnological objects in conditions of lack of information, that they should be attributed to priority tasks. The results of research on the application of a neuro-fuzzy system for controlling fermentation processes under conditions of uncertainty and multimode of processes, as well as a forecasting algorithm using nonlinear sets and neural networks are presented.

Huizhou resident population, Guangdong resident population and elderly population forecast based on the NAR neural network Markov model

<abstract> <p>We propose a nonlinear auto regressive neural network Markov model (NARMKM) to predict the annual Huizhou resident population, Guangdong resident population and elderly population in China, and improve the accuracy of population forecasting. The new model is built upon the traditional neural network model and utilized matrix perturbation theory to study the natural and response characteristics of a system when the structural parameters change slightly. The delay order and hidden layer number of neurons has a greater effect the prediction result of NAR neural network model. Therefore, we make full use of prior information to constrain and test when making predictions. We choose reasonable parameter settings to obtain more reliable prediction results. Three experiments are conducted to validate the high prediction accuracy of the NARMKM model, with mean absolute percentage error (MAPE), root mean square error (RMSE), <italic>STD</italic> and <italic>R</italic><sup>2</sup>. These results demonstrate the superior fitting performance of the NARMKM model when compared to other six competitive models, including GM (1, 1), ARIMA, Multiple regression, FGM (1, 1), FANGBM and NAR. Our study provides a scientific basis and technical references for further research in the finance as well as population fields.</p> </abstract>

A Transistor Operations Model for Deep Learning Energy Consumption Scaling Law

Deep Neural Networks (DNN) has transformed the automation of a wide range of industries and finds increasing ubiquity in society. The high complexity of DNN models and its widespread adoption has led to global energy consumption doubling every 3-4 months. Current energy consumption measures largely monitor system wide consumption or make linear assumptions of DNN models. The former approach captures other unrelated energy consumption anomalies, whilst the latter does not accurately reflect nonlinear computations. In this paper, we are the first to develop a bottom-up Transistor Operations (TOs) approach to expose the role of non-linear activation functions and neural network structure. As there will be inevitable energy measurement errors at the core level, we statistically model the energy scaling laws as opposed to absolute consumption values. We offer models for both feedforward DNNs and convolution neural networks (CNNs) on a variety of data sets and hardware configurations - achieving a 93.6% - 99.5% precision. This outperforms existing FLOPs-based methods and our TOs method can be further extended to other DNN models.

A novel activation function based on DNA enzyme-free hybridization reaction and its implementation on nonlinear molecular learning systems.

With the advent of the post-Moore's Law era, the development of traditional silicon-based computers has reached its limit, and there is an urgent need to develop new computing technologies to meet the needs of science, technology, and daily life. Due to its super-strong parallel computing capability and outstanding data storage capacity, DNA computing has become an important branch and hot research topic of new computer technology. DNA enzyme-free hybridization reaction technology is widely used in DNA computing, showing excellent performance in computing power and information processing. Studies have shown that DNA molecules not only have the computing function of electronic devices, but also exhibit certain human brain-like functions. In the field of artificial intelligence, activation functions play an important role as they enable artificial intelligence systems to fit and predict non-linear and complex variable relationships. Due to the difficulty of implementing activation functions in DNA computing, DNA circuits cannot easily achieve all the functions of artificial intelligence. DNA circuits need to rely on electronic computers to complete the training and learning process. Based on the parallel computing characteristics of DNA computing and the kinetic features of DNA molecule displacement reactions, this paper proposes a new activation function. This activation function can not only be easily implemented by DNA enzyme-free hybridization reaction reactions, but also has good nesting properties in DNA circuits, and can be cascaded with other DNA reactions to form a complete DNA circuit. This paper not only provides the mathematical analysis of the proposed activation function, but also provides a detailed analysis of its kinetic features. The activation function is then nested into a nonlinear neural network for DNA computing. This system is capable of fitting and predicting a certain nonlinear function.

Multi-step ahead ozone level forecasting using a component-based technique: A case study in Lima, Peru

<abstract><p>The rise in global ozone levels over the last few decades has harmed human health. This problem exists in several cities throughout South America due to dangerous levels of particulate matter in the air, particularly during the winter season, making it a public health issue. Lima, Peru, is one of the ten cities in South America with the worst levels of air pollution. Thus, efficient and precise modeling and forecasting are critical for ozone concentrations in Lima. The focus is on developing precise forecasting models to anticipate ozone concentrations, providing timely information for adequate public health protection and environmental management. This work used hourly O$ _{3} $ data in metropolitan areas for multi-step-ahead (one-, two-, three-, and seven-day-ahead) O$ _{3} $ forecasts. A multiple linear regression model was used to represent the deterministic portion, and four-time series models, autoregressive, nonparametric autoregressive, autoregressive moving average, and nonlinear neural network autoregressive, were used to describe the stochastic component. The various horizon out-of-sample forecast results for the considered data suggest that the proposed component-based forecasting technique gives a highly consistent, accurate, and efficient gain. This may be expanded to other districts of Lima, different regions of Peru, and even the global level to assess the efficacy of the proposed component-based modeling and forecasting approach. Finally, no analysis has been undertaken using a component-based estimation to forecast ozone concentrations in Lima in a multi-step-ahead manner.</p></abstract>

Photonic neural networks and optics-informed deep learning fundamentals

The recent explosive compute growth, mainly fueled by the boost of artificial intelligence (AI) and deep neural networks (DNNs), is currently instigating the demand for a novel computing paradigm that can overcome the insurmountable barriers imposed by conventional electronic computing architectures. Photonic neural networks (PNNs) implemented on silicon integration platforms stand out as a promising candidate to endow neural network (NN) hardware, offering the potential for energy efficient and ultra-fast computations through the utilization of the unique primitives of photonics, i.e., energy efficiency, THz bandwidth, and low-latency. Thus far, several demonstrations have revealed the huge potential of PNNs in performing both linear and non-linear NN operations at unparalleled speed and energy consumption metrics. Transforming this potential into a tangible reality for deep learning (DL) applications requires, however, a deep understanding of the basic PNN principles, requirements, and challenges across all constituent architectural, technological, and training aspects. In this Tutorial, we, initially, review the principles of DNNs along with their fundamental building blocks, analyzing also the key mathematical operations needed for their computation in photonic hardware. Then, we investigate, through an intuitive mathematical analysis, the interdependence of bit precision and energy efficiency in analog photonic circuitry, discussing the opportunities and challenges of PNNs. Followingly, a performance overview of PNN architectures, weight technologies, and activation functions is presented, summarizing their impact in speed, scalability, and power consumption. Finally, we provide a holistic overview of the optics-informed NN training framework that incorporates the physical properties of photonic building blocks into the training process in order to improve the NN classification accuracy and effectively elevate neuromorphic photonic hardware into high-performance DL computational settings.

Coordinated adaptive cruise control with integration of driving behaviors based on prediction for surrounding vehicles status

The adaptive cruise is an important vehicle control process for unmanned vehicles. This paper studies coordinated adaptive cruise control (ACC) of energy-saving in an intelligent networked environment, aiming to integrate control for common driving behaviors (cruising, following, lane-changing) by hierarchical control. The decision-making layer for integrating three driving behaviors is designed to improve the economy and passing time. Respectively, cruising strategy is optimized by considering efficient working area of motor; following strategy is constructed by dynamically adjusting the following distance with the state changes of the surrounding vehicles; lane-changing strategy is developed by considering the motor working efficiency of target speed. To improve the predictability of decision-making, non-linear autoregressive neural network (NARX) is introduced to predict the surrounding vehicle state. Furthermore, receiving the control reference from the decision layer, the controller is designed to realizes the tracking control of vehicle speed and following distance based on model predictive control (MPC), which is co-optimized for tracking, economy, and comfort. Eventually, the effectiveness of the coordinated ACC control is verified based on the hardware in the loop (HiL) test. The experiment results show that the vehicles with coordinated ACC can shorten the passing time and reduce the driving energy consumption to a certain extent, especially in the medium and low speed conditions.

Nonlinear autoregression-based non-destructive evaluation approach for railway tracks using an ultrasonic fiber bragg grating array

The safe operation of railway systems relies on rapid and accurate non-destructive damage detection and evaluation techniques to help rule out potential threats brought by rail track damages. In this paper, an innovative fiber Bragg grating (FBG) array-based sensing system is developed for rail defect inspection by integrating the ultrasonic testing technology, and a nonlinear autoregressive neural network with exogenous inputs (NARX) is proposed for rail damage detection based on measured ultrasonic signals. The developed sensing system can realize the high-speed and multiplexed ultrasonic signals demodulation. The NARX is proposed to process the ultrasonic signals obtained by the developed sensing system, and a specified probabilistic damage sensitive feature (DSF) is tailored to indicate rail health status, derived from the probability density function (PDF) of NARX prediction errors. To verify the proposed sensing system and damage identification algorithm, experimental studies considering four rail conditions (i.e., intact, cracked, bump, and welded conditions) are conducted in this study. First, baseline NARX models are priorly trained using ultrasonic signals acquired on an intact rail segment; then, DSFs corresponding to each condition are calculated enabling effective rail damage diagnosis. To optimize the training parameters of NARX models, three different training functions are utilized to select the most appropriate parameters that are sensitive to rail damages. Last, large-scale testing is conducted, further proving the applicability and reliability of the proposed rail damage evaluation approach. Through this study, it is suggested that integrating high-performance sensing technologies with effective data processing tools would maximumly enable efficient and robust non-destructive testing, especially on important infrastructures such as rails.