Nonlinear Neural Network Research Articles

Explainable and Reduced-Feature Machine learning models for shape and drag prediction of a freely moving drop in the sub-critical Weber number regime

Accurately predicting the shape and drag of a moving drop is crucial in many spray applications. However, due to the complex interaction between the drag force and drop shape deformation, accurate prediction of drop shape and drag from Computational Fluid Dynamics (CFD) simulation requires large computational resources and time. A novel data-driven approach using NARXNN (Non-linear Auto Regressive eXogenous input Neural Network) is proposed in this study, which recurrently predicts the drop shape and drag for a given Weber number (We) and Reynolds number (Re), in the sub-critical We regime where there is no drop breakup. The average error in radius and drag coefficient for the test cases, as low as 0.36% and 0.49%, respectively, is achieved. Most importantly, SHAP (SHapley Additive exPlanations) analysis is performed to identify important features for the predictions, hence enabling the explainability of the physics involved in the predictions of machine learning (ML) models for drop deformation and the interaction between the drop and gas flows. Global interpretation of SHAP values identifies We and Re as the most important features to predict the drop shape and drag. Based on the SHAP analysis, several reduced-feature models are developed. All the reduced-feature models can predict drop shape and drag coefficient within 4% average error in radius and drag coefficient for any We and Re combinations within the prescribed parameter space. The ML models developed in this study demonstrate a tremendous computational advantage over the traditional CFD simulations. The drop shape and drag coefficient evolution can be predicted within seconds for a single case, whereas the CFD simulation takes several days to a week on a single processor.

Heterogeneous Hierarchical Fusion Network for Multimodal Sentiment Analysis in Real-World Environments

Multimodal sentiment analysis models can determine users’ sentiments by utilizing rich information from various sources (e.g., textual, visual, and audio). However, there are two key challenges when deploying the model in real-world environments: (1) the limitations of relying on the performance of automatic speech recognition (ASR) models can lead to errors in recognizing sentiment words, which may mislead the sentiment analysis of the textual modality, and (2) variations in information density across modalities complicate the development of a high-quality fusion framework. To address these challenges, this paper proposes a novel Multimodal Sentiment Word Optimization Module and a heterogeneous hierarchical fusion (MSWOHHF) framework. Specifically, the proposed Multimodal Sentiment Word Optimization Module optimizes the sentiment words extracted from the textual modality by the ASR model, thereby reducing sentiment word recognition errors. In the multimodal fusion phase, a heterogeneous hierarchical fusion network architecture is introduced, which first utilizes a Transformer Aggregation Module to fuse the visual and audio modalities, enhancing the high-level semantic features of each modality. A Cross-Attention Fusion Module then integrates the textual modality with the audiovisual fusion. Next, a Feature-Based Attention Fusion Module is proposed that enables fusion by dynamically tuning the weights of both the combined and unimodal representations. It then predicts sentiment polarity using a nonlinear neural network. Finally, the experimental results on the MOSI-SpeechBrain, MOSI-IBM, and MOSI-iFlytek datasets show that the MSWOHHF outperforms several baselines, demonstrating better performance.

Hybrid wind speed optimization forecasting system based on linear and nonlinear deep neural network structure and data preprocessing fusion

Wind speed time series forecasting has been widely used in wind power generation. However, the nonlinear and non-stationary characteristics of wind speed make accurate wind speed forecasting a difficult task. In recent years, the rapid development of artificial intelligence and machine learning technology provides a new solution to the problem of wind speed forecasting. Combining the advance of artificial intelligence and data analysis strategy, this paper proposes a wind speed forecasting system based on multi-model fusion and integrated learning. Considering the differences in data observation and training principles of various algorithms and exploiting the advantages of each model, a wind speed forecasting system with multiple machine learning algorithms embedded in integrated learning is constructed, which includes three modules: data preprocessing, optimization and forecasting. The data preprocessing module can conduct quantitative analysis through input data decomposition and feature extraction, and the combination of multi-objective intelligent optimization algorithm and combined forecasting method can effectively forecast the wind speed time series. The validity of the algorithm is verified using the data of Shandong wind farm in China. The forecasting results show that compared with the traditional single model forecasting, the proposed integrated wind speed forecasting system based on the fusion of multiple models has higher forecasting accuracy.

Uncovering factors predicting BRAF mutation in cutaneous melanoma

Abstract Introduction/Objective BRAF mutations are considered to be the most common genetic alteration in cutaneous melanoma (CM). Patient management relies on assessing molecular biomarkers enabling the personalization of patients’ treatment. The goal of this study was to develop the tools for predicting BRAF mutation using routine clinical and histological features. Methods/Case Report A total of 2041 CM cases were enrolled in the study to identify factors associated with BRAF mutation. Variables included sex, age, primary location, stage, histological type, ulceration, mitosis, Clark level, Breslow thickness, lymphocytes, lymphovascular invasion (LVI), perineural invasion (PNI), regression, microsatellite, association with nevus. Results (if a Case Study enter NA) The Ukrainian population demonstrated a high rate of BRAF mutation in CM, associated with the younger age and location at non-sun-expose skin. This study also revealed gender-specific differences in CM anatomic distribution and BRAF mutation subtype prevalence. By using the genetic selection method, the minimal set of variables related to BRAF mutations was defined and included age, primary tumor location, histological type, ulceration, LVI, and association with nevus. To encounter non- linear links, the non-linear neural network modeling was applied. For this aim multilayer perceptron (MLP) with one hidden layer was used. The hidden layer architecture included 4 neurons with a logistic activating function. The AUROCMLP6 of the model was 0.79 (0.74 – 0.84). When applying the optimal threshold, the following characteristics of the model were reached: sensitivity - 89.4% (84.5 – 93.1%); specificity - 50.7% (42.2% – 59.1%); positive predictive value (PPV) - 73.1% (69.6% – 76.3%) and NPV - 76.0% (67.6% – 82.8%) respectively. The developed MLP model allows the prediction of BRAF status in CM, facilitating decisions concerning further patient management. Conclusion The developed MLP model, relying on 6 variables analysis allows the prediction of BRAF status in melanoma, facilitating decisions concerning further patient management.

Synchronization of nonlinear neural networks with hybrid couplings and uncertain time-varying perturbations: A novel distributed-delay impulsive comparison principle

This paper investigates the synchronization of nonlinear drive-response neural networks subject to uncertain time-varying perturbations, non-delayed coupling, and distributed delay coupling. To address the influence of distributed and discrete delays on the system, we establish a novel impulsive comparison principle, extending the Halanay inequality. By leveraging Lyapunov stability theory, we derive sufficient conditions for the exponential synchronization of the neural networks using a delayed impulsive controller with historical status information. This approach relaxes the conventional constraint that impulsive delays must be smaller than impulsive intervals, thereby generalizing existing synchronization results for distributed delay networks. Numerical simulations for chaotic neural networks validate the theoretical results and demonstrate the sensitivity of the control gain matrix.

Adaptive robust fault-tolerant control of spinning tether system for space debris removal

The Spinning Tether System (STS) is currently recognized as one of the most effective and ideal platforms for removing space debris. It offers several advantages, including a non-contact mechanism, rapid deorbit capabilities, stable spinning, and a short preparation time for subsequent missions. The system primarily depends on the thrusters of the main spacecraft and the capture device for its spin-up process. Although modern technology and quality management methods strive to minimize thruster failures, the capture of debris significantly affects the STS. This impact can induce tether oscillations and cause actuator failures, thereby destabilizing the spin-up process. Given these challenges, this paper explores the design of fault-tolerant control strategies to enhance system reliability during the post-capture spin-up phase. Initially, a dynamic model using quaternions as generalized coordinates was established based on the constrained Lagrange equation. This model facilitated a detailed analysis of potential thruster failure modes. Subsequently, the optimal trajectory for the spin-up process was planned using the pseudo-spectral method, ensuring efficiency and safety in control execution. To address the identified vulnerabilities, the study introduces an adaptive robust fault-tolerant controller. This controller, based on a BLF and integrated with a nonlinear neural network disturbance observer, is designed to operate effectively under fault conditions. Numerical simulations were conducted to validate the performance of the controller, demonstrating its capability to maintain stable control of the system, both after detecting a fault signal and following an actuator failure.

Renewable generation forecasting with DNN-based ensemble method in electricity scheduling

The generation of renewable energy encounters numerous obstacles, chiefly the unpredictability of renewable sources. When new energy generation prediction is lower than expected, it needs to be supplemented by other energy sources, which may lead to instability in the power grid if the deviation is large. When the prediction of new energy generation exceeds expectations, it will lead to energy waste. To address these issues, this paper proposes a Deep Neural Network-based fusion framework, which can improve the prediction accuracy of new energy and achieve a low-carbon, economical, and stable power grid. Within this structure, feature engineering is conducted initially. Subsequently, a combination of traditional tree algorithms like the Gradient Boosting Decision Tree, linear approaches such as the Least Squares Method, and nonlinear neural networks, for instance, Recurrent Neural Networks, are employed for individual model regression purposes. In the final step, both the original time-series data and the outcomes from the individual models are integrated into a deep neural network to derive the ultimate forecasting outcomes. By using our method, the electricity cost has been reduced by 26.5% and the carbon emissions have been decreased by 14.2%. Experiments have been carried out using actual community data, confirming the effectiveness of the proposed approach. The findings indicate that the integration of DNN with traditional and modern machine learning techniques can significantly improve the forecasting of renewable energy generation. This advancement contributes to the creation of a more sustainable, economical, and stable power grid.

Data-driven soliton solution implementation based on nonlinear adaptive physics-informed neural networks

Data-driven soliton solution implementation based on nonlinear adaptive physics-informed neural networks

Estimation of reservoir properties using pre-stack seismic inversion and neural network in mature oil field, Upper Assam basin, India

The mature oil fields require comprehensive characterization for enhanced hydrocarbon production, and subsequently demands estimation of reservoir properties. The key properties viz. volume of clay, effective-porosity, hydrocarbon-saturation has been evaluated for an aging Oligocene reservoir of Upper Assam basin, located in northeastern India from seismic and well log data. Elastic properties (acoustic and shear impedance) and density are derived from pre-stack inversion of 3D seismic data. These elastic properties are analyzed for their sensitivity for discrimination of lithology and fluid-content, and many derived attributes are computed from elastic properties. These attributes are assessed for their predictability to predict the target reservoir properties using multi-attribute analysis. For each of the target property neural network is trained with the most predictable attributes, and multi-dimensional, non-linear neural network models are created using multilayered feed forward neural network (MLFN), followed by Probabilistic neural network (PNN). The specific neural network models for each target property are employed for quantitative estimate of volume of clay, effective-porosity, hydrocarbon-saturation in inter-well regions. The estimated properties leverage the identification of untapped oil reserves and provide promising opportunity for enhanced production through drilling of infill wells.

Thermal coal futures trading volume predictions through the neural network

Purpose Predicting commodity futures trading volumes represents an important matter to policymakers and a wide spectrum of market participants. The purpose of this study is to concentrate on the energy sector and explore the trading volume prediction issue for the thermal coal futures traded in Zhengzhou Commodity Exchange in China with daily data spanning January 2016–December 2020. Design/methodology/approach The nonlinear autoregressive neural network is adopted for this purpose and prediction performance is examined based upon a variety of settings over algorithms for model estimations, numbers of hidden neurons and delays and ratios for splitting the trading volume series into training, validation and testing phases. Findings A relatively simple model setting is arrived at that leads to predictions of good accuracy and stabilities and maintains small prediction errors up to the 99.273th quantile of the observed trading volume. Originality/value The results could, on one hand, serve as standalone technical trading volume predictions. They could, on the other hand, be combined with different (fundamental) prediction results for forming perspectives of trading trends and carrying out policy analysis.

Short-term PM2.5 forecasting using a unique ensemble technique for proactive environmental management initiatives

Particulate matter with a diameter of 2.5 microns or less (PM2.5) is a significant type of air pollution that affects human health due to its ability to persist in the atmosphere and penetrate the respiratory system. Accurate forecasting of particulate matter is crucial for the healthcare sector of any country. To achieve this, in the current work, a new time series ensemble approach is proposed based on various linear (autoregressive, simple exponential smoothing, autoregressive moving average, and theta) and nonlinear (nonparametric autoregressive and neural network autoregressive) models. Three ensemble models are also developed, each employing distinct weighting strategies: equal distribution of weight among all single models (ESME), weight assignment based on training average accuracy errors (ESMT), and weight assignment based on validation mean accuracy measures (ESMV). This technique was applied to daily PM2.5 concentration data from 1 January 2019, to 31 May 2023, in Pakistan’s main cities, including Lahore, Karachi, Peshawar, and Islamabad, to forecast short-term PM2.5 concentrations. When compared to other models, the best ensemble model (ESMV) demonstrated mean errors ranging from 3.60% to 25.79% in Islamabad, 0.81%–13.52% in Lahore, 1.08%–7.06% in Karachi, and 1.09%–12.11% in Peshawar. These results indicate that the proposed ensemble approach is more efficient and accurate for short-term PM2.5 forecasting than existing models. Furthermore, using the best ensemble model, a forecast was made for the next 15 days (June 1 to 15 June 2023). The forecast showed that in Lahore, the highest PM2.5 value (236.00 μg/m3) was observed on 8 June 2023. Other days also displayed higher and poor air quality throughout the 15 days. Conversely, Karachi experienced moderate PM2.5 concentration levels between 50 μg/m3 and 80 μg/m3. In Peshawar, the PM2.5 concentration levels were consistently unhealthy, with the highest peak (153.00 μg/m3) observed on 9 June 2023. This forecasting experience can assist environmental monitoring organizations in implementing cost-effective planning to minimize air pollution.

A Novel Model for Prediction of Flashover 150kV Polluted Insulator Based on Nonlinear Autoregressive External Input Neural Network

A Novel Model for Prediction of Flashover 150kV Polluted Insulator Based on Nonlinear Autoregressive External Input Neural Network

Stacking integrated learning-based inverse design of four-nanopore high-Q all-dielectric metasurface

In this paper, we establish a four-nanophole hollowed out all-dielectric metasurface structure as the object of reverse design, which can realize many types of high Q-value Fano resonance effects through dynamic tuning of a single structure, with Q value up to 4401. We used the time domain finite difference method for data collection, and then select appropriate models according to the characteristics of the data from the perspective of the data itself. We select in small sample prediction excellent support vector regression (SVR), gradient promotion decision tree (GBDT), prediction generalization performance and stability of high random forest (RF) as a base learners, with good nonlinear fitting ability BP neural network (BPNN) yuan learners, establish a fusion model based on Stacking integrated learning strategy. Based on this fusion model, the multi-parameter all-dielectric metasurface structure is reverse designed instead. The results show that the proposed method has high prediction accuracy with an absolute average error of only 0.0043, and excellent average accuracy of 82.5%, 77.9% and 14% compared with the combined model GBDT-SVM-BP, GBDT-RF-BP, and RF-SVM-BP. This study provides a new perspective on the reverse design of all-dielectric metasurface structures.

Complete Controllability of Nonlinear Neural Network Control Systems

Complete Controllability of Nonlinear Neural Network Control Systems

Degradation assessment of wind turbine based on additional load measurements

In recent years, there has been an increasing focus on the degradation assessment of wind turbines using supervisory control and data acquisition (SCADA) data, establishing condition monitoring as the preferred approach. However, SCADA data primarily focuses on general condition variables, such as temperature, while overlooking critical metrics like load factors for assessing the mechanical properties of wind turbines. The load parameters can be utilized to assess the mechanical stresses and strains experienced by wind turbine components, thereby offering more precise information on degradation. To address this gap, we explore the feasibility of implementing additional load monitoring for the degradation assessment of wind turbines. Specifically, we design a cost-effective load sensor system for collecting load data from wind turbines. On this basis, a novel degradation assessment method that incorporates additional load data and a nonlinear dynamic state-space neural network model is proposed to extract degradation information from wind turbines. Then, we introduce a dynamic degradation index, derived through trend analysis, which effectively captures and quantifies the degradation levels of wind turbines over time. Finally, a case study using a dataset collected from a real wind farm is provided to validate the proposed method. The results demonstrate a significant enhancement in degradation assessment. By incorporating additional load data, the proposed method exhibits heightened responsiveness to degradation levels and more effectively captures the progression of degradation trends.

Predictive Models for Aggregate Available Capacity Prediction in Vehicle-to-Grid Applications

The integration of vehicle-to-grid (V2G) technology into smart energy management systems represents a significant advancement in the field of energy suppliers for Industry 4.0. V2G systems enable a bidirectional flow of energy between electric vehicles and the power grid and can provide ancillary services to the grid, such as peak shaving, load balancing, and emergency power supply during power outages, grid faults, or periods of high demand. In this context, reliable prediction of the availability of V2G as an energy source in the grid is fundamental in order to optimize both grid stability and economic returns. This requires both an accurate modeling framework that includes the integration and pre-processing of readily accessible data and a prediction phase over different time horizons for the provision of different time-scale ancillary services. In this research, we propose and compare two data-driven predictive modeling approaches to demonstrate their suitability for dealing with quasi-periodic time series, including those dealing with mobility data, meteorological and calendrical information, and renewable energy generation. These approaches utilize publicly available vehicle tracking data within the floating car data paradigm, information about meteorological conditions, and fuzzy weekend and holiday information to predict the available aggregate capacity with high precision over different time horizons. Two data-driven predictive modeling approaches are then applied to the selected data, and the performance is compared. The first approach is Hankel dynamic mode decomposition with control (HDMDc), a linear state-space representation technique, and the second is long short-term memory (LSTM), a deep learning method based on recurrent nonlinear neural networks. In particular, HDMDc performs well on predictions up to a time horizon of 4 h, demonstrating its effectiveness in capturing global dynamics over an entire year of data, including weekends, holidays, and different meteorological conditions. This capability, along with its state-space representation, enables the extraction of relationships among exogenous inputs and target variables. Consequently, HDMDc is applicable to V2G integration in complex environments such as smart grids, which include various energy suppliers, renewable energy sources, buildings, and mobility data.

Improving carbon flux estimation in tea plantation ecosystems: A machine learning ensemble approach

Tea plant (Camellia sinensis) is a major global crop consumed as a drink after water. Quantifying carbon flux, specifically the net ecosystem exchange (NEE), in tea plantations is essential for determining carbon sequestration and ecosystem carbon balance. The Eddy covariance (EC) system is widely used for continuous monitoring of carbon flux but high costs associated with installation and maintenance limit its widespread adoption. In addition, EC flux data is often discarded due to malfunction of instruments caused by adverse weather conditions. Therefore, additional approaches for estimating NEE are necessary to overcome these challenges and ensure accurate NEE measurement. For this purpose, three standalone tree-based machine learning (ML) models were used for NEE estimation using EC flux data collected from tea ecosystem located in subtropical region (Danyang county of Zhenjiang city) of China. To address the accuracy limitations inherent in standalone ML models, the ensemble mechanism based on voting regressor method was proposed. In addition, k-fold cross-validation based on early stopping process was also used to enhance the performance of standalone ML models. Based on visual plots (scatter diagram, heatMap, Taylor diagram) and performing indices (root-mean-square error (RMSE), determination coefficient (R2), mean absolute error (MAE), mean absolute percentage error (MAPE), Nash–Sutcliffe efficiency (NSE), correlation coefficient (r), Kling Gupta Efficiency (KGE) and index of agreement (d)), the findings indicated that non-linear ensemble-generalized regression neural network (NLE-GRNN) significantly improved standalone ML model's results. In current study, the highest NSE, r and d in case of standalone ML model (DT) achieved 0.49, 0.73 and 0.75 respectively while our proposed NLE-GRNN model improved 48 % in NSE value (NSE = 0.97), 25 % in r value (r = 0.98) and 24 % in d value (d = 0.99). Likewise, NLE-GRNN significantly reduce errors (MAE, MAPE and RMSE) and provides NEE estimate closet to the observed value. The impact of climatic variables on NEE using shapley additive explanations (SHAP) analysis revealed that Rg (solar radiation) and Tair (air temperature) were the prime factors controlling NEE variation in the tea ecosystem. Considering the high accuracy and stability of the studied ML models, it is recommended to apply developed ensemble ML model (NLE-GRNN) for significant improvement of NEE estimate in the tea biomes or other ecosystems.

A Nonlinear Convolutional Neural Network Algorithm for Autonomous Vehicle Lane Line Detection

A Nonlinear Convolutional Neural Network Algorithm for Autonomous Vehicle Lane Line Detection

Nonlinear-Model-Inversion Control for Stall-Flutter Suppression of an Airfoil via Camber Morphing

This paper presents a nonlinear-model-inversion control law to suppress stall flutter of an airfoil with active trailing-edge morphing. First, a nonlinear aeroelastic model is proposed utilizing two nonlinear autoregressive neural networks with exogenous inputs , which are used to predict aerodynamic moments on an airfoil due to large-amplitude oscillation and camber morphing, respectively. Afterward, a nonlinear-model-inversion control system is designed upon the mentioned aeroelastic system to suppress stall flutter via the camber morphing. A fluid–structure–control (FSC) coupling strategy is developed with structure and control systems embedded in the high-fidelity computational-fluid-dynamics environment to validate the control effect. The FSC high-fidelity simulations show that the nonlinear-model-inversion controller can suppress pitching oscillation completely, whereas a linear proportional–derivative controller without time delay only performs a limited suppression rate by 25.6%. The flowfield evolution result infers that the active camber morphing can generate a converse training-edge vortex, which counteracts the leading-edge vortex during stall flutter. From the perspective of an energy hysteresis, active camber morphing works well by converting injected aerodynamic energy from positive to negative.

A Coding Framework and Benchmark Towards Low-Bitrate Video Understanding.

Video compression is indispensable to most video analysis systems. Despite saving the transportation bandwidth, it also deteriorates downstream video understanding tasks, especially at low-bitrate settings. To systematically investigate this problem, we first thoroughly review the previous methods, revealing that three principles, i.e., task-decoupled, label-free, and data-emerged semantic prior, are critical to a machine-friendly coding framework but are not fully satisfied so far. In this paper, we propose a traditional-neural mixed coding framework that simultaneously fulfills all these principles, by taking advantage of both traditional codecs and neural networks (NNs). On one hand, the traditional codecs can efficiently encode the pixel signal of videos but may distort the semantic information. On the other hand, highly non-linear NNs are proficient in condensing video semantics into a compact representation. The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved w.r.t. the coding procedure, which is spontaneously learned from unlabeled data in a self-supervised manner. The videos collaboratively decoded from two streams (codec and NN) are of rich semantics, as well as visually photo-realistic, empirically boosting several mainstream downstream video analysis task performances without any post-adaptation procedure. Furthermore, by introducing the attention mechanism and adaptive modeling scheme, the video semantic modeling ability of our approach is further enhanced. Fianlly, we build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach. All codes, data, and models will be open-sourced for facilitating future research.