Robust Neural Network Research Articles

Robust Adaptive Rescaled Lncosh Neural Network Regression Toward Time-Series Forecasting

In time series forecasting with outliers and random noise, parameter estimation in a neural network via minimizing the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{2}$</tex-math> </inline-formula> loss is unreliable. Therefore, an adaptive rescaled lncosh loss function is proposed in this article to handle time series modeling with outliers and random noise. It overcomes the limitation of the single distribution of traditional loss functions and can switch among <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{1}$</tex-math> </inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{2}$</tex-math> </inline-formula> , and the Huber losses. A tuning parameter in the loss function is estimated by using a “working” likelihood approach according to estimated residuals. From the proposed loss function, a robust adaptive rescaled lncosh neural network (RARLNN) regression model is developed for highly accurate predictions. In the training phase of the model, an iterative learning procedure is presented to estimate the tuning parameter and train the neural network in iterations. A new prediction interval construction method is also developed based on quantile theory. The proposed RARLNN model is applied to two groups of wind speed forecasting tasks. The results show that the proposed RARLNN model is more conducive to enhancing forecasting accuracy and stability from the perspectives of noise distribution and outliers.

A Model Proposal for Enhancing Leaf Disease Detection Using Convolutional Neural Networks (CNN)

Deep learning has gained significant popularity due to its exceptional performance in various machine learning and artificial intelligence applications. In this paper, we propose a comprehensive methodology for enhancing leaf disease detection using Convolutional Neural Networks (CNNs). Our approach leverages the power of CNNs and introduces innovative techniques to improve accuracy and provide insights into the inner workings of the models. The methodology encompasses multiple stages. We describe the methodology as follows: Firstly, we employ advanced preprocessing techniques to enhance the leaf image dataset, including data augmentation methods to augment the training data and improve model accuracy. Secondly, we design and implement a robust Convolutional Neural Network architecture with multiple layers and ReLU activation, enabling the network to effectively learn complex patterns and features from the input images. To facilitate monitoring and control of the CNN processes, we introduce a novel network visualization module. This module offers a filter-level 2D embedding view, providing real-time insights into the inner workings of the network and aiding in the interpretation of the learned features. Additionally, we develop an interactive module that enables real-time model control, allowing researchers and practitioners to fine-tune the model parameters and optimize its performance. To evaluate the effectiveness of our proposed methodology, we conduct extensive experiments using the PlantVillage dataset, which contains a diverse range of plant diseases captured through a large number of leaf images. Through rigorous analysis and evaluation, we demonstrate the superior performance of our approach, achieving classification accuracy exceeding 99%.

RNGC-VIWO: Robust Neural Gyroscope Calibration Aided Visual-Inertial-Wheel Odometry for Autonomous Vehicle

Accurate and robust localization using multi-modal sensors is crucial for autonomous driving applications. Although wheel encoder measurements can provide additional velocity information for visual-inertial odometry (VIO), the existing visual-inertial-wheel odometry (VIWO) still cannot avoid long-term drift caused by the low-precision attitude acquired by the gyroscope of a low-cost inertial measurement unit (IMU), especially in visually restricted scenes where the visual information cannot accurately correct for the IMU bias. In this work, leveraging the powerful data processing capability of deep learning, we propose a novel tightly coupled monocular visual-inertial-wheel odometry with neural gyroscope calibration (NGC) to obtain accurate, robust, and long-term localization for autonomous vehicles. First, to cure the drift of the gyroscope, we design a robust neural gyroscope calibration network for low-cost IMU gyroscope measurements (called NGC-Net). Following a carefully deduced mathematical calibration model, NGC-Net leverages the temporal convolutional network to extract different scale features from raw IMU measurements in the past and regress the gyroscope corrections to output the de-noised gyroscope. A series of experiments on public datasets show that our NGC-Net has better performance on gyroscope de-noising than learning methods and competes with state-of-the-art VIO methods. Moreover, based on the more accurate de-noised gyroscope, an effective strategy for combining the advantages of VIWO and NGC-Net outputs is proposed in a tightly coupled framework, which significantly improves the accuracy of the state-of-the-art VIO/VIWO methods. In long-term and large-scale urban environments, our RNGC-VIWO tracking system performs robustly, and experimental results demonstrate the superiority of our method in terms of robustness and accuracy.

ANN model for predicting the elastic critical buckling coefficients of prismatic tapered steel web plates under stress gradients

Tapered steel plate girders are commonly used in large span industrial structures and composite bridges. The tapered thin steel web plates under stress gradients in such structures may be susceptible to elastic local buckling. An efficient and robust Artificial Neural Network (ANN) model is developed and presented in this paper for predicting the elastic critical buckling coefficients of prismatic tapered steel webs under stress gradients. More than 4000 samples are used in extensive parametric analyses of tapered web plates with simply supported edges. The ANN model recognizes important features associated with tapered plates, including the tapering ratio of the plate, aspect ratio, and stress gradient. A dataset containing 4665 records is developed for the training, validation, and testing of the ANN model developed. Two main ANN models are developed. The first model consists of a single hidden layer with neurons varying from 5 to 11. The second model consists of two hidden layers, with 8 neurons in the first layer and 6 neurons in the second layer. The accuracy of these models is thoroughly evaluated. It is found that the developed ANN model with two-hidden layers has superior performance than other models. The developed ANN can be employed to accurately predict the elastic critical buckling coefficients of prismatic tapered steel webs under stress gradients.

Identification and practical validation of spectrally efficient non-orthogonal frequency shaping waveform

Signal waveform is the basic physical layer element that fundamentally determines the most important key performance indicator in communication systems, namely spectral efficiency. Traditional waveforms rely on orthogonal Nyquist shaping but result in restricted spectral efficiency. Non-orthogonal waveforms have been proposed to enhance spectral efficiency via compressing either time or frequency resources. However, beyond 25% efficiency improvements in this way, classical Mazo theory states that performance will start to degrade. Here we use a robust and low-complexity neural network modulator to create asymmetric sub-carrier shapes, for multi-carrier non-orthogonal frequency shaping (NOFS) signals. The result is a non-Nyquist waveform which achieves 150% spectral efficiency improvement and still operates well with simple receiver processing. We set up a hardware communication link with over-the-air image transmission that practically validates the spectral efficiency improvement. This spectral efficiency will save communication resources for future communications systems such as 6G services.

DeepCredNet: Empowering Credibility Detection in Online Media Posts with Robust Deep Neural Networks

The rampant dissemination of misinformation and fake news through online media necessitates effective methods for detecting credibility. This study proposes a novel approach that employs a deep neural network (DNN) architecture, specifically Long Short-Term Memory (LSTM), to accurately assess the credibility of online media posts. The proposed LSTM-based DNN model leverages the inherent sequential nature of textual content and metadata associated with the posts. By capturing long-term dependencies and temporal dynamics, the model effectively learns intricate patterns and features crucial for credibility detection. A large labelled dataset comprising both credible and non-credible posts is employed to train the LSTM-based DNN. The input features encompass textual content, user information, and contextual details such as the post source and timing. The LSTM layers within the network enable the model to capture and retain relevant information over extended periods, enhancing its discriminative capabilities. Experimental evaluations validate the efficacy of the proposed approach, showcasing its ability to discern between credible and non-credible online media posts with high accuracy and robust performance. The real-time applicability of this method enables prompt credibility assessment, offering valuable support in combating misinformation and aiding users in making informed decisions while consuming online media.

Towards an adversarially robust convolutional neural network for automated diagnosis of malaria infection from microscopy images

Malaria is a life-threatening mosquito-borne disease of global importance, and it is most prevalent in the low-income countries of the developing world. The diagnosis of malaria infection is conventionally performed by laboratorians through visual examination of blood smear images under a microscope. However, the results of such diagnostic testing depend on the laboratory technicians' experience with staining and image interpretation, which can be severely lacking in low-resource settings. Convolutional neural networks (CNNs) have great potential for automated malaria diagnosis from images without the need of human expert interaction. However, studies have shown that current CNNs are not robust to adversarial noises which are small input perturbations that cause CNNs to incorrectly classify medical images. In this study, we developed a CNN for malaria parasite identification and evaluated it on a dataset of 27,558 cell images through patient-level cross validation. To improve robustness against adversarial noises for this application, the structure of the CNN was specially designed to limit the receptive field of the output, and adversarial training was applied with gradually increasing level of noises. Compared to three baseline networks (ResNet-18, MobileNet, and MnasNet) and two variants of the CNN, our custom-designed CNN achieved a substantially lower computation cost and better adversarial robustness against PGD-generated adversarial noises on the malaria image dataset. Our study holds the promise of developing a robust and affordable solution for automated diagnosis of malarial infection.

Self-supervised robust Graph Neural Networks against noisy graphs and noisy labels

Self-supervised robust Graph Neural Networks against noisy graphs and noisy labels

A Low-Power DNN Accelerator Enabled by a Novel Staircase RRAM Array.

Enhancing the ubiquitous sensors and connected devices with computational abilities to realize visions of the Internet of Things (IoT) requires the development of robust, compact, and low-power deep neural network accelerators. Analog in-memory matrix-matrix multiplications enabled by emerging memories can significantly reduce the accelerator energy budget while resulting in compact accelerators. In this article, we design a hardware-aware deep neural network (DNN) accelerator that combines a planar-staircase resistive random access memory (RRAM) array with a variation-tolerant in-memory compute methodology to enhance the peak power efficiency by 5.64× and area efficiency by 4.7× over state-of-the-art DNN accelerators. Pulse application at the bottom electrodes of the staircase array generates a concurrent input shift, which eliminates the input unfolding, and regeneration required for convolution execution within typical crossbar arrays. Our in-memory compute method operates in charge domain and facilitates high-accuracy floating-point computations with low RRAM states, device requirement. This work provides a path toward fast hardware accelerators that use low power and low area.

Developing Robust Neural Networks for High-Resolution TEM Image Analysis.

Developing Robust Neural Networks for High-Resolution TEM Image Analysis.

Robust Convolutional Neural Network Based on UNet for Iris Segmentation

Robust Convolutional Neural Network Based on UNet for Iris Segmentation

An approach towards building robust neural networks models using multilayer perceptron through experimentation on different photovoltaic thermal systems

The study is aimed at training an artificial neural network to determine the specific Photo voltaic thermal (PV/T) system that will function optimally in a specific environmental situation by optimizing the input parameters. In this regard, four different PV/T systems based on the arrangement of the tubes and the presence of the copper absorber sheet, namely, vertically oscillating tube, vertically oscillating sheet tube, rectangular spiral tube and horizontally oscillating tube PV/T’s were considered. In the experimental study, wind velocity, solar irradiation, ambient temperature, relative humidity, and inlet temperature to the PV/T are considered the influencing factors. The response factors for evaluating the performance of the PV/T system were temperature outlet and power output. Water was used as the working fluid for the thermal management of the systems for all four test cases. The investigation discloses that the rectangular spiral tube photovoltaic thermal system is the best among the studied systems as it maintains both average thermal efficiency and average electrical efficiency at 50.7% and 12.15%, respectively, considering the trade-off between electrical and thermal efficiency. The ideal design can then be selected for a specific area that will be more appropriate with its environmental fluctuations by approximating the production for each scenario. The artificial neural network was designed to be capable of being successfully trained for this application based on Artificial Neural Network (ANN), Multilayer Perceptron using Levenberg Marquardt learning algorithm through MATLAB software based on the experimental data. The ANN model for the rectangular spiral tube PV/T system achieved an exceptional overall R-value of 0.99843 among the proposed prediction models. For the artificial neural network model formulated for vertically oscillating tube, vertically oscillating sheet tube and horizontally oscillating tube PV/T system, the R-value was found to be 0.99807, 0.9943, and 0.99593, respectively. In the training phase, the MSE for vertically oscillating tube, vertically oscillating sheet tube, rectangular spiral tube and horizontally oscillating tube PV/T’s using the multilayer perceptron artificial neural network models were found to be 1.1328, 3.5082, 0.3134 and 1.1328, respectively. For the same PV/T’s models, the MSE values for Cross validation phase were found to be 1.8497, 3.9072, 5.1419, and 4.6402, respectively. Finally, the suggested linear prediction models assist in quickly and easily determining the ideal conditions for every solar system while lowering the error in furcating future results.

Deep learning design of functionally graded porous electrode of proton exchange membrane fuel cells

For the next generation of proton exchange membrane (PEM) fuel cells, the conventional electrode with uniform distribution of functional components is urged to be replaced by functional graded electrode for the prominent performance, efficiency and avoid exacerbated catalyst cost. Due to the complex and non-linear behaviours of PEM fuel cell system, rapid and effective computational model and optimisation algorithm are required to handle such a complex relationship between electrode design parameters and cell performance. In this work, a multi-physics model with multi-directionally graded electrode is developed, in which a deep machine learning approach is embedded, to create a surrogate model for multi-objective optimisation empowered by non-dominated sort genetic algorithm (NSGA-II). A robust prediction deep neural network model with the mean square error lower than 0.01 is obtained from training and then coupling with NSGA-II to evaluate and optimise the fuel cell performances and cost. Remarkably, the Pareto front is successfully defining the trade-off relationship between the objectives where it aids to identify an optimum point where it satisfies the cost effectiveness while maintaining relatively high cell performances. Our work presents a promising strategy to optimise the fuel cell system with underlying interaction and allow rapid and accurate prediction and optimisation.

Data‐driven stability of stochastic mean‐field type games via noncooperative neural network adversarial training

AbstractWe propose an approach to neural network stochastic differential games of mean‐field type and its corresponding stochastic stability analysis by means of adversarial training (aka adversarial attacks). This is a class of data‐driven differential games where the distribution of the variables such as the system states and the decision‐makers' strategies (control inputs) is incorporated into the problem. This work casts the cooperative/noncooperative game terminology into the deep learning framework where we talk about cooperative and noncooperative neural network computations that involve learning capabilities and neural network architectures. We suggest a method to computationally validate the feasibility of the approximated solutions via neural networks and evaluate the stochastic stability of the associated closed‐loop system (state feedback Nash). Moreover, we enhance the stochastic stability by enlarging the training set with adversarial initial states to obtain a more robust neural network for a particular decision‐maker. Finally, a worked‐out example based on the linear‐quadratic mean‐field type game (LQ‐MTG) that illustrates our methodology is presented.

Transferable learning on analog hardware.

While analog neural network (NN) accelerators promise massive energy and time savings, an important challenge is to make them robust to static fabrication error. Present-day training methods for programmable photonic interferometer circuits, a leading analog NN platform, do not produce networks that perform well in the presence of static hardware errors. Moreover, existing hardware error correction techniques either require individual retraining of every analog NN (which is impractical in an edge setting with millions of devices), place stringent demands on component quality, or introduce hardware overhead. We solve all three problems by introducing one-time error-aware training techniques that produce robust NNs that match the performance of ideal hardware and can be exactly transferred to arbitrary highly faulty photonic NNs with hardware errors up to five times larger than present-day fabrication tolerances.

Accelerating the Discovery of Transition Metal Borides by Machine Learning on Small Data Sets.

Accurate and efficient prediction of the stability and structure-stability relationship is important to discover materials; however, it requires tremendous efforts via traditional trial-and-error schemes. Here, we presented a small-data set machine learning (ML) method to accelerate the discovery of promising ternary transition metal boride (MAB) candidates. Based on data sets obtained by ab initio calculations, we developed three robust neural networks to predict the decomposition energy (ΔHd) and assess the thermodynamic stability of 212-typed MABs (M2AB2). The quantitative relation between ΔHd and stability was unraveled by several composition-and-structure descriptors. Three hexagonal M2AB2, i.e., Nb2PB2, Nb2AsB2, and Zr2SB2, were discovered to be stable with negative ΔHd, and 75 metastable MABs were identified with ΔHd less than 70 meV/atom. Finally, the dynamical stability and mechanical properties of MABs were investigated by ab initio calculations, whose results further verified the reliability of our ML models. This work provided a ML approach on small data sets to accelerate the discovery of compounds and expanded the MAB phase family to VA and VIA groups.

Robust CNN-based flow pattern identification for horizontal gas-liquid pipe flow using flow-induced vibration

Determination of flow patterns is crucial for the prediction of unsteady flow parameters, which is important for multiphase flow pipeline integrity management. Thus far, various methods have been put forward, however, their robustness has yet to be thoroughly tested across varying environments. Despite recent progress in flow-induced vibration (FIV) of multiphase flows, variability due to measurement approach and pipeline conditions remains an under-investigated area. This paper proposes a non-intrusive, robust convolutional neural network (CNN) flow pattern identification method based on FIV analysis. FIV for horizontal gas–liquid pipe flow under various flow patterns is investigated experimentally by simultaneously analyzing acquired FIV signals and high-speed video. A range of pipeline scenarios is thoroughly studied, encompassing variations in flow conditions, measurement axes, pipe material, and installation conditions. Both Hilbert-Huang Transform (HHT) and Short Time Fourier Transform (STFT) are used to investigate the characteristics of FIV. The results show distinct flow patterns with the FIV signal varying with measurement axes and pipe conditions. Three processes are applied to increase identification robustness: HHT images to eliminate variability due to structure signatures and flow conditions; Ostu’s threshold to eliminate image magnitude difference; and data augmentation, applied to both time signals and images, to increase database diversity. From these, the morphological features associated with flow patterns are extracted. Pattern identification results using a neural network architecture trained by GoogLeNet show over 95% accuracy. To test the generalizability of the proposed flow pattern identification method, the trained method is employed to make predictions in various pipeline scenarios. The results show that the trained model attains an accuracy of over 90% when predicting the flow pattern using data from different axes, and an accuracy above 80% when predicting with different pipe material and installation conditions. During the model training phase, the integration of multiple databases or the implementation of transfer learning can result in a substantial improvement in accuracy, reaching a respectable level of at least 90%.

Battery Safety Risk Assessment in Real-World Electric Vehicles Based on Abnormal Internal Resistance Using Proposed Robust Estimation Method and Hybrid Neural Networks

Battery safety issue is developing as one of the main hinders restricting the further application of real-world electric vehicles (EVs). Internal resistance (IR) is one of the important parameters to reflect battery safety, because bigger abnormal IR will cause more heat generation and make the battery easier to cross the critical condition of thermal runway. Safety risk assessment based on abnormal IR can locate this kind unsafe batteries and ensure the safe operation of EVs. Existing studies mostly focus on IR estimation in laboratory, yet is insufficient in real-world abnormal IR detection, which is influenced by random conditions and coupled factors. To cope with this issue, a method is proposed to detect unsafe battery, thereby predicting the thermal runaway. The method can be divided into three parts, i.e., IR estimation, normal IR prediction, and IR evolution law construction and safety risk assessment. First, we analyze the problems of battery IR estimation in real-world scenarios, and propose a robust method to estimate the IR only based on sparse voltage and current. Second, a novel hybrid neural network model is designed and trained to predict normal IRs inputted by temperature, mileage, and state-of-charge. The model combines the advantages of different neural network structures to improve the performance. A double-layer grid searching method is proposed to optimize hyperparameters of the hybrid neural network, aiming to achieve the best accuracy under constructed neural network structure. Finally, the safety boundary indicated by IR is formed by statistic method and the IR evolution law is constructed, then a strategy is proposed to make residual evaluation for safety risk assessment. All the proposed methods are driven by EV historical data to reflect real-world conditions. The results showcase that the method is effective in the cloud for the small IR prediction error with the mean-square-error, mean-relative-error, and max error of 2.45*e-4, 1.78%, and 0.15. In addition, the method can periodically classify the unsafe and normal batteries with 98.6% accuracy for the EVs with the same specification.

The robust physics-informed neural networks for a typical fourth-order phase field model

The robust physics-informed neural networks for a typical fourth-order phase field model

A Robust Complex-Valued Deep Neural Network for Target Recognition of UAV SAR Imagery

Unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) plays an important role in modern remote sensing for its characteristics of all weather, all day-and-night, zero casualty, flying flexibility and low cost. However, the atmospheric turbulence will cause motion errors to UAV SAR, resulting in unmodeled phase errors. The phase errors will degrade the focusing quality of the image and bring difficulties to the recognition task. Meanwhile, it is difficult for convolution neural network (CNN) to extract and utilize the back-scattering information for the target recognition. To this end, a novel Defocusing Adaptive Complex Convolution Neural Network (DA-CCNN) is proposed, which can realize the overall computation of the network in the complex-valued data domain and effectively extract the phase history information of the complex-valued data. Furthermore, it is the first time that the image entropy metric is introduced into the fully complex deep neural network to improve the focusing quality of the image and the interpretability of the network. The experiment is carried out using the benchmark dataset of MSTAR 10. In order to simulate the defocused images generated by UAV SAR and certificate the robustness to phase errors, datasets with the contamination are also applied. The results show that on the benchmark data, the recognition accuracy of DA-CCNN is comparable to that of the existing methods. On the data with phase errors, DA-CCNN shows stronger robustness and higher accuracy in terms of recognition than the reported networks.