Noise Injection Research Articles

Physics-Informed Deep Learning Inversion with Application to Noisy Magnetotelluric Measurements

Despite demonstrating exceptional inversion production for synthetic data, the application of deep learning (DL) inversion methods to invert realistic magnetotelluric (MT) measurements, which are inevitably contaminated by noise in acquisition, poses a significant challenge. Hence, to facilitate DL inversion for realistic MT measurements, this work explores developing a noise-robust MT DL inversion method by generating targeted noisy training datasets and constructing a physics-informed neural network. Different from most previous works that only considered the noise of one fixed distribution and level, we propose three noise injection strategies and compare their combinations to mitigate the adverse effect of measurement noise on MT DL inversion results: (1) add synthetic relative noise obeying Gaussian distribution; (2) propose a multiwindow Savitzky–Golay (MWSG) filtering scheme to extract potential and possible noise from the target field data and then introduce them into training data; (3) create an augmented training dataset based on the former two strategies. Moreover, we employ the powerful Swin Transformer as the backbone network to construct a U-shaped DL model (SwinTUNet), based on which a physics-informed SwinTUNet (PISwinTUNet) is implemented to further enhance its generalization ability. In synthetic examples, the proposed noise injection strategies demonstrate impressive inversion effects, regardless of whether they are contaminated by familiar or unfamiliar noise. In a field example, the combination of three strategies drives PISwinTUNet to produce considerably faithful reconstructions for subsurface resistivity structures and outperform the classical deterministic Occam inversions. The experimental results show that the proposed noise-robust DL inversion method based on the noise injection strategies and physics-informed DL architecture holds great promise in processing MT field data.

Convolutional neural network for classifying cartoon images augmented by DCGAN

Convolutional Neural Network (CNN) tend to have better results on large data sets and poor performance on small data sets, so the data augmentation is crucial for a CNN to get better performance based on the dataset with limited size. In this paper, Deep Convolution Generative Adversarial Network (DCGAN) was used to augment data to make the AlexNet perform better on an image classification task with small data sets. AlexNet was trained on a small anime face training set with only 160 samples to determine whether the anime face was male or female, and then tested its accuracy on a test set with 240 sample. Then, a pre-trained DCGAN was transferred to train on the male and female training sets respectively. And 2 DCGANs were obtained, one could generate male cartoon faces and another could generate female cartoon faces. The images generated by DCGANs were put in train set, which was used to train AlexNet again and the result was recorded. Other data augmentation methods such as cutout, cutmix and Noise Injection were compared as well. Finally, it is found that AlexNet has the best performance when using the DCGAN augmentation method, which can significantly improve the verification accuracy of the model.

Optimized injection of noise in activation functions to improve generalization of neural networks

This paper proposes a flexible probabilistic activation function that enhances the training and operation of artificial neural networks by intentionally injecting noise to gain additional control over the response of each neuron. During the learning phase, the level of injected noise is iteratively optimized by gradient-descent, realizing a form of adaptive stochastic resonance. From simple hard-threshold non-differentiable neuronal responses, controlled injection of noise gives access to a wide range of useful activation functions, with sufficient differentiability to enable gradient-descent learning for both the neuron and the injected-noise levels. Experimental results on function approximation demonstrate injected noise generally converging to non-vanishing optimal levels associated with improved generalization abilities in the neural networks. A theoretical explanation of the generalization improvement based on the path norm bound is presented. With injected noise in the deep neural network, experimental results on classifying images also obtain non-vanishing optimal noise levels to achieve better testing accuracies. The proposed probabilistic activation functions show the potential of adaptive stochastic resonance for useful applications in machine learning.

Data Hiding With Deep Learning: A Survey Unifying Digital Watermarking and Steganography

The advancement of secure communication and identity verification fields has significantly increased through the use of deep learning techniques for data hiding. By embedding information into a noise-tolerant signal, such as audio, video, or images, digital watermarking and steganography techniques can be used to protect sensitive intellectual property (IP) and enable confidential communication, ensuring that the information embedded is only accessible to authorized parties. This survey provides an overview of recent developments in deep learning techniques deployed for data hiding, categorized systematically according to model architectures and noise injection methods. In addition, potential future research directions that unite digital watermarking and steganography on software engineering to enhance security and mitigate risks are suggested and deliberated. This contribution furthers the creation of a more trustworthy digital world and advances responsible artificial intelligence (AI).

A robust auxiliary noise power scheduling strategy for online secondary path modeling in active noise control systems

Auxiliary noise injected in active noise control (ANC) systems for online secondary path modeling (SPM) contributes to the residual noise, and thus deteriorates the noise control performance of ANC systems. Moreover, sudden changes in secondary path may lead to easy divergence of the existing online SPM methods. This paper improves the existing methods from three aspects to further mitigate these problems: 1) the noise power is varied on the basis of the convergence status of SPM filter so that large-power noise is injected at the start or when secondary path changes and the power is reduced as the filter converges. 2) a novel ON/OFF controlling strategy is used in the method so that the noise injection is suspended at the optimum point when the modeling accuracy is sufficient. 3) continuous check for error outliers is applied in the method to detect changes in secondary path so that the injection of noise can be reactivated in the case of path perturbation. Comparative simulations show the effectiveness of the improved method both in convergence and noise reduction performance. In addition, the improved method is robust against sudden changes in secondary path.

Novel Noise Injection Scheme to Guarantee Zero Secrecy Outage under Imperfect CSI.

The paper proposes a novel artificial noise (AN) injection strategy in multiple-input single-output multiple-antenna-eavesdropper (MISOME) systems under imperfect channel estimation at the legitimate channel to achieve zero secrecy outage probability under any circumstance. The zero secrecy outage is proved to always be achievable regardless of the eavesdropper's number of antennas or location when the pair secrecy and codeword rates are chosen properly. The results show that when there is perfect channel state information, the zero-outage secrecy throughput increases with the transmit power, which is important for secrecy design. Additionally, an analysis of the secrecy throughput and secrecy energy efficiency gives further insight into the effectiveness of the proposed scheme.

A Multi-Agent Deep-Reinforcement-Learning-Based Strategy for Safe Distributed Energy Resource Scheduling in Energy Hubs

An energy hub (EH) provides an effective solution to the management of local integrated energy systems (IES), supporting the optimal dispatch and mutual conversion of distributed energy resources (DER) in multi-energy forms. However, the intrinsic stochasticity of renewable generation intensifies fluctuations in the system’s energy production when integrated into large-scale grids and increases peak-to-valley differences in large-scale grid integration, leading to a significant reduction in the stability of the power grid. A distributed privacy-preserving energy scheduling method based on multi-agent deep reinforcement learning is presented for the EH cluster with renewable energy generation. Firstly, each EH is treated as an agent, transforming the energy scheduling problem into a Markov decision process. Secondly, the objective function is defined as minimizing the total economic cost while considering carbon trading costs, guiding the agents to make low-carbon decisions. Lastly, differential privacy protection is applied to sensitive data within the EH, where noise is introduced using energy storage systems to maintain the same gas and electricity purchases while blurring the original data. The experimental simulation results demonstrate that the agents are able to train and learn from environmental information, generating real-time optimized strategies to effectively handle the uncertainty of renewable energy. Furthermore, after the noise injection, the validity of the original data is compromised while ensuring the protection of sensitive information.

Guaranteeing Zero Secrecy Outage in Relaying Systems under Eavesdropper’s Arbitrary Location and Unlimited Number of Antennas

This paper proposes a three-phase transmission scheme to ensure zero secrecy outage in decode-and-forward relay systems by using the strategies of artificial noise (AN) injection and channel state information (CSI) leakage avoidance. The zero-outage secrecy spectral efficiency (ZOSSE) and energy efficiency (ZOSEE) of the scheme are then analyzed. Finally, the paper demonstrates that the scheme can always achieve zero secrecy outage even when the eavesdropper has an unlimited number of antennas or is in an arbitrary location, which shows its practical applicability. The paper also shows that the ZOSSE increases with the transmit power and that both the ZOSSE and the ZOSEE are maximized when the relay is halfway between the transmitter and the receiver. This suggests that the placement of the helper node is important in securing the communication of two distant nodes.

Quantum Multiagent Actor–Critic Networks for Cooperative Mobile Access in Multi-UAV Systems

This paper proposes a novel algorithm, named quantum multi-agent actor-critic networks (QMACN) for autonomously constructing a robust mobile access system employing multiple unmanned aerial vehicles (UAVs). In the context of facilitating collaboration among multiple unmanned aerial vehicles (UAVs), the application of multi-agent reinforcement learning (MARL) techniques is regarded as a promising approach. These methods enable UAVs to learn collectively, optimizing their actions within a shared environment, ultimately leading to more efficient cooperative behavior. Furthermore, the principles of a quantum computing (QC) are employed in our study to enhance the training process and inference capabilities of the UAVs involved. By leveraging the unique computational advantages of quantum computing, our approach aims to boost the overall effectiveness of the UAV system. However, employing a QC introduces scalability challenges due to the near intermediate-scale quantum (NISQ) limitation associated with qubit usage. The proposed algorithm addresses this issue by implementing a quantum centralized critic, effectively mitigating the constraints imposed by NISQ limitations. Additionally, the advantages of the QMACN with performance improvements in terms of training speed and wireless service quality are verified via various data-intensive evaluations. Furthermore, this paper validates that a noise injection scheme can be used for handling environmental uncertainties in order to realize robust mobile access.

Abstract 18317: Implications of Noise on Deep Learning Models for 12-Lead ECG Construction

Introduction: ECG construction with deep learning can help standardize ECGs and remove noise and artifacts, transforming potentially unusable ECGs into clinically useful ones. However, noise levels, often unpredictable in clinical settings, may impact model performance. Hypothesis: This study explores the relationship between ECG noise levels and deep learning model performance in constructing noise-free ECGs. We hypothesize that beyond a certain noise threshold, the model's output ceases to be clinically usable. Aims: Our study aims to determine noise effects on the performance of deep learning models in constructing 12-Lead ECGs, providing insights into the model's robustness and the optimal conditions for its application. Methods: Using 250 digital 12-Lead ECGs from the PTB database, we injected Gaussian noise (mean 0.0, standard deviations between 0.0 and 1.0) into these ECGs, all processed and scaled to unitless values (-1 to +1) and resampled at 100Hz. A model was asked to construct a noise-free ECG from a full 12-lead ECG with noise injection, which was compared to the original noise-free ECG. Results: The findings reveal two distinct linear phases in the relationship between noise and model performance (measured by mean absolute error, MAE). Between standard deviations of 0.0 and 0.03, the MAE increased marginally, while a sharp increase occurred between 0.03 and 1.0. Importantly, critical features of the original ECG were retained for noise levels up to 0.08, implying a noise threshold for clinical usability. Conclusions: Deep learning models, though not entirely resistant to noise, demonstrate efficacy in constructing ECGs up to certain noise levels (standard deviation 0.03). Beyond this, performance declines markedly. These findings outline the limits and potential of deep learning models like ECGio in clinical practice. Future research should focus on resilience to higher noise levels or noise-reduction preprocessing strategies.

A feature-weighted suppressed possibilistic fuzzy c-means clustering algorithm and its application on color image segmentation

The possibilistic fuzzy c-means clustering (PFCM) algorithm is a hybridization of possibilistic c-means clustering (PCM) and fuzzy c-means clustering (FCM) algorithms. However, there are two main problems in PFCM. One is that the Euclidean distance employed in PFCM always disregards the imbalance among sample features because it treats all features of data equally, which easily causes misclassification for feature-imbalanced multidimensional data. The other is that PFCM always produces significant center deviations and overlapping centers for multiclass datasets with strong noise injection, due to the difficulty of PFCM in the membership-weight parameter setting and the lack of between-class relationships in possibilistic memberships. Therefore, a feature-weighted suppressed possibilistic fuzzy c-means clustering (FW-S-PFCM) algorithm is presented by introducing a feature-weighted method and “suppressed competitive learning” strategy into the PFCM algorithm in this paper. First, the FW-S-PFCM algorithm introduces a feature-weight matrix into the objective function that can automatically assign feature-weight values to different features and different clusters according to the distribution of samples, thus overcoming the influence of feature imbalance and improving clustering effects for noisy multidimensional datasets. Second, combined with the feature-weight matrix, a “suppressed competitive learning” strategy is designed to resolve the center-overlapping problem in noisy multiclass dataset clustering. Specifically, partial crucial points of each class near the center are selected according to a cluster core generated by a cross-section of a threshold on the possibilistic membership surface. Third, their possibilistic memberships participate in the suppressed learning process to overcome the lack of between-class relationships. Last, a segmentation algorithm for noisy color images based on FW-S-PFCM is proposed combined with the feature-weight method and noise-identification ability of possibilistic memberships. Experiments on synthetic data, UCI data and color image segmentation demonstrate that the proposed FW-S-PFCM algorithm can overcome the partial center-overlapping problem and improve clustering performance on complex datasets with feature imbalance and strong noise injection. The proposed algorithm can also reduce the iteration number, sensitivity to membership weights, and initializations of PFCM.

TB-MFCC multifuse feature for emergency vehicle sound classification using multistacked CNN – Attention BiLSTM

Vehicles equipped for emergencies like ambulances, fire engines, and police cruisers play a vital role in society by responding quickly to emergencies and helping to prevent loss of life and maintain order. Vehicle sound identification and classification are very important in the cities to identify emergency vehicles easily and to clear the traffic effectively. Convolutional Neural Network plays an important role in the accurate prediction of vehicles during an emergency. The main motive of this paper is to develop a suitable model and algorithms for data augmentation, feature extraction, and classification. The proposed TB-MFCC multifuse feature is comprised of data augmentation and feature extraction. First, in the proposed signal augmentation, each audio signal uses noise injection, stretching, shifting, and pitching separately and this process increases the number of instances in the dataset. The proposed augmentation reduces the overfitting problem in the network. Second, Triangular Bluestein Mel Frequency Cepstral Coefficients (TB-MFCC) are proposed and fused with Zero Crossing Rate (ZCR), Mel-frequency cepstral coefficients (MFCC), Root Mean Square (RMS), Chroma, and Tempogram to extract the exact feature which increases the accuracy and reduces the Mean Squared Error (MSE) of the model during classification. Finally, the proposed Multi-stacked Convolutional Neural Network (MCNN) with Attention-based Bidirectional Long Short Term Memory (A-BiLSTM) improves the nonlinear relationship among the features. The proposed Pooled Multifuse Feature Augmentation (PMFA) with MCNN & A-BiLSTM increases the accuracy (98.66 %), reduces the False Positive Rate (FPR) by 1.01 %, and loss (0 %). Thus the model predicts the sound without overfitting, underfitting, and vanishing gradient problems.

Mamdani vs. Takagi-Sugeno Fuzzy Inference Systems in the Calibration of Continuous-Time Car-Following Models.

The transition to intelligent transportation systems (ITSs) is necessary to improve traffic flow in urban areas and reduce traffic congestion. Traffic modeling simplifies the understanding of the traffic paradigm and helps researchers to estimate traffic behavior and identify appropriate solutions for traffic control. One of the most used traffic models is the car-following model, which aims to control the movement of a vehicle based on the behavior of the vehicle ahead while ensuring collision avoidance. Differences between the simulated and observed model are present because the modeling process is affected by uncertainties. Furthermore, the measurement of traffic parameters also introduces uncertainties through measurement errors. To ensure that a simulation model fully replicates the observed model, it is necessary to have a calibration process that applies the appropriate compensation values to the simulation model parameters to reduce the differences compared to the observed model parameters. Fuzzy inference techniques proved their ability to solve uncertainties in continuous-time models. This article aims to provide a comparative analysis of the application of Mamdani and Takagi-Sugeno fuzzy inference systems (FISs) in the calibration of a continuous-time car-following model by proposing a methodology that allows for parallel data processing and the determination of the simulated model output resulting from the application of both fuzzy techniques. Evaluation of their impact on the follower vehicle considers the running distance and the dynamic safety distance based on the observed behavior of the leader vehicle. In this way, the identification of the appropriate compensation values to be applied to the input of the simulated model has a great impact on the development of autonomous driving solutions, where the real-time processing of sensor data has a crucial impact on establishing the car-following strategy while ensuring collision avoidance. This research performs a simulation experiment in Simulink (MATLAB R2023a, Natick, MA, USA: The MathWorks Inc.) and considers traffic data collected by inductive loops as parameters of the observed model. To emphasize the role of Mamdani and Takagi-Sugeno FISs, a noise injection is applied to the model parameters with the help of a band-limited white-noise Simulink block to simulate sensor measurement errors and errors introduced by the simulation process. A discussion based on performance evaluation follows the simulation experiment, and even though both techniques can be successfully applied in the calibration of the car-following models, the Takagi-Sugeno FIS provides more accurate compensation values, which leads to a closer behavior to the observed model.

Boosting image denoising effect via low-level noise injection

Boosting image denoising effect via low-level noise injection

AdaNI: Adaptive Noise Injection to improve adversarial robustness

Deep Neural Networks (DNNs) have been proven vulnerable to adversarial perturbations, which narrow their applications in safe-critical scenarios such as video surveillance and autonomous driving. To counter this threat, a very recent line of adversarial defense methods is proposed to increase the uncertainty of DNNs via injecting random noises in both the training and testing process. Note the existing defense methods usually inject noises uniformly to DNNs. We argue that the magnitude of noises is highly correlated with the response of corresponding features and the randomness on important feature spots can further weaken adversarial attacks. As such, we propose a new method, namely AdaNI, which can increase feature randomness via Adaptive Noise Injection to improve the adversarial robustness. Compared to existing methods, our method creates non-unified random noises guided by features and then injects them into DNNs adaptively. Extensive experiments are conducted on several datasets (e.g., CIFAR10, CIFAR100, Mini-ImageNet) with comparisons to state-of-the-art defense methods, which corroborates the efficacy of our method against a variety of powerful white-box attacks (e.g., FGSM, PGD, C&W, Auto Attack) and black-box attacks (e.g., Transferable, ZOO, Square Attack). Moreover, our method is adapted to improve the robustness of DeepFake detection to demonstrate its applicability.

Generative adversarial networks via a composite annealing of noise and diffusion

Generative adversarial network (GAN) is a framework for generating fake data using a set of real examples. However, GAN is unstable in the training stage. In order to stabilize GANs, the noise injection has been used to enlarge the overlap of the real and fake distributions at the cost of increasing variance. The diffusion process (or data smoothing in its spatial domain) removes fine details in order to capture the structure and important patterns in data but it suppresses the capability of GANs to learn high-frequency information in the training procedure. Based on these observations, we propose a data representation for the GAN training, called noisy scale-space (NSS), that recursively applies the smoothing with a balanced noise to data in order to replace the high-frequency information by random data, leading to a coarse-to-fine training of GANs. We experiment with NSS using DCGAN and StyleGAN2 based on benchmark datasets in which the NSS-based GANs outperforms the state-of-the-arts in most cases.

A Robust and Energy Efficient Hyperdimensional Computing System for Voltage-scaled Circuits

Voltage scaling is one of the most promising approaches for energy efficiency improvement but also brings challenges to fully guaranteeing stable operation in modern VLSI. To tackle such issues, we further extend the DependableHD to the second version DependableHDv2 , a HyperDimensional Computing (HDC) system that can tolerate bit-level memory failure in the low voltage region with high robustness. DependableHDv2 introduces the concept of margin enhancement for model retraining and utilizes noise injection to improve the robustness, which is capable of application in most state-of-the-art HDC algorithms. We additionally propose the dimension-swapping technique, which aims at handling the stuck-at errors induced by aggressive voltage scaling in the memory cells. Our experiment shows that under 8% memory stuck-at error, DependableHDv2 exhibits a 2.42% accuracy loss on average, which achieves a 14.1 × robustness improvement compared to the baseline HDC solution. The hardware evaluation shows that DependableHDv2 supports the systems to reduce the supply voltage from 430mV to 340mV for both item Memory and Associative Memory, which provides a 41.8% energy consumption reduction while maintaining competitive accuracy performance.

Multichannel One-Dimensional Data Augmentation with Generative Adversarial Network.

Data augmentation is one of the most important problems in deep learning. There have been many algorithms proposed to solve this problem, such as simple noise injection, the generative adversarial network (GAN), and diffusion models. However, to the best of our knowledge, these works mainly focused on computer vision-related tasks, and there have not been many proposed works for one-dimensional data. This paper proposes a GAN-based data augmentation for generating multichannel one-dimensional data given single-channel inputs. Our architecture consists of multiple discriminators that adapt deep convolution GAN (DCGAN) and patchGAN to extract the overall pattern of the multichannel generated data while also considering the local information of each channel. We conducted an experiment with website fingerprinting data. The result for the three channels' data augmentation showed that our proposed model obtained FID scores of 0.005,0.017,0.051 for each channel, respectively, compared to 0.458,0.551,0.521 when using the vanilla GAN.

Noise-Boosted Convolutional Neural Network for Edge-Based Motor Fault Diagnosis With Limited Samples

Convolutional neural networks (CNNs) have been widely applied in motor fault diagnosis. However, to obtain high recognition accuracy, massive training data are typically required and transmitted to the cloud/local server for training, which may suffer from security and privacy problems. In this study, a noise-boosted CNN (NBCNN) model is developed to achieve accelerated training and improved recognition accuracy with limited training samples. First, the NBCNN model with a noise-injection fully connected layer is established. Then, a strategy for noise selection and injection is proposed to obtain an optimal matching among the data, model, and noise. Finally, the optimal injected noise accelerates the convergence of model training and improves the accuracy of motor fault diagnosis. Compared with the conventional CNN without noise injection and the state-of-the-art models, the effectiveness and superiority of the proposed NBCNN model are validated by two benchmark datasets. In addition, the algorithm is deployed onto an edge device and the results show that the training speed of the developed NBCNN can reach nine times faster than the conventional CNN. The proposed method shows remarkable potential for distributed model training, federal learning, and real-time motor fault diagnosis.

Combined frequency and voltage control of two-area multi-source interconnected microgrids via the 2DOF-TIDμ controller

The focus of this study is the utilization of a new two degree of freedom fractional controller, namely the two degree of freedom tilt-integral-derivative controller with fractional derivative (2DOF-TIDμ) and filter, optimized by the coot algorithm, for voltage and frequency regulation in two interconnected microgrids comprising of various sustainable and renewable sources. Both microgrids comprise of a biomass combined heat and power, biodiesel generator, micro-hydro turbine generator and biogas turbine generator along with a battery and superconducting energy storage system. In addition, the first microgrid includes a geothermal power plant, whereas the second microgrid a wind power plant. Furthermore, a power system stabilizer and a unified power flow controller are included in the system. The study aims to present an appropriate controller for the frequency and voltage control of interconnected microgrids which could be isolated communities and/or farms. The controller is compared with the 2DOF-PID and 2DOF-TID controllers, in various scenarios. The scenarios include various simultaneous step load perturbations in both microgrids, penetration of renewable energy sources in one or both areas and injection of noise in the control channels. The results indicate that the 2DOF-TIDμ controller has better transient response and is more effective at suppressing deviations than the other controllers.