Clean Input Research Articles

Motor imagery electroencephalography channel selection based on deep learning: A shallow convolutional neural network

Electroencephalography (EEG) motor imagery (MI) signals have recently attracted much attention because of their potential to communicate with the surrounding environment in a specific way without the need for muscular and physical movement. Despite these advantages, these signals are difficult to detect due to their low signal to noise (SNR) rate and non-stationary and dynamic nature. Convolutional neural networks (CNN) can extract appropriate spatial and temporal features without the need for separate feature extraction and classification steps. Clean input data for CNN significantly improves its performance, but sometimes, common interference occurs between multi-channel signals. This challenge, along with the noisy EEG signals, degrades the performance of CNN networks. This paper presents a channel selection method based on convolutional neural networks to obviate these challenges. The proposed shallow convolutional neural network (SCNN) is designed with temporal convolution and pointwise convolution (using 1×1 convolution) to select the best channel with minimal computational load. Following the channel selection step, the multi-layer fusion CNN model was used to classify the signals. Choosing suitable and relevant EEG channels for motor imagery tasks makes it possible to create appropriate inputs for the classification model so that the model can be improved in the end. For High Gamma Dataset and Brian-Computer Interface (BCI) Competition IV-2a, the following accuracies were obtained: 81.15% and 72.01%, which are higher than when the other channel selections such as Mutual Information, Sequential Feature Forward Selection (SFFS) and wrapper methods are used.

Towards assessing the synthetic-to-measured adversarial vulnerability of SAR ATR

Recently, there has been increasing concern about the vulnerability of deep neural network (DNN)-based synthetic aperture radar (SAR) automatic target recognition (ATR) to adversarial attacks, where a DNN could be easily deceived by clean input with imperceptible but aggressive perturbations. This paper studies the synthetic-to-measured (S2M) transfer setting, where an attacker generates adversarial perturbation based solely on synthetic data and transfers it against victim models trained with measured data. Compared with the current measured-to-measured (M2M) transfer setting, our approach does not need direct access to the victim model or the measured SAR data. We also propose the transferability estimation attack (TEA) to uncover the adversarial risks in this more challenging and practical scenario. The TEA makes full use of the limited similarity between the synthetic and measured data pairs for blind estimation and optimization of S2M transferability, leading to feasible surrogate model enhancement without mastering the victim model and data. Comprehensive evaluations based on the publicly available synthetic and measured paired labeled experiment (SAMPLE) dataset demonstrate that the TEA outperforms state-of-the-art methods and can significantly enhance various attack algorithms in computer vision and remote sensing applications. Codes and data are available at https://github.com/scenarri/S2M-TEA.

How constant is constant elasticity of substitution? Endogenous substitution between clean and dirty energy

The degree of substitutability between clean and dirty energy plays a central role in leading economic analyses of optimal environmental policy. Despite the importance, assuming a constant and exogenous elasticity of substitution has been a dominant theoretical approach. We challenge this assumption by developing a dynamic general equilibrium model with an endogenous elasticity of substitution that interacts with the relative share of clean inputs in the economy. We find strong dynamic feedback effects arising from endogenous substitution capacity that amplifies the impact of directed technical change and accelerates the transition to a green economy.

A Dual Stealthy Backdoor: From Both Spatial and Frequency Perspectives

Backdoor attacks pose serious security threats to deep neural networks (DNNs). Backdoored models make arbitrarily (targeted) incorrect predictions on inputs containing well-designed triggers, while behaving normally on clean inputs. Prior researches have explored the invisibility of backdoor triggers to enhance attack stealthiness. However, most of them only focus on the invisibility in the spatial domain, neglecting the generation of invisible triggers in the frequency domain. This limitation renders the generated poisoned images easily detectable by recent defense methods. To address this issue, we propose a DUal stealthy BAckdoor attack method named DUBA, which simultaneously considers the invisibility of triggers in both the spatial and frequency domains, to achieve desirable attack performance, while ensuring strong stealthiness. Specifically, we first use Wavelet Transform to embed the high-frequency information of the trigger image into the clean image to ensure attack effectiveness. Then, to attain strong stealthiness, we incorporate Fourier Transform and Cosine Transform to mix the poisoned image and clean image in the frequency domain. Moreover, DUBA adopts a novel attack strategy, training the model with weak triggers and attacking with strong triggers to further enhance attack performance and stealthiness. DUBA is evaluated extensively on four datasets against popular image classifiers, showing significant superiority over state-of-the-art backdoor attacks in attack success rate and stealthiness.

Poison Attack and Poison Detection on Deep Source Code Processing Models

In the software engineering (SE) community, deep learning (DL) has recently been applied to many source code processing tasks, achieving state-of-the-art results. Due to the poor interpretability of DL models, their security vulnerabilities require scrutiny. Recently, researchers have identified an emergent security threat to DL models, namely, poison attacks . The attackers aim to inject insidious backdoors into DL models by poisoning the training data with poison samples. The backdoors mean that poisoned models work normally with clean inputs but produce targeted erroneous results with inputs embedded with specific triggers. By using triggers to activate backdoors, attackers can manipulate poisoned models in security-related scenarios (e.g., defect detection) and lead to severe consequences. To verify the vulnerability of deep source code processing models to poison attacks, we present a poison attack approach for source code named CodePoisoner as a strong imaginary enemy. CodePoisoner can produce compilable and functionality-preserving poison samples and effectively attack deep source code processing models by poisoning the training data with poison samples. To defend against poison attacks, we further propose an effective poison detection approach named CodeDetector . CodeDetector can automatically identify poison samples in the training data. We apply CodePoisoner and CodeDetector to six deep source code processing models, including defect detection, clone detection, and code repair models. The results show that ❶ CodePoisoner conducts successful poison attacks with a high attack success rate (average: 98.3%, maximum: 100%). It validates that existing deep source code processing models have a strong vulnerability to poison attacks. ❷ CodeDetector effectively defends against multiple poison attack approaches by detecting (maximum: 100%) poison samples in the training data. We hope this work can help SE researchers and practitioners notice poison attacks and inspire the design of more advanced defense techniques.

Impact of Renewable Energy and Globalization on Environmental Pollution in Asian Countries: A Review

The main purpose of this work is to investigate the impacts of globalization (GL), renewable energy (RE) natural resources (NR) and value-added agriculture (AG) on ecological footprints (EF) and CO2 emissions. For this research paper includes yearly data from 1990–2022 for four South Asian nations: Bangladesh, India, Pakistan, and Sri Lanka. These countries are most vulnerable to climate hazards and rapid economic transitions. The. Globalization and AG are contributing to environmental degradation in selected South Asian countries. Therefore, these countries need to exploit solar energy to its full capacity. Moreover, these countries need to explore more RE resources to reduce their dependence on non-RE sources. These countries can make their agricultural sectors sustainable by following efficient farming practices. Environmental awareness should be enhanced among the farmers. Farmers can use animal fertilizers and clean inputs in AG to achieve sustainable agricultural products. Overall, this work suggests that these countries can achieve a cleaner environment by adopting RE and by promoting efficient technologies through globalization.

FCNet: Learning Noise-Free Features for Point Cloud Denoising

The acquisition of point clouds is usually accompanied by noise due to imperfect laser scanning or image-based reconstruction techniques. Deep learning-based methods have achieved impressive performance in point cloud denoising. However, the features captured by a denoising network from noisy point clouds are usually contaminated by noise during training. The feature noise will lead to the oscillation of back-propagated gradients, which interferes with parameter optimization and reduces the denoising performance. In this paper, we propose to explicitly clean up feature noise for point cloud denoising from two aspects: feature noise cleaning and network training. From the first aspect, we propose the feature clean network (FCNet for short) to explicitly clean up the feature noise. From the second aspect, we train FCNet by a teacher-student learning model to learn the noise-free features under the guidance of feature domain losses. Specifically, FCNet is designed with emphasis on two modules: non-local self-similarity (NSS) and weighted average pooling (WAP). NSS module smooths features through a non-local filter based on the inherent non-local self-similarity of point clouds. WAP module applies original weights calculated by the statistical outlier removal algorithm to suppress the feature noise induced by outliers. In the teacher-student learning model, we introduce a clean input using the noisy point and its clean neighbors. The teacher network accepts the clean input to capture noise-free features. The student network is trained to imitate the teacher network to learn noise-free features by minimizing the feature loss. The experiments on synthetic and real scanned point clouds show that FCNet outperforms state-of-the-art point cloud denoising methods.

Sparse Backdoor Attack Against Neural Networks

Abstract Recent studies show that neural networks are vulnerable to backdoor attacks, in which compromised networks behave normally for clean inputs but make mistakes when a pre-defined trigger appears. Although prior studies have designed various invisible triggers to avoid causing visual anomalies, they cannot evade some trigger detectors. In this paper, we consider the stealthiness of backdoor attacks from input space and feature representation space. We propose a novel backdoor attack named sparse backdoor attack, and investigate the minimum required trigger to induce the well-trained networks to make incorrect results. A U-net-based generator is employed to create triggers for each clean image. Considering the stealthiness of the trigger, we restrict the elements of the trigger between −1 and 1. In the aspect of the feature representation domain, we adopt an entanglement cost function to minimize the gap between feature representations of benign and malicious inputs. The inseparability of benign and malicious feature representations contributes to the stealthiness of our attack against various model diagnosis-based defences. We validate the effectiveness and generalization of our method by conducting extensive experiments on multiple datasets and networks.

B 3 : Backdoor Attacks against Black-box Machine Learning Models

Backdoor attacks aim to inject backdoors to victim machine learning models during training time, such that the backdoored model maintains the prediction power of the original model towards clean inputs and misbehaves towards backdoored inputs with the trigger. The reason for backdoor attacks is that resource-limited users usually download sophisticated models from model zoos or query the models from MLaaS rather than training a model from scratch, thus a malicious third party has a chance to provide a backdoored model. In general, the more precious the model provided (i.e., models trained on rare datasets), the more popular it is with users. In this article, from a malicious model provider perspective, we propose a black-box backdoor attack, named B 3 , where neither the rare victim model (including the model architecture, parameters, and hyperparameters) nor the training data is available to the adversary. To facilitate backdoor attacks in the black-box scenario, we design a cost-effective model extraction method that leverages a carefully constructed query dataset to steal the functionality of the victim model with a limited budget. As the trigger is key to successful backdoor attacks, we develop a novel trigger generation algorithm that intensifies the bond between the trigger and the targeted misclassification label through the neuron with the highest impact on the targeted label. Extensive experiments have been conducted on various simulated deep learning models and the commercial API of Alibaba Cloud Compute Service. We demonstrate that B 3 has a high attack success rate and maintains high prediction accuracy for benign inputs. It is also shown that B 3 is robust against state-of-the-art defense strategies against backdoor attacks, such as model pruning and NC.

Enhancing adversarial robustness of quantum neural networks by adding noise layers

Abstract The rapid advancements in machine learning and quantum computing have given rise to a new research frontier: quantum machine learning. Quantum models designed for tackling classification problems possess the potential to deliver speed enhancements and superior predictive accuracy compared to their classical counterparts. However, recent research has revealed that quantum neural networks (QNNs), akin to their classical deep neural network-based classifier counterparts, are vulnerable to adversarial attacks. In these attacks, meticulously designed perturbations added to clean input data can result in QNNs producing incorrect predictions with high confidence. To mitigate this issue, we suggest enhancing the adversarial robustness of quantum machine learning systems by incorporating noise layers into QNNs. This is accomplished by solving a Min-Max optimization problem to control the magnitude of the noise, thereby increasing the QNN’s resilience against adversarial attacks. Extensive numerical experiments illustrate that our proposed method outperforms state-of-the-art defense techniques in terms of both clean and robust accuracy.

Con-Detect: Detecting adversarially perturbed natural language inputs to deep classifiers through holistic analysis

Deep Learning (DL) algorithms have shown wonders in many Natural Language Processing (NLP) tasks such as language-to-language translation, spam filtering, fake-news detection, and comprehension understanding. However, research has shown that the adversarial vulnerabilities of deep learning networks manifest themselves when DL is used for NLP tasks. Most mitigation techniques proposed to date are supervised—relying on adversarial retraining to improve the robustness—which is impractical. This work introduces a novel, unsupervised detection methodology for detecting adversarial inputs to NLP classifiers. In summary, we note that minimally perturbing an input to change a model’s output—a major strength of adversarial attacks—is a weakness that leaves unique statistical marks reflected in the cumulative contribution scores of the input. Particularly, we show that the cumulative contribution score, called CF-score, of adversarial inputs is generally greater than that of the clean inputs. We thus propose Con-Detect—a Contribution based Detection method—for detecting adversarial attacks against NLP classifiers. Con-Detect can be deployed with any classifier without having to retrain it. We experiment with multiple attackers—Text-bugger, Text-fooler, PWWS—on several architectures—MLP, CNN, LSTM, Hybrid CNN-RNN, BERT—trained for different classification tasks—IMDB sentiment classification, fake-news classification, AG news topic classification—under different threat models—Con-Detect-blind attacks, Con-Detect-aware attacks, and Con-Detect-adaptive attacks—and show that Con-Detect can reduce the attack success rate (ASR) of different attacks from 100% to as low as 0% for the best cases and ≈70% for the worst case. Even in the worst case, we note a 100% increase in the required number of queries and a 50% increase in the number of words perturbed, suggesting that Con-Detect is hard to evade.

Universal backdoor attack on deep neural networks for malware detection

Backdoor attacks targeting the deep neural network are flourishing recently and are more stealthy than existing adversarial attacks. A deep understanding of the backdoor attacks targeting malware detection models is still missing. We design a highly transferable backdoor attack targeting three benchmark convolutional neural networks (CNNs) for malware detection. The designed backdoor attack involves two steps: trigger generation and trigger insertion. Firstly, based on the computation of the most significant byte sub-sequence from samples of a chosen target label, the trigger patterns are generated by training a class activation mapping-based deep neural network (CAM-DNN). Then, the byte sequence with the maximum class activation mapping score is chosen as the candidate trigger pattern. The computed trigger pattern is then inserted into an index-based place that satisfies the minimum distance between a predefined feature space to the target label. Through detailed experiments, the CAM-DNN-based backdoor considers many influential factors, including the number of backdoor triggers, the degree of perturbations applied on a single trigger pattern, the length of the inserted trigger, etc. The experiments demonstrate that the CAM-DNN-based backdoor attack achieves an 89.58% success rate on average at the cost of a 2.25% accuracy drop on clean inputs. More importantly, the poisoned malware ensures high integrity because the original malicious functions are preserved to a large extent.

Clean-label poisoning attack with perturbation causing dominant features

Numerous studies have proved that deep learning models are sensitive to attacks from adversarial examples. However, due to the high computationally demanding and limiting explanation for deep neural networks (DNNs), research on poisoning attacks against deep models is insufficient. In this work, considering the interpretation that DNNs are feature extractors, we propose a clean-label poisoning attack against DNNs by dominant image features added to the original data. Each category of clean data is augmented with imperceptible perturbations which we call “dominant image features” of another category. The dominant image features determine which class the input belongs to, while the clean input behaves like noise. Well-trained DNNs on these poisoning data will get extremely low accuracy on clean test data. The attack can make personal data unlearnable for DNNs, which preserves an individual's privacy. Besides, our data poisoning attack can be expanded into a powerful clean-label backdoor attack in that the poisoned samples and their labels are consistent for humans. Experimental results show that our proposed attack can exceed the previous advanced poisoning attacks which demonstrate the effectiveness of our presented attack. Additionally, our work provides a new perspective on the interpretation of DNNs as feature extractors.

Self-adaptive logit balancing for deep neural network robustness: Defence and detection of adversarial attacks

With the widespread applications of Deep Neural Networks (DNNs), the safety of DNNs has become a significant issue. The vulnerability of the neural networks against adversarial examples deepens concerns about the safety of DNNs applications. This paper proposed a novel defence method to improve the adversarial robustness of DNN classifiers without using adversarial training. This method introduces two new loss functions. First, a zero-cross-entropy loss is used to punish overconfidence and find the appropriate confidence for different instances. Second, a logit balancing loss is proposed to protect DNNs from non-targeted attacks by regularising incorrect classes’ logits distribution. This method achieved competitive adversarial robustness compared to advanced adversarial training methods. Meanwhile, a novel robustness diagram is proposed to analyse, interpret and visualise the robustness of DNN classifiers against adversarial attacks. Furthermore, a Log-Softmax-pattern-based adversarial attack detection method is proposed. This detection method can distinguish clean inputs and multiple adversarial attacks via one multi-classification MLP. In particular, it is state-of-the-art in identifying white-box gradient-based attacks; it achieved at least 95.5% accuracy for classifying four white-box gradient-based attacks with maximum 0.1% false positive ratio.

Climate Change and Uncertainty: An Asset Pricing Perspective

Climate change and uncertainty about its potential consequences has become a central concern for economists, investors, and policymakers alike. I use a stochastic, dynamic general equilibrium model where final output is produced using a mix of cheap, dirty inputs and expensive, clean inputs and preferences incorporate aversion to climate model misspecification to analyze the implications of climate change and climate model uncertainty on economic and financial market outcomes. I find that climate change leads to increased clean input production and reduced emissions and that there is a negative price of climate risk that is significantly amplified and increasing in magnitude as climate change increases due to aversion to climate model uncertainty. Existing empirical estimates are consistent with the model implications, highlighting the potentially significant influence of climate model uncertainty on macroeconomic and asset pricing outcomes. This paper was accepted by Elke Weber, Special Section of Management Science on Business and Climate Change. Funding: This work was supported by the Energy Policy Institute at Chicago [Dissertation Support Fellowship], the Fama-Miller Center [Liew Fama-Miller Fellowship], the National Science Foundation [Graduate Research Fellowship Award D], and the University of Chicago [Harper (Provost) Dissertation Fellowship]. Supplemental Material: The data files and online appendices are available at https://doi.org/10.1287/mnsc.2022.4635 .

Robust Automatic Modulation Classification in Low Signal to Noise Ratio

In a non-cooperative communication environment, automatic modulation classification (AMC) is an essential technology for analyzing signals and classifying different kinds of signal modulation before they are demodulated. Deep learning (DL)-based AMC has been proposed as an efficient method of achieving high classification performance. However, most current DL-AMC methods have limited generalization capabilities under varying noise conditions, especially at low signal-to-noise ratios (SNRs). Therefore, these methods can not be directly applied to practical systems. In this paper, we propose a threshold autoencoder denoiser convolutional neural network (TADCNN), which consists of a threshold autoencoder denoiser (TAD) and a convolutional neural network (CNN). TADs reduce noise power and clean input signals, which are then passed on to CNN for classification. The TAD network generally consists of three components: the batch normalization layer, the autoencoder, and the threshold denoise. The threshold denoise component uses an auto-learning threshold sub-network to compute thresholds automatically. According to experiments, AMC with TAD improved classification accuracy by 70% at low SNR compared with a model without a denoiser. Additionally, our model achieves an average accuracy of 66.64% on the RML2016.10A dataset, which is 6% to 18% higher than the current AMC model.

Towards secure private and trustworthy human-centric embedded machine learning: An emotion-aware facial recognition case study

The use of artificial intelligence (AI) at the edge is transforming every aspect of the lives of human beings from scheduling daily activities to personalized shopping recommendations. Since the success of AI is to be measured ultimately in terms of how it benefits human beings, and that the data driving the deep learning-based edge AI algorithms are intricately and intimately tied to humans, it is important to look at these AI technologies through a human-centric lens. However, despite the significant impact of AI design on human interests, the security and trustworthiness of edge AI applications are not foolproof and ethicalneither foolproof nor ethical; Moreover, social norms are often ignored duringin the design, implementation, and deployment of edge AI systems. In this paper, we make the following two contributions: Firstly, we analyze the application of edge AI through a human-centric perspective. More specifically, we present a pipeline to develop human-centric embedded machine learning (HC-EML) applications leveraging a generic human-centric AI (HCAI) framework. Alongside, we also analyzediscuss the privacy, trustworthiness, robustness, and security aspects of HC-EML applications with an insider look at their challenges and possible solutions along the way. Secondly, to illustrate the gravity of these issues, we present a case study on the task of human facial emotion recognition (FER) based on AffectNet dataset, where we analyze the effects of widely used input quantization on the security, robustness, fairness, and trustworthiness of an EML model. We find that input quantization partially degrades the efficacy of adversarial and backdoor attacks at the cost of a slight decrease in accuracy over clean inputs. By analyzing the explanations generated by SHAP, we identify that the decision of a FER model is largely influenced by features such as eyes, alar crease, lips, and jaws. Additionally, we note that input quantization is notably biased against the dark skin faces, and hypothesize that low-contrast features of dark skin faces may be responsible for the observed trends. We conclude with precautionary remarks and guidelines for future researchers.

Increasing depth, distribution distillation, and model soup: erasing backdoor triggers for deep neural networks

Deep neural networks are vulnerable to backdoor attacks, in which the adversary injects a trigger embedded set into the training process. Inputs marked with the trigger provide incorrect predictions, whereas clean inputs remain unaffected. To erase the latent triggers in models, increasing depth, distribution distillation, and model soup (ID3MS), a defensive solution that operates without prior knowledge of triggers and relies on a small clean set is introduced. The depth of the backdoor model is increased by adding fully connected layer(s) at the penultimate layer. Without a classification layer, the original backdoor and increased depth models are considered as teacher and student, respectively. The student model applies distribution distillation to refit the distribution of the clean set and erase the backdoor triggers. The distilled student model is then recovered with the classification layer and model soup is used to ensemble a collection of models generated by various fine-tuning hyperparameters. The experimental results validate the superior performance of the ID3MS compared with existing defensive techniques against several attacks across datasets.

A defense method against backdoor attacks on neural networks

Due to computational complexities of artificial neural networks (ANNs), there is an increasing demand for third parties and MLaaS (machine learning as a service) to take charge of the training procedure. Therefore, making ANNs robust against adversarial attacks has received a lot of attention. Backdoor attacks, which causes targeted mis-classification while the accuracy on clean data is not affected, are among the most efficient attacks. In this paper, we propose a method called link-pruning with scale-freeness (LPSF), in which the dormant threatening links from the neurons in the input layer to other neurons of feed-forward neural network are eliminated according to the information gained from a portion of clean input data and the essential links are strengthened by changing the fully-connected networks to scale-free structures. To the best of our knowledge, it is the first defense method that makes the network significantly robust against backdoor attack (BD) before the network is attacked. LPSF is evaluated on feed-forward neural networks and with malicious MNIST, FMNIST, handwritten Chinese characters and HODA datasets. Through LPSF strategy, we achieve a sufficiently high and stable accuracy on clean data and an exceeding reduction range of 50%−94% for attack success rate.

ASCL: Adversarial supervised contrastive learning for defense against word substitution attacks

Attacks with adversarial examples can tremendously worsen the performance of deep neural networks (DNNs). Hence, defending against such adversarial attacks is crucial for nearly all DNN-based applications. Adversarial training is an effective and extensively adopted approach for increasing the robustness of DNNs in which benign examples and their adversarial counterparts are considered together in the training stage. However, this may result in a decrease in accuracy on benign examples because it does not account for the inter-class distance of benign examples. To overcome the aforementioned dilemma, we devise a novel defense approach named adversarial supervised contrastive learning (ASCL), which combines adversarial training with supervised contrastive learning to enhance the robustness of DNN-based models while maintaining their clean accuracy. We validate the effectiveness of the proposed ASCL approach in the scenario of defending against word substitution attacks by means of extensive experiments on benchmark tasks and datasets. The experimental results show that ASCL reduces the attack success rate to 20% while maintaining the accuracy for clean inputs within a 2% margin.