Input Examples Research Articles

A comparative analysis of powder ENDOR spectra of aromatic and aliphatic radicals by exact and 1st order simulation

Exact and perturbation methods were employed in simulations of powder ENDOR spectra to obtain the anisotropic hyperfine ( hfc ) and nuclear quadrupole ( nqc ) coupling constants of certain organic radicals of interest in fundamental research and in applications in radiation research, surface chemistry and biophysics. The principal hfc values of the ring protons and methyl substituents for several aromatic cations trapped in disordered matrices might be more accurate than those previously reported using regular EPR. Only one of the earlier assignments of the naphthalene cation spectrum was in acceptable agreement with the simulations. The proton couplings at the β-position of alkyl radicals were deduced by simulations while the spectrum due to α-couplings with appreciable anisotropy was weak. Accurate simulation of the 14 N (I = 1) spectra in bio-radicals was obtained by adjustment of the relative orientation of the principal hfc and nuclear quadrupole coupling ( nqc ) tensors as well as the principal values. Adjustment of the excitation width parameter employed in the software was also required in a few cases to improve the agreement with the experimental spectra. The hfc patterns due to matrix nuclei ( 1 H and 7 Li) around the radicals of X-irradiated samples were simulated to elucidate the nature of the trapping sites in materials used for EPR dosimetry. ENDOR simulation programs known to us are presented in an Appendix. The performance of the ENDORF2 program used in previous works was examined by comparison with exact treatment. Input and output examples, source and executable codes used in this work can be downloaded at https://github.com/EndorF2/Simulation .

An Adversarial Framework for Generating Unseen Images by Activation Maximization

Activation maximization (AM) refers to the task of generating input examples that maximize the activation of a target class of a classifier, which can be used for class-conditional image generation and model interpretation. A popular class of AM method, GAN-based AM, introduces a GAN pre-trained on a large image set, and performs AM over its input random seed or style embeddings, so that the generated images are natural and adversarial attacks are prevented. Most of these methods would require the image set to contain some images of the target class to be visualized. Otherwise they tend to generate other seen class images that most maximizes the target class activation. In this paper, we aim to tackle the case where information about the target class is completely removed from the image set. This would ensure that the generated images truly reflect the target class information residing in the classifier, not the target class information in the image set, which contributes to a more faithful interpretation technique. To this end, we propose PROBEGAN, a GAN-based AM algorithm capable of generating image classes unseen in the image set. Rather than using a pre-trained GAN, PROBEGAN trains a new GAN with AM explicitly included in its training objective. PROBEGAN consists of a class-conditional generator, a seen-class discriminator, and an all-class unconditional discriminator. It can be shown that such a framework can generate images with the features of the unseen target class, while retaining the naturalness as depicted in the image set. Experiments have shown that PROBEGAN can generate unseen-class images with much higher quality than the baselines. We also explore using PROBEGAN as a model interpretation tool. Our code is at https://github.com/csmiler/ProbeGAN/.

Optimizing Binary Decision Diagrams with MaxSAT for Classification

The growing interest in explainable artificial intelligence(XAI) for critical decision making motivates the need for interpretable machine learning (ML) models. In fact, due to their structure (especially with small sizes), these models are inherently understandable by humans. Recently, several exact methods for computing such models are proposed to overcome weaknesses of traditional heuristic methods by providing more compact models or better prediction quality. Despite their compressed representation of Boolean functions, Binary decision diagrams (BDDs) did not gain enough interest as other interpretable ML models. In this paper, we first propose SAT-based models for learning optimal BDDs (in terms of the number of features) that classify all input examples. Then, we lift the encoding to a MaxSAT model to learn optimal BDDs in limited depths, that maximize the number of examples correctly classified. Finally, we tackle the fragmentation problem by introducing a method to merge compatible subtrees for the BDDs found via the MaxSAT model. Our empirical study shows clear benefits of the proposed approach in terms of prediction quality and interpretability (i.e., lighter size) compared to the state-of-the-art approaches.

A Sparsity-Limitation-Based High-Dimensional Distribution Searching Algorithm for Adversarial Attack

Deep neural networks-based image classification systems could suffer from adversarial attack algorithms, which generate input examples by adding deliberately crafted yet imperceptible noise to original inputs. To reduce the impact on human visual sense and to ensure adversarial attack ability, the input image needs to be modified by pixels in considerable iterations which is time consuming. By using sparse mapping network to map the input into a higher dimensional space, searching space of adversarial perturbation distribution is enlarged to better acquire perturbation information. Taking both searching speed and searching effectiveness into consideration, sparsity limitation is introduced to suppress unnecessary neurons during parameter updating process. Based on different eye sensitivity of different colors, maps of each color channel are disturbed by perturbations with different strengths to reduce visual perception. Numerical experiments confirm that compared with the state-of-the-art adversarial attack algorithms, the proposed SparseAdv performs a relatively high attack ability, better imperceptible visualization, and faster generation speed.

Texture Synthesizability Assessment via Deep Siamese-Type Network

Example-based texture synthesis plays a significant role in many fields, including computer graphics, computer vision, multimedia, and image and video editing and processing. However, it is not easy for all textures to synthesize high-quality outputs of any size from a small input example. Hence, the assessment of the synthesizability of the example textures deserves more attention. Inspired by the broad studies in image quality assessment, we propose a texture synthesizability assessment approach based on a deep Siamese-type network. To our best knowledge, this is the first attempt to evaluate the synthesizability of sample textures through end-to-end training. We first train a Siamese-type network to compare the example texture and the synthesized texture in terms of their similarity and then transfer the experience knowledge obtained in the Siamese-type network to a traditional CNN by fine-tuning, so that to give an absolute score to a single example texture, representing its synthesizability. Not relying on laborious human selection and annotation, these synthesized textures can be generated automatically by example-based synthesis algorithms. We demonstrate that our approach is completely data-driven without hand-crafted features and/or prior knowledge in the field of expertise. Experiments show that our approach improves the accuracy of texture synthesizability assessment qualitatively and quantitatively and outperforms the manual feature-based method.

Visual Analytics for Predicting Disease Outcomes Using Laboratory Test Results

Laboratory tests play an essential role in the early and accurate diagnosis of diseases. In this paper, we propose SUNRISE, a visual analytics system that allows the user to interactively explore the relationships between laboratory test results and a disease outcome. SUNRISE integrates frequent itemset mining (i.e., Eclat algorithm) with extreme gradient boosting (XGBoost) to develop more specialized and accurate prediction models. It also includes interactive visualizations to allow the user to interact with the model and track the decision process. SUNRISE helps the user probe the prediction model by generating input examples and observing how the model responds. Furthermore, it improves the user’s confidence in the generated predictions and provides them the means to validate the model’s response by illustrating the underlying working mechanism of the prediction models through visualization representations. SUNRISE offers a balanced distribution of processing load through the seamless integration of analytical methods with interactive visual representations to support the user’s cognitive tasks. We demonstrate the usefulness of SUNRISE through a usage scenario of exploring the association between laboratory test results and acute kidney injury, using large provincial healthcare databases from Ontario, Canada.

Training with growing sets: A comparative study

AbstractBeing similar to and inspired from the process of human education, curriculum learning methods ‐or CL methods‐sort the input examples from easy to difficult, then add them to the training set in that order. Considering the fact that CL research is most concerned with determining the direction (from easy to difficult vs. from difficult to easy) and the criteria of this sorting, vast and various studies have emerged in the literature addressing both types of sorting. However, this results in a contradiction that demands finding a common aspect of ordering in both directions. This study argues that this required common aspect lies in the gradual enlargement of training. In other words, it is claimed that the success of CL methods does not depend on which criteria or in which direction the ordering is made. Extensive experiments have been conducted on various datasets using different deep learning models in order to test this claim. It was observed that random ordering had achieved competitive results with CL methods. Moreover, random ordering proved to be faster than other CL methods as it eliminates the cost of sorting computation. Based on these results, using random ordered growing sets as a baseline in future CL studies is recommended. Moreover, the possibly to improve the optimization performance via training with growing sets in theoretical perspective is also explained.

White-box attacks on hate-speech BERT classifiers in German with explicit and implicit character-level defense

Attention-based transformer models have achieved state-of-the-art results in natural language processing (NLP). However, recent work shows that the underlying attention mechanism can be exploited by adversaries to craft malicious inputs designed to induce spurious outputs, thereby harming model performance and trustworthiness. Unlike in the vision domain, the literature examining neural networks under adversarial conditions in the NLP domain is limited and most of it focuses mainly on the English language. In this article, we first analyze the adversarial robustness of Bidirectional Encoder Representations from Transformers (BERT) models for German data sets. Second, we introduce two novel NLP attacks: a character-level and a word-level attacks, both of which utilize attention scores to calculate where to inject character-level and word-level noise, respectively. Finally, we present two defense strategies against the attacks above. The first implicit character-level defense is a variant of adversarial training, which trains a new classifier capable of abstaining/rejecting certain (ideally adversarial) inputs. The other explicit character-level defense learns a latent representation of the complete training data vocabulary and then maps all tokens of an input example to the same latent space, enabling the replacement of all out-of-vocabulary tokens with the most similar in-vocabulary tokens based on the cosine similarity metric.

Performing Genome-Wide Association Studies with Multiple Models Using GAPIT.

Genome-wide association study (GWAS) is based on the linkage disequilibrium (LD) between phenotypes and genetic markers covering the whole genome. Besides the genetic linkage between the genetic markers and the causal mutations, many other factors contribute to the LD, including selection and nonrandom mating formatting population structure. Many methods have been developed with accompany of corresponding software such as multiple loci mixed model (MLMM). There are software packages that implement multiple methods to reduce the learning curve. One of them is the Genomic Association and Prediction Integrated Tool (GAPIT), which implemented eight models including GLM (General Linear Model), Mixed Linear Model (MLM), Compressed MLM, MLMM, SUPER (Settlement of mixed linear models Under Progressively Exclusive Relationship), FarmCPU (Fixed and random model Circulating Probability Unification), and BLINK (Bayesian-information and Linkage-disequilibrium Iteratively Nested Keyway). Besides the availability of multiple models, GAPIT provides comprehensive functions for data quality control, data visualization, and publication-ready quality graphic outputs, such as Manhattan plots in rectangle and circle formats, quantile-quantile (QQ) plots, principal component plots, scatter plot of minor allele frequency against GWAS signals, plots of LD between associated markers and the adjacent markers. GAPIT developers and users established a community through the GAPIT forum ( https://groups.google.com/g/gapit-forum ) with over 600 members for asking questions, making comments, and sharing experiences. In this chapter, we detail the GAPIT functions, input data frame, output files, and example codes for each GWAS model. We also interpret parameters, functional algorithms, and modules of GAPIT implementation.

Prediksi Pemilihan Jurusan Siswa Kelas 1 SMK Menggunakan Jaringan Syaraf Tiruan dengan Metode Backpropagation

Student learning potential can be seen or meansured in various ways, one of them by concluding it’s success in learning. The value of subjects is one indicator that isused as a references to conclude studentlearning success academically. Artificial Neural Networks have the ability to learn and produce rules or operations from some input examples that are entered and make predictions. Backpropagation isone method that is often used in solving complex problems because the network with this method is trained using guided learning methods.

Review of machine learning applications to power systems studies

The complexity of electric power networks from generation, transmission and distribution stations in modern times has resulted to generation of big and more complex data that requires more technical and mathematical analysis because it deals with monitoring, supervisory control and data acquisition all in real time. This has necessitated the need for more accurate analysis and predictions in power systems studies especially under transient, uncertainty or emergency conditions without interference of humans. This is necessary so as to minimize errors with the aim targeted towards improving the overall performance and the need to use more technical but very intelligent predictive tools has become very relevant. Machine learning (ML) is a powerful tool which can be utilized to make accurate predictions about the future nature of data based on past experiences. ML algorithms operate by building a model (mathematical or pictorial) from input examples to make data driven predictions or decisions for the future. ML can be used in conjunction with big data to build effective predictive systems or to solve complex data analytic problems. Electricity generation forecasting systems that could predict the amount of power required at a rate close to the electricity consumption have been proposed in several works. This study seeks to review machine learning applications to power system studies. This paper reviewed applications of ML tools in power systems studies.

Research Progress and Challenges on Application-Driven Adversarial Examples: A Survey

Great progress has been made in deep learning over the past few years, which drives the deployment of deep learning–based applications into cyber-physical systems. But the lack of interpretability for deep learning models has led to potential security holes. Recent research has found that deep neural networks are vulnerable to well-designed input examples, called adversarial examples . Such examples are often too small to detect, but they completely fool deep learning models. In practice, adversarial attacks pose a serious threat to the success of deep learning. With the continuous development of deep learning applications, adversarial examples for different fields have also received attention. In this article, we summarize the methods of generating adversarial examples in computer vision, speech recognition, and natural language processing and study the applications of adversarial examples. We also explore emerging research and open problems.

Research on an Adaptive Neural Network K-Pixel Adversarial Example Generation Algorithm

Neural network technology has achieved good results in many tasks, such as image classification. However, for some input examples of neural networks, after the addition of designed and imperceptible perturbations to the examples, these adversarial examples can change the output results of the original examples. For image classification problems, we derive low-dimensional attack perturbation solutions on multidimensional linear classifiers and extend them to multidimensional nonlinear neural networks. Based on this, a new adversarial example generation algorithm is designed to modify a specified number of pixels. The algorithm adopts a greedy iterative strategy, and gradually iteratively determines the importance and attack range of pixel points. Finally, experiments demonstrate that the algorithm-generated adversarial example is of good quality, and the effects of key parameters in the algorithm are also analyzed.

Beating Attackers At Their Own Games: Adversarial Example Detection Using Adversarial Gradient Directions

Adversarial examples are input examples that are specifically crafted to deceive machine learning classifiers. State-of-the-art adversarial example detection methods characterize an input example as adversarial either by quantifying the magnitude of feature variations under multiple perturbations or by measuring its distance from estimated benign example distribution. Instead of using such metrics, the proposed method is based on the observation that the directions of adversarial gradients when crafting (new) adversarial examples play a key role in characterizing the adversarial space. Compared to detection methods that use multiple perturbations, the proposed method is efficient as it only applies a single random perturbation on the input example. Experiments conducted on two different databases, CIFAR-10 and ImageNet, show that the proposed detection method achieves, respectively, 97.9% and 98.6% AUC-ROC (on average) on five different adversarial attacks, and outperforms multiple state-of-the-art detection methods. Results demonstrate the effectiveness of using adversarial gradient directions for adversarial example detection.

EnsembleFool: A method to generate adversarial examples based on model fusion strategy

Deep neural networks have been shown vulnerable to adversarial attacks launched by adversarial examples. These examples’ transferability makes an attack in the real-world feasible, which poses a security threat to deep learning. Considering the limited representation capacity of a single deep model, the transferability of an adversarial example generated by a single attack model would cause the failure of attacking other different models. In this paper, we propose a new adversarial attack method, named EnsembleFool, which flexibly integrates multiple models to enhance adversarial examples’ transferability. Specifically, the model confidence concerning an input example reveals the risk of a successful attack. In an iterative attacking case, the result of a previous attack could guide us to enforce a new attack that possesses a higher probability of success. Regarding this, we design a series of integration strategies to improve the adversarial examples in each iteration. Extensive experiments on ImageNet indicate that the proposed method has superior attack performance and transferability than state-of-the-art methods.

AC/DC Analysis: Broad and Comprehensive Approach to Analyze Infrared Intensities at the Atomic Level.

We present a complete theoretical protocol to partition infrared intensities into terms owing to individual atoms by two different but related approaches: the atomic contributions (ACs) show how the entire molecular vibrational motion affects the electronic structure of a single atom and the total infrared intensity. On the other hand, the dynamic contributions (DCs) show how the displacement of a single atom alters the electronic structure of the entire molecule and the total intensity. The two analyses are complementary ways of partitioning the same total intensity and conserve most of the features of the total intensity itself. Combined, they are called the AC/DC analysis. These can be further partitioned following the CCTDP (or CCT) models according to the population analysis chosen by the researcher. The main conceptual features of the equations are highlighted, and representative numerical results are shown to support the interpretation of the equations. The results are invariant to rotation and translation and can readily be extended to molecules of any size, shape, or symmetry. Although the AC/DC analysis requires the choice of a charge model, all charge models that correctly reproduce the total molecular dipole moment can be used. A fully automated protocol managed by the Placzek program is made available, free of charge and with input examples.

Regularizing deep networks with prior knowledge: A constraint-based approach

Deep Learning architectures can develop feature representations and classification models in an integrated way during training. This joint learning process requires large networks with many parameters, and it is successful when a large amount of training data is available. Instead of making the learner develop its entire understanding of the world from scratch from the input examples, the injection of prior knowledge into the learner seems to be a principled way to reduce the amount of require training data, as the learner does not need to induce the rules from the data. This paper presents a general framework to integrate arbitrary prior knowledge into learning. The domain knowledge is provided as a collection of first-order logic (FOL) clauses, where each task to be learned corresponds to a predicate in the knowledge base. The logic statements are translated into a set of differentiable constraints, which can be integrated into the learning process to distill the knowledge into the network, or used during inference to enforce the consistency of the predictions with the prior knowledge. The experimental results have been carried out on multiple image datasets and show that the integration of the prior knowledge boosts the accuracy of several state-of-the-art deep architectures on image classification tasks.

Generalization ability of online pairwise support vector machine

Pairwise classification depends on two input examples instead of one example and predicts whether the two input examples belong to the same class or to different classes. In this paper we investigate online pairwise support vector machine (OPSVM) algorithm with independently and identically distributed (i.i.d.) and non-i.i.d. samples. We first establish the convergence rates of the last iteration for OPSVM algorithm and obtain the fast convergence rates of OPSVM algorithm with strongly mixing sequence or i.i.d. samples for the case of polynomially decaying step size. We also introduce a novel OPSVM algorithm based on Markov selective sampling (OPSVM-MSS). The experimental results based on benchmark repository display that the classifier induced by OPSVM-MSS algorithm not only has smaller misclassification rate, but also the sampling and training total time is shorter compared to the classifier induced by the classical OPSVM algorithm based on randomly independent sampling (OPSVM-RIS).

Adversarial examples: attacks and defenses in the physical world

Deep learning technology has become an important branch of artificial intelligence. However, researchers found that deep neural networks, as the core algorithm of deep learning technology, are vulnerable to adversarial examples. The adversarial examples are some special input examples which were added small magnitude and carefully crafted perturbations to yield erroneous results with extremely confidence. Hence, they bring serious security risks to deep-learning-based systems. Furthermore, adversarial examples exist not only in the digital world, but also in the physical world. This paper presents a comprehensive overview of adversarial attacks and defenses in the real physical world. First, we reviewed these works that can successfully generate adversarial examples in the digital world, analyzed the challenges faced by applications in real environments. Then, we compare and summarize the work of adversarial examples on image classification tasks, target detection tasks, and speech recognition tasks. In addition, the relevant feasible defense strategies are summarized. Finally, relying on the reviewed work, we propose potential research directions for the attack and defense of adversarial examples in the physical world.

White-Box Attacks on Hate-speech BERT Classifiers in German with Explicit and Implicit Character Level Defense

Attention based Transformer models have achieved state-of-the-art results in natural language processing (NLP). However, recent work shows that the underlying attention mechanism can be exploited by adversaries to craft malicious inputs designed to induce spurious outputs, thereby harming model performance and trustworthiness. Unlike in the vision domain, the literature examining neural networks under adversarial conditions in the NLP domain is limited and most of it focuses mainly on the English language. In this paper, we first analyze the adversarial robustness of Bidirectional Encoder Representations from Transformers (BERT) models for German datasets. Second, we introduce two novel NLP attacks. Namely, a character-level and a word-level attacks, both of which utilize attention scores to calculate where to inject character-level and word-level noise, respectively. Finally, we present two defense strategies against the attacks above. The first implicit character-level defense is a variant of adversarial training, which trains a new classifier capable of abstaining/rejecting certain (ideally adversarial) inputs. The other explicit character-level defense learns a latent representation of the complete training data vocabulary and then maps all tokens of an input example to the same latent space, enabling the replacement of all out of vocabulary tokens with the most similar in-vocabulary tokens based on the cosine similarity metric.