Input Examples Research Articles

Adversarial Dropout for Supervised and Semi-Supervised Learning

Recently, training with adversarial examples, which are generated by adding a small but worst-case perturbation on input examples, has improved the generalization performance of neural networks. In contrast to the biased individual inputs to enhance the generality, this paper introduces adversarial dropout, which is a minimal set of dropouts that maximize the divergence between 1) the training supervision and 2) the outputs from the network with the dropouts. The identified adversarial dropouts are used to automatically reconfigure the neural network in the training process, and we demonstrated that the simultaneous training on the original and the reconfigured network improves the generalization performance of supervised and semi-supervised learning tasks on MNIST, SVHN, and CIFAR-10. We analyzed the trained model to find the performance improvement reasons. We found that adversarial dropout increases the sparsity of neural networks more than the standard dropout. Finally, we also proved that adversarial dropout is a regularization term with a rank-valued hyper-parameter that is different from a continuous-valued parameter to specify the strength of the regularization.

Perception-driven procedural texture generation from examples

Abstract Procedural textures are widely used in computer games and animations for efficiently rendering natural scenes. They are generated using mathematical functions, and users need to tune the model parameters to produce desired texture. However, unless one has a good knowledge of these procedural models, it is difficult to predict which model can produce what types of textures. This paper proposes a framework for generating new procedural textures from examples. The new texture can have the same perceptual attributes as those of the input example or re-defined by the users. To achieve this goal, we first introduce a PCA-based Convolutional Network (PCN) to effectively learn texture features. These PCN features can be used to accurately predict the perceptual scales of the input example and a procedural model that can generate the input. Perceptual scales of the input can be redefined by users and further mapped to a point in the perceptual texture space, which has been established in advance by using a training dataset. Finally, we determine the parameters of the procedural generation model by performing perceptual similarity measurement in the perceptual texture space. Extensive experiments show that our method has produced promising results.

Example image-based feature extraction for face recognition

This paper proposes a novel method for recognizing facial images based on the relative distances between an input image and example images. Example facial images can be easily collected online, and a large example database can span new possible facial variations not sufficiently learned during the learning phase. We first extract facial features using a baseline classifier that has a certain degree of accuracy. To achieve a better performance of the proposed method, we divide the collected examples into groups using a clustering method (e.g., k-means), where each clustered group contains examples with similar characteristics. We then hierarchically partition a group formed in the previous level into other groups to analyze more specific facial characteristics, which represent an example pyramid. To describe the characteristics of a group using the clustered examples, we divide the example group into a number of sub-groups. We calculate the averages of the sub-groups and select an example most similar to the average in each sub-group because we assume that the averages of the sub-groups can directly represent their characteristics. Using the selected examples, we build example code words for a novel feature extraction. The example code words are used to measure the distances to an input image and serve as anchors to analyze a facial image in the example domain. The distance values are normalized for each group at all pyramid levels, and are concatenated to form novel features for face recognition. We verified the effectiveness of the proposed example pyramid framework using well-known proposed features, including LBP, HOG, Gabor, and the deep learning method, on the LFW database, and showed that it can yield significant improvements in recognition performance.

Procedural modeling of rivers from single image toward natural scene production

The rapid and flexible design of natural environments is an important yet challenging task in graphics simulation, virtual reality, and video game productions. This is particularly difficult for natural river modeling due to its complex topology, geometric diversity, and its natural interaction with the complicated terrain. In this paper, we introduce an integrated method for example-based procedural modeling to overcome such difficulties. First, we propose a compact parametric model to represent the certain river, which inherits typical features of natural rivers such as tributary, distributary, tortuosity, possible lakes adjacent to the river. Then, we demonstrate our method for generating 3D river scene solely based on the parametric model. However, choosing appropriate parameters is a tedious undertaking in practice. To further enhance our method's functionality, we rely upon a natural river image to extract meaningful parameters toward the rapid procedural production of the new river scene. Finally, we design a new method to compare two river scenes and iteratively optimize the river network by using the simulated annealing technique. Our method can produce natural river scenes from an example river network and single terrain image with little interaction, and the synthesized scene is visually consistent with the input example in terms of feature similarity. We also demonstrate that our procedural modeling approach is highly automatic toward rapid scene production through various graphics examples.

Synthesizing program input grammars

We present an algorithm for synthesizing a context-free grammar encoding the language of valid program inputs from a set of input examples and blackbox access to the program. Our algorithm addresses shortcomings of existing grammar inference algorithms, which both severely overgeneralize and are prohibitively slow. Our implementation, GLADE, leverages the grammar synthesized by our algorithm to fuzz test programs with structured inputs. We show that GLADE substantially increases the incremental coverage on valid inputs compared to two baseline fuzzers.

Genetic Counselors' Perception of the Effect on Practice of Laws Restricting Abortion.

In 2013, twenty-two states enacted seventy provisions restricting access to abortion. The legislation restricted access to abortions by instituting more regulations on providers and facilities, by prohibiting abortion prior to viability, by restricting funding available to patients and by requiring patients to wait a mandatory time period before having a procedure. Genetic counselors are trained to provide unbiased, comprehensive information in a non-directive style in order to allow patients to exercise their reproductive freedom. We developed a survey of 37 questions for genetic counselors to gauge the potential impact these provisions will have on their ability to be a patient advocate. A total of 286 individuals completed the survey; however, not all respondents answered all questions. Qualitative questions complemented quantitative survey entries, allowing respondents to input thoughts and examples. Results indicate genetic counselors in all regions share similar professional opinions about the provisions. More genetic counselors in the South and Midwest noticed changes impacting patients since the provisions have been enacted. These regional differences correlate with the location of states that have seem the greatest increase in antiabortion provisions.

Linear programming twin support vector regression

In this paper, a new linear programming formulation of a 1-norm twin support vector regression is proposed whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton method. The idea of our formulation is to reformulate TSVR as a strongly convex problem by incorporated regularization technique and then derive a new 1-norm linear programming formulation for TSVR to improve robustness and sparsity. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point and can be easily implemented in MATLAB without using any optimization packages. The efficiency of the proposed method is demonstrated by experimental results on a number of interesting synthetic and real-world datasets.

Semi-supervised learning through adaptive Laplacian graph trimming

Graph-based semi-supervised learning (GSSL) attracts considerable attention in recent years. The performance of a general GSSL method relies on the quality of Laplacian weighted graph (LWR) composed of the similarity imposed on input examples. A key for constructing an effective LWR is on the proper selection of the neighborhood size K or ε on the construction of KNN graph or ε-neighbor graph on training samples, which constitutes the fundamental elements in LWR. Specifically, too large K or ε will result in “shortcut” phenomenon while too small ones cannot guarantee to represent a complete manifold structure underlying data. To this issue, this study attempts to propose a method, called adaptive Laplacian graph trimming (ALGT), to make an automatic tuning to cut improper inter-cluster shortcut edges while enhance the connection between intra-cluster samples, so as to adaptively fit a proper LWR from data. The superiority of the proposed method is substantiated by experimental results implemented on synthetic and UCI data sets.

Machine Learning with Big Data An Efficient Electricity Generation Forecasting System

Abstract Machine Learning (ML) is a powerful tool that can be used to make predictions on the future nature of data based on the past history. ML algorithms operate by building a model from input examples to make data-driven predictions or decisions for the future. The growing concept “Big Data” has brought much success in the field of data science; it provides data scalability in a variety of ways that empower data science. ML can also be used in conjunction with Big Data to build effective predictive systems or to solve complex data analytic problems. In this work, we propose an electricity generation forecasting system that could predict the amount of power required at a rate close to the electricity consumption for the United States. The proposed scheme uses Big Data analytics to process the data collected on power management in the past 20 years. Then, it applies a ML model to train the system for the prediction stage. The model can forecast future power generation based on the collected data, and our test results show that the proposed system can predict the required power generation close to 99% of the actual usage. Our results indicate that the ML with Big Data can be integrated in forecasting techniques to improve the efficiency and solve complex data analytic problems existing in the power management systems.

Enhancing evolutionary fuzzy systems for multi-class problems: Distance-based relative competence weighting with truncated confidences (DRCW-TC)

Classification problems with multiple classes suppose a challenge in Data Mining tasks. There is a difficulty inherent to the learning process when trying to find the most adequate discrimination functions among the different concepts within the dataset. Using Fuzzy Rule Based Classification Systems in general, and Evolutionary Fuzzy Systems in particular, provide the advantage of describing smoother borderline areas, thanks to the linguistic label-based representation.In multi-classification, the pairwise learning approach (One-vs-One) has gained a notorious attention. However, there is certain dependence between the goodness of the confidence degrees or scores of binary classifiers, and the final performance shown by the global model. Regarding this fact, the problem of non-competent classifiers is of special relevance. It occurs when a binary classifier outputs a positive score for a couple of classes unrelated with the input example, which may degrade the final accuracy. Precisely, the previously exposed properties of fuzzy classifiers make them more prone to the former condition.In this paper, we propose an extension of the distance-based combination strategy to overcome this non-competence problem. It is based on the truncation of the confidence degrees of the classes prior to the distance-based tuning. This allows taking advantage of the good classification abilities of Evolutionary Fuzzy Systems, while diminishing the adverse effect of the aforementioned non-competence. Experimental results, using FARC-HD with overlap functions as the fuzzy learning algorithm, show that this new adaptation of the Distance-based Relative Competence Weighting model outperforms both the OVO and standard distance-based approaches, and it is competitive with robust classifiers such as Support Vector Machines.

An Improved Multi-Level Adaptive Control Algorithm Based on Parameters Self-Tuning for a Class of SISO Nonlinear Systems

In this paper, a multi-level adaptive control algorithm using parameters self-tuning is proposed for improving the tracking performance of a class of SISO nonlinear systems. This scheme is comprised of the pseudo partial derivative estimation law and the adaptive control law. The adaptive control law is derived from a novel higher order weighted one-step-ahead criterion function which exploits more previous control information. In order to identify the laws’ penalty factors which partly imply the dynamics of the system, an approach of parameters self-tuning is proposed with on-line recursive gradient algorithm. This design is model free and depends directly on the pseudo partial derivative updating on-line with the input and output data Simulation examples are provided to illustrate the effectiveness of the proposed method.

A computer-based automated algorithm for assessing acinar cell loss after experimental pancreatitis.

The change in exocrine mass is an important parameter to follow in experimental models of pancreatic injury and regeneration. However, at present, the quantitative assessment of exocrine content by histology is tedious and operator-dependent, requiring manual assessment of acinar area on serial pancreatic sections. In this study, we utilized a novel computer-generated learning algorithm to construct an accurate and rapid method of quantifying acinar content. The algorithm works by learning differences in pixel characteristics from input examples provided by human experts. HE-stained pancreatic sections were obtained in mice recovering from a 2-day, hourly caerulein hyperstimulation model of experimental pancreatitis. For training data, a pathologist carefully outlined discrete regions of acinar and non-acinar tissue in 21 sections at various stages of pancreatic injury and recovery (termed the “ground truth”). After the expert defined the ground truth, the computer was able to develop a prediction rule that was then applied to a unique set of high-resolution images in order to validate the process. For baseline, non-injured pancreatic sections, the software demonstrated close agreement with the ground truth in identifying baseline acinar tissue area with only a difference of 1%±0.05% (p = 0.21). Within regions of injured tissue, the software reported a difference of 2.5%±0.04% in acinar area compared with the pathologist (p = 0.47). Surprisingly, on detailed morphological examination, the discrepancy was primarily because the software outlined acini and excluded inter-acinar and luminal white space with greater precision. The findings suggest that the software will be of great potential benefit to both clinicians and researchers in quantifying pancreatic acinar cell flux in the injured and recovering pancreas.

Application of smoothing techniques for linear programming twin support vector machines

In this paper, a new unconstrained minimization problem formulation is proposed for linear programming twin support vector machine (TWSVM) classifiers. The proposed formulation leads to two smaller-sized unconstrained minimization problems having their objective functions piecewise differentiable. However, since their objective functions contain the non-smooth "plus" function, two new smoothing approaches are assumed to solve the proposed formulation, and then apply Newton-Armijo algorithm. The idea of our formulation is to reformulate TWSVM as a strongly convex problem by incorporated regularization techniques and then derive smooth 1-norm linear programming formulation for TWSVM to improve robustness. One significant advantage of our proposed algorithm over TWSVM is that the structural risk minimization principle is implemented in the primal problems which embodies the marrow of statistical learning theory. In addition, the solution of two modified unconstrained minimization problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems in TWSVM and TBSVM, which leads to extremely simple and fast algorithm. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point that can be easily implemented in MATLAB without using any optimization packages. The performance of our proposed method is verified experimentally on several benchmark and synthetic datasets. Experimental results show the effectiveness of our methods in both training time and classification accuracy.

The PAG Crowd: A Graph Based Approach for Efficient Data‐Driven Crowd Simulation

AbstractWe present a data‐driven method for the real‐time synthesis of believable steering behaviours for virtual crowds. The proposed method interlinks the input examples into a structure we call the perception‐action graph (PAG) which can be used at run‐time to efficiently synthesize believable virtual crowds. A virtual character's state is encoded using a temporal representation, the Temporal Perception Pattern (TPP). The graph nodes store groups of similar TPPs whereas edges connecting the nodes store actions (trajectories) that were partially responsible for the transformation between the TPPs. The proposed method is being tested on various scenarios using different input data and compared against a nearest neighbours approach which is commonly employed in other data‐driven crowd simulation systems. The results show up to an order of magnitude speed‐up with similar or better simulation quality.

Robust and Sparse Linear Programming Twin Support Vector Machines

In this paper, we propose a new linear programming formulation of exact 1-norm twin support vector machine (TWSVM) for classification whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton–Armijo algorithm. The idea of our formulation is to reformulate TWSVM as a strongly convex problem by incorporated regularization techniques and then derive an exact 1-norm linear programming formulation for TWSVM to improve robustness and sparsity. The solution of two modified unconstrained minimization problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems in TWSVM and TBSVM, which leads to extremely simple and fast algorithm. One significant advantage of our proposed method is the implementation of structural risk minimization principle. However, only empirical risk is considered in the primal problems of TWSVM due to its complex structure and thus may incur overfitting and suboptimal in some cases. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point that can be easily implemented in MATLAB without using any optimization packages. Computational comparisons of our proposed method against original TWSVM, GEPSVM and SVM have been made on both synthetic and benchmark datasets. Experimental results show that our method is better or comparable in both computation time and classification accuracy.

SAFE: Structure‐aware facade editing

AbstractMany man‐made objects, in particular building facades, exhibit dominant structural relations such as symmetry and regularity. When editing these shapes, a common objective is to preserve these relations. However, often there are numerous plausible editing results that all preserve the desired structural relations of the input, creating ambiguity. We propose an interactive facade editing framework that explores this structural ambiguity. We first analyze the input in a semi‐automatic manner to detect different groupings of the facade elements and the relations among them. We then provide an incremental editing process where a set of variations that preserve the detected relations in a particular grouping are generated at each step. Starting from one input example, our system can quickly generate various facade configurations.

An Instance of Constructing Conceptual Model of Statistical Machine Translation for Processing Natural Language

The basic frame of Example based machine translation is concerned in this paper. Some key issues, such as bilingual alignment, similarity measure between input sentence and example, and template acquisition, are introduced.

Surprise! Infants consider possible bases of generalization for a single input example.

Infants have been shown to generalize from a small number of input examples. However, existing studies allow two possible means of generalization. One is via a process of noting similarities shared by several examples. Alternatively, generalization may reflect an implicit desire to explain the input. The latter view suggests that generalization might occur when even a single input example is surprising, given the learner's current model of the domain. To test the possibility that infants are able to generalize based on a single example, we familiarized 9-month-olds with a single three-syllable input example that contained either one surprising feature (syllable repetition, Experiment 1) or two features (repetition and a rare syllable, Experiment 2). In both experiments, infants generalized only to new strings that maintained all of the surprising features from familiarization. This research suggests that surprise can promote very rapid generalization.

A numerical solution to the point kinetic equations using Taylor–Lie series combined with a scaling and squaring technique

A system of differential equations is derived to model the dynamics of neutron density and the delayed neutron precursors within a point kinetics equation modeling framework for a nuclear reactor. The point kinetic equations are mathematically characterized as stiff, occasionally nonlinear, ordinary differential equations, posing significant challenges when numerical solutions are sought, and traditionally resulting in the need for smaller time step intervals within various computational schemes. In light of the above realization, the present paper proposes a new discretization/simulation method that is: (i) conceptually inspired by system-theoretic notions used in the analysis of sampled-date system dynamics (discrete-time system dynamics) under potentially irreducible large sampling periods/time steps, and (ii) technically based on Taylor–Lie series and the zero-order hold (ZOH). Within the proposed time discretization framework, the sampled-data representation of the original point kinetic system of equations is derived for an arbitrary size of the time-step used. Furthermore, within the context of the proposed approach, the integration of a scaling-and-squaring technique is pursued for the attainment of further performance enhancement of the proposed simulation technique. The performance of the proposed approach is evaluated in several case studies involving step and ramp-like reactivity profiles as well as multiple input examples that simultaneously account for variations in reactivity and neutron source function profiles. In particular, it is demonstrated, that by applying the proposed method, the inherent stiffness problem associated with the simulation challenges of the point kinetic equations can be adequately addressed within a wide range of reactor operating conditions, including large sampling periods dictated by physical and/or technical limitations associated with the current state of sensor and digital reactor control system technology.

Time discretization of the point kinetic equations using matrix exponential method and First-Order Hold

A system of nonlinear differential equations is derived to model the dynamics of neutron density and the delayed neutron precursors within a point kinetics equation modeling framework for a nuclear reactor. The point kinetic equations are mathematically characterized as stiff, occasionally nonlinear, ordinary differential equations, posing significant challenges when numerical solutions are sought and traditionally resulting in the need for smaller time step intervals within various computational schemes. In light of the above realization, the present paper proposes a new discretization method inspired by system-theoretic notions and technically based on a combination of the matrix exponential method (MEM) and the First-Order Hold (FOH) assumption. Under the proposed time discretization structure, the sampled-data representation of the nonlinear point kinetic system of equations is derived. The performance of the proposed time discretization procedure is evaluated using several case studies with sinusoidal reactivity profiles and multiple input examples (reactivity and neutron source function). It is shown, that by applying the proposed method under a First-Order Hold for the neutron density and the precursor concentrations at each time step interval, the stiffness problem associated with the point kinetic equations can be adequately addressed and resolved. Finally, as evidenced by the aforementioned detailed simulation studies, the proposed method retains its validity and accuracy for a wide range of reactor operating conditions, including large sampling periods dictated by physical and/or technical limitations associated with the current state of sensor and digital reactor control system technology.