Pairs Of Instances Research Articles

Bayesian network classifiers for probability-based metrics

A large number of distance metrics have been proposed to measure the difference of two instances. Among these metrics, Short and Fukunaga metric (SFM) and minimum risk metric (MRM) are two probability-based metrics which are widely used to find reasonable distance between each pair of instances with nominal attributes only. For simplicity, existing works use naive Bayesian (NB) classifiers to estimate class membership probabilities in SFM and MRM. However, it has been proved that the ability of NB classifiers to class probability estimation is poor. In order to scale up the classification performance of NB classifiers, many augmented NB classifiers are proposed. In this paper, we study the class probability estimation performance of these augmented NB classifiers and then use them to estimate the class membership probabilities in SFM and MRM. The experimental results based on a large number of University of California, Irvine (UCI) data-sets show that using these augmented NB classifiers to estimate the class membership probabilities in SFM and MRM can significantly enhance their generalisation ability.

Integrating local and global topological structures for semi-supervised dimensionality reduction

Dimensionality reduction plays an important role in many machine learning tasks. This paper studies semi-supervised dimensionality reduction using pairwise constraints. In this setting, domain knowledge is given in the form of pairwise constraint, which specifies whether a pair of instances belongs to the same class (must-link constraint) or different classes (cannot-link constraint). In this paper, a novel semi-supervised dimensionality reduction method called LGS3DR is proposed, which can integrate both local and global topological structures of the data as well as pairwise constraints. The LGS3DR method is effective and has a closed form solution. Experiments on data visualization and face recognition show that LGS3DR is superior to many existing dimensionality reduction methods.

Selective Value Difference Metric

Value Difference Metric (VDM) is one of the widely used distance functions to define the distance between a pair of instances with nominal attributes only. Many approaches have been proposed to improve the performance of VDM. In this paper, we focus on the attribute selection approach and propose another improved Value Difference Metric. We call it Selective Value Difference Metric (SVDM). In order to learn SVDM, we investigate the attribute independence assumption held by VDM and then single out two effective attribute selection methods for SVDM. The experimental results on 36 UCI benchmark datasets validate the effectiveness of the proposed SVDM.

An Improved Cop-Kmeans Clustering for Solving Constraint Violation Based on MapReduce Framework

Clustering with pairwise constraints has received much attention in the clustering community recently. Particularly, must-link and cannot-link constraints between a given pair of instances in the data set are common prior knowledge incorporated in many clustering algorithms today. This approach has been shown to be successful in guiding a number of famous clustering algorithms towards more accurate results. However, recent work has also shown that the incorporation of must-link and cannot-link constraints makes clustering algorithms too much sensitive to “the assignment order of instances” and therefore results in consequent constraint violation. The major contributions of this paper are two folds. One is to address the issue of constraint violation in Cop-Kmeans by emphasizing a sequenced assignment of cannot-link instances after conducting a Breadth-First Search of the cannot-link set. The other is to reduce the computational complexity of Cop-Kmeans for massive data sets by adopting a MapReduce Framework. Experimental results show that our approach performs well on massive data sets while may overcome the problem of constraint violation.

PairMotif+: A Fast and Effective Algorithm for De Novo Motif Discovery in DNA sequences

The planted (l, d) motif search is one of the most widely studied problems in bioinformatics, which plays an important role in the identification of transcription factor binding sites in DNA sequences. However, it is still a challenging task to identify highly degenerate motifs, since current algorithms either output the exact results with a high computational cost or accomplish the computation in a short time but very often fall into a local optimum. In order to make a better trade-off between accuracy and efficiency, we propose a new pattern-driven algorithm, named PairMotif+. At first, some pairs of l-mers are extracted from input sequences according to probabilistic analysis and statistical method so that one or more pairs of motif instances are included in them. Then an approximate strategy for refining pairs of l-mers with high accuracy is adopted in order to avoid the verification of most candidate motifs. Experimental results on the simulated data show that PairMotif+ can solve various (l, d) problems within an hour on a PC with 2.67 GHz processor, and has a better identification accuracy than the compared algorithms MEME, AlignACE and VINE. Also, the validity of the proposed algorithm is tested on multiple real data sets.

A New Unsupervised Feature Ranking Method for Gene Expression Data Based on Consensus Affinity

Feature selection is widely established as one of the fundamental computational techniques in mining microarray data. Due to the lack of categorized information in practice, unsupervised feature selection is more practically important but correspondingly more difficult. Motivated by the cluster ensemble techniques, which combine multiple clustering solutions into a consensus solution of higher accuracy and stability, recent efforts in unsupervised feature selection proposed to use these consensus solutions as oracles. However,these methods are dependent on both the particular cluster ensemble algorithm used and the knowledge of the true cluster number. These methods will be unsuitable when the true cluster number is not available, which is common in practice. In view of the above problems, a new unsupervised feature ranking method is proposed to evaluate the importance of the features based on consensus affinity. Different from previous works, our method compares the corresponding affinity of each feature between a pair of instances based on the consensus matrix of clustering solutions. As a result, our method alleviates the need to know the true number of clusters and the dependence on particular cluster ensemble approaches as in previous works. Experiments on real gene expression data sets demonstrate significant improvement of the feature ranking results when compared to several state-of-the-art techniques.

Pre-organizing Shape Instances for Landmark-Based Shape Correspondence

The major challenge in constructing a statistical shape model for a structure is shape correspondence, which identifies a set of corresponded landmarks across a population of shape instances to accurately estimate the underlying shape variation. Both global or pairwise shape-correspondence methods have been developed to automatically identify the corresponded landmarks. For global methods, landmarks are found by optimizing a comprehensive objective function that considers the entire population of shape instances. While global methods can produce very accurate shape correspondence, they tend to be very inefficient when the population size is large. For pairwise methods, all shape instances are corresponded to a given template independently. Therefore, pairwise methods are usually very efficient. However, if the population exhibits a large amount of shape variation, pairwise methods may produce very poor shape correspondence. In this paper, we develop a new method that attempts to address the limitations of global and pairwise methods. In particular, we first construct a shape tree to globally organize the population of shape instances by identifying similar shape instance pairs. We then perform pairwise shape correspondence between such similar shape instances with high accuracy. Finally, we combine these pairwise correspondences to achieve a unified correspondence for the entire population of shape instances. We evaluate the proposed method by comparing its performance to five available shape correspondence methods, and show that the proposed method achieves the accuracy of a global method with the efficiency of a pairwise method.

Generalization Bounds for Certain Class of Ranking Algorithm

The quality of ranking determines the success or failure of information retrieval and the goal of ranking is to learn a real-valued ranking function that induces a ranking or ordering over an instance space. We focus on a ranking setting which uses truth function to label each pair of instances and the ranking preferences are given randomly from some distributions on the set of possible undirected edge sets of a graph. The contribution of this paper is the given generalization bounds for such ranking algorithm via strong and weak stability. Such stabilities have lower demand than uniform stability and fit for more real applications.

One Dependence Value Difference Metric

Many distance-related algorithms depend upon a good distance metric to be successful. The Value Difference Metric, simply VDM, is proposed to find reasonable distance metric between each pair of instances with nominal attribute values only. In VDM, all of the attributes are assumed to be fully independent, and the difference between two values of an attribute is only considered to be closer if they have more similar correlation with the output classes. It is obvious that the attribute independence assumption in VDM is rarely true in reality, which would harm its performance in the applications with complex attribute dependencies. In this paper, we single out an improved Value Difference Metric by relaxing its unrealistic attribute independence assumption. We call it One Dependence Value Difference Metric, simply ODVDM. In ODVDM, the structure learning algorithms for Bayesian network classifiers, such as tree augmented naive Bayes, are used to find the dependence relationships among the attributes. Our experimental results validate its effectiveness in terms of classification accuracy.

Connecting instances to promote children’s relational reasoning

The practice of learning from multiple instances seems to allow children to learn about relational structure. The experiments reported here focused on two issues regarding relational learning from multiple instances: (a) what kind of perceptual situations foster such learning and (b) how particular object properties, such as complexity and similarity, interact with relational learning. Two kinds of perceptual situations were of interest here: simultaneous view, where instances are viewed at once, and sequential view, where instances are viewed one at a time (one right after the other). We examined the influence of particular perceptual situations and object properties using two tests of relational reasoning: a common match-to-sample task, where new instances are compared with a common sample, and a variable match-to-sample task, where new instances are compared with a sample that varies on each trial. Experiments 1 and 2 indicate that simultaneous presentation of even highly dissimilar instances, one simple and one complex, effectively connects them together and improves relational generalization in both match-to-sample tasks. Experiment 3 shows that simple samples are more effective than complex ones in the common match-to-sample task. However, when one instance is not used a common sample and various pairs of instances are simply compared, as in Experiment 4, simple and rich instances are equally effective at promoting relational learning. These results bear on our understanding of how children connect instances and how those initial connections affect learning and generalization.

Learning a tensor subspace for semi-supervised dimensionality reduction

The high-dimensional data is frequently encountered and processed in real-world applications and unlabeled samples are readily available, but labeled or pairwise constrained ones are fairly expensive to capture. Traditionally, when a pattern itself is an n 1 × n 2 image, the image first has to be vectorized to the vector pattern in $$ \Re^{{n_{1} \times n_{2} }} $$ by concatenating its pixels. However, such a vector representation fails to take into account the spatial locality of pixels in the images, which are intrinsically matrices. In this paper, we propose a tensor subspace learning-based semi-supervised dimensionality reduction algorithm (TS2DR), in which an image is naturally represented as a second-order tensor in $$ \Re^{{n_{1} }} \otimes \Re^{{n_{2} }} $$ and domain knowledge in the forms of pairwise similarity and dissimilarity constraints is used to specify whether pairs of instances belong to the same class or different classes. TS2DR has an analytic form of the global structure preserving embedding transformation, which can be easily computed based on eigen-decomposition. We also verify the efficiency of TS2DR by conducting unbalanced data classification experiments based on the benchmark real-word databases. Numerical results show that TS2DR tends to capture the intrinsic structure characteristics of the given data and achieves better classification accuracy, while being much more efficient.

A Survey of Distance Metrics for Nominal Attributes

Many distance-related algorithms, such as k- nearest neighbor learning algorithms, locally weighted learn- ing algorithms etc, depend upon a good distance metric to be successful. In this kind of algorithms, a key problem is how to measure the distance between each pair of instances. In this paper, we provide a survey on distance metrics for nominal attributes, including some basic distance metrics and their improvements based on attribute weighting and attribute selection. The experimental results on the whole 36 UCI datasets published on the main web site of Weka platform validate their effectiveness.

Effective semi-supervised nonlinear dimensionality reduction for wood defects recognition

Dimensionality reduction is an important preprocessing step in high-dimensional data analysis without losing intrinsic information. The problem of semi-supervised nonlinear dimensionality reduction called KNDR is considered for wood defects recognition. In this setting, domain knowledge in forms of pairs constraints are used to specify whether pairs of instances belong to the same class or different classes. KNDR can project the data onto a set of 'useful' features and preserve the structure of labeled and unlabeled data as well as the constraints defined in the embedding space, under which the projections of the original data can be effectively partitioned from each other. We demonstrate the practical usefulness of KNDR for data visualization and wood defects recognition through extensive experiments. Experimental results show it achieves similar or even higher performances than some existing methods.

Bagging Constraint Score for feature selection with pairwise constraints

Constraint Score is a recently proposed method for feature selection by using pairwise constraints which specify whether a pair of instances belongs to the same class or not. It has been shown that the Constraint Score, with only a small amount of pairwise constraints, achieves comparable performance to those fully supervised feature selection methods such as Fisher Score. However, one major disadvantage of the Constraint Score is that its performance is dependent on a good selection on the composition and cardinality of constraint set, which is very challenging in practice. In this work, we address the problem by importing Bagging into Constraint Score and a new method called Bagging Constraint Score (BCS) is proposed. Instead of seeking one appropriate constraint set for single Constraint Score, in BCS we perform multiple Constraint Score, each of which uses a bootstrapped subset of original given constraint set. Diversity analysis on individuals of ensemble shows that resampling pairwise constraints is helpful for simultaneously improving accuracy and diversity of individuals. We conduct extensive experiments on a series of high-dimensional datasets from UCI repository and gene databases, and the experimental results validate the effectiveness of the proposed method.

On the Expressive Power of the Relational Algebra on Finite Sets of Relation Pairs

We give a language-independent characterization of the expressive power of the relational algebra on finite sets of source-target relation instance pairs. The associated decision problem is shown to be co-graph-isomorphism hard and in co NP. The main result is also applied in providing a new characterization of the generic relational queries.

Neighbourhood preserving based semi-supervised dimensionality reduction

A semi-supervised linear dimensionality reduction method based on side information and neighbourhood preserving is proposed. In this problem, only must-link constraints (pairs of instances belong to the same class) and cannot-link constraints (pairs of instances belong to different classes) are given. The proposed neighbourhood preserving based semi-supervised dimensionality reduction algorithm can not only preserve the must-link and cannot-link constraints but can preserve the local structure of the input data in the low dimensional embedding subspace. Experimental results on several datasets demonstrate the effectiveness of the method.

Constructing Low-Order Discriminant Neural Networks Using Statistical Feature Selection

CONSTRUCTING LOW-ORDER DISCRIMINANT NEURAL NETWORKS USING STATISTICAL FEATURE SELECTION The selection of relevant inputs, and determining an appropriate network topology, are two critical issues faced when applying neural networks to classification problems. This paper presents an algorithm called Pair Attribute Learning (PAL) for addressing both input selection, and the determination of network topology. The PAL algorithm uses a preprocessing stage to search for features derived from pairs of training instances. A statistical rank is used to select a good set of features, and these features are then used to drive the construction of a single hidden layer neural network. Only inputs relevant within the context of a feature are used in constructing the network. This results in a sparsely connected hidden layer, and lower-order discriminants. Results on nine learning problems demonstrate that PAL constructed networks are 70% less complex on average than networks built using other constructive techniques, without a significant loss of predictive accuracy. In addition, the PAL algorithm does not use iterative construction, or suffer from bias mismatch. Because it addresses both input selection and network topology, it provides an end-to-end solution for applying neural networks to classification problems.

Implementation and evaluation of a hypercube-based method for spatiotemporal exploration and analysis

This paper presents the results obtained with a new type of spatiotemporal topological dimension implemented within a hypercube, i.e., within a multidimensional database (MDDB) structure formed by the conjunction of several thematic, spatial and temporal dimensions. Our goal is to support efficient SpatioTemporal Exploration and Analysis (STEA) in the context of Automatic Position Reporting System (APRS), the worldwide amateur radio system for position report transmission. Mobile APRS stations are equipped with GPS navigation systems to provide real-time positioning reports. Previous research about the multidimensional approach has proved good potential for spatiotemporal exploration and analysis despite a lack of explicit topological operators (spatial, temporal and spatiotemporal). Our project implemented such operators through a hierarchy of operators that are applied to pairs of instances of objects. At the top of the hierarchy, users can use simple operators such as “same place”, “same time” or “same time, same place”. As they drill down into the hierarchy, more detailed topological operators are made available such as “adjacent immediately after”, “touch during” or more detailed operators. This hierarchy is structured according to four levels of granularity based on cognitive models, generalized relationships and formal models of topological relationships. In this paper, we also describe the generic approach which allows efficient STEA within the multidimensional approach. Finally, we demonstrate that such an implementation offers query run times which permit to maintain a “train-of-thought” during exploration and analysis operations as they are compatible with Newell's cognitive band (query runtime<10 s) (Newell, A., 1990. Unified theories of cognition. Harvard University Press, Cambridge MA, 549 p.).

The polymorphous concept of money

Three experiments were carried out to investigate people's concept of money. In the first experiment, subjects were asked to rate items for their typicality as money, and subsequently to say, as quickly as possible, whether or not those items were ‘a kind of money’. Subjects were quicker to categorize the more typical instances. In the second experiment, subjects were asked to rate the similarities between pairs of instances of the money concept. The mean ratings were well described by a single-link cluster analysis whose structure was dominated by the factor of typicality. These two experiments were both carried out in England. The third experiment was conducted in both England and the Netherlands, and was preceded by a postal questionnaire aiming to identify items which were widely agreed to be instances of money. It combined the manipulations of the first two experiments, and the same patterns of categorization times and similarity ratings were found. In addition, multidimensional scaling showed that typical value is an important dimension in determining the similarities between kinds of money. There were interpretable differences in the similarity ratings between the English and Dutch samples. The three experiments show that money is a typical ‘polymorphous concept’, that is, a concept whose definition and boundaries cannot be specified precisely, but which nonetheless can be used consistently and efficiently.

Preference ranking of discrete multiattribute instances

This paper presents a procedure for preference ranking of discrete multiattribute instances based on the observations that (1) a difficult question could be answered in terms of answers to a sequence of easier ones, (2) in general, the decision maker's preferences can easily be expressed only relative to pairs of instances that differ over one attribute, and (3) the decision maker is able and willing to express his uncertainty regarding a quantity via a subjective probability distribution. The procedure is illustrated through a hypothetical situation.