Regions Of Feature Space Research Articles

Improved Generative Semisupervised Learning Based on Finely Grained Component-Conditional Class Labeling

We introduce new inductive, generative semisupervised mixtures with more finely grained class label generation mechanisms than in previous work. Our models combine advantages of semisupervised mixtures, which achieve label extrapolation over a component, and nearest-neighbor (NN)/nearest-prototype (NP) classification, which achieve accurate classification in the vicinity of labeled samples or prototypes. For our NN-based method, we propose a novel two-stage stochastic data generation, with all samples first generated using a standard finite mixture and then all class labels generated, conditioned on the samples and their components of origin. This mechanism entails an underlying Markov random field, specific to each mixture component or cluster. We invoke the pseudo-likelihood formulation, which forms the basis for an approximate generalized expectation-maximization model learning algorithm. Our NP-based model overcomes a problem with the NN-based model that manifests at very low labeled fractions. Both models are advantageous when within-component class proportions are not constant over the feature space region “owned by” a component. The practicality of this scenario is borne out by experiments on UC Irvine data sets, which demonstrate significant gains in classification accuracy over previous semisupervised mixtures and also overall gains, over KNN classification. Moreover, for very small labeled fractions, our methods overall outperform supervised linear and nonlinear kernel support vector machines.

A dynamical model of perceptual categorization

In visual and auditory scenes, we are able to identify shared features among sensory objects and group them according to their similarity. We consider the case of temporally streamed input: sequentially presented or sampled objects, disregarding (in the case of vision) spatial arrangement. The process of grouping by similarity is easy, fast, and does not require attention or training when the stimuli are sufficiently simple. What neural mechanisms govern such a grouping process remains an open question. We propose a high-level, neuro-mechanistic model of perceptual grouping based on the framework of continuous attractor networks [1,2]. The network model is able to dynamically cluster the features of the sensory objects present in the input stream, thereby performing an elementary categorization of the sensory scene. The resulting categories are represented by self-sustained selective patterns of neuronal activity, or “bumps”, in which only a fraction of neurons with similar selectivity properties are active. Such activity patterns can be thought of as localized in feature space, and persist in the absence of inputs due to recurrent excitation. The connectivity of the network is sufficiently structured for these non-homogeneous patterns to be stable, while is mild enough to maintain the stability of a homogeneous, low-activity state, associated with spontaneous activity. The bistability between the homogeneous and the localized states allows for the formation of categories conditioned on the statistics of the external inputs, idealized as temporal sequences of single feature values. We suppose that the presentation of a single item in the sequence elicits a subthreshold response that, compared to the stimulus presentation rate, decays slowly after the stimulus is removed. Because of the slow decay, the network responds to the sequence of stimuli by effectively summing temporally the responses triggered separately by each item. Only when the input stream consists of an adequate number of similar feature values, will there be successive hits within a critical time window in one or more localized regions of feature space that will be sufficient to trigger the emergence of one or more bump states. Each such bump state encodes approximately the mean feature value of the items belonging to a particular category, and hence can be regarded as the neural representation of a category prototype. Using a combination of analytical and numerical methods, we study the conditions for the network to be capable of forming categories. More specifically, we investigate the necessary conditions that the connectivity has to satisfy so that multiple localized states can coexist, while preserving the bistability with a homogeneous state. These conditions can be fulfilled when the connectivity has a narrow Mexican-hat profile, and there is an approximate balance between excitation and inhibition. A generalization of the model for multi-dimensional features is also analyzed and leads to similar results. We characterize the ability of the network to extract clusters of features as a function of the statistics of the input stream, and show that results are consistent with the psychophysics of perceptual categorization and ensemble statistics [3].

A dynamical model of perceptual grouping

In a visual scene, we perceive similar visual objects as being grouped together. We propose a high-level, mechanistic model of perceptual grouping where stimuli that are sufficiently similar are dynamically clustered forming a visual category. The model assumes that each stimulus can be characterized by a continuous feature, like an orientation, which is encoded in the neuronal activity of a recurrent network. The connectivity of the network is sufficiently structured to allow for the generation of feature-selective stable states, or bumps, while maintaining the stability of a non-specific, low-activity state associated with spontaneous activity. The emergence of bumps corresponds to the formation of stimulus categories and is triggered by external inputs, idealized as temporal sequences of feature values. We suppose that each feature value in the sequence is sub-threshold; it leads to a response that, compared to the stimulus presentation rate, decays slowly after the stimulus is removed. When the sequence of stimuli is presented, the network sums temporally the responses triggered separately by each item. Only if the input stream consists of an adequate number of similar feature values, will there be successive hits within a critical time window in one or more localized regions of feature space that will give rise to the emergence of bump activity states. The network encodes therefore the distribution of feature values of the temporal input stream and, by extension, the visual scene.

FUZZY C-MEANS ALGORITHM WITH LOCAL THRESHOLDING FOR GRAY-SCALE IMAGES

An improved fuzzy C-means (FCM) clustering method is proposed. It incorporates Otsu thresholding with conventional FCM to reduce FCM's susceptibility to local minima, as well as its tendency to derive a threshold that is biased towards the component with larger probability, and derive threshold values with greater accuracy. Thresholding is performed at the cluster boundary region in feature space. A comparison of the results produced by improved and conventional algorithms confirms the superior performance of the former.

Medical Image Categorization and Retrieval for PACS Using the GMM-KL Framework

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.

Content-Based Copy Retrieval Using Distortion-Based Probabilistic Similarity Search

Content-based copy retrieval (CBCR) aims at retrieving in a database all the modified versions or the previous versions of a given candidate object. In this paper, we present a copy-retrieval scheme based on local features that can deal with very large databases both in terms of quality and speed. We first propose a new approximate similarity search technique in which the probabilistic selection of the feature space regions is not based on the distribution in the database but on the distribution of the features distortion. Since our CBCR framework is based on local features, the approximation can be strong and reduce drastically the amount of data to explore. Furthermore, we show how the discrimination of the global retrieval can be enhanced during its post-processing step, by considering only the geometrically consistent matches. This framework is applied to robust video copy retrieval and extensive experiments are presented to study the interactions between the approximate search and the retrieval efficiency. Largest used database contains more than 1 billion local features corresponding to 30000 h of video

Extraction of classification rules characterized by ellipsoidal regions using soft-computing techniques

This article presents a soft-computing-based data mining technique that addresses methodology aspects on extracting classification rules characterized by ellipsoidal regions in feature space. Self-organizing mapping and statistical techniques are employed to initialize the rules. A regularization model embedding some information on recognition rate and generalization ability is presented for refining the initial rules. Rule optimization is implemented for each individual rule using an evolutionary strategy. To generate rules for patterns with low probability of occurrence but considerable conceptual importance, a multilayer structure of rule generation and use is proposed. Simulation results are carried out by three benchmark data sets, and compared with other data mining tools and classifiers, such as decision trees, BRAINNE (Building Representations of Artificial Intelligence (AI) using Neural Networks), support vector machine, and neural networks. Our technique demonstrates its power and potential for real-world applications.

Documenting surface magmatic activity at Mount Etna using ATSR remote sensing

Analysis of thermally generated night-time volcanic radiances recorded with a 1-km pixel size at 1.6 and 11 µm during 1991–1993 and 1996–1999 for Mount Etna shows that lava flows extending beyond the summit craters can be distinguished from vent activity. The two phenomena plot in different regions of feature space when the mean volcanic radiance (per anomalous pixel) at 11 µm is plotted against the mean volcanic radiance at 1.6 µm. The distinct feature space characteristics of lava flow fields are apparent within 1–2 days of the onset of each effusive event. Such a plot also enables lava flow fields being fed by open channels to be distinguished from tube-fed flow fields. Rank order analysis of the total 1.6-µm volcanic radiance series shows that vent activity and lava flows belong to different populations, and offers further scope for remotely identifying changes in eruptive state.

Combination of multiple classifiers using local accuracy estimates

This paper presents a method for combining classifiers that uses estimates of each individual classifier's local accuracy in small regions of feature space surrounding an unknown test sample. An empirical evaluation using five real data sets confirms the validity of our approach compared to some other combination of multiple classifiers algorithms. We also suggest a methodology for determining the best mix of individual classifiers.

Automatic construction of decision trees for classification

An algorithm for learning decision trees for classification and prediction is described which converts real-valued attributes into intervals using statistical considerations. The trees are automatically pruned with the help of a threshold for the estimated class probabilities in an interval. By means of this threshold the user can control the complexity of the tree, i.e. the degree of approximation of class regions in feature space. Costs can be included in the learning phase if a cost matrix is given. In this case class dependent thresholds are used. Some applications are described, especially the task of predicting the high water level in a mountain river.

Dual-echo MRI segmentation using vector decomposition and probability techniques: a two-tissue model.

We combined a vector decomposition technique with Gaussian probability thresholding in feature space to segment normal brain tissues, tumors, or other abnormalities on dual-echo MR images. The vector decomposition technique assigns to each voxel a fractional volume for each of two tissues. A probability threshold, based on an assumed Gaussian probability density function describing random noise, isolates a region in feature space for fractional volume calculation that minimizes contamination from other tissues. The calculated fractional volumes are unbiased estimates of the true fractional volumes. The contrast-to-noise ratio (CNR) between tissues on the segmented images is the same as the Euclidean norm of CNRs in the original images. The method is capable of segmenting more than two tissues from a set of dual-echo images by sequentially analyzing different pairs of tissues. The model is analyzed mathematically and in experiments with a phantom. Two clinical examples are presented.

A comparison of decision tree classifiers with backpropagation neural networks for multimodal classification problems

Multi-modal classification problems involve the recognition of patterns where the patterns associated with each class can come from disjoint regions in feature space. Traditional linear discriminant methods cannot cope with these problems. While a number of approaches exist for classifying patterns with multiple modes, decision trees and backpropagation neural networks represent leading algorithms with special capabilities for dealing with this problem class. This paper provides a comparison of decision trees with backpropagation neural networks for three distinct multi-modal problems: two from emitter classification and one from digit recognition. These real-world problems provide an interesting range of problem characteristics for our comparison: one emitter classification problem has few features and a large data set; and the other has many features and a small data set. Additionally, both emitter classification problems have real-valued features, while the digit recognition problem has binary-valued features. The results show that both methods produce comparable error rates but that direct application of either method will not necessarily produce the lowest error rate. In particular, we improve decision tree results with multi-variable splits and we improve backpropagation neural networks with feature selection and mode identification.