Set Of Visual Features Research Articles

The importance of “what”: Infants use featural information to index events

Dynamic spatial indexing is the ability to encode, remember, and track the location of complex events. For example, in a previous study, 6-month-old infants were familiarized to a toy making a particular sound in a particular location, and later they fixated that empty location when they heard the sound presented alone (Journal of Experimental Psychology: General, 2004, Vol. 133, pp. 46–62). The basis and developmental trajectory of this ability are currently unclear. We investigated dynamic spatial indexing across the first year after birth and tested the hypothesis that the structure of visual cues supports infants’ learning of sound and location associations. In our study, 3-, 6-, and 10-month-olds were tested in a dynamic spatial indexing eye movement paradigm that paired two sounds with two locations. In one condition, these were reliably paired with two sets of visual features (two toys condition), replicating the original studies. We also presented a single set of visual cues in both locations (one toy condition) and multiple sets of visual features in both locations (six toys condition). Infants from 3months of age onward showed evidence of dynamic spatial indexing in the two toys condition, but only the 10-month-olds succeeded in the one toy and six toys conditions. We argue that this may reflect a broader developmental trajectory, whereby infants first make use of multiple cue integration but with age are able to learn from a narrow set of cues.

The role of textures to improve the detection accuracy of Rumex obtusifolius in robotic systems

Hiremath S, Van Der Heijden G, Van Evert FK & Stein A (2012) The role of textures to improve the detection accuracy of Rumex obtusifolius in robotic systems. Weed Research.52, 430–440.SummaryRumex obtusifolius is a common weed that is difficult to control in organic farming systems. Among the proposed non‐chemical treatment methods, robotic systems to mechanically remove the weed have potential for its automatic detection and removal. This article considers a recently developed robot with a real‐time vision system capable of detecting R. obtusifolius in a pasture with the objective of improving its detection accuracy. We show that the texture measure used by the current system is equivalent to local variance and has limited value for detecting R. obtusifolius in a complex background like pasture. To improve the system, two different sets of visual texture features corresponding to Gray level Co‐occurence Matrix (GLCM) and Laws’ filter masks were investigated. Through feature selection, we determined that GLCM features of contrast, entropy and correlation were the best among the two sets of features and were 25% more accurate in estimating the taproot location than the current system. We incorporated these texture features in a new segmentation algorithm and demonstrated its robustness by testing it on a data set of 92 images with high complexity in terms of variation in illumination and weed size. The new segmentation algorithm had a detection accuracy of 90%, with an average error of 141 mm in the estimation of the location of the taproot of R. obtusifolius, compared with 308 mm with the former algorithm.

Data space reduction, quality assessment and searching of seismograms: autoencoder networks for waveform data

SUMMARY What makes a seismogram look like a seismogram? Seismic data sets generally contain waveforms sharing some set of visual characteristics and features—indeed, seismologists routinely exploit this when performing quality control ‘by hand’. Understanding and harnessing these characteristics offers the prospect of a deeper understanding of seismic waveforms, and opens up many potential new techniques for processing and working with data. In addition, the fact that certain features are shared between waveforms suggests that it may be possible to transform the data away from the time domain, and represent the same information using fewer parameters. If so, this would be a significant step towards making fully non-linear tomographic inversions computationally tractable. Hinton & Salakhutdinov showed that a particular class of neural network, termed ‘autoencoder networks’, may be used to find lower-dimensional encodings of complex binary data sets. Here, we adapt their work to the continuous case to allow the use of autoencoders for seismic waveforms, and offer a demonstration in which we compress 512-point waveforms to 32-element encodings. We also demonstrate that the mapping from data to encoding space, and its inverse, are well behaved, as required for many applications. Finally, we sketch a number of potential applications of the technique, which we hope will be of practical interest across all seismological disciplines, and beyond.

Image classification by multimodal subspace learning

In recent years we witnessed a surge of interest in subspace learning for image classification. However, the previous methods lack of high accuracy since they do not consider multiple features of the images. For instance, we can represent a color image by finding a set of visual features to represent the information of its color, texture and shape. According to the “Patch Alignment” Framework, we developed a new subspace learning method, termed Semi-Supervised Multimodal Subspace Learning (SS-MMSL), in which we can encode different features from different modalities to build a meaningful subspace. In particular, the new method adopts the discriminative information from the labeled data to construct local patches and aligns these patches to get the optimal low dimensional subspace for each modality. For local patch construction, the data distribution revealed by unlabeled data is utilized to enhance the subspace learning. In order to find a low dimensional subspace wherein the distribution of each modality is sufficiently smooth, SS-MMSL adopts an alternating and iterative optimization algorithm to explore the complementary characteristics of different modalities. The iterative procedure reaches the global minimum of the criterion due to the strong convexity of the criterion. Our experiments of image classification and cartoon retrieval demonstrate the validity of the proposed method.

IntentSearch: Capturing User Intention for One-Click Internet Image Search

Web-scale image search engines (e.g., Google image search, Bing image search) mostly rely on surrounding text features. It is difficult for them to interpret users' search intention only by query keywords and this leads to ambiguous and noisy search results which are far from satisfactory. It is important to use visual information in order to solve the ambiguity in text-based image retrieval. In this paper, we propose a novel Internet image search approach. It only requires the user to click on one query image with minimum effort and images from a pool retrieved by text-based search are reranked based on both visual and textual content. Our key contribution is to capture the users' search intention from this one-click query image in four steps. 1) The query image is categorized into one of the predefined adaptive weight categories which reflect users' search intention at a coarse level. Inside each category, a specific weight schema is used to combine visual features adaptive to this kind of image to better rerank the text-based search result. 2) Based on the visual content of the query image selected by the user and through image clustering, query keywords are expanded to capture user intention. 3) Expanded keywords are used to enlarge the image pool to contain more relevant images. 4) Expanded keywords are also used to expand the query image to multiple positive visual examples from which new query specific visual and textual similarity metrics are learned to further improve content-based image reranking. All these steps are automatic, without extra effort from the user. This is critically important for any commercial web-based image search engine, where the user interface has to be extremely simple. Besides this key contribution, a set of visual features which are both effective and efficient in Internet image search are designed. Experimental evaluation shows that our approach significantly improves the precision of top-ranked images and also the user experience.

A Tensor-Based Algorithm for High-Order Graph Matching

This paper addresses the problem of establishing correspondences between two sets of visual features using higher order constraints instead of the unary or pairwise ones used in classical methods. Concretely, the corresponding hypergraph matching problem is formulated as the maximization of a multilinear objective function over all permutations of the features. This function is defined by a tensor representing the affinity between feature tuples. It is maximized using a generalization of spectral techniques where a relaxed problem is first solved by a multidimensional power method and the solution is then projected onto the closest assignment matrix. The proposed approach has been implemented, and it is compared to state-of-the-art algorithms on both synthetic and real data.

Bimodal fusion of low-level visual features and high-level semantic features for near-duplicate video clip detection

The detection of near-duplicate video clips (NDVCs) is an area of current research interest and intense development. Most NDVC detection methods represent video clips with a unique set of low-level visual features, typically describing color or texture information. However, low-level visual features are sensitive to transformations of the video content. Given the observation that transformations tend to preserve the semantic information conveyed by the video content, we propose a novel approach for identifying NDVCs, making use of both low-level visual features (this is, MPEG-7 visual features) and high-level semantic features (this is, 32 semantic concepts detected using trained classifiers). Experimental results obtained for the publicly available MUSCLE-VCD-2007 and TRECVID 2008 video sets show that bimodal fusion of visual and semantic features facilitates robust NDVC detection. In particular, the proposed method is able to identify NDVCs with a low missed detection rate (3% on average) and a low false alarm rate (2% on average). In addition, the combined use of visual and semantic features outperforms the separate use of either of them in terms of NDVC detection effectiveness. Further, we demonstrate that the effectiveness of the proposed method is on par with or better than the effectiveness of three state-of-the-art NDVC detection methods either making use of temporal ordinal measurement, features computed using the Scale-Invariant Feature Transform (SIFT), or bag-of-visual-words (BoVW). We also show that the influence of the effectiveness of semantic concept detection on the effectiveness of NDVC detection is limited, as long as the mean average precision (MAP) of the semantic concept detectors used is higher than 0.3. Finally, we illustrate that the computational complexity of our NDVC detection method is competitive with the computational complexity of the three aforementioned NDVC detection methods.

Multimodal Information Integration and Fusion for Histology Image Classification

Biomedical imaging technology has become an important tool for medical research and clinical practice. A large amount of imaging data is generated and collected every day. Managing and analyzing these data sets require the corresponding development of the computer based algorithms for automatic processing. Histology image classification is one of the important tasks in the bio-image informatics field and has broad applications in phenotype description and disease diagnosis. This study proposes a novel framework of histology image classification. The original images are first divided into several blocks and a set of visual features is extracted for each block. An array of C-RSPM (Collateral Representative Subspace Projection Modeling) models is then built that each model is based on one block from the same location in original images. Finally, the C-Value Enhanced Majority Voting (CEWMV) algorithm is developed to derive the final classification label for each testing image. To evaluate this framework, the authors compare its performance with several well-known classifiers using the benchmark data available from IICBU data repository. The results demonstrate that this framework achieves promising performance and performs significantly better than other classifiers in the comparison.

Saliency and Gist Features for Target Detection in Satellite Images

Reliably detecting objects in broad-area overhead or satellite images has become an increasingly pressing need, as the capabilities for image acquisition are growing rapidly. The problem is particularly difficult in the presence of large intraclass variability, e.g., finding "boats" or "buildings," where model-based approaches tend to fail because no good model or template can be defined for the highly variable targets. This paper explores an automatic approach to detect and classify targets in high-resolution broad-area satellite images, which relies on detecting statistical signatures of targets, in terms of a set of biologically-inspired low-level visual features. Broad-area images are cut into small image chips, analyzed in two complementary ways: "attention/saliency" analysis exploits local features and their interactions across space, while "gist" analysis focuses on global nonspatial features and their statistics. Both feature sets are used to classify each chip as containing target(s) or not, using a support vector machine. Four experiments were performed to find "boats" (Experiments 1 and 2), "buildings" (Experiment 3) and "airplanes" (Experiment 4). In experiment 1, 14 416 image chips were randomly divided into training (300 boat, 300 nonboat) and test sets (13 816), and classification was performed on the test set (ROC area: 0.977 ± 0.003). In experiment 2, classification was performed on another test set of 11 385 chips from another broad-area image, keeping the same training set as in experiment 1 (ROC area: 0.952 ± 0.006). In experiment 3, 600 training chips (300 for each type) were randomly selected from 108 885 chips, and classification was conducted (ROC area: 0.922 ± 0.005). In experiment 4, 20 training chips (10 for each type) were randomly selected to classify the remaining 2581 chips (ROC area: 0.976 ± 0.003). The proposed algorithm outperformed the state-of-the-art SIFT, HMAX, and hidden-scale salient structure methods, and previous gist-only features in all four experiments. This study shows that the proposed target search method can reliably and effectively detect highly variable target objects in large image datasets.

A decision-theoretic saliency, its biological plausibility and implications for pre-attentive vision

A decision-theoretic formulation of visual saliency, first proposed for top-down processing (object recognition) in (Gao & Vasconcelos, 2005) is extended to the problem of bottom-up saliency. Under this formulation, optimality is defined in the minimum probability of error sense, under a constraint of computational parsimony. The saliency of the visual features at a given location of the visual field is defined as the power of those features to discriminate between the stimulus at the location and a null hypotheses. For bottom-up saliency, this is the set of visual features that surround the location under consideration. Discrimination is defined in an information-theoretic sense and the optimal saliency detector derived for a class of stimuli that complies with known statistical properties of natural images. It is shown that the optimal detector consists of what is usually referred to as the standard architecture of V1 (Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, et al., 2005): a cascade of linear filtering, divisive normalization, rectification and spatial pooling. The optimal detector is also shown to replicate the fundamental properties of the psychophysics of saliency (Treisman & Gelade, 1980): stimulus pop-out, saliency asymmetries for stimulus presence vs. absence, disregard of feature conjunctions, and Weber’s law. Finally, it is shown that the optimal saliency architecture can be applied to the solution of generic inference problems. In particular, for the class stimuli studied, it performs the three fundamental operations of statistical inference: assessment of probabilities, implementation of Bayes decision rule, and feature selection.

Multiview Spectral Embedding

In computer vision and multimedia search, it is common to use multiple features from different views to represent an object. For example, to well characterize a natural scene image, it is essential to find a set of visual features to represent its color, texture, and shape information and encode each feature into a vector. Therefore, we have a set of vectors in different spaces to represent the image. Conventional spectral-embedding algorithms cannot deal with such datum directly, so we have to concatenate these vectors together as a new vector. This concatenation is not physically meaningful because each feature has a specific statistical property. Therefore, we develop a new spectral-embedding algorithm, namely, multiview spectral embedding (MSE), which can encode different features in different ways, to achieve a physically meaningful embedding. In particular, MSE finds a low-dimensional embedding wherein the distribution of each view is sufficiently smooth, and MSE explores the complementary property of different views. Because there is no closed-form solution for MSE, we derive an alternating optimization-based iterative algorithm to obtain the low-dimensional embedding. Empirical evaluations based on the applications of image retrieval, video annotation, and document clustering demonstrate the effectiveness of the proposed approach.

Visual Features Involving Motion Seen from Airport Control Towers

Visual motion cues are used by tower controllers to support both visual and anticipated separation. Some of these cues are tabulated as part of the overall set of visual features used in towers to separate aircraft. An initial analyses of one motion cue, landing deceleration, is provided as a basis for evaluating how controllers detect and use it for spacing aircraft on or near the surface. Understanding cues like it will help determine if they can be safely used in a remote/virtual tower in which their presentation may be visually degraded.

Improvements on Visual Servoing From Spherical Targets Using a Spherical Projection Model

This paper is concerned with improvements to visual feature modeling using a spherical projection model. Three spherical targets are considered: a sphere, a sphere marked with a tangent vector to a point on its surface, and a sphere marked with two points on its surface. A new minimal and decoupled set of visual features is proposed for each target using any central catadioptric camera. Using the proposed set for a sphere, a classical control law is proved to be globally asymptotically stable in the presence of modeling errors and locally asymptotically stable in the presence of calibration errors, considering that perspective and paracatadioptric cameras were used. Simulation and experimental results with perspective, paracatadioptric, and fish-eye cameras validate the proposed theoretical results.

Human detection using a mobile platform and novel features derived from a visual saliency mechanism

Human detection is a key ability to an increasing number of applications that operates in human inhabited environments or needs to interact with a human user. Currently, most successful approaches to human detection are based on background substraction techniques that apply only to the case of static cameras or cameras with highly constrained motions. Furthermore, many applications rely on features derived from specific human poses, such as systems based on features derived from the human face which is only visible when a person is facing the detecting camera. In this work, we present a new computer vision algorithm designed to operate with moving cameras and to detect humans in different poses under partial or complete view of the human body. We follow a standard pattern recognition approach based on four main steps: (i) preprocessing to achieve color constancy and stereo pair calibration, (ii) segmentation using depth continuity information, (iii) feature extraction based on visual saliency, and (iv) classification using a neural network. The main novelty of our approach lies in the feature extraction step, where we propose novel features derived from a visual saliency mechanism. In contrast to previous works, we do not use a pyramidal decomposition to run the saliency algorithm, but we implement this at the original image resolution using the so-called integral image. Our results indicate that our method: (i) outperforms state-of-the-art techniques for human detection based on face detectors, (ii) outperforms state-of-the-art techniques for complete human body detection based on different set of visual features, and (iii) operates in real time onboard a mobile platform, such as a mobile robot (15 fps).

Decision-Theoretic Saliency: Computational Principles, Biological Plausibility, and Implications for Neurophysiology and Psychophysics

A decision-theoretic formulation of visual saliency, first proposed for top-down processing (object recognition) (Gao & Vasconcelos, 2005a), is extended to the problem of bottom-up saliency. Under this formulation, optimality is defined in the minimum probability of error sense, under a constraint of computational parsimony. The saliency of the visual features at a given location of the visual field is defined as the power of those features to discriminate between the stimulus at the location and a null hypothesis. For bottom-up saliency, this is the set of visual features that surround the location under consideration. Discrimination is defined in an information-theoretic sense and the optimal saliency detector derived for a class of stimuli that complies with known statistical properties of natural images. It is shown that under the assumption that saliency is driven by linear filtering, the optimal detector consists of what is usually referred to as the standard architecture of V1: a cascade of linear filtering, divisive normalization, rectification, and spatial pooling. The optimal detector is also shown to replicate the fundamental properties of the psychophysics of saliency: stimulus pop-out, saliency asymmetries for stimulus presence versus absence, disregard of feature conjunctions, and Weber's law. Finally, it is shown that the optimal saliency architecture can be applied to the solution of generic inference problems. In particular, for the class of stimuli studied, it performs the three fundamental operations of statistical inference: assessment of probabilities, implementation of Bayes decision rule, and feature selection.

A connectionist architecture for view-independent grip-aperture computation

This paper addresses the problem of extracting view-invariant visual features for the recognition of object-directed actions and introduces a computational model of how these visual features are processed in the brain. In particular, in the test-bed setting of reach-to-grasp actions, grip aperture is identified as a good candidate for inclusion into a parsimonious set of hand high-level features describing overall hand movement during reach-to-grasp actions. The computational model NeGOI (neural network architecture for measuring grip aperture in an observer-independent way) for extracting grip aperture in a view-independent fashion was developed on the basis of functional hypotheses about cortical areas that are involved in visual processing. An assumption built into NeGOI is that grip aperture can be measured from the superposition of a small number of prototypical hand shapes corresponding to predefined grip-aperture sizes. The key idea underlying the NeGOI model is to introduce view-independent units ( VIP units) that are selective for prototypical hand shapes, and to integrate the output of VIP units in order to compute grip aperture. The distinguishing traits of the NEGOI architecture are discussed together with results of tests concerning its view-independence and grip-aperture recognition properties. The overall functional organization of NEGOI model is shown to be coherent with current functional models of the ventral visual stream, up to and including temporal area STS. Finally, the functional role of the NeGOI model is examined from the perspective of a biologically plausible architecture which provides a parsimonious set of high-level and view-independent visual features as input to mirror systems.

Incremental GRLVQ: Learning relevant features for 3D object recognition

We present a new variant of generalized relevance learning vector quantization (GRLVQ) in a computer vision scenario. A version with incrementally added prototypes is used for the non-trivial case of high-dimensional object recognition. Training is based upon a generic set of standard visual features, the learned input weights are used for iterative feature pruning. Thus, prototypes and input space are altered simultaneously, leading to very sparse and task-specific representations. The effectiveness of the approach and the combination of the incremental variant together with pruning was tested on the COIL100 database. It exhibits excellent performance with regard to codebook size, feature selection and recognition accuracy.

Using Biologically Inspired Features for Face Processing

In this paper, we show that a new set of visual features, derived from a feed-forward model of the primate visual object recognition pathway proposed by Riesenhuber and Poggio (R&P Model) (Nature Neurosci. 2(11):1019---1025, 1999) is capable of matching the performance of some of the best current representations for face identification and facial expression recognition. Previous work has shown that the Riesenhuber and Poggio Model features can achieve a high level of performance on object recognition tasks (Serre, T., et al. in IEEE Comput. Vis. Pattern Recognit. 2:994---1000, 2005). Here we modify the R&P model in order to create a new set of features useful for face identification and expression recognition. Results from tests on the FERET, ORL and AR datasets show that these features are capable of matching and sometimes outperforming other top visual features such as local binary patterns (Ahonen, T., et al. in 8th European Conference on Computer Vision, pp. 469---481, 2004) and histogram of gradient features (Dalal, N., Triggs, B. in International Conference on Computer Vision & Pattern Recognition, pp. 886---893, 2005). Having a model based on shared lower level features, and face and object recognition specific higher level features, is consistent with findings from electrophysiology and functional magnetic resonance imaging experiments. Thus, our model begins to address the complete recognition problem in a biologically plausible way.

Robust online orientation correction for radiographs in PACS environments

In picture archiving and communications systems (PACS), images need to be displayed in standardized ways for radiologists' interpretations. However, for most radiographs acquired via computed radiography (CR), digital radiography (DR), or digitized films, the image orientation is undetermined because of the variations in examination conditions and patients' situations. To address this problem, an automatic orientation correction method is developed. It first detects the most indicative region in a radiograph for image orientation, and then extracts a set of low-level visual features from the region. Based on these features, a well-trained classifier, using support vector machines, is employed to recognize the correct orientation of the radiograph and reorient it to the desired position. A large-scale experiment was conducted on more than 12 000 radiographs, which covered a wide variety of exam types, to validate the method. The overall success rate of orientation correction was 96.1%. A workflow study on the method also demonstrated a significant improvement in efficiency for image display. To our knowledge, this work represents the first robust system designed to handle all radiographic exam types using a unified framework instead of using dedicated strategies for different exam types.

Optimizing plane-to-plane positioning tasks by image-based visual servoing and structured light

This paper considers the problem of positioning an eye-in-hand system so that it becomes parallel to a planar object. Our approach to this problem is based on linking to the camera a structured light emitter designed to produce a suitable set of visual features. The aim of using structured light is not only for simplifying the image processing and allowing low-textured objects to be considered, but also for producing a control scheme with nice properties like decoupling, convergence, and adequate camera trajectory. This paper focuses on an image-based approach that achieves decoupling in all the workspace, and for which the global convergence is ensured in perfect conditions. The behavior of the image-based approach is shown to be partially equivalent to a 3-D visual servoing scheme, but with a better robustness with respect to image noise. Concerning the robustness of the approach against calibration errors, it is demonstrated both analytically and experimentally