MPEG-7 Dataset Research Articles

Skeleton based Non rigid Shape Matching

This paper offers a new feature for matching deformable objects based on skeleton. Effectiveness of the skeleton-based methods depends to a large extent on the quality and perfection of the extracted skeleton. When the skeleton is extracted, additional branches of which, with the shape contour segmentation, are omitted by DCE method. The pruned skeleton is robust to the complex deformations and noise. After the pruned skeleton is obtained, minimum distance of contour points of the shape from the skeleton is calculated. This distance almost is constant against the articulated changes which this feature is used for finding the corresponding points. In order to carry out matching, the suggested method is applied on the MPEG7 dataset. Results show that the proposed algorithm is highly effective for matching objects.

Shape recognition using orientational and morphological scale‐spaces of curvatures

In this study, a scale‐invariant representation for closed planar curves (silhouettes) is proposed. The orientations of all points within the Gaussian scale‐space of the curve are extracted. This orientation scale‐space is used to create the silhouette orientation image in which the positions of each pixel indicate the curve's pixel positions and scales, whereas the colour represents orientation. The representation is extracted for multiple levels of the morphological scale‐space of the silhouette. The proposed representation is invariant to scale and transformable under planar rotation. Using linear and non‐linear distance learning methods, experiments on the MPEG7, ETH80 and Kimia shape datasets were conducted, with results indicating an advanced recognition capability.

Discriminative outlines parts for shape retrieval

In this paper we propose a new method for shape retrieval using only discriminative parts which are sufficient for the recognition of many objects and their classes. The discriminative outline shape is firstly determined by performing psycho-visual tests and then described with geometric attributes of high curvature points located along the outline. This obtained description is invariant to scale change, rotation, mirroring and deformation. To study the performance of our approach, global approach which deals with the whole contour and partial approach which is based on the discriminative part are compared. Experiments conducted on our data set and a selection of MPEG-7 dataset demonstrated the usefulness of this approach and the results obtained are presented and discussed.

Contextual Distance Refining for Image Retrieval

Recently, a number of methods have been proposed to improve image retrieval accuracy by capturing context information. These methods try to compensate for the fact that a visually less similar image might be more relevant because it depicts the same object. We propose a new quick method for refining any pairwise distance metric, it works by iteratively discovering the object in the image from the most similar images, and then refine the distance metric accordingly. Test show that our technique improves over the state of art in terms of accuracy over the MPEG7 dataset.

Performance Analysis of Minimum Distance Classifier and Clustering Algorithm in CBIR

Content-based image retrieval (CBIR) system can be used to effectively and precisely retrieve the desired images from a large image database, and the development has become an important research issue.Statistical methods like, gray level co-occurrence matrix (GLCM) and the autocorrelation function are used to extract texture feature. Region-based methods utilize information from both boundaries and interior regions of the shape. Shape features like perimeter, area, centroid, circularity, solidity based on region can be extracted in the feature space. Similar images can be retrieved using minimum distance classifiers with and without clustering algorithm .Time complexity and the retrieval efficiency has been analyed and compared on both the methods. The experiments have been conducted on MPEG-7 dataset.

Angular Pattern and Binary Angular Pattern for Shape Retrieval

In this paper, we propose two novel shape descriptors, angular pattern (AP) and binary angular pattern (BAP), and a multiscale integration of them for shape retrieval. Both AP and BAP are intrinsically invariant to scale and rotation. More importantly, being global shape descriptors, the proposed shape descriptors are computationally very efficient, while possessing similar discriminability as state-of-the-art local descriptors. As a result, the proposed approach is attractive for real world shape retrieval applications. The experiments on the widely used MPEG-7 and TARI-1000 data sets demonstrate the effectiveness of the proposed method in comparison with existing methods.

Multi-feature structure fusion of contours for unsupervised shape classification

Nonlinear distortion, especially structure distortion, is one of the main reasons for the poor performance of shape contour classification. The structure fusion of multiple features provides a new solution for the structure distortion. How is this structure fusion performed? To answer the question, in this letter, the multi-feature of a contour is defined. Second, the structure of each feature is measured by similarity. Then, the fusion structure is obtained using the algebraic operation of the respective structure, the specific form of which is deduced based on locality-preserving projection (LPP). Finally, the combined feature is mapped into the new structure-fusion feature in terms of the fusion structure. The experiment demonstrates that this structure fusion method is superior to other state-of-the-art methods that address geometrical transformations and nonlinear distortion for classification in Kimia or MPEG-7 datasets.

Symmetry-driven shape description for image retrieval

We propose a novel symmetry-driven Bayesian framework to incorporate structural shape into conventional geometrical shape descriptor of an image indexing and retrieval. We use rotation and reflection symmetries for structural shape description. Symmetry detection on each shape image provides a qualitative and a quantitative categorization of the types and the degrees of symmetry level. The posterior shape similarity enhances the shape matching performance based on the symmetry structural discrimination. Experimental results show statistically significant improvement on retrieval accuracy over the state of the art methods on MPEG-7 data set.

Perceptually motivated morphological strategies for shape retrieval

In this paper, two perceptually motivated morphological strategies (PMMS) are proposed to enhance the retrieval performance of common shape matching methods. Firstly, two human perception customs are introduced, which have important relations to shape retrieval. Secondly, these two customs are properly modeled by morphological operations. Finally, the proposed PMMS is applied to improve the retrieval performances of a popular shape matching method named Inner-Distance Shape Contexts (IDSC), and then the Locally Constrained Diffusion Process (LCDP) method is exploited to further enhance the retrieval performance. This combination achieves a retrieval rate of 98.56% on MPEG-7 dataset. We also conduct the experiments on Swedish Leaf dataset, the ETH-80 dataset and the Natural Silhouette dataset. The experimental results obtained from four datasets demonstrate clearly the effectiveness of the proposed method.

Shape matching and classification using height functions

We propose a novel shape descriptor for matching and recognizing 2D object silhouettes. The contour of each object is represented by a fixed number of sample points. For each sample point, a height function is defined based on the distances of the other sample points to its tangent line. One compact and robust shape descriptor is obtained by smoothing the height functions. The proposed descriptor is not only invariant to geometric transformations such as translation, rotation and scaling but also insensitive to nonlinear deformations due to noise and occlusion. In the matching stage, the Dynamic Programming (DP) algorithm is employed to find out the optimal correspondence between sample points of every two shapes. The height function provides an excellent discriminative power, which is demonstrated by excellent retrieval performances on several popular shape benchmarks, including MPEG-7 data set, Kimia’s data set and ETH-80 data set.

Co-Transduction for Shape Retrieval

In this paper, we propose a new shape/object retrieval algorithm, namely, co-transduction. The performance of a retrieval system is critically decided by the accuracy of adopted similarity measures (distances or metrics). In shape/object retrieval, ideally, intraclass objects should have smaller distances than interclass objects. However, it is a difficult task to design an ideal metric to account for the large intraclass variation. Different types of measures may focus on different aspects of the objects: for example, measures computed based on contours and skeletons are often complementary to each other. Our goal is to develop an algorithm to fuse different similarity measures for robust shape retrieval through a semisupervised learning framework. We name our method co-transduction, which is inspired by the co-training algorithm. Given two similarity measures and a query shape, the algorithm iteratively retrieves the most similar shapes using one measure and assigns them to a pool for the other measure to do a re-ranking, and vice versa. Using co-transduction, we achieved an improved result of 97.72% (bull's-eye measure) on the MPEG-7 data set over the state-of-the-art performance. We also present an algorithm called tri-transduction to fuse multiple-input similarities, and it achieved 99.06% on the MPEG-7 data set. Our algorithm is general, and it can be directly applied on input similarity measures/metrics; it is not limited to object shape retrieval and can be applied to other tasks for ranking/retrieval.

Matching sequences of salient contour points characterized by Voronoi region features

In this paper, we introduce a shape matching method by matching sequences of salient contour points that are characterized by Voronoi region features. The proposed approach is summarized as follows: (1) a sequence of salient contour points is selected using the Voronoi diagram of the contour point set, (2) the features of the salient points are computed based on the interior and exterior regions of the Voronoi diagram, and (3) a cyclic edit distance is used to match two shapes. Tests on the MPEG-7 and ETH-80 datasets demonstrated the effectiveness and efficiency of the proposed method.

Complex Object Correspondence Construction in Two-Dimensional Animation

Correspondence construction of objects in key frames is the precondition for inbetweening and coloring in 2-D computer-assisted animation production. Since each frame of an animation consists of multiple layers, objects are complex in terms of shape and structure. Therefore, existing shape-matching algorithms specifically designed for simple structures such as a single closed contour cannot perform well on objects constructed by multiple contours with an open shape. This paper introduces a semisupervised patch alignment framework for complex object correspondence construction. In particular, the new framework constructs local patches for each point on an object and aligns these patches in a new feature space, in which correspondences between objects can be detected by the subsequent clustering. For local patch construction, pairwise constraints, which indicate the corresponding points (must link) or unfitting points (cannot link), are introduced by users to improve the performance of correspondence construction. This kind of input is convenient for animation software users via user-friendly interfaces. A dozen of experimental results on our cartoon data set that is built on industrial production suggest the effectiveness of the proposed framework for constructing correspondences of complex objects. As an extension of our framework, additional shape retrieval experiments on MPEG-7 data set show that its performance is comparable with that of a prominent algorithm published in T-PAMI 2009.

Effective Improved Graph Transduction

In this paper, we focus on the problem of shape retrieval and clustering. We put two questions together because they are based on the same method, called Improved Graph Transduction. For shape retrieval, we regard the shape as a node in a graph and the similarity of shapes is represented by the edge of the graph. Then we learn a new distance measure between the query shape and the testing shapes. The main contribution of our work is to merge the most likely node with the query node during the learning process. The appending process helps us to mine the latent information in the propagation. The experimental results on the MPEG-7 data set show that comparing with the existing methods, our method can complete shape retrieval with similar correct rate in less time；For clustering task，the existing literatures in this domain often use the distance measure between the testing data point individual which is proved not enough in the real applications. In this paper, we think about the core concept in semi-supervised learning method, and use a graph to reflect the original distance measure, and combine the density information of the data distribution with the distance measure. Given a set of testing data, we select the original data randomly and use graph transduction iterative on the defined graph. The given algorithm is rapid and steady comparing with the existing clustering method. The experiments show that the novel algorithm is effective for the clustering task.

Learning Context-Sensitive Shape Similarity by Graph Transduction

Shape similarity and shape retrieval are very important topics in computer vision. The recent progress in this domain has been mostly driven by designing smart shape descriptors for providing better similarity measure between pairs of shapes. In this paper, we provide a new perspective to this problem by considering the existing shapes as a group, and study their similarity measures to the query shape in a graph structure. Our method is general and can be built on top of any existing shape similarity measure. For a given similarity measure, a new similarity is learned through graph transduction. The new similarity is learned iteratively so that the neighbors of a given shape influence its final similarity to the query. The basic idea here is related to PageRank ranking, which forms a foundation of Google Web search. The presented experimental results demonstrate that the proposed approach yields significant improvements over the state-of-art shape matching algorithms. We obtained a retrieval rate of 91.61 percent on the MPEG-7 data set, which is the highest ever reported in the literature. Moreover, the learned similarity by the proposed method also achieves promising improvements on both shape classification and shape clustering.

Shape L'Âne Rouge: Sliding Wavelets for Indexing and Retrieval.

Shape representation and retrieval of stored shape models are becoming increasingly more prominent in fields such as medical imaging, molecular biology and remote sensing. We present a novel framework that directly addresses the necessity for a rich and compressible shape representation, while simultaneously providing an accurate method to index stored shapes. The core idea is to represent point-set shapes as the square root of probability densities expanded in a wavelet basis. We then use this representation to develop a natural similarity metric that respects the geometry of these probability distributions, i.e. under the wavelet expansion, densities are points on a unit hypersphere and the distance between densities is given by the separating arc length. The process uses a linear assignment solver for non-rigid alignment between densities prior to matching; this has the connotation of "sliding" wavelet coefficients akin to the sliding block puzzle L'Âne Rouge. We illustrate the utility of this framework by matching shapes from the MPEG-7 data set and provide comparisons to other similarity measures, such as Euclidean distance shape distributions.

Nonparametric motion characterization for robust classification of camera motion patterns

Motion characterization plays a critical role in video indexing. An effective way of characterizing camera motion facilitates the video representation, indexing and retrieval tasks. This paper describes a novel nonparametric motion representation to achieve an effective and robust recognition of parts of the video in which camera is static, or panning, or tilting, or zooming, etc. This representation employs the mean shift filtering and the vector histograms to produce a compact description of a motion field. The basic idea is to perform spatio-temporal mode-seeking in the motion feature space and use the histograms-based spatial distributions of dominant motion modes to represent a motion field. Unlike most existing approaches, which focus on the estimation of a parametric motion model from a dense optical flow field (OFF) or a block matching-based motion vector field (MVF), the proposed method combines the motion representation and machine learning techniques (e.g., support vector machines) to perform camera motion analysis from the classification point of view. The main motivation lies in the impossibility of uniformly securing a proper parametric assumption in a wide range of video scenarios. The diverse camera shot sizes and frequent occurrences of bad OFF/MVF necessitates a learning mechanism, which can not only capture the domain-independent parametric constraints, but also acquire the domain-dependent knowledge to tolerate the influence of bad OFF/MVF. In order to improve performance, we can use this learning-based method to train enhanced classifiers aiming at a certain context (i.e., shot size, neighbor OFF/MVFs, and video genre). Other visual cues (e.g., dominant color) can also be incorporated for further motion analysis. Our main aim is to use a generic feature space analysis method to explore a flexible OFF/MVF representation in a nonparametric technique, which could be fed into a learning framework to robustly capture the global motion by incorporating the context information. Results on videos with various types of content (23 191 MVFs culled from MPEG-7 dataset, and 20 000 MVFs culled from broadcast tennis, soccer, and basketball videos) are reported to validate the proposed approach.

A cognitive evaluation procedure for contour based shape descriptors

Present image processing algorithms are unable to extract a neat and closed contour of an object of interest from a natural image. Advanced contour detection algorithms extract the contour of an object of interest from a natural scene with a side effect of depletion of the contour. Hence in order to perform well in a real world scenario, object recognition algorithms should be robust to contour incompleteness. With inspiration from psychophysical studies of the human cognitive abilities we propose a novel method to evaluate the performance of object recognition algorithms in terms of their robustness to incomplete contour representations. Complete contour representations of objects are used as a reference (training) set. Incomplete contour representations of the same objects are used as a test set. The performance of an algorithm is evaluated using the recognition rate as a function of the percentage of contour retained. The test framework is illustrated by using two contour based shape recognition algorithms which use a shape context and a distance multiset as shape descriptors. Three types of contour incompleteness, viz. segment-wise contour deletion, occlusion and random pixel depletion, are considered. In our experiments we use images from the COIL and MPEG-7 datasets. Both algorithms qualitatively perform similar to the human visual system in the sense that recognition performance monotonously increases with the degree of completeness and that they perform best in the case of random depletion and worst in the case of occluded contours. The distance multiset shape descriptor outperforms the shape context in this test especially for high levels of incompleteness.

Component-based LDA face description for image retrieval and MPEG-7 standardisation

We propose a method of face description for facial image retrieval from a large data base and for MPEG-7 (Moving Picture Experts Group) standardisation. The novel descriptor is obtained by decomposing a face image into several components and then combining the component features. The decomposition combined with LDA (Linear Discriminant Analysis) provides discriminative facial descriptions that are less sensitive to light and pose changes. Each facial component is represented in its Fisher space and another LDA is then applied to compactly combine the features of the components. To enhance retrieval accuracy further, a simple pose classification and transformation technique is performed, followed by recursive matching. Our algorithm has been developed to deal with the problem of face image retrieval from huge databases such as those found in Internet environments. Such retrieval requires a compact face representation which has robust recognition performance under lighting and pose variations. The partitioning of a face image into components offers a number of benefits that facilitate the development of an efficient and robust face retrieval algorithm. Variation in image statistics due to pose and/or illumination changes within each component region can be simplified and more easily captured by a linear encoding than that of the whole image. So an LDA encoding at the component level facilitates better classification. Furthermore, a facial component can be weighted according to its importance. The component with a large variation is weighted less in the matching stage to yield a more reliable decision. The experimental results obtained on the MPEG-7 data set show an impressive accuracy of our algorithm as compared with other methods including conventional PCA (Principal Component Analysis)/ICA (Independent Component Analysis)/LDA methods and the previous MPEG-7 proposals.

Shape-sequence-based key image generation algorithm for browsing and retrieval of video clips

A new shape-sequence-based algorithm that can effectively generate key images from video clips is introduced. Generated key images can be used as the feature information in the browsing and retrieval of video clips from a multimedia database. Experiments with MPEG-7 data sets were performed, and the results are compared with existing methods.