MPEG-7 Shape Dataset Research Articles

Plant Species Recognition Using Triangle-Distance Representation

Plant species recognition using leaf images is a highly important and challenging issue in botany and pattern recognition. A center problem of this task is how to accurately extract leaf image characteristics and quickly calculate the similarity between them. This article presents a new shape description approach called triangle-distance representation (TDR) for plant leaf recognition. The TDR descriptor is represented by two matrices: a sign matrix and a triangle center distance matrix. The sign matrix is used to characterize the convex/concave property of a shape contour, while the triangle center distance matrix is used to represent the bending degree and spatial information of a shape contour. This method can effectively capture the detailed and global characteristics of a leaf shape while keeping the similarity transformations (translation, rotation, and scaling) unchanged. In addition, this method is quite compact and has low computational complexity. We tested our method on four standard plant leaf datasets, including the famous Swedish, Smithsonian, Flavia, and ImageCLEF2012 datasets. The results confirm that our approach exceeds the prior state-of-the-art shape-based plant leaf recognition approaches. An extra experiment on the MPEG-7 shape dataset further shows that our method can be applied to general shape recognition.

V4 shape features for contour representation and object detection

Cortical area V4 lies in the middle of the visual pathway involved with object recognition. Neurons in V4 selectively respond to different curve fragments along the object contour. In this paper, we propose a computational model that captures the shape features extracted by V4 neurons. The computational model emulated the information processing mechanism in the visual cortex. It extracted curve segments that V4 neurons respond to and quantitatively represented features of the curve segments. The proposed V4 shape features could describe object contours accurately and efficiently. With quantitative evaluation using the MPEG7 shape dataset, we showed that complex shapes could be represented with a very limited number of V4 shape features. Based on V4 features, we further developed a self-organizing map neural network to learn object shape models. The shape model was defined by a group of V4 features with constraints on their spatial relationships. The model was evaluated in object detection experiments using ETHZ objects and INRIA horses datasets. The proposed model could learn to recognize objects by shapes and accurately outline the object contour in the images. Thus, this model provides insight into the neural mechanisms of shape-based object recognition.

Object matching with hierarchical skeletons

The skeleton of an object provides an intuitive and effective abstraction which facilitates object matching and recognition. However, without any human interaction, traditional skeleton-based descriptors and matching algorithms are not stable for deformable objects. Specifically, some fine-grained topological and geometrical features would be discarded if the skeleton was incomplete or only represented significant visual parts of an object. Moreover, the performance of skeleton-based matching highly depends on the quality and completeness of skeletons. In this paper, we propose a novel object representation and matching algorithm based on hierarchical skeletons which capture the shape topology and geometry through multiple levels of skeletons. For object representation, we reuse the pruned skeleton branches to represent the coarse- and fine-grained shape topological and geometrical features. Moreover, this can improve the stability of skeleton pruning without human interaction. We also propose an object matching method which considers both global shape properties and fine-grained deformations by defining singleton and pairwise potentials for similarity computation between hierarchical skeletons. Our experiments attest our hierarchical skeleton-based method a significantly better performance than most existing shape-based object matching methods on six datasets, achieving a 99.21% bulls-eye score on the MPEG7 shape dataset.

Beyond diffusion process: Neighbor set similarity for fast re-ranking

Measuring the similarity between two instances reliably, shape or image, is a challenging problem in shape and image retrieval. In this paper, a simple yet effective method called Neighbor Set Similarity (NSS) is proposed, which is superior to both traditional pairwise similarity and diffusion process. NSS makes full use of contextual information to capture the geometry of the underlying manifold, and obtains a more precise measure than the original pairwise similarity. Moreover, based on NSS, we propose a powerful fusion process to utilize the complementarity of different descriptors to further enhance the retrieval performance. The experimental results on MPEG-7 shape dataset, N-S image dataset and ORL face dataset demonstrate the effectiveness of the proposed method. In addition, the time complexity of NSS is much lower than diffusion process, which suggests that NSS is more suitable for large scale image retrieval than diffusion process.

Unsupervised manifold learning using Reciprocal kNN Graphs in image re-ranking and rank aggregation tasks

In this paper, we present an unsupervised distance learning approach for improving the effectiveness of image retrieval tasks. We propose a Reciprocal kNN Graph algorithm that considers the relationships among ranked lists in the context of a k-reciprocal neighborhood. The similarity is propagated among neighbors considering the geometry of the dataset manifold. The proposed method can be used both for re-ranking and rank aggregation tasks. Unlike traditional diffusion process methods, which require matrix multiplication operations, our algorithm takes only a subset of ranked lists as input, presenting linear complexity in terms of computational and storage requirements. We conducted a large evaluation protocol involving shape, color, and texture descriptors, various datasets, and comparisons with other post-processing approaches. The re-ranking and rank aggregation algorithms yield better results in terms of effectiveness performance than various state-of-the-art algorithms recently proposed in the literature, achieving bull's eye and MAP scores of 100% on the well-known MPEG-7 shape dataset.

Shape Similarity Analysis by Self-Tuning Locally Constrained Mixed-Diffusion

Similarity analysis is a powerful tool for shape matching/retrieval and other computer vision tasks. In the literature, various shape (dis)similarity measures have been introduced. Different measures specialize on different aspects of the data. In this paper, we consider the problem of improving retrieval accuracy by systematically fusing several different measures. To this end, we propose the locally constrained mixed-diffusion method, which partly fuses the given measures into one and propagates on the resulted locally dense data space. Furthermore, we advocate the use of self-adaptive neighborhoods to automatically determine the appropriate size of the neighborhoods in the diffusion process, with which the retrieval performance is comparable to the best manually tuned kNNs. The superiority of our approach is empirically demonstrated on both shape and image datasets. Our approach achieves a score of 100% in the bull's eye test on the MPEG-7 shape dataset, which is the best reported result to date.

Affinity Learning with Diffusion on Tensor Product Graph

In many applications, we are given a finite set of data points sampled from a data manifold and represented as a graph with edge weights determined by pairwise similarities of the samples. Often the pairwise similarities (which are also called affinities) are unreliable due to noise or due to intrinsic difficulties in estimating similarity values of the samples. As observed in several recent approaches, more reliable similarities can be obtained if the original similarities are diffused in the context of other data points, where the context of each point is a set of points most similar to it. Compared to the existing methods, our approach differs in two main aspects. First, instead of diffusing the similarity information on the original graph, we propose to utilize the tensor product graph (TPG) obtained by the tensor product of the original graph with itself. Since TPG takes into account higher order information, it is not a surprise that we obtain more reliable similarities. However, it comes at the price of higher order computational complexity and storage requirement. The key contribution of the proposed approach is that the information propagation on TPG can be computed with the same computational complexity and the same amount of storage as the propagation on the original graph. We prove that a graph diffusion process on TPG is equivalent to a novel iterative algorithm on the original graph, which is guaranteed to converge. After its convergence we obtain new edge weights that can be interpreted as new, learned affinities. We stress that the affinities are learned in an unsupervised setting. We illustrate the benefits of the proposed approach for data manifolds composed of shapes, images, and image patches on two very different tasks of image retrieval and image segmentation. With learned affinities, we achieve the bull's eye retrieval score of 99.99 percent on the MPEG-7 shape dataset, which is much higher than the state-of-the-art algorithms. When the data- points are image patches, the NCut with the learned affinities not only significantly outperforms the NCut with the original affinities, but it also outperforms state-of-the-art image segmentation methods.

Affine-invariant Shape Recognition Using Grassmann Manifold

Traditional Kendall shape space theory is only applied to similar transform. However, geometric transforms of the object in the imaging process should be represented by affine transform at most situations. We analyze the nonlinear geometry structure of the affine invariant shape space and propose an affine-invariant shape recognition algorithm based on Grassmann manifold geometry. Firstly, we compute the mean shape and covariance for every shape class in the train sets. Then, we construct their norm probability models on the tangent space at each mean shape. Finally, we compute the maximum likelihood class according to the measured object and prior learned shape models. We use the proposed algorithm to recognize shapes in standard shape dataset and real images. Experiment results on MPEG-7 shape dataset show that our recognition algorithm outperforms the algorithm based on Procrustean metric in traditional Kendall shape space theory. Experiment results on real images also show that the proposed algorithm exhibits higher capacity to affine transform than the Procrustean metric based algorithm and can recognize object classes with higher posterior probability.

Formula omitted] based shape approximation for image content representation

Shape approximation is usually a prerequisite step to image content analysis and understanding and has been well studied in the passed decades. However, those approaches show their deficiencies while facing the factors such as the representation efficiency, the variation of image scale and the initial estimation. To alleviate these issues, we propose a novel method for ε-isometry based shape approximation. We first analyze the descending property on approximating error and its relation with salient geometric features. After that, we approximate the polygonal shape and detect the feature point based on the ε-isometric construction. In the experiments, we employ traditional shape benchmarks, MPEG7 shape dataset, SQUID dataset and other real contours to evaluate the visual effects and quantitative performances of the proposed method. Experimental results demonstrate that our method is not only robust to the initial estimation, but also outperforms the state-of-the-art methods with respect to the compactness and scale variability.

ART의 위상정보를 이용한 회전각도 추정 방법

본 논문에서는 두 영상간의 회전각도를 추정하는 방법들 중에서, 정확도에서 높은 성능을 제공하는 저니키 모멘트의 위상을 이용한 방법들의 회전각도 추정 결과의 정확도를 비교하여 제시하고, 기존의 방법들보다 정확하게 각도를 추정하는 angular radial transform(ART) 계수들의 위상성분을 이용한 회전각도 추정방법을 제안한다. 제안하는 방법은 Revaud가 제안한 저니키 모멘트를 이용한 회전각도 추정 방법[1]을 ART로 확장한 방법이다. ART는 저니키 모멘트에 비해서 회전에 의한 영상의 변화를 보다 효과적으로 서술할 수 있는 기저함수의 생성이 가능하기 때문에 저니키 모멘트보다 두 영상간의 회전각도를 정확히 추정하는 것을 가능하게 한다. MPEG-7 데이터셋을 이용한 실험 결과, 제안하는 방법이 제곱평균제곱근오차(root mean square error) 대 커버리지(coverage)를 기준으로한 성능비교에서 가장 우수한 성능을 보였다. Several methods which utilize the phase of Zernike moments (ZMs) to estimate the rotation angle have shown good performance in terms of accuracy. In this paper, we provides the performance comparison results of the existing rotation angle estimation methods based on ZMs and propose an extension of Revaud et al.'s method [1] which utilizes the phase of ZMs; the proposed method uses angular radial transform coefficients instead of ZMs and yields better performance than the ZMs based methods in terms of accuracy. A set of ART can describe angular variation of image more intensively than ZMs, it enables more accurate estimation of the rotation angle than ZMs. In the experiments, the proposed method outperforms ZMs based method. Comparisons were made in terms of the root mean square error vs. the coverage on MPEG-7 shape dataset.

ART의 위상 정보를 이용한 형태기반 영상 검색 방법

영상에 포함된 객체의 형태는 내용 기반 영상검색에 있어서 중요한 정보를 가지고 있기 때문에, 이를 이용하여 영상을 검색하는 방법들이 활발히 연구되어 왔다. 그중에서도 최근에 제안된 저니키 모멘트의 위상과 크기를 이용하는 회전불변 서술자(IZMD: Invariant Zernike moment descriptor)을 이용한 영상 검색 방법은 기존의 크기 정보만을 이용한 저니키 모멘트 서술자보다 높은 영상 검색 성능을 보인다. 본 논문에서는 IZMD를 이용한 방법 보다 향상된 영상 검색 성능을 얻기 위해서 ART(Angular Radial Transform)의 크기와 위상을 이용한 회전 불변 특징 서술자(IARTD: Invariant Angular Radial Transform Descriptor)와 이를 이용해서 영상을 정합하는 방법을 제안한다. IARTD는 ART 기저함수의 특성을 이용해서 정렬된 ART 계수의 위상과 크기로 구성된 특징벡터이다. 영상의 검색은 두 IARTD의 크기차이와 위상차이의 곱을 이용하여 정의된 거리 계산 방법을 이용해서 수행한다. MPEG-7 데이터셋을 이용한 실험 결과, 제안하는 방법의 평균 BEP(Bull's Eye performance)는 0.5806으로서, ARTD나 IZMD를 이용한 영상 검색 결과의 평균 BEP 0.3574, 0.4234보다 우수한 검색 성능을 제공하는 것을 확인하였다. Since shape of an object in an image carries important information in contents based image retrieval (CBIR), many shape description methods have been proposed to retrieve images using shape information. Among the existing shape based image retrieval methods, the method which employs invariant Zernike moment desciptor (IZMD) showed better performance compared to other methods which employ traditional Zernike moments descriptor in CBIR. In this paper, we propose a new image retrieval method which applies invariant angular radial transform descriptor (IARTD) to obtain higher performance than the method which employs IZMD in CBIR. IARTD is a rotationally invariant feature which consists of magnitudes and alligned phases of angular radial transform coefficients. To produce rotationally invariant phase coefficients, a phase correction scheme is performed while extracting the IARTD. The distance between two IARTDs is defined by combining the differences of the magnitudes and the aligned phases. Through the experiment using MPEG-7 shape dataset, the average bull's eye performance (BEP) of the proposed method is 0.5806 while the average BEPs of the exsiting methods which employ IZMD and traditional ART are 0.4234 and 0.3574, respectively.

Improving Shape Retrieval by Spectral Matching and Meta Similarity

We propose two computational approaches for improving the retrieval of planar shapes. First, we suggest a geometrically motivated quadratic similarity measure, that is optimized by way of spectral relaxation of a quadratic assignment. By utilizing state-of-the-art shape descriptors and a pairwise serialization constraint, we derive a formulation that is resilient to boundary noise, articulations and nonrigid deformations. This allows both shape matching and retrieval. We also introduce a shape meta-similarity measure that agglomerates pairwise shape similarities and improves the retrieval accuracy. When applied to the MPEG-7 shape dataset in conjunction with the proposed geometric matching scheme, we obtained a retrieval rate of 92.5%.

An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison

We propose EMD-L1: a fast and exact algorithm for computing the Earth Mover's Distance (EMD) between a pair of histograms. The efficiency of the new algorithm enables its application to problems that were previously prohibitive due to high time complexities. The proposed EMD-L1 significantly simplifies the original linear programming formulation of EMD. Exploiting the L1 metric structure, the number of unknown variables in EMD-L1 is reduced to O(N) from O(N2) of the original EMD for a histogram with N bins. In addition, the number of constraints is reduced by half and the objective function of the linear program is simplified. Formally, without any approximation, we prove that the EMD-L1 formulation is equivalent to the original EMD with a L1 ground distance. To perform the EMD-L1 computation, we propose an efficient tree-based algorithm, Tree-EMD. Tree-EMD exploits the fact that a basic feasible solution of the simplex algorithm-based solver forms a spanning tree when we interpret EMD-L1 as a network flow optimization problem. We empirically show that this new algorithm has an average time complexity of O(N2), which significantly improves the best reported supercubic complexity of the original EMD. The accuracy of the proposed methods is evaluated by experiments for two computation-intensive problems: shape recognition and interest point matching using multidimensional histogram-based local features. For shape recognition, EMD-L1 is applied to compare shape contexts on the widely tested MPEG7 shape data set, as well as an articulated shape data set. For interest point matching, SIFT, shape context and spin image are tested on both synthetic and real image pairs with large geometrical deformation, illumination change, and heavy intensity noise. The results demonstrate that our EMD-L1-based solutions outperform previously reported state-of-the-art features and distance measures in solving the two tasks.