Manifold Ranking Research Articles

Salient Superpixel Visual Tracking with Graph Model and Iterative Segmentation

Visual object tracking is to locate an object of interest in a sequence of consecutive video frames, which is widely applied in many high-level computer vision tasks such as intelligent video surveillance and robotics. It is of great challenges for visual tracking methods to handle large target appearance variations caused by pose deformation, fast motion, occlusion, and surrounding environments in real-time videos. In this paper, inspired by human attention cognitive saliency model, we propose a visual tracking method based on salient superpixels which integrates the target appearance similarity and cognitive saliency, and helps to location inference and appearance model updating. The saliency of superpixel is detected by graph model and manifold ranking. We cluster the superpixels of the first four target boxes into a set corresponding to object foreground and model the target appearance with color descriptors. While tracking, the relevance is computed between the candidate superpixels and the target appearance set. We also propose an iterative threshold segmentation method to distinguish the foreground and background of superpixels based on saliency and relevance. To increase the accuracy of location inference, we explore particle filter in both confidence estimation and sampling procedures. We compared our method with the existing techniques in OTB100 dataset in terms of precision based on center location error and success rate based on overlap, and the experimental results show that our proposed method achieved substantially better performance. Promising results have shown that the proposed salient superpixel-based approach is effective to deformation, occlusion, and other challenges in object tracking.

Manifold ranking graph regularization non-negative matrix factorization with global and local structures

Non-negative matrix factorization (NMF) is a recently popularized technique for learning parts-based, linear representations of non-negative data. Although the decomposition rate of NMF is very fast, it still suffers from the following deficiency: It only revealed the local geometry structure; global geometric information of data set is ignored. This paper proposes a manifold ranking graph regularization non-negative matrix factorization with local and global geometric structure (MRLGNMF) to overcome the above deficiency. In particular, MRLGNMF induces manifold ranking to the non-negative matrix factorization with Sinkhorn distance. Numerical results show that the new algorithm is superior to the existing algorithm.

Multimedia Retrieval Through Unsupervised Hypergraph-Based Manifold Ranking

Accurately ranking images and multimedia objects are of paramount relevance in many retrieval and learning tasks. Manifold learning methods have been investigated for ranking mainly due to their capacity of taking into account the intrinsic global manifold structure. In this paper, a novel manifold ranking algorithm is proposed based on the hypergraphs for unsupervised multimedia retrieval tasks. Different from traditional graph-based approaches, which represent only pairwise relationships, hypergraphs are capable of modeling similarity relationships among a set of objects. The proposed approach uses the hyperedges for constructing a contextual representation of data samples and exploits the encoded information for deriving a more effective similarity function. An extensive experimental evaluation was conducted on nine public datasets including diverse retrieval scenarios and multimedia content. Experimental results demonstrate that high effectiveness gains can be obtained in comparison with the state-of-the-art methods.

Salient Object Detection via Two-Stage Graphs

Despite recent advances made in salient object detection using graph theory, the approach still suffers from accuracy problems when the image is characterized by a complex structure, either in the foreground or background, causing erroneous saliency segmentation. This fundamental challenge is mainly attributed to the fact that most existing graph-based methods take only the adjacently spatial consistency among graph nodes into consideration. In this paper, we tackle this issue from a coarse-to-fine perspective and propose a two-stage-graphs approach for salient object detection, in which two graphs having the same nodes but different edges are employed. Specifically, a weighted joint robust sparse representation model, rather than the commonly used manifold ranking model, helps to compute the saliency value of each node in the first-stage graph, thereby providing a saliency map at the coarse level. In the second-stage graph, along with the adjacently spatial consistency , a new regionally spatial consistency among graph nodes is considered in order to refine the coarse saliency map, assuring uniform saliency assignment even in complex scenes. Particularly, the second stage is generic enough to be integrated in existing salient object detectors, enabling improved performance. Experimental results on benchmark data sets validate the effectiveness and superiority of the proposed scheme over related state-of-the-art methods.

Low-rank weighted co-saliency detection via efficient manifold ranking

Co-saliency detection, which aims to detect common salient objects in a group of images, has attracted much attention in the field of computer vision. In this paper, we present an effective co-saliency detection approach that first exploits an efficient manifold ranking scheme to extract a set of co-saliency regions, and then renders rank constraint to the feature matrix of the extracted regions to achieve a high-quality co-saliency map. Specifically, for each input image, we first develop a two-stage manifold ranking algorithm to generate multiple coarse co-saliency maps, and then we extract a group of co-salient regions from each image by fusing the co-saliency maps and the superpixels extracted from it. Then, we design an adaptive weight for each co-saliency map based on the sparse error matrix that is obtained by rendering rank constraint on the feature matrix of the salient regions. Finally, we multiply the coarse co-saliency maps with their corresponding weights to get the fine fusion results, which are further optimized by Graph cuts. Extensive evaluations on the iCoseg dataset demonstrate favorable performance of the proposed approach over some state-of-art methods in terms of both qualitative and quantitative results.

A novel multi-graph framework for salient object detection

Graph-based methods have been widely adopted for predicting the most attractive region in an image. Most of the existing graph-based methods only utilize single graph to describe the image information, and thus cannot adapt for complex scenes. In this paper, a novel multi-graph framework for salient object detection is proposed. The proposed method is divided into three steps. Firstly, an image is divided into superpixels and described as a multi-graph, where superpixels are represented as nodes and their information is computed by color space and location space. Secondly, the multiple graphs are combined into a novel multi-graph-based manifold ranking propagation framework to obtain a coarse map. Finally, a map refinement model is developed to improve the quality of the coarse map. Experimental results on four challenging datasets show that the proposed method performs favorably against the state-of-the-art salient object detection methods.

Fast Denoising of Sonar Image Based on Saliency Detection

Sonar image is inevitably contaminated by noise during the acquisition process, while the noise reduction algorithms with good performance are usually of high time complexity. The human visual attention mechanism usually guides the eyes to salient region and gives priority to those visual information. In view of this, saliency detection based on manifold ranking was introduced into sonar image processing, to divide the image into two parts:salient region and non-significant region. For the salient region with small proportion, block-matching and 3-D filtering (BM3D) algorithm was adopted to reduce noise and protect the main information of the image; for the non-significant background which was not very concerned, high efficiency mean filtering was used. On the collected sonar image data set, the present algorithm was compared with the classic MF and BM3D algorithms through the subjective and 2 objective evaluation indexes. The experimental results show that the efficiency of the present algorithm is much higher than that of BM3D, while the image visual effect is guaranteed, which is satisfied with the real-time application requirement of autonomous underwater vehicles.

Research on Scene Understanding-Based Encrypted Image Retrieval Algorithm

Aiming to improve the precision and security, and reduce complexity of the encrypted image retrieval algorithm, a scene understanding-based encrypted image retrieval algorithm is proposed in this paper, which encrypts images by chaotic system-based image encryption algorithm, extracts image features-based on two-dimensional multi-scale hidden Markov model, and classifies the indoor scene to establish an offline database. In the online stage, the query image captured by the smart device is first encrypted, and the multi-scale feature extraction and the image scene matching are performed. Then images in the scene are matched one by one using manifold ranking. Finally, the optimal encrypted matching image is obtained by the homography-based image matching algorithm, and the plaintext image is also obtained by the decryption algorithm. Performance analysis and simulation results show that the proposed algorithm can improve the precision of image retrieval and protect the security of users’ privacy.

Accurate Underwater ATR in Forward-Looking Sonar Imagery Using Deep Convolutional Neural Networks

Underwater automatic target recognition (ATR) is a challenging task for marine robots due to the complex environment. The existing recognition methods basically use hand-crafted features and classifiers to recognize targets, which are difficult to achieve ideal recognition accuracy. In this paper, we proposed a novel method to realize accurate multiclass underwater ATR by using forward-looking sonar—Echoscope and deep convolutional neural networks (DCNNs). A complete recognition process from data preprocessing to network training and image recognition was realized. Firstly, we established a real, measured Echoscope sonar image dataset. Inspired by the human visual attention mechanism, the suspected target region was extracted via the graph-based manifold ranking method in image preprocessing. Secondly, an end-to-end DCNNs model, named EchoNet, was designed for Echoscope sonar image feature extraction and recognition. Finally, a network training method based on transfer learning was developed to solve the problem of insufficient training data, and mini-batch gradient descent was used for network optimization. Experimental results demonstrated that our method can implement efficiently, and the recognition accuracy on a nine-class underwater ATR task reached 97.3%, outperforming traditional feature-based methods. The proposed method is expected to be a potential novel technology for the intelligent perception of autonomous underwater vehicles.

LOSGSR: Learned Opinion Score Guided Shoeprint Retrieval

As a kind of forensic evidence, a shoeprint conveys many important human characteristics, and it plays a vital role in forensic investigations. Millions of shoeprints are acquired from crime scenes, and it is a challenging task to retrieve the most similar shoeprints for a query shoeprint. Most shoeprint retrieval methods sort shoeprint images by feature similarities with respect to the query shoeprint; however, the results are not always what the investigator expects because the retrieval algorithm cannot determine what the investigator prefers based on only the content of the query shoeprint. This paper proposes a method to guide the shoeprint retrieval process to approximate what the user wants by applying learned opinion scores. Additionally, this paper improves shoeprint retrieval effectiveness by implementing the following four perspectives: 1) using the opinion scores of multiple examples to guide the results to meet the forensic experts’ expectations; 2) proposing a learning-based method to refine opinion scores, which corrects the labeled opinion scores of multiple examples and their neighbors; 3) using a manifold ranking method to propagate the opinion scores to other dataset shoeprints; and 4) introducing a coefficient matrix to prevent the tendency of the ranking scores to become low values. The experiments show that the cumulative match scores of the proposed method are more than 96.6% in the top 2% of the dataset composed of 10,096 crime scene shoeprints.

Research on Gesture Recognition Based on Improved GBMR Segmentation and Multiple Feature Fusion

Aiming at addressing the problem of interactive gesture recognition between lunar robot and astronaut, a novel gesture detection and recognition algorithm is proposed. In gesture detection stage, based on saliency detection via Graph-Based Manifold Ranking (GBMR) algorithm, the depth information of foreground is added to the calculation of superpixel. By increasing the weight of connectivity domains in graph theory model, the foreground boundary is highlighted and the impact of background is weakened. In gesture recognition stage, Pyramid Histogram of Oriented Gradient (PHOG) feature and Gabor amplitude also phase feature of image samples are extracted. To highlight the Gabor amplitude feature, we propose a novel feature calculation by fusing feature in different directions at the same scale. Because of the strong classification capability and not-easy-to-fit advantage of Adaboosting, this paper applies it as the classifier to realize gesture recognition. Experimental results show that the improved gesture detection algorithm can maintain the robustness to influences of complex environment. Based on multi-feature fusion, the error rate of gesture recognition remains at about 4.2%, and the recognition rate is around 95.8%.

Saliency Detection Based on Improved Manifold Ranking via Convex Hull

Saliency Detection Based on Improved Manifold Ranking via Convex Hull

Salient object detection of dairy goats in farm image based on background and foreground priors

In order to achieve object detection of dairy goats in farm image efficiently and accurately, this paper proposes a novel salient object detection approach based on background and foreground priors. First, we improve the FastMBD algorithm to generate background-prior-based saliency map, which eliminate background interference initially. Second, the approach of seed selection is optimized to determine the foreground seeds more accurately, and then saliency map is generated based on manifold ranking to further eliminate the background interference. Finally, two saliency maps are fused, and then combined with post-processing to achieve saliency object detection of farm image. Subsequently, we also use the threshold segmentation based on K-Means algorithm to separate the objects and background. Moreover, the horizontal scanning method and contour extraction are employed to realize the extraction and counting of dairy goats. Experimental results show that the proposed approach based on background and foreground priors has promising performance, and the accuracy of the proposed segmentation method was up to 89.503%.

Bayberry image segmentation based on manifold ranking salient object detection method

A bayberry image segmentation method based on salient object detection is proposed in this study. First, an image enhancement algorithm, homomorphic filtering, was used for light compensation. Second, superpixels were used as the basic unit in the subsequent significant calculation with the SLIC (simple linear iterative clustering) algorithm and were constructed as nodes in an Undirected Graphical Model G ( V , E ). Third, a two-stage scheme of saliency detection was proposed. In the first stage, the nodes on each boundary were separately ranked as query nodes. Then, four saliency maps were obtained using a MR (manifold ranking) method. After combining the four saliency maps, the saliency map of the first stage was acquired. In the second stage, binary segmentation was performed, based on the saliency map of the first stage to obtain the target query. Finally, binary segmentation was performed based on the saliency map with the Otsu's method to achieve the segmentation of bayberry images. The average values of Precision, Recall and F-measure in the proposed algorithm are 0.93, 0.83 and 0.90, respectively. Experimental results show that the proposed segmentation algorithm performs more effectively than Otsu's and K-means segmentation algorithms.

An adaptive segmentation method combining MSRCR and mean shift algorithm with K-means correction of green apples in natural environment

During the recognition and localization process of green apple targets, problems such as uneven illumination, occlusion of branches and leaves need to be solved. In this study, the multi-scale Retinex with color restoration (MSRCR) algorithm was applied to enhance the original green apple images captured in an orchard environment, aiming to minimize the impacts of varying light conditions. The enhanced images were then explicitly segmented using the mean shift algorithm, leading to a consistent gray value of the internal pixels in an independent fruit. After that, the fuzzy attention based on information maximization algorithm (FAIM) was developed to detect the incomplete growth position and realize threshold segmentation. Finally, the poorly segmented images were corrected using the K-means algorithm according to the shape, color and texture features. The users intuitively acquire the minimum enclosing rectangle localization results on a PC. A total of 500 green apple images were tested in this study. Compared with the manifold ranking algorithm, the K-means clustering algorithm and the traditional mean shift algorithm, the segmentation accuracy of the proposed method was 86.67%, which was 13.32%, 19.82% and 9.23% higher than that of the other three algorithms, respectively. Additionally, the false positive and false negative errors were 0.58% and 11.64%, respectively, which were all lower than the other three compared algorithms. The proposed method accurately recognized the green apples under complex illumination conditions and growth environments. Additionally, it provided effective references for intelligent growth monitoring and yield estimation of fruits.

Robust pixelwise saliency detection via progressive graph rankings

In this paper, we propose a novel saliency detection method based on superpixel-to-pixel level optimization. First, we segment the input image into superpixels under four scales. For each scale, we construct a k-regular basic graph with these superpixels as nodes. Furthermore, we enlarge the basic graph with virtual absorbing nodes and utilize absorbing Markov chain ranking to calculate background-based saliency. Second, for each scale, we obtain robust foreground queries from the previous result, and use manifold ranking to obtain foreground-based saliency. Third, a regularized random walk ranking based on the pixelwise graph for each scale is used to diffuse the saliency values among pixels. Finally, we obtain four saliency maps for the input image and integrate them together for the final saliency map. Extensive experiments on several challenging datasets reveal that the proposed method performs better in terms of precision, recall and F-measure values. Despite complex backgrounds, our method performs better in detecting small and/or multiple salient objects than other state-of-the-art methods as a whole.

COSUM: Text summarization based on clustering and optimization

AbstractText summarization is a process of extracting salient information from a source text and presenting that information to the user in a condensed form while preserving its main content. In the text summarization, most of the difficult problems are providing wide topic coverage and diversity in a summary. Research based on clustering, optimization, and evolutionary algorithm for text summarization has recently shown good results, making this a promising area. In this paper, for a text summarization, a two‐stage sentences selection model based on clustering and optimization techniques, called COSUM, is proposed. At the first stage, to discover all topics in a text, the sentences set is clustered by using k‐means method. At the second stage, for selection of salient sentences from clusters, an optimization model is proposed. This model optimizes an objective function that expressed as a harmonic mean of the objective functions enforcing the coverage and diversity of the selected sentences in the summary. To provide readability of a summary, this model also controls length of sentences selected in the candidate summary. For solving the optimization problem, an adaptive differential evolution algorithm with novel mutation strategy is developed. The method COSUM was compared with the 14 state‐of‐the‐art methods: DPSO‐EDASum; LexRank; CollabSum; UnifiedRank; 0–1 non‐linear; query, cluster, summarize; support vector machine; fuzzy evolutionary optimization model; conditional random fields; MA‐SingleDocSum; NetSum; manifold ranking; ESDS‐GHS‐GLO; and differential evolution, using ROUGE tool kit on the DUC2001 and DUC2002 data sets. Experimental results demonstrated that COSUM outperforms the state‐of‐the‐art methods in terms of ROUGE‐1 and ROUGE‐2 measures.

Graph model-based salient object detection using objectness and multiple saliency cues

Recent years have witnessed increasing interest in salient object detection, which aims at stimulating the human visual attention mechanism to detect and segment the most attractive object in natural scenes, and can be widely applied in numerous computer vision tasks. In this paper, by considering both objectness cue and saliency detection, we propose a graph model-based bottom-up salient object detection framework by fusing multiple saliency maps using low-level features and objectness features under a manifold ranking framework. Specifically, for each feature, we utilize geodesic distance between any two superpixels to construct the affinity matrix and un-normalized Laplacian matrix of the graph. Then, we apply saliency optimization to refine each saliency map generated by manifold ranking with the first-stage query, and integrate saliency maps corresponding to different features by multilayer cellular automata in the final stage. Extensive experimental results demonstrate that our method can deliver promising performance in comparison to several state-of-the-art bottom-up methods on many benchmark datasets.

Local features and manifold ranking coupled method for sketch-based 3D model retrieval

3D model retrieval can benefit many downstream virtual reality applications. In this paper, we propose a new sketch-based 3D model retrieval framework by coupling local features and manifold ranking. At technical fronts, we exploit spatial pyramids based local structures to facilitate the efficient construction of feature descriptors. Meanwhile, we propose an improved manifold ranking method, wherein all the categories between arbitrary model pairs will be taken into account. Since the smooth and detail-preserving line drawings of 3D model are important for sketch-based 3D model retrieval, the Difference of Gaussians (DoG) method is employed to extract the line drawings over the projected depth images of 3D model, and Bezier Curve is then adopted to further optimize the extracted line drawing. On that basis, we develop a 3D model retrieval engine to verify our method. We have conducted extensive experiments over various public benchmarks, and have made comprehensive comparisons with some state-of-the-art 3D retrieval methods. All the evaluation results based on the widely-used indicators prove the superiority of our method in accuracy, reliability, robustness, and versatility.

Video co-segmentation based on directed graph

This paper proposes a novel video co-segmentation method, which aims to extract multi-class objects from a group of videos. A set of tracklets are first generated based on object proposals, and then a novel directed graph is constructed to connect object tracklets. The directed graph is transformed to an undirected graph, and the extraction of common object tracklets is solved by using maximum weighted clique. The obtained common object tracklets are used as seed regions to perform manifold ranking and to generate the object-level saliency maps. Based on common object tracklets and object-level saliency maps, GrabCut is exploited to get the refined co-segmentation results. Experimental results on a public video dataset show that the proposed video co-segmentation method consistently outperforms the state-of-the-art methods.