Co-saliency Detection Research Articles

Revisiting Co-Saliency Detection: A Novel Approach Based on Two-Stage Multi-View Spectral Rotation Co-clustering.

With the goal of discovering the common and salient objects from the given image group, co-saliency detection has received tremendous research interest in recent years. However, as most of the existing co-saliency detection methods are performed based on the assumption that all the images in the given image group should contain co-salient objects in only one category, they can hardly be applied in practice, particularly for the large-scale image set obtained from the Internet. To address this problem, this paper revisits the co-saliency detection task and advances its development into a new phase, where the problem setting is generalized to allow the image group to contain objects in arbitrary number of categories and the algorithms need to simultaneously detect multi-class co-salient objects from such complex data. To solve this new challenge, we decompose it into two sub-problems, i.e., how to identify subgroups of relevant images and how to discover relevant co-salient objects from each subgroup, and propose a novel co-saliency detection framework to correspondingly address the two sub-problems via two-stage multi-view spectral rotation co-clustering. Comprehensive experiments on two publically available benchmarks demonstrate the effectiveness of the proposed approach. Notably, it can even outperform the state-of-the-art co-saliency detection methods, which are performed based on the image subgroups carefully separated by the human labor.

Two Birds With One Stone: A Unified Approach to Saliency and Co-Saliency Detection via Multi-Instance Learning

Saliency detection on an individual image as well as co-saliency detection from a group of images are currently popular topics or reflect future trends more recently due to their importance and challenging roles in computer vision. In many cases, co-saliency detection is usually dependent on the single-image saliency detection results. Nevertheless, most efforts have been made to tackle them separately and not much attention has been paid to tackling them together in a unified idea. Being aware of these two tasks are highly related, the difference from previous surveys is that this paper applies a unified framework by employing a multi-instance learning (MIL) algorithm to resolve both the issues, and formulating single-image saliency and co-saliency detection as top-down weakly supervised learning paradigm. Specifically, for single-image saliency detection, we utilize the evidence confidence-support vector machine algorithm to learn a discriminant model to predict the saliency on test images. For co-saliency detection from image group, we concatenate the EC values and saliency scores to obtain the final results of co-saliency detection. By observing the importance of the selection of negative bags in the MIL framework, we also introduce a novel selection strategy of negative bags to improve the robustness of the proposed method. Experimental results on publicly available image benchmark data sets have demonstrated that the proposed unified framework can achieve competitive performances as compared with the state-of-the-art algorithms in terms of accuracy and effectiveness.

RGBD Co-saliency Detection via Bagging-Based Clustering

With the additional depth information, RGBD co-saliency detection, which is an emerging and interesting issue in saliency detection, aims to discover the common salient objects in a set of RGBD images. This letter proposes a novel RGBD co-saliency model using bagging-based clustering. First, candidate object regions are generated based on RGBD single saliency maps and region pre-segmentation. Then, in order to make regional clustering more robust to different image sets, the feature bagging method is introduced to randomly generate multiple clustering results and the cluster-level weak co-saliency maps. Finally, a clustering quality (CQ) criterion is devised to adaptively integrate the weak co-saliency maps into the final co-saliency map for each image. Experimental results on a public RGBD co-saliency dataset show that the proposed co-saliency model significantly outperforms the state-of-the-art co-saliency models.

Co-Saliency Detection via a Self-Paced Multiple-Instance Learning Framework.

As an interesting and emerging topic, co-saliency detection aims at simultaneously extracting common salient objects from a group of images. On one hand, traditional co-saliency detection approaches rely heavily on human knowledge for designing hand-crafted metrics to possibly reflect the faithful properties of the co-salient regions. Such strategies, however, always suffer from poor generalization capability to flexibly adapt various scenarios in real applications. On the other hand, most current methods pursue co-saliency detection in unsupervised fashions. This, however, tends to weaken their performance in real complex scenarios because they are lack of robust learning mechanism to make full use of the weak labels of each image. To alleviate these two problems, this paper proposes a new SP-MIL framework for co-saliency detection, which integrates both multiple instance learning (MIL) and self-paced learning (SPL) into a unified learning framework. Specifically, for the first problem, we formulate the co-saliency detection problem as a MIL paradigm to learn the discriminative classifiers to detect the co-saliency object in the "instance-level". The formulated MIL component facilitates our method capable of automatically producing the proper metrics to measure the intra-image contrast and the inter-image consistency for detecting co-saliency in a purely self-learning way. For the second problem, the embedded SPL paradigm is able to alleviate the data ambiguity under the weak supervision of co-saliency detection and guide a robust learning manner in complex scenarios. Experiments on benchmark datasets together with multiple extended computer vision applications demonstrate the superiority of the proposed framework beyond the state-of-the-arts.

Co-saliency detection via inter and intra saliency propagation

The goal of salient object detection from an image is to extract the regions which capture the attention of the human visual system more than other regions of the image. In this paper a novel method is presented for detecting salient objects from a set of images, known as co-saliency detection. We treat co-saliency detection as a two-stage saliency propagation problem. The first inter-saliency propagation stage utilizes the similarity between a pair of images to discover common properties of the images with the help of a single image saliency map. With the pairwise co-salient foreground cue maps obtained, the second intra-saliency propagation stage refines pairwise saliency detection using a graph-based method combining both foreground and background cues. A new fusion strategy is then used to obtain the co-saliency detection results. Finally an integrated multi-scale scheme is employed to obtain pixel-level co-saliency maps. The proposed method makes use of existing saliency detection models for co-saliency detection and is not overly sensitive to the initial saliency model selected. Extensive experiments on three benchmark databases show the superiority of the proposed co-saliency model against the state-of-the-art methods both subjectively and objectively.

BSFCoS: Block and Sparse Principal Component Analysis-Based Fast Co-Saliency Detection Method

Co-saliency detection, an emerging research area in saliency detection, aims to extract the common saliency from the multi images. The extracted co-saliency map has been utilized in various applications, such as in co-segmentation, co-recognition and so on. With the rapid development of image acquisition technology, the original digital images are becoming more and more clearly. The existing co-saliency detection methods processing these images need enormous computer memory along with high computational complexity. These limitations made it hard to satisfy the demand of real-time user interaction. This paper proposes a fast co-saliency detection method based on the image block partition and sparse feature extraction method (BSFCoS). Firstly, the images are divided into several uniform blocks, and the low-level features are extracted from Lab and RGB color spaces. In order to maintain the characteristics of the original images and reduce the number of feature points as well as possible, Truncated Power for sparse principal components method are employed to extract sparse features. Furthermore, K-Means method is adopted to cluster the extracted sparse features, and calculate the three salient feature weights. Finally, the co-saliency map was acquired from the feature fusion of the saliency map for single image and multi images. The proposed method has been tested and simulated on two benchmark datasets: Co-saliency Pairs and CMU Cornell iCoseg datasets. Compared with the existing co-saliency methods, BSFCoS has a significant running time improvement in multi images processing while ensuring detection results. Lastly, the co-segmentation method based on BSFCoS is also given and has a better co-segmentation performance.

Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining.

As an interesting and emerging topic, cosaliency detection aims at simultaneously extracting common salient objects in multiple related images. It differs from the conventional saliency detection paradigm in which saliency detection for each image is determined one by one independently without taking advantage of the homogeneity in the data pool of multiple related images. In this paper, we propose a novel cosaliency detection approach using deep learning models. Two new concepts, called intrasaliency prior transfer and deep intersaliency mining, are introduced and explored in the proposed work. For the intrasaliency prior transfer, we build a stacked denoising autoencoder (SDAE) to learn the saliency prior knowledge from auxiliary annotated data sets and then transfer the learned knowledge to estimate the intrasaliency for each image in cosaliency data sets. For the deep intersaliency mining, we formulate it by using the deep reconstruction residual obtained in the highest hidden layer of a self-trained SDAE. The obtained deep intersaliency can extract more intrinsic and general hidden patterns to discover the homogeneity of cosalient objects in terms of some higher level concepts. Finally, the cosaliency maps are generated by weighted integration of the proposed intrasaliency prior, deep intersaliency, and traditional shallow intersaliency. Comprehensive experiments over diverse publicly available benchmark data sets demonstrate consistent performance gains of the proposed method over the state-of-the-art cosaliency detection methods.

Co-Saliency Detection via Co-Salient Object Discovery and Recovery

This letter proposes a novel co-saliency model to effectively discover and highlight co-salient objects in a set of images. Based on the gross similarity which combines color features and SIFT descriptors, some co-salient object regions are first discovered in each image as exemplars, which are exploited to generate the exemplar saliency maps with the use of single-image saliency model. Then both local recovery and global recovery of co-salient object regions are performed by propagating the exemplar saliency to the matched regions, and border connectivity is further exploited to generate the region-level co-saliency maps. Finally, the foci of attention area based pixel-level saliency derivation is used to generate the pixel-level co-saliency maps with even better quality. Experimental results on two benchmark datasets demonstrate that the proposed co-saliency model outperforms the state-of-the-art co-saliency models.

Efficient Saliency-Model-Guided Visual Co-Saliency Detection

This letter proposes a novel framework to detect common salient objects in a group of images automatically and efficiently. Different from most existing co-saliency models which directly redesign algorithms for multiple images, the saliency model for a single image is fully exploited under the proposed framework to guide the co-saliency detection. Given single image saliency maps, a two-stage guided detection pipeline led by queries is proposed to obtain the guided saliency maps of the image set through a ranking scheme. Then the guided saliency maps generated by different queries are fused in a way that takes advantages of both averaging and multiplication. The proposed model makes existing saliency models work well in co-saliency scenarios. Experimental results on two benchmark databases demonstrate that the proposed framework outperforms the state-of-the-art models in terms of both accuracy and efficiency.

A Review on Co-Saliency Detection Methods for Complex Background Images

Human eye is perceptually more sensitive to certain colors and intensities and objects with such features are considered more salient. Co-saliency detection aims at discovering the common salient objects existing in multiple images, which is a relatively under-explored area. A review of various techniques used in co-saliency detection like cluster based co-saliency detection, Hierarchical segmentation, Base reconstruction, Low rank matrix recovery etc. are done in this paper. Detection of Salient image regions is useful in applications such as co-segmentation, common pattern discovery, co recognition, Image retrieval etc. The survey is represented in a tabular form for quick reference and also describes the advantages and disadvantages of papers.

Detecting Co-Salient Objects in Large Image Sets

Co-salient object detection has attracted much more attention recently as it is useful for many problems in vision computing. However, most of existing methods emphasize detecting the common salient objects in a small group of images and the objects of interest in those images have clear borders with respect to the backgrounds. In this work, we propose a novel co-saliency detection method, which aims at discovering the common objects in a large and diverse image set composed of hundreds of images. First, we search a group of similar images for each image in the set. Our method is based on the overlapped groups. We handle each group with an unsupervised random forest to extract the rough contours of the common objects. Then a contrast-based measure is utilized to produce the saliency map for an individual image. For each image in the set, we collect all the maps from the groups that contain the image and fuse them together as the inter-saliency map for the image. The final co-saliency map is computed by combining the inter-saliency map with the single saliency map of this image. Experimental evaluation on an established large dataset demonstrates that our approach attains superior results and outperforms the state-of-the-art methods.

Co-saliency Detection Linearly Combining Single-View Saliency and Foreground Correspondence

Co-saliency Detection Linearly Combining Single-View Saliency and Foreground Correspondence

Co-Saliency Detection Based on Hierarchical Segmentation

Co-saliency detection, an emerging and interesting issue in saliency detection, aims to discover the common salient objects in a set of images. This letter proposes a hierarchical segmentation based co-saliency model. On the basis of fine segmentation, regional histograms are used to measure regional similarities between region pairs in the image set, and regional contrasts within each image are exploited to evaluate the intra-saliency of each region. On the basis of coarse segmentation, an object prior for each region is measured based on the connectivity with image borders. Finally, the global similarity of each region is derived based on regional similarity measures, and then effectively integrated with intra-saliency map and object prior map to generate the co-saliency map for each image. Experimental results on two benchmark datasets demonstrate the better co-saliency detection performance of the proposed model compared to the state-of-the-art co-saliency models.

Self-adaptively Weighted Co-saliency Detection via Rank Constraint.

Co-saliency detection aims at discovering the common salient objects existing in multiple images. Most existing methods combine multiple saliency cues based on fixed weights, and ignore the intrinsic relationship of these cues. In this paper, we provide a general saliency map fusion framework, which exploits the relationship of multiple saliency cues and obtains the self-adaptive weight to generate the final saliency/cosaliency map. Given a group of images with similar objects, our method firstly utilizes several saliency detection algorithms to generate a group of saliency maps for all the images. The feature representation of the co-salient regions should be both similar and consistent. Therefore, the matrix jointing these feature histograms appears low rank. We formalize this general consistency criterion as the rank constraint, and propose two consistency energy to describe it, which are based on low rank matrix approximation and low rank matrix recovery, respectively. By calculating the self-adaptive weight based on the consistency energy, we highlight the common salient regions. Our method is valid for more than two input images and also works well for single image saliency detection. Experimental results on a variety of benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art methods.

Cluster-based co-saliency detection.

Co-saliency is used to discover the common saliency on the multiple images, which is a relatively underexplored area. In this paper, we introduce a new cluster-based algorithm for co-saliency detection. Global correspondence between the multiple images is implicitly learned during the clustering process. Three visual attention cues: contrast, spatial, and corresponding, are devised to effectively measure the cluster saliency. The final co-saliency maps are generated by fusing the single image saliency and multiimage saliency. The advantage of our method is mostly bottom-up without heavy learning, and has the property of being simple, general, efficient, and effective. Quantitative and qualitative experiments result in a variety of benchmark datasets demonstrating the advantages of the proposed method over the competing co-saliency methods. Our method on single image also outperforms most the state-of-the-art saliency detection methods. Furthermore, we apply the co-saliency method on four vision applications: co-segmentation, robust image distance, weakly supervised learning, and video foreground detection, which demonstrate the potential usages of the co-saliency map.

A Co-Saliency Model of Image Pairs

In this paper, we introduce a method to detect co-saliency from an image pair that may have some objects in common. The co-saliency is modeled as a linear combination of the single-image saliency map (SISM) and the multi-image saliency map (MISM). The first term is designed to describe the local attention, which is computed by using three saliency detection techniques available in literature. To compute the MISM, a co-multilayer graph is constructed by dividing the image pair into a spatial pyramid representation. Each node in the graph is described by two types of visual descriptors, which are extracted from a representation of some aspects of local appearance, e.g., color and texture properties. In order to evaluate the similarity between two nodes, we employ a normalized single-pair SimRank algorithm to compute the similarity score. Experimental evaluation on a number of image pairs demonstrates the good performance of the proposed method on the co-saliency detection task.