Saliency Detection Research Articles

GVVST: Image-Driven Style Extraction From Graph Visualizations for Visual Style Transfer.

Incorporating automatic style extraction and transfer from existing well-designed graph visualizations can significantly alleviate the designer's workload. There are many types of graph visualizations. In this paper, our work focuses on node-link diagrams. We present a novel approach to streamline the design process of graph visualizations by automatically extracting visual styles from well-designed examples and applying them to other graphs. Our formative study identifies the key styles that designers consider when crafting visualizations, categorizing them into global and local styles. Leveraging deep learning techniques such as saliency detection models and multi-label classification models, we develop end-to-end pipelines for extracting both global and local styles. Global styles focus on aspects such as color scheme and layout, while local styles are concerned with the finer details of node and edge representations. Through a user study and evaluation experiment, we demonstrate the efficacy and time-saving benefits of our method, highlighting its potential to enhance the graph visualization design process.

Retraction Note: Saliency detection in stereoscopic images using adaptive Gaussian Kernel and Gabor filter

Retraction Note: Saliency detection in stereoscopic images using adaptive Gaussian Kernel and Gabor filter

Infrared and visible image fusion based on hybrid multi-scale decomposition and adaptive contrast enhancement

Effectively fusing infrared and visible images enhances the visibility of infrared target information while capturing visual details. Balancing the brightness and contrast of the fusion image adequately has posed a significant challenge. Moreover, preserving detailed information in fusion images has been problematic. To address these issues, this paper proposes a fusion algorithm based on multi-scale decomposition and adaptive contrast enhancement. Initially, we present a hybrid multi-scale decomposition method aimed at extracting valuable information comprehensively from the source image. Subsequently, we advance an adaptive base layer optimization approach to regulate the brightness and contrast of the resultant fusion image. Lastly, we design a weight mapping rule grounded in saliency detection to integrate small-scale layers, thereby conserving the edge structure within the fusion outcome. Both qualitative and quantitative experimental results affirm the superiority of the proposed method over 11 state-of-the-art image fusion methods. Our method excels in preserving more texture and achieving higher contrast, which proves advantageous for monitoring tasks.

Graph-based Saliency Detection: Integrating Multilevel Features and Analyzing the Distraction Using DNN

Graph-based Saliency Detection: Integrating Multilevel Features and Analyzing the Distraction Using DNN

What Is Faster than Where in Vocal Emotional Perception.

Voices carry a vast amount of information about speakers (e.g., emotional state; spatial location). Neuroimaging studies postulate that spatial ("where") and emotional ("what") cues are processed by partially independent processing streams. Although behavioral evidence reveals interactions between emotion and space, the temporal dynamics of these processes in the brain and its modulation by attention remain unknown. We investigated whether and how spatial and emotional features interact during voice processing as a function of attention focus. Spatialized nonverbal vocalizations differing in valence (neutral, amusement, anger) were presented at different locations around the head, whereas listeners discriminated either the spatial location or emotional quality of the voice. Neural activity was measured with ERPs of the EEG. Affective ratings were collected at the end of the EEG session. Emotional vocalizations elicited decreased N1 but increased P2 and late positive potential amplitudes. Interactions of space and emotion occurred at the salience detection stage: neutral vocalizations presented at right (vs. left) locations elicited increased P2 amplitudes, but no such differences were observed for emotional vocalizations. When task instructions involved emotion categorization, the P2 was increased for vocalizations presented at front (vs. back) locations. Behaviorally, only valence and arousal ratings showed emotion-space interactions. These findings suggest that emotional representations are activated earlier than spatial representations in voice processing. The perceptual prioritization of emotional cues occurred irrespective of task instructions but was not paralleled by an augmented stimulus representation in space. These findings support the differential responding to emotional information by auditory processing pathways.

Saliency detection for underwater moving object with sonar based on motion estimation and multi-trajectory analysis

Due to the limited detection range of optical sensors in the ocean, sonar stands out as the predominant method for detecting dynamic objects in the deep sea. Since there is stronger reflection underwater, the objects are more salient in sonar images compared to the background. In this paper, we propose a saliency detection framework for underwater moving object, which consists of three stages. In the first stage, we use optical flow to get rough global motion cues under background interference in unstable sonar platforms. In the second stage, we propose a trajectory analysis paradigm, which converts the motion cues into isolated connected domains for tracking, and then evaluates the trajectories to get the path of real target. In the third stage, we propose a salient object detection (SOD) model that predicts local saliency maps through feature fusion and multi-level supervision. The global saliency map is then obtained by remapping. Finally, extensive experiments conducted on eight sonar videos demonstrate that our proposed methodology outperforms fifteen other saliency object detection approaches.

Attenuated color channel adaptive correction and bilateral weight fusion for underwater image enhancement

Due to the absorption and scattering of light and the influence of suspended particles, underwater images commonly exhibit color distortions, reduced contrast, and diminished details. This paper proposes an attenuated color channel adaptive correction and bilateral weight fusion approach called WLAB to address the aforementioned degradation issues. Specifically, a novel white balance method is first applied to balance the color channel of the input image. Moreover, a local-block-based fast non-local means method is proposed to obtain a denoised version of the color-corrected image. Then, an adaptive stretching method that considers the histogram's local features to get a contrast-enhanced version of the color-corrected image. Finally, a bilateral weight fusion method is proposed to fuse the above two image versions to obtain an output image with complementary advantages. Experimental studies are conducted on three benchmark underwater image datasets and compared with ten state-of-the-art methods. The results show that WLAB has a significant advantage over the comparative methods. Notably, WLAB exhibits a degree of independence from camera settings and enhances the precision of various image processing applications, including key points and saliency detection. Additionally, it demonstrates commendable adaptability in improving low-light and foggy images.

Efficient fusion of spatio-temporal saliency for frame wise saliency identification

<p>Video saliency detection is a rapidly growing subject that has seen very few contributions. The most common technique used nowadays is to perform frame-by-frame saliency detection. The modified Spatio-temporal fusion method presented in this paper offers a novel approach to saliency detection and mapping. It uses frame-wise overall motion color saliency as well as pixel-based consistent Spatio-temporal diffusion for its temporal uniformity. Additionally, a variety of techniques is advocated as a way to increase the saliency maps' overall accuracy and precision. The video is divided into groups of frames, and each frame temporarily goes through diffusion and integration in order to compute the color saliency mapping, as covered in the proposed method section. Then, with the aid of a permutation matrix, the inter-group frame is used to format the pixel-based saliency fusion, after which the features, or the fusion of pixel saliency and color information, direct the diffusion of the spatiotemporal saliency. The result is tested using five publicly accessible global saliency evaluation metrics, and it is determined that the proposed algorithm outperforms numerous saliency detection techniques with an improvement in accuracy margin. The robustness, dependability, adaptability, and precision are all demonstrated by the results.</p>

CsdlFusion: An Infrared and Visible Image Fusion Method Based on LatLRR-NSST and Compensated Saliency Detection

CsdlFusion: An Infrared and Visible Image Fusion Method Based on LatLRR-NSST and Compensated Saliency Detection

Unsupervised Object Cosegmentation Method Devoted to Image Classification

Rich heterogeneous data provided by social networks can be very big, which imposes considerable challenges for object extraction and image classification. Therefore, the objective of this work is to propose an unsupervised object cosegmentation method that could be notably efficient to improve image classification performance. The main goal of cosegmentation is to extract the salient common objects within each image. To this end, we propose to minimize an energy function based on the Markov Random Field using the saliency detection, while considering linear dependence of generated foreground histograms of the input image collection. In fact, the saliency detection is processed in two steps. In each image, we detect salient objects, by considering appearance similarity and spatial distributions of image pixels. Then, fuzzy quantification is used to correct the belonging of pixels to the foreground objects. Finally, an iterative optimization permits to enhance the final segmentation results. The proposed method has been validated as a preprocessing step for image classification. Indeed, to enhance cosegmentation-based classification performance, we have applied a semi-supervised object classification based on ensemble projection. Qualitative and quantitative evaluations of the proposed cosegmentation and classification techniques on the iCoseg, CDS and Oxford Flowers 17 datasets demonstrate the effectiveness of the proposed framework.

Enhancing IIoT Vision Data Transmission and Processing via Spatial-Difference-Attention-Guided Saliency Detection

Enhancing IIoT Vision Data Transmission and Processing via Spatial-Difference-Attention-Guided Saliency Detection

A virtual-reality spatial matching algorithm and its application on equipment maintenance support: System design and user study

Equipment maintenance support is an important technical measure to maintain the equipment’s expected performance. However, the current maintenance supports are mainly completed by maintainers under the guidance of technical manual or additional experts, which may be insufficient for some advanced equipment with rapid update rate and complex inner structure. The rising technology of augmented reality (AR) provides a new solution for equipment maintenance support, while one of the key issues limiting the practical application of AR in maintenance field is the spatial matching issue between virtual space and reality space. In this paper, a virtual-reality spatial matching algorithm is designed to accurately superimpose the virtual information to the corresponding actual scene on the AR glasses. In this algorithm, two methods are proposed to help achieve the stable matching of virtual space and reality space. In detail, to obtain the saliency map with less background interference and improved saliency detection accuracy, a saliency detection method is designed based on the super-pixel segmentation. To deal with the problems of uneven distribution on the feature points and weak robustness to the light changes, a feature extraction and matching method is proposed for acquiring the feature point matching set with the utilization of the obtained saliency map. Finally, an immersive equipment maintenance support system (IEMSS) is developed based on this spatial matching algorithm, which provides the maintainers with immediate and immersive guidance to improve the efficiency and safety in the maintenance task, as well as offers maintenance training for inexperienced maintainers with expanded virtual information in case of limited experts. Several comparative experiments are implemented to verify the effectiveness of proposed methods, and a user study of real system application is carried out to further evaluate the superiority of these methods when applied in the IEMSS.

Research on image saliency detection based on deep neural network

AbstractAs a hot research field at present, computer vision is devoted to the rapid acquisition and application of target information from images or videos by simulating human visual mechanism. In order to improve the accuracy and efficiency of image detection, image saliency region detection technology has received more and more attention in the field of computer vision research; an important research content in the field, the core part of which lies in the research on algorithms related to feature extraction and saliency calculation of targets. This paper analyzes the multi‐feature fusion saliency detection model and visual saliency calculation process, and based on the existing algorithm, by improving the VGG16 network, a fully convolutional network saliency detection algorithm is proposed. The qualitative and quantitative experimental results show that compared with the four mainstream methods of BL, GS, SF, and RFCN, our algorithm not only improves the accuracy of salient object detection, but also effectively solves the problem of target edge blur. Therefore, this study has improved the accuracy and efficiency of saliency detection, which can not only promote the development of computer vision technology, but also provide support for research in the field of image processing.

Cascaded Aggregation Convolution Network for Salient Grain Pests Detection.

Pest infestation poses significant threats to grain storage due to pests' behaviors of feeding, respiration, excretion, and reproduction. Efficient pest detection and control are essential to mitigate these risks. However, accurate detection of small grain pests remains challenging due to their small size, high variability, low contrast, and cluttered background. Salient pest detection focuses on the visual features that stand out, improving the accuracy of pest identification in complex environments. Drawing inspiration from the rapid pest recognition abilities of humans and birds, we propose a novel Cascaded Aggregation Convolution Network (CACNet) for pest detection and control in stored grain. Our approach aims to improve detection accuracy by employing a reverse cascade feature aggregation network that imitates the visual attention mechanism in humans when observing and focusing on objects of interest. The CACNet uses VGG16 as the backbone network and incorporates two key operations, namely feature enhancement and feature aggregation. These operations merge the high-level semantic information and low-level positional information of salient objects, enabling accurate segmentation of small-scale grain pests. We have curated the GrainPest dataset, comprising 500 images showcasing zero to five or more pests in grains. Leveraging this dataset and the MSRA-B dataset, we validated our method's efficacy, achieving a structure S-measure of 91.9%, and 90.9%, and a weighted F-measure of 76.4%, and 91.0%, respectively. Our approach significantly surpasses the traditional saliency detection methods and other state-of-the-art salient object detection models based on deep learning. This technology shows great potential for pest detection and assessing the severity of pest infestation based on pest density in grain storage facilities. It also holds promise for the prevention and control of pests in agriculture and forestry.

Deep underwater image compression for enhanced machine vision applications

Underwater image compression is fundamental in underwater visual applications. The storage resources of autonomous underwater vehicles (AUVs) and underwater cameras are limited. By employing effective image compression methods, it is possible to optimize the resource utilization of these devices, thereby extending the operational time underwater. Current image compression methods neglect the unique characteristics of the underwater environment, thus failing to support downstream underwater visual tasks efficiently. We propose a novel underwater image compression framework that integrates frequency priors and feature decomposition fusion in response to these challenges. Our framework incorporates a task-driven feature decomposition fusion module (FDFM). This module enables the network to understand and preserve machine-friendly information during the compression process, prioritizing task relevance over human visual perception. Additionally, we propose a frequency-guided underwater image correction module (UICM) to address noise issues and accurately identify redundant information, enhancing the overall compression process. Our framework effectively preserves machine-friendly features at a low bit rate. Extensive experiments across various downstream visual tasks, including object detection, semantic segmentation, and saliency detection, consistently demonstrated significant improvements achieved by our approach.

SFD-SLAM: a novel dynamic RGB-D SLAM based on saliency region detection

Abstract In dynamic environments, several simultaneous localization and mapping (SLAM) systems effectively utilize optical flow fields to distinguish dynamic from static feature points. Commonly, these systems leverage the amplitude information within the optical flow field to develop adaptive thresholding segmentation models for identifying dynamic scene regions. Nevertheless, designing adaptive thresholding models typically necessitates meticulous planning and extensive experimentation. This study introduces a dynamic RGBD SLAM system, SFD-SLAM, which innovates by employing a saliency detection network for the direct extraction of dynamic regions via scene flow. This approach notably streamlines the design process associated with conventional adaptive thresholding models. Furthermore, SFD-SLAM incorporates a geometric module that merges depth residuals with hyperpixel segmentation to enhance the refinement of the dynamic mask. This is followed by integration with FCM clustering for the precise identification of moving objects. The efficacy of SFD-SLAM is assessed using the widely recognized TUM dynamic dataset. Experimental results demonstrate that the proposed system surpasses DGFlow-SLAM, which relies on an adaptive thresholding model for dynamic object segmentation, in terms of trajectory accuracy. It also achieves comparable localization accuracy to DynaSLAM. Moreover, SFD-SLAM maintains robust tracking capabilities, even in scenarios where DynaSLAM experiences tracking loss, thereby augmenting the robustness of RGBD-SLAM in dynamic settings.

Computer-Vision-Oriented Adaptive Sampling in Compressive Sensing.

Compressive sensing (CS) is recognized for its adeptness at compressing signals, making it a pivotal technology in the context of sensor data acquisition. With the proliferation of image data in Internet of Things (IoT) systems, CS is expected to reduce the transmission cost of signals captured by various sensor devices. However, the quality of CS-reconstructed signals inevitably degrades as the sampling rate decreases, which poses a challenge in terms of the inference accuracy in downstream computer vision (CV) tasks. This limitation imposes an obstacle to the real-world application of existing CS techniques, especially for reducing transmission costs in sensor-rich environments. In response to this challenge, this paper contributes a CV-oriented adaptive CS framework based on saliency detection to the field of sensing technology that enables sensor systems to intelligently prioritize and transmit the most relevant data. Unlike existing CS techniques, the proposal prioritizes the accuracy of reconstructed images for CV purposes, not only for visual quality. The primary objective of this proposal is to enhance the preservation of information critical for CV tasks while optimizing the utilization of sensor data. This work conducts experiments on various realistic scenario datasets collected by real sensor devices. Experimental results demonstrate superior performance compared to existing CS sampling techniques across the STL10, Intel, and Imagenette datasets for classification and KITTI for object detection. Compared with the baseline uniform sampling technique, the average classification accuracy shows a maximum improvement of 26.23%, 11.69%, and 18.25%, respectively, at specific sampling rates. In addition, even at very low sampling rates, the proposal is demonstrated to be robust in terms of classification and detection as compared to state-of-the-art CS techniques. This ensures essential information for CV tasks is retained, improving the efficacy of sensor-based data acquisition systems.

Two-stage fusion strategy and residual shrinkage capsule network for SAR image change detection

ABSTRACT Acquiring precise change map for synthetic aperture radar (SAR) images poses challenges due to the absence of labelled datasets and the presence of speckle noise interference. This paper presents a two-stage fusion strategy coupled with capsule network classification detection method. Initially, to enhance the structure of change areas within the difference image (DI), we propose an energy ratio (ER) algorithm based on the mean ratio (MR). Subsequently, employing a two-stage fusion strategy integrating multiplication and discrete wavelet transform (DWT), we fuse the DIs generated by the three algorithms to effectively complement change information across distinct DIs. In addition, a saliency detection algorithm is introduced to mitigate noise interference and balance the number of samples. Hierarchical fuzzy c-means clustering (HFCM) is then used to pre-classify the salient fusion difference image (SFDI) and derive pseudo-labels. Finally, we apply the residual shrinkage capsule network (RSCN) to reclassify pre-classification results and obtain the change detection outcome, capable of extracting the multi-scale refinement features and adaptively learning discriminant information from changed and unchanged regions. Experimental evaluations on various SAR image datasets demonstrate the exceptional detection performance of the proposed method.

Individual differences in face salience and rapid face saccades.

Humans saccade to faces in their periphery faster than to other types of objects. Previous research has highlighted the potential importance of the upper face region in this phenomenon, but it remains unclear whether this is driven by the eye region. Similarly, it remains unclear whether such rapid saccades are exclusive to faces or generalize to other semantically salient stimuli. Furthermore, it is unknown whether individuals differ in their face-specific saccadic reaction times and, if so, whether such differences could be linked to differences in face fixations during free viewing. To explore these open questions, we invited 77 participants to perform a saccadic choice task in which we contrasted faces as well as other salient objects, particularly isolated face features and text, with cars. Additionally, participants freely viewed 700 images of complex natural scenes in a separate session, which allowed us to determine the individual proportion of first fixations falling on faces. For the saccadic choice task, we found advantages for all categories of interest over cars. However, this effect was most pronounced for images of full faces. Full faces also elicited faster saccades compared with eyes, showing that isolated eye regions are not sufficient to elicit face-like responses. Additionally, we found consistent individual differences in saccadic reaction times toward faces that weakly correlated with face salience during free viewing. Our results suggest a link between semantic salience and rapid detection, but underscore the unique status of faces. Further research is needed to resolve the mechanisms underlying rapid face saccades.

A Two-branch Edge Guided Lightweight Network for infrared image saliency detection

In the dynamic landscape of saliency detection, convolutional neural networks have emerged as catalysts for innovation, but remain largely tailored for RGB imagery, falling short in the context of infrared images, particularly in memory-restricted environments. These existing approaches tend to overlook the wealth of contour information vital for a nuanced analysis of infrared images. Addressing this notable gap, we introduce the novel Two-branch Edge Guided Lightweight Network (TBENet), designed explicitly for the robust analysis of infrared image saliency detection. The main contributions of this paper are as follows. First, we formulate the saliency detection task as two subtasks, contour enhancement and foreground segmentation. Therefore, the TBENet is divided into two specialized branches: a contour prediction branch for extracting target contour and a saliency map generation branch for separating the foreground from the background. The first branch employs an encoder–decoder architecture to meticulously delineate object contours, serving as a guiding blueprint for the second branch. This latter segment adeptly integrates spatial and semantic data, creating a precise saliency map that is refined further by an innovative edge-weighted contour loss function. Second, to enhance feature integration capabilities, we propose depthwise multi-scale and multi-cue modules, facilitating sophisticated feature aggregation. Third, a high-level linear bottleneck module is devised to ensure the extraction of rich semantic information, and by replacing the standard convolution with the depthwise convolution, it is beneficial to reduce model complexity. Additional, we reduce the number of channels of the feature maps from each stage of the decoder to further enhance the lightweight of the model. Last, we construct a novel infrared ship dataset Small-IRShip to train and evaluate our proposed model. Experimental results on the homemade dataset Small-IRShip and two publicly available datasets, namely RGB-T and IRSTD-1k, demonstrate TBENet’s superior performance over state-of-the-art methods, affirming its effectiveness in harnessing edge information and incorporating advanced feature integration strategies.