Traditional Saliency Research Articles

Cascaded Aggregation Convolution Network for Salient Grain Pests Detection.

Pest infestation poses significant threats to grain storage due to pests' behaviors of feeding, respiration, excretion, and reproduction. Efficient pest detection and control are essential to mitigate these risks. However, accurate detection of small grain pests remains challenging due to their small size, high variability, low contrast, and cluttered background. Salient pest detection focuses on the visual features that stand out, improving the accuracy of pest identification in complex environments. Drawing inspiration from the rapid pest recognition abilities of humans and birds, we propose a novel Cascaded Aggregation Convolution Network (CACNet) for pest detection and control in stored grain. Our approach aims to improve detection accuracy by employing a reverse cascade feature aggregation network that imitates the visual attention mechanism in humans when observing and focusing on objects of interest. The CACNet uses VGG16 as the backbone network and incorporates two key operations, namely feature enhancement and feature aggregation. These operations merge the high-level semantic information and low-level positional information of salient objects, enabling accurate segmentation of small-scale grain pests. We have curated the GrainPest dataset, comprising 500 images showcasing zero to five or more pests in grains. Leveraging this dataset and the MSRA-B dataset, we validated our method's efficacy, achieving a structure S-measure of 91.9%, and 90.9%, and a weighted F-measure of 76.4%, and 91.0%, respectively. Our approach significantly surpasses the traditional saliency detection methods and other state-of-the-art salient object detection models based on deep learning. This technology shows great potential for pest detection and assessing the severity of pest infestation based on pest density in grain storage facilities. It also holds promise for the prevention and control of pests in agriculture and forestry.

Visual Saliency Detection using Deep Learning

Salient object discovery models mimic the gesture of human beings and capture the most salient region/ object from the images or scenes. This field has numerous important operations in both computer vision and pattern recognition tasks. Despite hundreds of models proposed in this field, it still has a large room for exploration. This paper demonstrates a detailed overview of the recent progress of saliency discovery models in terms of heuristic- grounded ways and deep literacy- grounded ways. We've bandied and reviewed it’s co-related fields, similar as Eye obsession- vaticination, RGBD salient- object- discovery, co- saliency object discovery, and videotape- saliency- discovery models. Image saliency object discovery can fleetly prize useful information from image scenes and further assay it. At present, the traditional saliency target discovery technology still has the edge of outstanding target that can not be well saved. Convolutional neural network( CNN) can prize largely general deep features from the images and effectively express the essential point information of the images. This paper designs a model which applies CNN in deep saliency object discovery tasks. It can efficiently optimize the edges of focus objects and realize largely effective image saliency discovery through multilayer nonstop point birth, refinement of layered boundary, and original saliency point emulsion. The experimental result shows that the proposed system can achieve further robust saliency discovery to acclimate itself to complex background terrain.

Distracted by Previous Experience: Integrating Selection History, Current Task Demands and Saliency in an Algorithmic Model

AbstractAttention can be biased by previous learning and experience. We present an algorithmic-level model of this selection history bias in visual attention that predicts quantitatively how stimulus-driven processes, goal-driven control and selection history compete to control attention. In the model, the output of saliency maps as stimulus-driven guidance interacts with a history map that encodes learning effects and a goal-driven task control to prioritize visual features. The model works on coded features rather than image pixels which is common in many traditional saliency models. We test the model on a reaction time (RT) data from a psychophysical experiment. The model accurately predicts parameters of reaction time distributions from an integrated priority map that is comprised of an optimal, weighted combination of separate maps. Analysis of the weights confirms selection history effects on attention guidance. The model is able to capture individual differences between participants’ RTs and response probabilities per group. Moreover, we demonstrate that a model with a reduced set of maps performs worse, indicating that integrating history, saliency and task information are required for a quantitative description of human attention. Besides, we show that adding intertrial effect to the model (as another lingering bias) improves the model’s predictive performance.

Predicting Clinician Fixations on Glaucoma OCT Reports via CNN-Based Saliency Prediction Methods.

Goal: To predict physician fixations specifically on ophthalmology optical coherence tomography (OCT) reports from eye tracking data using CNN based saliency prediction methods in order to aid in the education of ophthalmologists and ophthalmologists-in-training. Methods: Fifteen ophthalmologists were recruited to each examine 20 randomly selected OCT reports and evaluate the likelihood of glaucoma for each report on a scale of 0-100. Eye movements were collected using a Pupil Labs Core eye-tracker. Fixation heat maps were generated using fixation data. Results: A model trained with traditional saliency mapping resulted in a correlation coefficient (CC) value of 0.208, a Normalized Scanpath Saliency (NSS) value of 0.8172, a Kullback-Leibler (KLD) value of 2.573, and a Structural Similarity Index (SSIM) of 0.169. Conclusions: The TranSalNet model was able to predict fixations within certain regions of the OCT report with reasonable accuracy, but more data is needed to improve model accuracy. Future steps include increasing data collection, improving quality of data, and modifying the model architecture.

Transformer-based network with difference information aware attention module

Existing great progress has been made through remote sensing image change detection (CD) methods, which use convolutions and transformations appropriately. However, due to the complexity of the objects photographed on satellites, the out-of-phase of the same object may exhibit a fine feature diversity and a wide range of variations. Meanwhile, in the previous work, pure convolutional networks and Transformer networks that focus on local information perform better on non-local information. Therefore, current methods still have the problem of insufficient discrimination between irregular changes and diverse boundaries. Here, to solve this biphasic dilemma, we put forward a network that uses the difference information between bitemporal images as a guide to refine features. We use one feature map to subtract another and use multi-scale convolution to refine the difference map. We then designed an attention module similar to self-attention using differential information. In addition, considering the need for both local and global information in change detection, we use the full Swin Transformer framework with shift window self-attention to pass information between local areas. In addition, in order to enhance the network’s ability to recognize, members of our team reformulate supervised, traditional saliency map prediction through saliency and edge prediction. Notably, our proposed model significantly outperforms existing single-branch methods and state-of-the-art performance was achieved on four famous public datasets in terms of the F1-score index, interest in joint indices, and population indices.

Class attention to regions of lesion for imbalanced medical image recognition

Automated medical image classification is the key component in intelligent diagnosis systems. However, most medical image datasets contain plenty of samples of common diseases and just a handful of rare ones, leading to major class imbalances. Currently, it is an open problem in intelligent diagnosis to effectively learn from imbalanced training data. In this paper, we propose a simple yet effective framework, named Class Attention to REgions of the lesion (CARE), to handle data imbalance issues by embedding attention into the training process of Convolutional Neural Networks (CNNs). The proposed attention module helps CNNs attend to lesion regions of rare diseases, therefore helping CNNs to learn their characteristics more effectively. In addition, this attention module works only during the training phase and does not change the architecture of the original network, so it can be directly combined with any existing CNN architecture. The CARE framework needs bounding boxes to represent the lesion regions of rare diseases. To alleviate the need for manual annotation, we further developed variants of CARE by leveraging the traditional saliency methods or a pretrained segmentation model for bounding box generation. Results show that the CARE variants with automated bounding box generation are comparable to the original CARE framework with manual bounding box annotations. A series of experiments on an imbalanced skin image dataset and a pneumonia dataset indicates that our method can effectively help the network focus on the lesion regions of rare diseases and remarkably improves the classification performance of rare diseases.

Similarity evaluation of graphic design based on deep visual saliency features

The creativity of an excellent design work generally comes from the inspiration and innovation of its main visual features. The similarity among primary visual elements stands as a paramount indicator when it comes to identifying plagiarism in design concepts. This factor carries immense importance, especially in safeguarding cultural heritage and upholding copyright protection. This paper aims to develop an efficient similarity evaluation scheme for graphic design. A novel deep visual saliency feature extraction generative adversarial network is proposed to deal with the problem of lack of training examples. It consists of two networks: One predicts a visual saliency feature map from an input image and the other takes the output of the first to distinguish whether a visual saliency feature map is a predicted one or ground truth. Unlike traditional saliency generative adversarial networks, a residual refinement module is connected after the encoding and decoding network. Design importance maps generated by professional designers are used to guide the network training. A saliency-based segmentation method is developed to locate the optimal layout regions and notice insignificant regions. Priorities are then assigned to different visual elements. Experimental results show the proposed model obtains state-of-the-art performance among various similarity measurement methods.

Transformer-Based Efficient Salient Instance Segmentation Networks With Orientative Query

Salient instance segmentation (SIS) can be consid-ered as the next generation task for saliency detection com-munity. As a new challenging task, most of the existing state-of-the-art methods are built on the mainstream Mask R-CNN architecture. However, this mechanism relies heavily on hand-designed anchors and NMS post-processing. In this paper, we provide a one stage SIS framework with transformers, termed Orientative Query Transformer (OQTR). To leverage long-range dependencies of transformer, a cross fusion module is designed to efficiently fuse the global features in encoder and salient query features for salient mask prediction. Furthermore, derived from the center prior in traditional saliency models, we propose an orientative query that is considered as the initial salient object query to accelerate convergence. In addition, to mitigate the issue of lacking large-scale dataset with salient instance labels, we collect a new SIS dataset (SIS10K) containing over 10K images elaborately annotated with both object- and instance-level labels to promote the community. Without any post-processing, our end-to-end OQTR framework significantly surpass the top-1 RDPNet by an average of 13.1% AP scores across all three challenging datasets, showing a strong performance of the proposed OQTR. The code and the dataset proposed in this work are available at: https://github.com/ssecv/OQTR.

Residual attentive feature learning network for salient object detection

Salient object detection is one of the basic challenging problems in the area of computer vision. Traditional saliency models usually utilize handcrafted features and various prior cues to locate and segment salient objects from complicated surroundings. Recently, with the development of convolutional neural networks (CNNs), most of existing approaches explore the combination of multi-level feature maps for improving detection performance. However, how to learn powerful feature is still a challenge, and the multi-level and multi-scale feature maps are usually aggregated without distinction. In this paper, we propose a novel salient object detection approach named residual attentive feature learning (RAFL) to uniformly highlight salient objects and effectively suppress background noises. In our proposed RAFL, a global perception (GP) module is designed to obtain rich global features of input image. A pyramid feature extraction (PFE) module is utilized to capture the rich context of side-output information. Moreover, an attentive feature selection (AFS) module is applied to refine these side-output features and generate attentive features, so as to find out the important feature channels having high responses to the salient objects. Finally, a residual feature learning (RFL) module is proposed to make the shallow layers generate more accurate saliency maps with the help of the deep layers which contain more global semantic information. Experimental results show that our proposed approach performs favorably against 18 state-of-the-art representative methods on five benchmark datasets.

A compact deep architecture for real-time saliency prediction

Saliency computation models aim to imitate the attention mechanism in the human visual system. The application of deep neural networks for saliency prediction has led to a drastic improvement over the last few years. However, deep models have a high number of parameters which makes them less suitable for real-time applications. Here we propose a compact yet fast model for real-time saliency prediction. Our proposed model consists of a modified U-net architecture, a location-dependent fully-connected layer, and central difference convolutional layers. The modification we made on U-Net architecture, promoted compactness and efficiency of the final model. Our location-dependent fully-connected layer facilitates implicit capturing of the location-dependent information. Using the central difference convolutional layers at different scales enables capturing more robust and biologically motivated features. We compare our model with state-of-the-art saliency models using traditional saliency scores as well as our newly devised scheme. Experimental results over four challenging saliency benchmark datasets demonstrate the effectiveness of our approach in striking a balance between accuracy and speed. Our model can be run in real-time which makes it appealing for edge devices and video processing.

Intelligent Image Saliency Detection Method Based on Convolution Neural Network Combining Global and Local Information

Image saliency object detection can rapidly extract useful information from image scenes and further analyze it. At present, the traditional saliency target detection technology still has the edge of outstanding target that cannot be well preserved. Convolutional neural network (CNN) can extract highly general deep features from the images and effectively express the essential feature information of the images. This paper designs a model which applies CNN in deep saliency object detection tasks. It can efficiently optimize the edges of foreground objects and realize highly efficient image saliency detection through multilayer continuous feature extraction, refinement of layered boundary, and initial saliency feature fusion. The experimental result shows that the proposed method can achieve more robust saliency detection to adjust itself to complex background environment.

A new frog leaping algorithm-oriented fully convolutional neural network for dance motion object saliency detection

Image saliency detection is an important research topic in the field of computer vision. With the traditional saliency detection models, the texture details are not obvious and the edge contour is not complete. The accuracy and recall rate of object detection are low, which are mostly based on the manual features and prior information. With the rise of deep convolutional neural networks, saliency detection has been rapidly developed. However, the existing saliency methods still have some common shortcomings, and it is difficult to uniformly highlight the clear boundary and internal region of the whole object in complex images, mainly because of the lack of sufficient and rich features. In this paper, a new frog leaping algorithm-oriented fully convolutional neural network is proposed for dance motion object saliency detection. The VGG (Visual Geometry Group) model is improved. The final full connection layer is removed, and the jump connection layer is used for the saliency prediction, which can effectively combine the multi-scale information from different convolution layers in the convolutional neural network. Meanwhile, an improved frog leaping algorithm is used to optimize the selection of initial weights during network initialization. In the process of network iteration, the forward propagation loss of convolutional neural network is calculated, and the anomaly weight is corrected by using the improved frog leaping algorithm. When the network satisfies the terminal conditions, the final weight is optimized by one frog leaping to make the network weight further optimization. In addition, the new network can combine high-level semantic information and low-level detail information in a data-driven framework. In order to preserve the unity of the object boundary and inner region effectively, the fully connected conditional random field (CRF) model is used to adjust the obtained saliency feature map. In this paper, the precision recall (PR) curve, F-measure, maximum F-measure, weighted F-measure and mean absolute error (MAE) are tested on six widely used public data sets. Compared with other most advanced and representative methods, the results show that the proposed method achieves better performance and it is superior to most representative methods. The presented method reveals that it has strong robustness for image saliency detection with various scenes, and can make the boundary and inner region of the saliency object more uniform and the detection results more accurate.

Multiple Kernel Boosting Saliency Detection of Flame Image of Sintering Section

<p indent=0mm>The flame image of the sintering section is usually interfered by smoke and halo, which would cause the blurring of flame edge and material layer in the image. In order to solve the problem that traditional saliency detection method based on two-dimensional image features is difficult to effectively obtain the actual saliency of the cross-sectional flame image, a saliency detection method based on boundary connectivity and multi-kernel Boosting is proposed. Firstly, the color de-correlation is used in the process of image color space conversion. Boundary connectivity and dark channel prior principle are used to obtain the initial saliency map. Secondly, the super-pixel region information, regional variance and regional contrast of the flame image are used to construct region descriptor, the multiple kernel Boosting algorithm based on support vector regression is used to generate the complementary saliency map on 4 scales. Finally, the initial saliency map and the complementary saliency map are fused to obtain the final saliency map. 600 flame images including manual labeling are used to compare the proposed method with other 5 existing methods and each stage of the proposed method is analyzed. <italic>P</italic>-<italic>R</italic> curve, <italic>F</italic>-measure, mean absolute error and running time are taken as evaluation indexes. The experimental results show that the proposed method is superior to the other 5 methods, and the detection performance of each stage is gradually enhanced, which lays a foundation for improving the effective information extraction of sintering flame section image.

Detection of dim and small infrared targets based on the most appropriate contrast saliency analysis

Aiming at the current situation of dim target detection in infrared image, a dim and small target detection method based on the most appropriate contrast saliency analysis was proposed. In sliding serial port, the non-linear processing technology was used to process the image, which avoided the saliency produced by traditional saliency analysis algorithm when processing the image at the scene edge. The problem of value interference does not affect the ability of target extraction in the target area. A large number of hardware-in-the-loop simulation experiments were carried out. The results show that although the proposed method can not improve significantly in the background suppression factor, the performance of target detection in the two indicators of mean signal-to-noise ratio and signal-to-noise ratio gain of the proposed method are significantly enhanced. And among the three methods of image processing, the effect is the best.

Landmark Extraction via Composite Features of Voronoi Diagram

Landmark extraction is a hotspot of spatial cognition and spatial knowledge organization research. Saliency measurement model is a most important method of landmark extraction, of which the principle is the weighted evaluation of the appearance, structure and semantics of objects. Considering the spatial distribution and geometric characteristics of objects, and it is difficult to extract landmarks with more consistent features, making it a complex process. From the viewpoint of composite features in Voronoi domain, a landmark extraction method is proposed to explore the geometric feature reflection of landmark from individual to its neighborhoods, based on the height of Voronoi element, Voronoi k-order neighborhood crevice and Voronoi k-order neighborhood visibility. Comparative experiments show that: (1) Compared with the landmark extracted by the existing methods with the results of traditional saliency measurement model, which verifies the effectiveness of this method; (2) As for the characteristics of ground property, the geometric features contained in Voronoi diagram can effectively represent the landmarks, reducing the complexity for feature selection of landmarks.

Robust Visual Saliency Optimization Based on Bidirectional Markov Chains

Saliency detection aims to automatically highlight the most important area in an image. Traditional saliency detection methods based on absorbing Markov chain only take into account boundary nodes and often lead to incorrect saliency detection when the boundaries have salient objects. In order to address this limitation and enhance saliency detection performance, this paper proposes a novel task-independent saliency detection method based on the bidirectional absorbing Markov chains that jointly exploits not only the boundary information but also the foreground prior and background prior cues. More specifically, the input image is first segmented into number of superpixels, and the four boundary nodes (duplicated as virtual nodes) are selected. Subsequently, the absorption time upon transition node’s random walk to the absorbing state is calculated to obtain the foreground possibility. Simultaneously, foreground prior (as the virtual absorbing nodes) is used to calculate the absorption time and get the background possibility. In addition, the two aforementioned results are fused to form a combined saliency map which is further optimized by using a cost function. Finally, the superpixel-level saliency results are optimized by a regularized random walks ranking model at multi-scale. The comparative experimental results on four benchmark datasets reveal superior performance of our proposed method over state-of-the-art methods reported in the literature. The experiments show that the proposed method is efficient and can be applicable to the bottom-up image saliency detection and other visual processing tasks.

Supersaliency: A Novel Pipeline for Predicting Smooth Pursuit-Based Attention Improves Generalisability of Video Saliency

Predicting attention is a popular topic at the intersection of human and computer vision. However, even though most of the available video saliency data sets and models claim to target human observers' fixations, they fail to differentiate them from smooth pursuits (SPs), a major eye movement type that is unique to perception of dynamic scenes. In this work, we strive for a more meaningful prediction and conceptual understanding of saliency in general. Because of the higher attentional selectivity of smooth pursuit compared to fixations modelled in traditional saliency research, we refer to the problem of SP prediction as “supersaliency”. To make this distinction explicit, we (i) use algorithmic and manual annotations of SPs and fixations for two well-established video saliency data sets, (ii) train Slicing Convolutional Neural Networks for saliency prediction on either fixation- or SP-salient locations, and (iii) evaluate our and 26 publicly available dynamic saliency models on three data sets against traditional saliency and supersaliency ground truth. Overall, our models outperform the state of the art in both the new supersaliency and the traditional saliency problem settings, for which literature models are optimised. Importantly, on two independent data sets, our supersaliency model shows greater generalisation ability than its counterpart saliency model and outperforms all other models, even for fixation prediction. Furthermore, we tested an end-to-end video saliency model, which also showed systematic improvements when smooth pursuit was predicted either exclusively or together with fixations, with the best performance achieved when the model was trained for the supersaliency problem. This demonstrates the practical benefits and the potential of principled training data selection based on eye movement analysis.

Attention distribution as a measure of issue salience

In spatial theory a central concept is salience, or the relative importance of issues in a voter’s mind in evaluating candidates’ platforms. Traditional, self-reported measures of salience have either been national in breadth (“which issues are most important to the nation as a whole?”) or personal (“which issues do you care most about personally?”). In the former case, the subjects are being asked to guess what issues other voters think are important; in the latter case, subjects are likely to report issues that are “socially” important to avoid seeming selfish or superficial. Unsurprisingly, such self-reported measures have not been found to explain actual candidate choices by individual voters very well. We introduce a simple process-tracing measure of salience, using mouse-tracking. Experimental participants were asked to rate three hypothetical candidates, using information accessed in a setting where the distribution of attention represents salience in the decision process. Four models were tested: standard city block distance and then the addition of each of the two measures of traditional salience—national and personal—and, finally, the attention distribution measure. Attention distribution improves model fit over the standard distance model and improves classification compared to the traditional salience measures.

Unsupervised Video Object Segmentation Based on Mixture Models and Saliency Detection

In this paper, we propose an unsupervised video object segmentation approach which is mainly based on a saliency detection method and the Gaussian mixture model with Markov random field. In our approach, the saliency detection method is developed as a preprocessing technique to calculate the probability of each pixel as the target object. In contrast to traditional saliency detection methods which are normally difficult to obtain the object’s precise boundary and are therefore hard to segment consistent objects, the developed saliency detection method can calculate the saliency of each frame in the video sequence and extract the position and region of the target object with more accurate object boundary. The refined extracted object region is then taken as the prior information and incorporated into the Gaussian mixture model with Markov random field to obtain the precise pixel-wise segmentation result of each frame. The effectiveness of the proposed unsupervised video object segmentation approach is validated through experimental results using both the SegTrack and the SegTrack v2 data sets.

Cervical Cancer Detection and Diagnosis Based on Saliency Single Shot MultiBox Detector in Ultrasonic Elastography.

Cervical cancer is increasingly threatening the health of women, then early screening and prevention of cervical cancer is very necessary. A traditional saliency cervical cancer detection method in Ultrasound image, assuming that there is only one salient object, is not conductive to practical application. Their effects are dependent on saliency threshold. Object detection model provides a kind of new solutions for multiple salient objects. Shot multiBox detector can accurately detect multi-objects with different scales simultaneously except for small cervical cancer regions. To overcome this drawback, this paper presents a new multi-saliency objects detection model, appending deconvolution module embedded within attention residual module. Experiments show that our proposed diagnosis algorithm achieves higher detection accuracy than comparison algorithms. Also, it improves detection performance for mult-saliency cervical cancer objects with small scales, which greatly improve the diagnosis accuracy of cervical cancer.