Saliency Calculation Research Articles

Research on image saliency detection based on deep neural network

AbstractAs a hot research field at present, computer vision is devoted to the rapid acquisition and application of target information from images or videos by simulating human visual mechanism. In order to improve the accuracy and efficiency of image detection, image saliency region detection technology has received more and more attention in the field of computer vision research; an important research content in the field, the core part of which lies in the research on algorithms related to feature extraction and saliency calculation of targets. This paper analyzes the multi‐feature fusion saliency detection model and visual saliency calculation process, and based on the existing algorithm, by improving the VGG16 network, a fully convolutional network saliency detection algorithm is proposed. The qualitative and quantitative experimental results show that compared with the four mainstream methods of BL, GS, SF, and RFCN, our algorithm not only improves the accuracy of salient object detection, but also effectively solves the problem of target edge blur. Therefore, this study has improved the accuracy and efficiency of saliency detection, which can not only promote the development of computer vision technology, but also provide support for research in the field of image processing.

A Fusion Underwater Salient Object Detection Based on Multi-Scale Saliency and Spatial Optimization

Underwater images contain abundant information, but many challenges remain for underwater object detection tasks. Various salient object detection methods may encounter low detection precision, and the segmented map has an incomplete region of the target object. To deal with blurry underwater scenes and vague detection problems, a novel fusion underwater salient object detection algorithm (FUSOD) is proposed based on multi-scale saliency and spatial optimization. Firstly, an improved underwater color restoration was utilized to restore the color information for afterward color contrast saliency calculation. Secondly, a more accurate multi-scale fusion saliency map was obtained by fully considering both the global and local feature contrast information. Finally, the fusion saliency was optimized by the proposed spatial optimization method to enhance the spatial coherence. The proposed FUSOD algorithm may process turbid and complex underwater scenes and preserve a complete structure of the target object. Experimental results on the USOD dataset show that the proposed FUSOD algorithm can segment the salient object with a comparatively higher detection precision than the other traditional state-of-the-art algorithms. An ablation experiment showed that the proposed spatial optimization method increases the detection precision by 0.0325 scores in the F-Measure.

Saliency detection for large-scale mesh decimation

Highly complex and dense models of 3D objects have recently become indispensable in digital industries. Mesh decimation then plays a crucial role in the production pipeline to efficiently get visually convincing yet compact expressions of complex meshes. However, the current pipeline typically does not allow artists control the decimation process, just a simplification rate. Thus a preferred approach in production settings splits the process into a first pass of saliency detection highlighting areas of greater detail, and allowing artists to iterate until satisfied before simplifying the model. We propose a novel, efficient multi-scale method to compute mesh saliency at coarse and finer scales, based on fast mesh entropy of local surface measurements. Unlike previous approaches, we ensure a robust and straightforward calculation of mesh saliency even for densely tessellated models with millions of polygons. Moreover, we introduce a new adaptive subsampling and interpolation algorithm for saliency estimation. Our implementation achieves speedups of up to three orders of magnitude over prior approaches. Experimental results showcase its resilience to problem scenarios that efficiently scales up to process multi-million vertex meshes. Our evaluation with artists in the entertainment industry also demonstrates its applicability to real use-case scenarios.

What Catch Your Attention in SAR Images: Saliency Detection Based on Soft-Superpixel Lacunarity Cue

In existing superpixel-wise saliency detection algorithms, superpixel generation often is an isolated preprocessing step. The performance of saliency maps is determined by the accuracy of superpixels to a certain extent. However, it is still a challenge to develop a stable superpixel generation method. In this article, we attempt to incorporate the superpixel generation and saliency calculation steps into an end-to-end trainable deep network. First, we employ a recently proposed differentiable superpixel generation method to over-segment the synthetic aperture radar (SAR) images, which outputs the possibility that the pixels assigned to neighbor superpixels (soft superpixel). In saliency calculation part, as one of our main contributions, we propose a differentiable and computationally simple saliency model, i.e., lacunarity cue. It is inspired by the fact that generally the backscattering intensity of regions of interest (ROIs) in SAR images irregularly fluctuates, while the areas with consistent pixels are often ignored as the clusters. We improve the pixelwise box differential dimension algorithm to measure the irregularity of scattering points in a superpixel. The superpixel generation and saliency calculation can be implemented under a unified deep network. Hence, the shapes of the superpixels can be iteratively adjusted according to the saliency maps until the ROIs are correctly detected. Experiments on real SAR images with different sizes and scenes show that the saliency maps can effectively highlight the target areas, thus outperforming the state-of-the-art saliency detection models.

Integration of Measurement Vector Insertion With Discontinuous PWM to Improve Saliency-Based Sensorless Drive Position Estimation

Saliency-based position sensorless machine drive with the pulsewidth modulation (PWM) voltage switching frequency injection has advantages of high estimation bandwidth and low acoustic noise. However, multiple currents must be sampled every PWM cycle for spatial saliency calculation. Besides, the switching frequency voltage injection distorts the symmetric PWM voltage pattern. For these saliency-based drives, the PWM injection is the tradeoff between the transient estimation bandwidth and steady state estimation error. This article proposes a modified PWM voltage injection through the measurement vector insertion (MVI) for sensorless position estimation. By integrating the MVI with discontinuous PWM, the proposed injection maintains the symmetric PWM voltage for machine control. This MVI-based PWM (MVIPWM) improves the saliency-based drive among voltage harmonics, current harmonics, and position estimation error. The proposed MVIPWM injection is compared to other PWM injection methods on an interior permanent magnet machine. All advantages of MVIPWM on the saliency-based drive are verified by experimental results.

IGWO-SS: Improved Grey Wolf Optimization Based on Synaptic Saliency for Fast Neural Architecture Search in Computer Vision

Neural Architecture Search (NAS) is the process of automating the design of neural network architectures for a given task. Although NAS provides automates the process of finding suitable neural network architectures for a specific task, the existing NAS algorithms are immensely time-consuming. The main bottleneck in NAS algorithms is the training time for each architecture. This study proposes an Improved Grey Wolf Optimization based on Synaptic Saliency (IGWO-SS), which is much faster than the existing NAS algorithms and provides better final performance. The IGWO-SS algorithm skips training the less promising architectures by creating a relative rank between the architectures based on synaptic saliency. The architectures that are lower in rank are considered less promising than the architectures that are higher in rank. Since the calculation of synaptic saliency is a very fast process, a significant amount of time is saved by skipping training of less promising architectures. We performed extensive experiments to determine the efficacy of synaptic saliency in improving NAS. Our experimental results suggest that the synaptic saliency of an untrained neural network positively correlates with its final accuracy. Hence, it can be used to identify untrained promising neural networks. The experimental results suggest that the IGWO-SS algorithm is almost 10<i>x</i> faster and achieves better final performance than five other bio-inspired algorithms. The IGWO-SS algorithm achieves higher mean accuracy than state-of-the-art NAS algorithms, including - REA, RS, RL, BOHB, DARTSV1, DARTSV2, GDAS, SETN, and ENAS. We hope that our work will make NAS more accessible and useful to researchers by reducing the time and resources required to perform NAS.

Image Segmentation Method for an Illumination Highlight Region of Interior Design Effects Based on the Partial Differential Equation

The saliency calculation model based on the principle of partial differential equations sometimes highlights areas with high contrast in the background, and the salient targets obtained occasionally have holes. The above problems can be solved by combining the improved convex hull calculation center saliency map. This paper designs a single-target color image segmentation algorithm based on partial differential equations. First, we calculate the basic saliency map according to the uniqueness of the color and the spatial distribution of the color; second, we then use the superpixel to improve the convex hull and calculate the central saliency map according to the principle; finally, the basic saliency map and the central saliency map are calculated. The weighted fusion is used to obtain the comprehensive saliency map, and the threshold method is used to segment the comprehensive saliency map to obtain the final target image. This paper designs an evaluation standard suitable for the segmentation of the illuminated highlight area of the effect image. It compares the experimental results of the segmentation method in this paper with the SLIC (Simple Linear Iterative Clustering) method and the traditional superpixel method to segment the illuminated highlight area. The segmentation method is applied to the image enhancement experiment. Based on the fuzzy means clustering algorithm, a fuzzy clustering objective function including brightness, color, and distance parameters is designed, which improves the weight of the brightness value in the clustering and improves the edge fit of the segmentation of the lighting highlight area of the rendering. The segmentation method produced by combining the clustering method with the superpixel biased clustering method can improve the output effect of the illuminated highlight area of the effect image after segmentation. We perform color equalization processing on the image to be segmented to reduce the impact of light, then set the closed value of the brightness information component, perform segmentation judgment, and expand the long and short axes of the ellipse model in the high-brightness area to further reduce the impact of light. The experimental results prove that the above method has a better segmentation effect than the traditional ellipse model and can accurately segment the gesture image. Compared with the existing mainstream saliency calculation models, this algorithm is closer to the true value image in terms of visual effects and has obvious advantages in terms of accuracy.

Quality is in the Salient Region of the Image

The ultimate goal in any proposed Image Quality Metrics (IQMs) is to accurately predict the subjective quality scores given by observers. In the case of most IQMs the quality score is calculated by pooling the quality scores from what is referred to as a quality map of an image. While different pooling methods have been proposed, most such approaches use various types of a weighting average over the quality map to calculate the image quality score. One such approach is to use saliency maps as a weighting factor in our pooling process. Such an approach will result in giving a higher weight to the salient regions of the image. In this work we study if we can evaluate the quality of an image by only calculating the quality of the most salient region in the image. Such an approach could possibly reduce the computational time and power needed for image quality assessment. Results show that in most cases, depending on the saliency calculation method used, we can improve the accuracy of IQMs by simply calculating the quality of a region in the image which covers as low as 20% of the salient energy.

The Identification and Assessment of Rail Corrugation Based on Computer Vision

The identification and assessment of rail corrugation are two of the essential tasks of daily railway inspection to guarantee the safety of train operation and promote the development of an efficient maintenance strategy. In view of the requirements for automatic identification and smart decision-making, computer vision-based rail corrugation identification and assessment methods are presented in this paper. Firstly, an improved Spatial Pyramid Matching (SPM) model, integrating multi-features and locality-constrained linear coding (IMFLLC), is proposed for rail corrugation identification. After that, an innovative period estimation method for rail corrugation is proposed based on the frequency domain analysis of each column in the corrugation region. Finally, the severity of the rail corrugation is assessed with the help of the wear saliency calculation and fuzzy theory. The experiment results demonstrate that the proposed corrugation identification method achieves a higher precision rate and recall rate than those of traditional methods, reaching 99.67% and 98.34%, respectively. Besides, the validity and feasibility of the proposed methods for the rail corrugation period estimation and severity assessment are also investigated.

Region of interest extraction via common salient feature analysis and feedback reinforcement strategy for remote sensing images

Saliency detection is an effective approach to extract regions of interest (ROIs) for remote sensing images. However, existing saliency detection models mainly focus on ROI extraction from a single image and usually are not able to produce satisfactory results because of complex background interference in remote sensing images. The employment of mutual information in a set of remote sensing images can provide an effective solution to this issue. In this paper, we propose a novel saliency detection model for multiple remote sensing images to simultaneously extract ROIs and identify images without ROIs. First, common salient feature analysis based on synthesized feature clustering and global contrast is implemented to exploit global correspondence in the synthesis feature domain, thereby highlighting preliminary ROIs against background interference and assigning lower saliency values to images without ROIs. Then, we design an exclusion criterion based on saliency value judgment to remove images without ROIs, and the remaining saliency maps are refined by an enhancement strategy. Finally, the enhanced maps serve as a feedback to yield a homogenous synthesized feature space in which integral ROIs with subtle borders are extracted by the reused cluster-based saliency calculation. Experiments reveal that our model outperforms seven state-of-the-art models by achieving the best ROC curve (AUC = 0.945) and maximal F-measure at 0.729.

Camera-Assisted Video Saliency Prediction and Its Applications.

Video saliency prediction is an indispensable yet challenging technique which can facilitate various applications, such as video surveillance, autonomous driving, and realistic rendering. Based on the popularity of embedded cameras, we in this paper predict region-level saliency from videos by leveraging human gaze locations recorded using a camera, (e.g., those equipped on an iMAC and laptop PC). Our proposed camera-assisted mechanism improves saliency prediction by discovering human attended regions inside a video clip. It is orthogonal to the current saliency models, i.e., any existing video/image saliency model can be boosted by our mechanism. First of all, the spatial-and temporal-level visual features are exploited collaboratively for calculating an initial saliency map. We notice that the current saliency models are not sufficiently adaptable to the variations in lighting, different view angles, and complicated backgrounds. Therefore, assisted by a camera tracking human gaze movements, a non-negative matrix factorization algorithm is designed to accurately localize the semantically/visually salient video regions perceived by humans. Finally, the learned human gaze locations as well as the initial saliency map are integrated to optimize video saliency calculation. Empirical results thoroughly demonstrated that: 1) our approach achieves the state-of-the-art video saliency prediction accuracy by outperforming 11 mainstream algorithms considerably and 2) our method can conveniently and successfully enhance video retargeting, quality estimation, and summarization.

Image structure-based saliency detection

We consider image intrinsic structure in two aspects: foreground saliency computation and background saliency calculation. On the one hand, a segmented image is represented as a weighted undirected graph, and we calculate the foreground saliency by choosing some superpixels as ranking queries, where we apply a robust background measure to select those for accuracy. On the other hand, we compute the posterior probabilities to measure the background saliency and in terms of the probability, we construct a probability tree via multimerging on superpixels, and then apply the optimization strategy to the background saliency. We evaluate our proposed algorithm on two benchmark datasets and our algorithm yields the competitive results when compared with nine state-of-the-art algorithms in terms of five evaluation metrics.

Automatic Polyp Detection via a Novel Unified Bottom-Up and Top-Down Saliency Approach.

In this paper, we propose a novel automatic computer-aided method to detect polyps for colonoscopy videos. To capture perceptually and semantically meaningful salient polyp regions, we first segment images into multilevel superpixels. Each level corresponds to different sizes of superpixels. Rather than adopting hand-designed features to describe these superpixels in images, we employ sparse autoencoder (SAE) to learn discriminative features in an unsupervised way. Then, a novel unified bottom-up and top-down saliency method is proposed to detect polyps. In the first stage, we propose a weak bottom-up (WBU) saliency map by fusing the contrast-based saliency and object-center-based saliency together. The contrast-based saliency map highlights image parts that show different appearances compared with surrounding areas, whereas the object-center-based saliency map emphasizes the center of the salient object. In the second stage, a strong classifier with multiple kernel boosting is learned to calculate the strong top-down (STD) saliency map based on samples directly from the obtained multilevel WBU saliency maps. We finally integrate these two-stage saliency maps from all levels together to highlight polyps. Experiment results achieve 0.818 recall for saliency calculation, validating the effectiveness of our method. Extensive experiments on public polyp datasets demonstrate that the proposed saliency algorithm performs better compared with state-of-the-art saliency methods to detect polyps.

A Novel Salient Region Detection Method Based on Hierarchical Spatial Information

Different patterns in one object will cause unequal saliency degree which makes it hard to highlight the object region uniformly. We propose a salient region detection method which mainly includes image abstraction, saliency calculation and integration. Under the detection framework, the hierarchical spatial information is introduced to improve the performance. The image abstraction with “pixel level” spatial information is applied to capture some meaningful elements. The local contrast is calculated with the “element level” spatial information. The “object level” spatial information is represented as compactness and background possibility, which further help to better pop out the object region and suppress the background. The results show that our method has a good performance even though the object consists of complex patterns.

Moving Targets Detection Based on Moving Saliency Calculation

Moving targets saliency extraction is the key technology in moving target detection, tracking and recognition systems. In order to solve the problems for moving targets detection tasks, such as the background interference, target occlusion and so on, a new moving target detection method by moving saliency calculation is proposed. By the Quaternion Fast Fourier Transform, the proposed method fuses four motion features as optical flows, direction vectors and frame difference. The moving saliency map, as the result, is generated. The experimental result demonstrates that our method has better ability to handle the problems in moving object detection, such as moving information loss in complex scene. The results with better accuracy score(Cgood=0.875 , Cfalse=0.028) are given by our method.

Vehicle detection based on visual saliency and deep sparse convolution hierarchical model

Traditional vehicle detection algorithms use traverse search based vehicle candidate generation and hand crafted based classifier training for vehicle candidate verification. These types of methods generally have high processing times and low vehicle detection performance. To address this issue, a visual saliency and deep sparse convolution hierarchical model based vehicle detection algorithm is proposed. A visual saliency calculation is firstly used to generate a small vehicle candidate area. The vehicle candidate sub images are then loaded into a sparse deep convolution hierarchical model with an SVM-based classifier to perform the final detection. The experimental results demonstrate that the proposed method is with 94.81% correct rate and 0.78% false detection rate on the existing datasets and the real road pictures captured by our group, which outperforms the existing state-of-the-art algorithms. More importantly, high discriminative multi-scale features are generated by deep sparse convolution network which has broad application prospects in target recognition in the field of intelligent vehicle.

DETECTING SALIENT OBJECTS OF NATURAL SCENE IN A VIDEO'S USING SPATIO-TEMPORAL SALIENCY & COLOUR MAP

Detection of spatio temporal saliency is the main aim of this paper. It uses system where it utilizes the human eye perceptive action. This uses the signal coming from human eye, for detection of saliency, for that a large database are used (about 15000 frames). For Saliency detection two pathways are decided. This uses the data on the basis of same thing on images and in accordance similarity will be calculated. According to it visual summary data are used. It gives much amount of data, which gives a lot of research in computer vision and results are used for calculation of saliency. We have take the view of all method used for calculation of saliency. So by using the data of different method we develop a Novel method which will give best results of detection of saliency. Here technique used is calculation of saliency by using spatio temporal saliency and colour map of video or images. In this method number of experiment are in progress for saliency. This method has fast and robust result.

Moving Athlete Detection for Multimedia Physical Education

In the process of the multimedia physical education, the moving athlete detection is the key to tracking and recognizing the athlete. This task contributes to the gesture and action modeling and simulation. However, since the scene of sports events is complicated, the background is commonly affected by the noise and illumination changes, making the background difficult to remove. In order to solve this problem, a novel motion saliency calculation based method is pro- posed for detecting the moving athlete in sport videos. In our method, four motion features, as the optic flows, direction vectors and frame difference are fused by the Quaternion Fast Fourier Transform. The motion saliency map is generated to identify the area of the moving athlete. The experimental result demonstrated that our method has better ability to de- tect the moving athletes, contributing to the multimedia physical education..

A Saliency Detection Method Based on Global Contrast

To highlight the saliency object clearly from the foreground, we propose a saliency detection method based on global contrast with cluster. Due to the fact that background pixels usually have similar patches, we use cluster analysis to merge the background regions. By using mean shift filter, the background pixels with similar color level are clustered and the saliency calculation can be decreased a lot. In the method, we use the contrast of color feature with all the other pixels to compute the saliency map. A weight coefficient is utilized to improve the detection accuracy in global contrast differences evaluation. The results of extensive experiments on public dataset show that our method perform well and can highlight the salient object clearly against the other five state-of-the-art methods. Besides, we demonstrate that the applications in image segmentation and fusion with our saliency map can get satisfactory results.

Low light level image target detection based on texture saliency

Owing to its low contrast, the target of low light level (LLL) image is not very salient, and it is difficult to detect automatically. Aimed at this problem, this paper proposes a noise robustness algorithm for computing the local texture coarseness (LTC) of textured images, and provides a texture saliency (TS) calculation method that is applicable to saliency analysis of LLL image. Firstly, we present a novel LTC algorithm, by which the LTC around a pixel using the best size of the pixel. Compared with coarseness measure based on local fractal dimension, the LTC algorithm shows much better noise robustness in the experiments of noised textured images. Then, a TS algorithm is given based on the extraction of texture coarseness feature map. Finally, we apply the TS algorithm to LLL image target detection, which is efficient proved by experimental results.