Saliency Detection Technique Research Articles

Efficient fusion of spatio-temporal saliency for frame wise saliency identification

<p>Video saliency detection is a rapidly growing subject that has seen very few contributions. The most common technique used nowadays is to perform frame-by-frame saliency detection. The modified Spatio-temporal fusion method presented in this paper offers a novel approach to saliency detection and mapping. It uses frame-wise overall motion color saliency as well as pixel-based consistent Spatio-temporal diffusion for its temporal uniformity. Additionally, a variety of techniques is advocated as a way to increase the saliency maps' overall accuracy and precision. The video is divided into groups of frames, and each frame temporarily goes through diffusion and integration in order to compute the color saliency mapping, as covered in the proposed method section. Then, with the aid of a permutation matrix, the inter-group frame is used to format the pixel-based saliency fusion, after which the features, or the fusion of pixel saliency and color information, direct the diffusion of the spatiotemporal saliency. The result is tested using five publicly accessible global saliency evaluation metrics, and it is determined that the proposed algorithm outperforms numerous saliency detection techniques with an improvement in accuracy margin. The robustness, dependability, adaptability, and precision are all demonstrated by the results.</p>

Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network

In recent years, there has been increased interest in multispectral pedestrian detection using visible and infrared image pairs. This is due to the complementary visual information provided by these modalities, which enhances the robustness and reliability of pedestrian detection systems. However, current research in multispectral pedestrian detection faces the challenge of effectively integrating different modalities to reduce miss rates in the system. This article presents an improved method for multispectral pedestrian detection. The method utilises a saliency detection technique to modify the infrared image and obtain an infrared-enhanced map with clear pedestrian features. Subsequently, a multiscale image features fusion network is designed to efficiently fuse visible and IR-enhanced maps. Finally, the fusion network is supervised by three loss functions for illumination perception, light intensity, and texture information in conjunction with the light perception sub-network. The experimental results demonstrate that the proposed method improves the logarithmic mean miss rate for the three main subgroups (all day, day and night) to 3.12%, 3.06%, and 4.13% respectively, at “reasonable” settings. This is an improvement over the traditional method, which achieved rates of 3.11%, 2.77%, and 2.56% respectively, thus demonstrating the effectiveness of the proposed method.

Semantic feature-based test selection for deep neural networks: A frequency domain perspective

While deep neural networks (DNNs) have great potential for applications in security and safety-critical domains, their limited robustness to adversarial samples and out-of-distribution (OOD) samples raise significant concerns. In the software engineering community, significant efforts have been devoted to devising testing techniques that verify the robustness of DNNs. This paper investigates semantic feature-based test selection for DNNs from a frequency domain perspective and propose a novel method called SaFeTS. Specifically, we leverage saliency detection techniques, such as Fourier Phase Transform to extract semantic features from test cases. These features are then clustered to select diverse test cases to evaluate the robustness of DNNs and model retraining. Experiments on CIFAR-10 and SVHN datasets demonstrate that SaFeTS exposes more varied model errors compared to baseline methods. Further, retraining with SaFeTS-selected samples significantly improves adversarial and out-of-distribution robustness over state-of-the-art test selection methods.

Modified ResNet50 model and semantic segmentation based image co-saliency detection

Co-salient object identification is a new and emerging branch of the visual saliency detection technique that tries to find salient pattern appearing in several image groups. The proposed work has the potential to benefit a wide variety of important applications including the detection of objects of interest, more robust object recognition and animation synthesis, handling input image query, 3D object reconstruction, object co-segmentation etc. To build modified ResNet50 model, the hyperparameters are adjusted in the current work to increase accuracy while minimizing loss. The modified network is trained on HOG features to mine more significant features along with their corresponding ground truth images. For a more streamlined outcome, the proposed system was built using the SGDM optimizer. During testing among the relevant and irrelevant image the network generates appropriate co-saliency map of relevant images. Integrating the associated and prominent characteristics of the image yields the appropriate ground truth for each image. The proposed method reports better F1 value 98.7% and MAE score 0.089 value when compared with SOTA model.

Dynamic Temporal–Spatial Regularization-Based Channel Weight Correlation Filter for Aerial Object Tracking

Correlation filter (CF) has drawn extensive interest in aerial object tracking due to its remarkable performance. Recently, the popular CF methods based on temporal–spatial regularization have been proved to be able to effectively improve the tracking results. However, the boundary effect and filter template degradation still influence the speed and accuracy of the trackers. To handle the two problems, a novel dynamic temporal–spatial regularization-based channel weighted tracking (DTSCT) method was proposed in this work. First, we attempted to employ the saliency detection technique to describe object variation for weakening the boundary effect. Then, the filter template was introduced to the temporal regularization to alleviate the template degradation. In addition, an adaptive weighting strategy was utilized to remove data redundancy in the feature channels. Experiments on three benchmark datasets showed the competitive performance of our DTSCT approach compared to the state-of-the-art methods.

Pic2PolyArt: Transforming a photograph into polygon-based geometric art

Geometric art, an artwork that is structured by geometric shapes, was first made popular by the introduction of Cubism paintings by Pablo Picasso and Georges Braque in the early-20th-century. With the recent advancement in digital imaging technology coupled with the rising popularity of social media such as Instagram, automatic geometric abstraction that can automatically transform a photograph into a piece of geometric art starts to gain research attention. Several state-of-the-art works have been proposed. Notably, despite the fact that Cubism artworks by renown artists illustrate the importance of the main subject in a painting to be recognizable and should be given more detailed representation, most of the state-of-the-art abstraction algorithms are not subject-focused. In this paper, we present Pic2PolyArt, a unified subject-focused geometric abstraction framework that can support both triangle-based and polygon-based abstraction. Given an input photograph, our proposed algorithm first identifies the main subject and important features of an image with a combination of saliency, edge, and face detection techniques. It then generates a set of seed points that are used by Delaunay Triangulation and Voronoi Tessellation to generate triangle-based and polygon-based geometric abstraction respectively. Results from qualitative evaluation on benchmark dataset and empirical user studies demonstrate the effectiveness of our proposed abstraction framework in generating pleasant geometric abstraction from photographs and provide insightful knowledge on users preference with regards to the type of polygons and the level of abstraction used to represent the resulting geometric art.

Interestingness Improvement of Face Images by Learning Visual Saliency

Connecting features of face images with the interestingness of a face may assist in a range of applications such as intelligent visual human-machine communication. To enable the connection, we use interestingness and image features in combination with machine learning techniques. In this paper, we use visual saliency of face images as learning features to classify the interestingness of the images. Applying multiple saliency detection techniques specifically to objects in the images allows us to create a database of saliency-based features. Consistent estimation of facial interestingness and using multiple saliency methods contribute to estimate, and exclusively, to modify the interestingness of the image. To investigate interestingness – one of the personal characteristics in a face image, a large benchmark face database is tested using our method. Taken together, the method may advance prospects for further research incorporating other personal characteristics and visual attention related to face images.

Saliency detection via multiple-morphological and superpixel based fast fuzzy C-mean clustering network

To model a human perception-based saliency detection algorithm in cluttered and noisy background images is a challenging problem in computer vision. Recently, many saliency detection algorithms have been proposed, which exploit the background information, or boundary priors of an image, to detect the salient object similar to the human attention system. These algorithms may not provide satisfying detection results for color images due to the assimilation of local spatial information. In this paper, we propose an unsupervised saliency detection technique, which uses multi-color space-based morphological gradient images. These gradient images contain different edge features, which are useful to obtain an accurate counter-based superpixel image containing both foreground and background clusters. A robust background measuring technique is implemented to remove background clusters, which describes the spatial information of an image cluster to image boundaries. This geometric clarification method effectively removes multiple low-level clues to produce a precise and uniform saliency map. These initially obtained saliency maps are fused using a multi-map fusion technique, and a compact saliency map prevails. The proposed algorithm is evaluated by executing different experiments on nine data sets. The results show that the proposed algorithm performs well for the detection of both single and multiple objects. The proposed algorithm is computationally efficient and provides better saliency detection results than state-of-the-art algorithms.

Coding structure of JMVDC along saliency mapping: a prespective compression technique

SummarySummarizing of multi‐see video and penetrating information depicts an augmentation of the high‐performance video coding (HPVC) criterion. Multi‐see recordings taken by numerous cameras of the same scene from various positions and edges progress towards becoming feasible since little estimated and less value cameras are accessible. Multi‐see video enables the customers to unreservedly change their viewpoints whenever they need to see the scene from any position. For profundity coding, new intracoding modes, an altered movement pay, and movement vector coding and also the idea of movement parameter legacy are a piece of the HPVC expansion. A novel encoder control utilizes see blend improvement, which ensures that amazing middle of the road perspectives can be produced in view of the decoded information. The bit stream arrange bolsters the extraction of halfway piece streams, with the goal that traditional latest video standards like 2D, stereo, multi‐see in addition to profundity organization can be de‐transmit from a solitary piece stream. We proposed a novel saliency detection technique for stereoscopic pictures, which depends on local‐global differentiation highlights, trailed by encompassing district improvement and stereo focus earlier upgrade and furthermore we have proposed an algorithm that effectively chooses a near‐optimalinterview PS and related texture and profundity QPs, at GOP level, when the MVD information organize is utilized for IMVS frameworks. Objective and subjective outcomes are introduced, exhibiting that the proposed approach gives half piece rate investment funds in correlation with JMVDC simulcast and 22% in examination with a direct multi‐see expansion of JMVDC without the recently created coding instruments.

Multi-Type Self-Attention Guided Degraded Saliency Detection

Existing saliency detection techniques are sensitive to image quality and perform poorly on degraded images. In this paper, we systematically analyze the current status of the research on detecting salient objects from degraded images and then propose a new multi-type self-attention network, namely MSANet, for degraded saliency detection. The main contributions include: 1) Applying attention transfer learning to promote semantic detail perception and internal feature mining of the target network on degraded images; 2) Developing a multi-type self-attention mechanism to achieve the weight recalculation of multi-scale features. By computing global and local attention scores, we obtain the weighted features of different scales, effectively suppress the interference of noise and redundant information, and achieve a more complete boundary extraction. The proposed MSANet converts low-quality inputs to high-quality saliency maps directly in an end-to-end fashion. Experiments on seven widely-used datasets show that our approach produces good performance on both clear and degraded images.

Webly-supervised learning for salient object detection

End-to-end training of a deep CNN-Based model for salient object detection usually requires a huge number of training samples with pixel-level annotations, which are costly and time-consuming to obtain. In this paper, we propose an approach that can utilize large amounts of web data for learning a deep salient object detection model. With thousands of images collected from the Web, we first employ several bottom-up saliency detection techniques to generate salient object masks for all images, and then use a novel quality evaluation method to pick out a subset of images with reliable masks for training. After that, we develop a self-training approach to boost the performance of our initial network, which iterates between the network training process and the training set updating process. Importantly, different from existing webly-supervised or weakly-supervised methods, our approach is able to automatically select reliable images for network training without requiring any human intervention (e.g., dividing images into different difficulty levels).Results of extensive experiments on several widely-used benchmarks demonstrate that our method has achieved state-of-the-art performance. It significantly outperforms existing unsupervised and weakly-supervised salient object detection methods, and achieves competitive or even better performance than fully supervised approaches.

Saliency-Based Diver Target Detection and Localization Method

Diver target automatic detection is indispensable for underwater defense systems, such as the unmanned harbor surveillance system. It is a very challenging task due to various poses and intensity features of diver target. In addition, the background noise in sonar images is complex, which also makes the task more difficult. In this paper, we propose a diver detection method based on saliency detection for sonar images. On the basis of studying the characteristics of diver sonar images, we first decompose the original sonar image and perform median filtering on it, which can significantly improve the quality of the sonar image saliency map. We employ saliency detection technique based on frequency analysis to segment the acoustic highlight region from its surroundings. This segmentation region roughly locates the diver target and generates a region of interest (ROI). We then extract the acoustic shadow region in ROI, which contributes to furtherly improve the localization accuracy. Finally, we merge the segmented highlight region and the extracted acoustic shadow region and compute the minimum outer rectangle of the merged region. Experimental results validate that the proposed method can well detect and locate the diver target, and it can also satisfy the demands of real-time application, and there is almost no false alarm in this method.

Saliency detection via outlier pursuit in compress domain (SDOPCD)

A saliency detection technique has been proposed in this article. The two major constituents, proximal gradient pursuit and compress sensing techniques, are the important requirement.The preliminery assumption is salient object adhered to minimal portion in an image and exist in large size along with other color based constraints. The concept of superpixel has been employed to divide the image into super pixel blocks. The similar colors within the predefined radius are used to reduce required color space. The blocks of super pixels are used as columns to constitute a matrix which is further compressed using compress sensing methods. The compressed matrix is used in outlier pursuit algorithm. The sparse matrix generated by outlier pursuit provides the required salient object without requiring any reconstruction method. The performance enhancement is verified by comparing with existing methods.

Dynamic Zero-Parallax-Setting Techniques for Multi-View Autostereoscopic Display

The objective of this paper is to research a dynamic computation of Zero-Parallax-Setting (ZPS) for multi-view autostereoscopic displays in order to effectively alleviate blurry 3D vision for images with large disparity. Saliency detection techniques can yield saliency map which is a topographic representation of saliency which refers to visually dominant locations. By using saliency map, we can predict what attracts the attention, or region of interest, to viewers. Recently, deep learning techniques have been applied in saliency detection. Deep learning-based salient object detection methods have the advantage of highlighting most of the salient objects. With the help of depth map, the spatial distribution of salient objects can be computed. In this paper, we will compare two dynamic ZPS techniques based on visual attention. They are 1) maximum saliency computation by Graphic-Based Visual Saliency (GBVS) algorithm and 2) spatial distribution of salient objects by a convolutional neural networks (CNN)-based model. Experiments prove that both methods can help improve the 3D effect of autostereoscopic displays. Moreover, the spatial distribution of salient objects-based dynamic ZPS technique can achieve better 3D performance than maximum saliency-based method.

Detecting action-relevant regions for action recognition using a three-stage saliency detection technique

Dense tracking has been proven successful in action recognition, but it may produce a large number of features in background, which are not so relevant to actions and may hurt recognition performance. To obtain the action-relevant features for action recognition, this paper proposes a three-stage saliency detection technique to recover action-relevant regions. In the first stage, low-rank matrix recovery optimization is employed to decompose the overall motion of each sub-video (temporally split video) into a low-rank part and a sparse part, and the latter is used to compute initial saliency to discriminate candidate foreground from definite background. In the second stage, using the dictionary formed by the patches in definite background, the sparse representation for each patch in candidate foreground is obtained based on motion and appearance information to compute the refined saliency, which ensures the action-relevant regions tend to be distinguished more clearly from background. In the third stage, the saliency is spatially updated based on the motion and appearance similarity so that the action-relevant regions can be better highlighted due to the increase of spatial saliency coherence. Finally, a binary saliency map is created by comparing the updated saliency with a given threshold to indicate action-relevant regions, which is fused into dense tracking to extract action-relevant trajectory features in a video for action recognition. Experimental results on four benchmark datasets demonstrate that the proposed method performs better than the conventional dense tracking and competitively with its improved versions.

Automatic Optic Disc Segmentation Based on Modified Local Image Fitting Model with Shape Prior Information.

Accurate optic disc (OD) detection is an essential yet vital step for retinal disease diagnosis. In the paper, an approach for segmenting OD boundary without manpower named full-automatic double boundary extraction is designed. There are two main advantages in it. (1) Since the performances and the computational cost produced by iterations of contour evolution of active contour models- (ACM-) based approaches greatly depend on the initialization, this paper proposes an effective and adaptive initial level set contour extraction approach using saliency detection and threshold techniques. (2) In order to handle unreliable information generated by intensity in abnormal retinal images caused by diseases, a modified LIF approach is presented by incorporating the shape prior information into LIF. We test the effectiveness of the proposed approach on a publicly available DIARETDB0 database. Experimental results demonstrate that our approach outperforms well-known approaches in terms of the average overlapping ratio and accuracy rate.

Underwater Object Recognition Using Transformable Template Matching Based on Prior Knowledge

Underwater object recognition in sonar images, such as mine detection and wreckage detection of a submerged airplane, is a very challenging task. The main difficulties include but are not limited to object rotation, confusion from false targets and complex backgrounds, and extensibility of recognition ability on diverse types of objects. In this paper, we propose an underwater object detection and recognition method using a transformable template matching approach based on prior knowledge. Specifically, we first extract features and construct a template from sonar video sequences based on the analysis of acoustic shadows and highlight regions. Then, we identify the target region in the objective image by fast saliency detection techniques based on FFT, which can significantly improve efficiency by avoiding an exhaustive global search. After affine transformation of the template according to the orientation of the target, we extract normalized gradient features and calculate the similarity between the template and the target region, which can solve various difficulties mentioned above using only one template. Experimental results demonstrate that the proposed method can well recognize different underwater objects, such as mine‐like objects and triangle‐like objects and can satisfy the demands of real‐time application.

RETRACTED ARTICLE: Saliency detection in stereoscopic images using adaptive Gaussian Kernel and Gabor filter

So far a number of saliency detection designs have been developed for various applications in multimedia scenario. However, certain applications of stereoscopic imaging necessitate enhancing saliency detection designs to extract the salient areas accurately. Saliency detection strategy still faces many challenges like complexity in natural images as well as lesser-scale patterns in salient objects. This paper is set to obtain the saliency detection technique in two stages: feature extraction and depth saliency detection. Gaussian Kernel model is used to extract the features and Gabor filter to get the depth saliency map. This paper optimizes two coefficients namely feature difference among image patches $$ U $$ , from feature evaluation and fine scale $$ c $$ , from which the accurate detection can be achieved. For this, a renowned optimization algorithm named grey wolf optimization (GWO) is used, which is a recently proposed intelligent optimization method inspired by hunting behavior of grey wolves. The saliency detection model detection model is implemented in MATLAB 2015a. To the next of implementation, the performance of the proposed model, GWO-3-dimensional saliency mapping is compared over other the conventional algorithms in terms of receiver operator curve, Pearson correlation coefficient, Kullback–Leibler divergence and area under the curve. The overall performance analysis in terms of above measures and a valuable statistical analysis are carried out for validating the effectiveness of proposed model. The entire analysis proves that the proposed saliency detection performs better than the conventional models.

Robust image hashing through DWT-SVD and spectral residual method

In the last few decades, the discovery of various methods for generating secure image hash has become revolutionary in the field of image hashing. This paper presents an efficient approach to obtain image hash through DWT-SVD and a saliency detection technique using spectral residual model. The latest image hashing technique based on ring partition and invariant vector distance is rotation invariant for the large angle at the cost of being insensitive to corner forgery. But, due to the use of the central orientation information, the proposed system is rotation invariant for arbitrary angles along with sensitiveness to corner changes. In addition, we have used the HSV color space that gives desirable classification performance. It provides satisfactory results against large degree rotation, JPEG compression, brightness and contrast adjustment, watermarking, etc. This technique is also sensitive to malicious activities. Moreover, it locates the forged areas of a forged image. We have applied the proposed algorithm to a large collection of images from various databases. The receiver operating characteristics shows that the proposed method is better than some state-of-the-art methods.

Utilising Kronecker Decomposition and Tensor-based Multi-view Learning to predict where people are looking in images

Eye movement data collection is very expensive and laborious. Moreover, there are usually missing values. Assuming that we are collecting eye movement data from a set of images viewed by different users, there is a possibility that we will not able to collect the data of every user from every image–one or more views may not be represented in the image. We assume that the relationships among the views can be learnt from the whole collection of views (or items). The task is then to reproduce the missing part of the incomplete items from the relationships derived from the complete items and the known part of these items. Using certain properties of tensor algebra, we showed that this problem can be formulated consistently as a regression type learning task. Furthermore, there is a maximum margin based optimisation framework in which this problem can be solved in a tractable way. This problem is similar to learning to predict where a person is looking in an image. Therefore, we proposed an algorithm called “Tensor-based Multi-View Learning”(TMVL) in this paper. Furthermore, we also proposed a technique for improving prediction by introducing a new feature set obtained from Kronecker decomposition of the image fused with user’s eye movement data. Using this new feature can improve prediction performance markedly. The proposed approach was proven to be more effective than two well-known saliency detection techniques.