Visual Saliency Method Research Articles

Fabric defect detection using cartoon–texture image decomposition model and visual saliency method

Fabric defect detection is of great importance in the modern textile industry. In actual production, subjective factors usually affect manual detection, which is prone to problems such as false and missed detection. With the development of computer vision, a large number of fabric defect automatic detection algorithms have been proposed. Traditional algorithms rely too heavily on setting parameters manually, while deep learning algorithms have expensive training and computing costs. Given these limitations, a new fabric defect detection method based on the latest cartoon texture image decomposition model, visual saliency algorithm, and mathematical morphology, is proposed in this article. A digital image acquisition system is also designed and constructed. Therefore, the self-made dataset used in the experiment is composed of self-collected images and network public images. To further evaluate the performance of the proposed method, this study conducted fabric defect detection experiments and comparison experiments based on the self-made dataset. The results show that this method can successfully detect fabric defects, having high detection accuracy and efficiency. Combining subjective vision and objective evaluation, comparison experiments prove that this method is superior to other common fabric defect detection methods and has the highest value of accuracy and F1-score. This research provides a new method and technical support for fabric defect detection and other fields.

SR-HorNetYOLO: an efficient algorithm for defect detection of metal gasket end-face

Metal gaskets are an essential component of the manufacturing industry, and detecting end-face defects is critical to ensuring their quality. However, current detection methods struggle to detect such defects due to the gaskets’ uneven end-face structure and small defect size. To address such issues, an approach called SR-HorNetYOLO has been proposed. This approach combines traditional machine vision and deep learning techniques. The visual saliency method is used to simplify the image and remove irrelevant features. YOLOv5 is then optimized in three ways: the GhostNet backbone replaces the CSPDarkNet53 backbone to reduce model weight, the self-attention mechanism and bottleneck transformer block improve the precision of detecting small targets, and the feature pyramid is improved through the HorNet network to enhance the predictive performance of the network. Finally, a classification module and position module are employed to classify and locate three-fusion proportion characteristic images. The experimental results show that the SR-HorNetYOLO method achieves a precision of 93.3% and a recall rate of 94.0%.

3D Visual Saliency: An Independent Perceptual Measure or a Derivative of 2D Image Saliency?

While 3D visual saliency aims to predict regional importance of 3D surfaces in agreement with human visual perception and has been well researched in computer vision and graphics, latest work with eye-tracking experiments shows that state-of-the-art 3D visual saliency methods remain poor at predicting human fixations. Cues emerging prominently from these experiments suggest that 3D visual saliency might associate with 2D image saliency. This paper proposes a framework that combines a Generative Adversarial Network and a Conditional Random Field for learning visual saliency of both a single 3D object and a scene composed of multiple 3D objects with image saliency ground truth to 1) investigate whether 3D visual saliency is an independent perceptual measure or just a derivative of image saliency and 2) provide a weakly supervised method for more accurately predicting 3D visual saliency. Through extensive experiments, we not only demonstrate that our method significantly outperforms the state-of-the-art approaches, but also manage to answer the interesting and worthy question proposed within the title of this paper.

TSGB: Target-Selective Gradient Backprop for Probing CNN Visual Saliency.

The explanation for deep neural networks has drawn extensive attention in the deep learning community over the past few years. In this work, we study the visual saliency, a.k.a. visual explanation, to interpret convolutional neural networks. Compared to iteration based saliency methods, single backward pass based saliency methods benefit from faster speed, and they are widely used in downstream visual tasks. Thus, we focus on single backward pass based methods. However, existing methods in this category struggle to successfully produce fine-grained saliency maps concentrating on specific target classes. That said, producing faithful saliency maps satisfying both target-selectiveness and fine-grainedness using a single backward pass is a challenging problem in the field. To mitigate this problem, we revisit the gradient flow inside the network, and find that the entangled semantics and original weights may disturb the propagation of target-relevant saliency. Inspired by those observations, we propose a novel visual saliency method, termed Target-Selective Gradient Backprop (TSGB), which leverages rectification operations to effectively emphasize target classes and further efficiently propagate the saliency to the image space, thereby generating target-selective and fine-grained saliency maps. The proposed TSGB consists of two components, namely, TSGB-Conv and TSGB-FC, which rectify the gradients for convolutional layers and fully-connected layers, respectively. Extensive qualitative and quantitative experiments on the ImageNet and Pascal VOC datasets show that the proposed method achieves more accurate and reliable results than the other competitive methods. Code is available at https://github.com/123fxdx/CNNvisualizationTSGB.

Learning Reliable Visual Saliency For Model Explanations

By highlighting important features that contribute to model prediction, visual saliency is used as a natural form to interpret the working mechanism of deep neural networks. Numerous methods have been proposed to achieve better saliency results. However, we find that previous visual saliency methods are not reliable enough to provide meaningful interpretation through a simple sanity check: saliency methods are required to explain the output of non-maximum prediction classes, which are usually not ground-truth classes. For example, let the methods interpret an image of “dog” given a wrong class label “fish” as the query. This procedure can test whether these methods reliably interpret model's predictions based on existing features that appear in the data. Our experiments show that previous methods failed to pass the test by generating similar saliency maps or scattered patterns. This false saliency response can be dangerous in certain scenarios, such as medical diagnosis. We find that these failure cases are mainly due to the attribution vanishing and adversarial noise within these methods. In order to learn reliable visual saliency, we propose a simple method that requires the output of the model to be close to the original output while learning an explanatory saliency mask. To enhance the smoothness of the optimized saliency masks, we then propose a simple Hierarchical Attribution Fusion (HAF) technique. In order to fully evaluate the reliability of visual saliency methods, we propose a new task Disturbed Weakly Supervised Object Localization (D-WSOL) to measure whether these methods can correctly attribute the model's output to existing features. Experiments show that previous methods fail to meet this standard, and our approach helps to improve the reliability by suppressing false saliency responses. After observing a significant layout difference in saliency masks between real and adversarial samples. we propose to train a simple CNN on these learned hierarchical attribution masks to distinguish adversarial samples. Experiments show that our method can improve detection performance over other approaches significantly.

Novel event analysis for human-machine collaborative underwater exploration

One of the main task for deep sea submersible is for human-machine collaborative scientific exploration, e.g., human ourselves drive the submersible and monitor cameras around the submersible to observe new species fish or strange topography in a tedious way. In this paper, by defining novel marine animals or any extreme events as novel events, we design a new deep sea novel visual event analysis framework to improve the efficiency of human-machine collaboration and improve the accuracy simultaneously. Specifically, our visual framework concerns diverse functions than most state-of-the-arts, including novel event detection, tracking and summarization. Due to the power and computation resource limitation of the submersible, we design an efficient deep learning based visual saliency method for novel event detection and propose an online object tracking strategy as well. All the experiments are depending on Chinese Jiaolong, the manned deep sea submersible, which mounts several PanCtiltCzoom (PTZ) camera and static cameras. We build a new novel deep sea event dataset and the results justify that our human-machine collaborative visual observation framework can automatically detect, track and summarize the novel deep sea event.

A Locally Weighted Fixation Density-Based Metric for Assessing the Quality of Visual Saliency Predictions.

With the increased focus on visual attention (VA) in the last decade, a large number of computational visual saliency methods have been developed over the past few years. These models are traditionally evaluated by using performance evaluation metrics that quantify the match between predicted saliency and fixation data obtained from eye-tracking experiments on human observers. Though a considerable number of such metrics have been proposed in the literature, there are notable problems in them. In this paper, we discuss shortcomings in the existing metrics through illustrative examples and propose a new metric that uses local weights based on fixation density, which overcomes these flaws. To compare the performance of our proposed metric at assessing the quality of saliency prediction with other existing metrics, we construct a ground-truth subjective database in which saliency maps obtained from 17 different VA models are evaluated by 16 human observers on a five-point categorical scale in terms of their visual resemblance with corresponding ground-truth fixation density maps obtained from eye-tracking data. The metrics are evaluated by correlating metric scores with the human subjective ratings. The correlation results show that the proposed evaluation metric outperforms all other popular existing metrics. In addition, the constructed database and corresponding subjective ratings provide an insight into which of the existing metrics and future metrics are better at estimating the quality of saliency prediction and can be used as a benchmark.

Visual Saliency Using Binary Spectrum of Walsh–Hadamard Transform and Its Applications to Ship Detection in Multispectral Imagery

Detection of visual saliency is valuable for applications like robot navigation, adaptive image compression, and object recognition. In this paper, we propose a fast frequency domain visual saliency method by use of the binary spectrum of Walsh---Hadamard transform (WHT). The method achieves saliency detection by simply exploiting the WHT components of the scene under view. Unlike space domain-based approaches, our method performs the cortical center-surround suppression in frequency domain and thus has implicit biological plausibility. By virtue of simplicity and speed of the WHT, the proposed method is very simple and fast in computation, and outperforms existing state-of-the-art saliency detection methods, when evaluated by using the capability of eye fixation prediction. In arduous tasks of ship detection in multispectral imagery, large amount of multispectral data require real-time processing and analyzing. As a very fast and effective technique for saliency detection, the proposed method is modified and applied to automatic ship detection in multispectral imagery. The robustness of the method against sea clutters is further proved.

Building up a Bio-Inspired Visual Attention Model by Integrating Top-Down Shape Bias and Improved Mean Shift Adaptive Segmentation

The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments.

Real-time visualization of 3D city models at street-level based on visual saliency

Street-level visualization is an important application of 3D city models. Challenges to street-level visualization include the cluttering of buildings due to fine detail and visualization performance. In this paper, a novel method is proposed for street-level visualization based on visual saliency evaluation. The basic idea of the method is to preserve these salient buildings in a scene while removing those that are non-salient. The method can be divided into pre-processing procedures and real-time visualization. The first step in pre-processing is to convert 3D building models at higher Levels of Detail (LoDs) into LoD1 models with simplified ground plans. Then, a number of index viewpoints are created along the streets; these indices refer to both the position and the direction of each street site. A visual saliency value is computed for each building, with respect to the index site, based on a visual difference between the original model and the generalized model. We calculate and evaluate three methods for visual saliency: local difference, global difference and minimum projection area. The real-time visualization process begins by mapping the observer to its closest indices. The street view is then generated based on the building information stored in those indexes. A user study shows that the local visual saliency method performs better than do the global visual saliency, area and image-based methods and that the framework proposed in this paper may improve the performance of 3D visualization.

Research of Smoke Detection on Visual Saliency Method

Smoke detection is the key to the early warning of the fire, and it is hard to reach the unified standard of the smoke detection because of different environments and different combustions. Considering the continuity of the occurrence of smoke and the more obvious visual saliency along with the long-time integration, the paper proposes the algorithm of multi-step accumulation of inter-frame difference in order to rapidly find out the regions in which moving targets in video can appear, which can reduce the detection range. In the small matrix in the motion region, the matrix of low rank and sparse decomposition are adopted to separate the moving foreground object from the background. In the complex outdoor scene, the smoke’s drift motility and the color’s translucence are more obvious, and the smoke target can be locked by means of the growth for all motion region and the saliency detection in HSV color space. The experiment compares the current mainstream salient algorithms which are applied to the smoke detection. The method of detecting speed and accuracy which is used in the paper has achieved a good effect. The method can be applied in different video scenes, even in the low-resolution and strong-noise scenes, it can also achieve a better detection result.