Saliency-based Approach Research Articles

SU2GE-Net: a saliency-based approach for non-specific class foreground segmentation

Salient object detection is vital for non-specific class subject segmentation in computer vision applications. However, accurately segmenting foreground subjects with complex backgrounds and intricate boundaries remains a challenge for existing methods. To address these limitations, our study proposes SU2GE-Net, which introduces several novel improvements. We replace the traditional CNN-based backbone with the transformer-based Swin-TransformerV2, known for its effectiveness in capturing long-range dependencies and rich contextual information. To tackle under and over-attention phenomena, we introduce Gated Channel Transformation (GCT). Furthermore, we adopted an edge-based loss (Edge Loss) for network training to capture spatial-wise structural details. Additionally, we propose Training-only Augmentation Loss (TTA Loss) to enhance spatial stability using augmented data. Our method is evaluated using six common datasets, achieving an impressive F_{beta } score of 0.883 on DUTS-TE. Compared with other models, SU2GE-Net demonstrates excellent performance in various segmentation scenarios.

Saliency-Aided Online RPCA for Moving Target Detection in Infrared Maritime Scenarios.

Moving target detection (MTD) is a crucial task in computer vision applications. In this paper, we investigate the problem of detecting moving targets in infrared (IR) surveillance video sequences captured using a steady camera in a maritime setting. For this purpose, we employ robust principal component analysis (RPCA), which is an improvement of principal component analysis (PCA) that separates an input matrix into the following two matrices: a low-rank matrix that is representative, in our case study, of the slowly changing background, and a sparse matrix that is representative of the foreground. RPCA is usually implemented in a non-causal batch form. To pursue a real-time application, we tested an online implementation, which, unfortunately, was affected by the presence of the target in the scene during the initialization phase. Therefore, we improved the robustness by implementing a saliency-based strategy. The advantages offered by the resulting technique, which we called "saliency-aided online moving window RPCA" (S-OMW-RPCA) are the following: RPCA is implemented online; along with the temporal features exploited by RPCA, the spatial features are also taken into consideration by using a saliency filter; the results are robust against the condition of the scene during the initialization. Finally, we compare the performance of the proposed technique in terms of precision, recall, and execution time with that of an online RPCA, thus, showing the effectiveness of the saliency-based approach.

Federated selective aggregation for on-device knowledge amalgamation

In this paper, we explore a new knowledge amalgamation problem, termed Federated Selective Aggregation for on-device knowledge amalgamation (FedSA). FedSA aims to train an on-device student model for a new task with the help of several decentralized teachers whose pre-training tasks and data are different and agnostic. Our motivation for investigating such a problem setup stems from a recent dilemma of model sharing. Due to privacy, security, or intellectual property issues, the pre-trained models are, however, not able to be shared, and the resources of devices are usually limited. The proposed FedSA offers a solution to this dilemma and makes it one step further, again, the method can be employed on low-power and resource-limited devices. To this end, we proposed a dedicated strategy for handling knowledge amalgamation. Specifically, our student-training process is driven by a novel saliency-based approach that adaptively selects teachers as the participants and integrates their representative capabilities into the student. To evaluate the effectiveness of FedSA, we conduct experiments on both single-task and multi-task settings. Experimental results demonstrate that FedSA effectively amalgamates knowledge from decentralized models and achieves competitive performance to centralized baselines.

A Visual Saliency-Based Approach for Content-Based Image Retrieval

During the past two decades an enormous amount of visual information has been generated; as a result, content-based image retrieval (CBIR) has received considerable attention. In CBIR the image is used as a query to find the most similar images. One of the biggest challenges in CBIR system is to fill up the “semantic gap,” which is the gap between low-level visual features and the high-level semantic concepts of an image. In this paper, the authors have proposed a saliency-based CBIR system that utilizes the semantic information of image and users search intention. In the proposed model, firstly a significant region is identified with the help of method structured matrix decomposition (SMD) using high-level priors that highlight the prominent area of the image. After that, a two-dimensional principal component analysis (2DPCA) is used as a feature, which is compact and effectively used for fast recognition. Experiment results are validated on different image dataset having an extensive collection of semantic classifications.

A saliency-based approach to event recognition

Over the last few years, a number of interesting solutions covering different aspects of event recognition have been proposed for event-based multimedia analysis. Existing approaches mostly focus on an efficient representation of the image and advanced classification schemes. However, it would be desirable to focus on the event-specific information available in the image, namely the so-called event saliency. In this paper, we propose a novel approach based on multiple instance learning (MIL) to learn the visual features contained in event-salient regions, extracted through a crowd-sourcing study. In total, we collect the salient regions for 76 different events from 4 large-scale datasets. The experimental results demonstrate the efficacy of using only event-related regions by achieving a significant gain in performance over the state-of-the-art.

SWVFS: a saliency weighted visual feature similarity metric for image quality assessment

In this paper, a saliency weighted visual feature similarity (SWVFS) metric is proposed for full reference image quality assessment (IQA). Instead of traditional spatial pooling strategies, a visual saliency-based approach is employed for better compliance with properties of the human visual system, where the saliency allocation is closely related to the activity of posterior parietal cortex and the pluvial nuclei of the thalamus. Assuming that the saliency map actually represents the contribution of locally computed visual distortions to the overall image quality, the gradient similarity and the textural congruency are merged into the final image quality indicator. The gradient and texture comparison play complementary roles in characterizing the local image distortion. Extensive experiments conducted on seven publicly available image databases show that the performance of SWVFS is competitive with the state-of-the-art IQA algorithms.

Saliency adaptive super-resolution image reconstruction

A saliency-based approach is proposed in this paper to perform super-resolution image reconstruction, with the aim to produce a higher-resolution image based on its low-resolution counterparts. The proposed approach adaptively adjusts the degree of regularization using the saliency measure of the local content of the image. Furthermore, a gradient-based criterion is proposed to measure the saliency of the image. Experiments are conducted to demonstrate the superior performance of the proposed approach.

An integrated visual saliency-based watermarking approach for synchronous image authentication and copyright protection

This paper proposes an integrated visual saliency-based watermarking approach, which can be used for both synchronous image authentication and copyright protection. Firstly, regions of interest (ROIs), which are not in a fixed size and can present the most important information of one image, would be extracted automatically using the proto-object based saliency attention model. Secondly, to resist common signal processing attacks, for each ROI, an improved quantization method is employed to embed the copyright information into its DCT coefficients. Finally, the edge map of one ROI is chosen as the fragile watermark, and is then embedded into the DWT domain of the watermarked image to further resist the tampering attacks. Using ROI-based visual saliency as a bridge, this proposed method can achieve image authentication and copyright protection synchronously, and it can also preserve much more robust information. Experimental results on standard benchmark demonstrate that compared with the state-of-the-art watermarking scheme, the proposed method is more robust to white noise, filtering and JPEG compression attacks. Furthermore, it also shows that the proposed method can effectively detect tamper and locate forgery.

INCORPORATING A WAVELET TRANSFORM INTO A SALIENCY-BASED METHOD FOR ONLINE HANDWRITING SEGMENTATION

In the framework of a saliency-based approach for segmenting cursive handwriting into elementary strokes, we propose a smoothing technique based on the use of a Wavelet Transform to describe the electronic ink at different resolutions. According to such an approach, derived in analogy with those proposed in the literature for early visual tasks in primates, curvature maxima corresponding to actual segmentation points are separated from those produced by the different source of noise affecting the handwriting generation process. This is obtained by evaluating the curvature maxima at different levels of resolution, and by arranging them into a pyramidal structure, from which a saliency map is eventually achieved by combining the values across all possible scales. Such maps enjoy the property of exhibiting higher values in correspondence of regions of the original ink where curvature maxima survive along difference scales, thus indicating that those regions should correspond to the parts of the ink where two strokes join. Experiments performed by using both standard databases and one of online words collected in our laboratory, have shown high performance in terms of correct segmentation, stability and predictability of the results.