Local Contrast Features Research Articles

Scene‐content‐sensitive real‐time adaptive foveated rendering

AbstractIn recent years, techniques for accelerating rendering by exploiting the limitations of the human visual system have become increasingly prevalent. The foveated rendering method significantly reduces the computational requirements during rendering by reducing image quality in peripheral regions. In this paper, we propose a scene‐content‐sensitive real‐time adaptive foveated rendering method. First, we pre‐render the three‐dimensional (3D) scene at a low resolution. Then, we utilize the low‐resolution pre‐rendered image as input to extract edge, local contrast, and color features. Subsequently, we generate a screen‐space region division map based on the gaze point position. Next, we calculate the visual importance of each 16 × 16 pixel tile based on edge, local contrast, color, and screen‐space region. We then map the visual importance to the shading rate to generate a shading rate control map for the current frame. Finally, we complete the rendering of the current frame based on variable rate shading technology. Experimental results demonstrate that our method effectively enhances the visual quality of images near the foveal region while generating high quality foveal region images. Furthermore, our method can significantly improve performance compared to per‐pixel shading method and existing scene‐content‐based foveated rendering methods.

Feature-level contrastive learning for full-reference light field image quality assessment

Light field (LF), as a new visual information representation for immersive experiences in virtual reality and augmented reality, has attracted extensive attention. It is of significance to establish the light field image quality assessment (LF-IQA) that conforms to the characteristics of human visual perception. Due to the spatial–angular inter-twined imaging mechanism, the quality evaluation for LF images faces more challenges than ordinary 2D/3D IQA. This paper proposes a feature-level contrastive learning based full-reference LF-IQA with a divide-and-conquer learning strategy. It first employs the cross-attention mechanism to learn the local contrast features related to the reference features from three subsets of distorted sub-aperture images (SAIs). Then a Transformer-based feature fusion is used to predict the overall perceptual quality. The evaluation experiments are carried out on three commonly used LF-IQA datasets: Win5-LID, NBU-LF1.0, and SHU. The experimental results demonstrate that our model outperforms the existing state-of-the-art LF-IQA methods.

The object-based shift direction anisotropy is modulated by the horizontal visual field meridian.

Reallocating object-based attention across the visual field meridians is significantly faster horizontally than vertically (termed the shift direction anisotropy; SDA), implicating the meridians in reorienting object-based attention. Here, we tested the modulatory role of the meridians in the emergence of the SDA by manipulating meridian local feature contrast. Considering the notion of separate pools of attentional resources in each cortical hemisphere, we hypothesised that manipulating the horizontal meridian would selectively modulate the SDA. In four experiments, participants were presented with an "L"-shaped object and detected a target that appeared at either a cued location or at one of two equidistant non-cued locations at the far end of the horizontal or vertical object arm. Meridian local feature contrast was manipulated with perceptually strong enhancements (visible lines and colour contrast borders) and perceptually weak enhancements (illusory borders from line texture patterns and inducers). Weak enhancements of the meridians did not significantly modulate SDA magnitude; however, during perceptually strong enhancements of the horizontal meridian, the SDA was significantly reduced compared with both vertical meridian enhancement and no-enhancement conditions. Moreover, horizontal and vertical shift RTs were statistically equivalent when the horizontal meridian was enhanced with a visible line, our strongest manipulation, indicating the SDA was eliminated. These results suggest that the SDA emerges due to reallocating object-based attention across the horizontal meridian. We interpret this finding as evidence in support of the theory by which anatomical segregations of the visual system determine how pools of attentional resources resolve competition between and within cortical hemispheres.

Star map spatial target detection method based on fast local contrast and target features

Star map spatial target detection method based on fast local contrast and target features

Feature-Specific Salience Maps in Human Cortex.

Priority map theory is a leading framework for understanding how various aspects of stimulus displays and task demands guide visual attention. Per this theory, the visual system computes a priority map, which is a representation of visual space indexing the relative importance, or priority, of locations in the environment. Priority is computed based on both salience, defined based on image-computable properties; and relevance, defined by an individual's current goals, and is used to direct attention to the highest-priority locations for further processing. Computational theories suggest that priority maps identify salient locations based on individual feature dimensions (e.g., color, motion), which are integrated into an aggregate priority map. While widely accepted, a core assumption of this framework, the existence of independent feature dimension maps in visual cortex, remains untested. Here, we tested the hypothesis that retinotopic regions selective for specific feature dimensions (color or motion) in human cortex act as neural feature dimension maps, indexing salient locations based on their preferred feature. We used fMRI activation patterns to reconstruct spatial maps while male and female human participants viewed stimuli with salient regions defined by relative color or motion direction. Activation in reconstructed spatial maps was localized to the salient stimulus position in the display. Moreover, the strength of the stimulus representation was strongest in the ROI selective for the salience-defining feature. Together, these results suggest that feature-selective extrastriate visual regions highlight salient locations based on local feature contrast within their preferred feature dimensions, supporting their role as neural feature dimension maps.SIGNIFICANCE STATEMENT Identifying salient information is important for navigating the world. For example, it is critical to detect a quickly approaching car when crossing the street. Leading models of computer vision and visual search rely on compartmentalized salience computations based on individual features; however, there has been no direct empirical demonstration identifying neural regions as responsible for performing these dissociable operations. Here, we provide evidence of a critical double dissociation that neural activation patterns from color-selective regions prioritize the location of color-defined salience while minimally representing motion-defined salience, whereas motion-selective regions show the complementary result. These findings reveal that specialized cortical regions act as neural "feature dimension maps" that are used to index salient locations based on specific features to guide attention.

An Infrared Small Target Detection Method Based on Attention Mechanism.

The human visual attention system plays an important role in infrared target recognition because it can quickly and accurately recognize infrared small targets and has good scene adaptability. This paper proposes an infrared small target detection method based on an attention mechanism, which consists of three modules: a bottom-up passive attention module, a top-down active attention module, and decision feedback equalization. In the top-down active attention module, given the Gaussian characteristics of infrared small targets, the idea of combining knowledge-experience Gaussian shape features is applied to implement feature extraction, and quaternion cosine transform is performed to achieve multi-dimensional fusion of Gaussian shape features, thereby achieving complementary fusion of multi-dimensional feature information. In the bottom-up passive attention module, considering that the difference in contrast and motion between the target and the background can attract attention easily, an optimal fast local contrast algorithm and improved circular pipeline filtering are adopted to find candidate target regions. Meanwhile, the multi-scale Laplacian of the Gaussian filter is adopted to estimate the optimal size of the infrared small target. The fast local contrast algorithm based on box filter acceleration and structure optimization is employed to extract local contrast features, and candidate target regions can be obtained by using an adaptive threshold. Besides, the mean gray, target size, Gaussian consistency, and circular region constraint are used in pipeline filtering to extract motion regions, and the false-alarm rate is reduced effectively. Finally, decision feedback equalization is adopted to obtain real targets. Experiments are conducted on some real infrared images involving complex backgrounds with sea, sky, and ground clutters, and the experimental results indicate that the proposed method can achieve better detection performance than conventional baseline methods, such as RLCM, ILCM, PQFT, MPCM, and ADMD. Also, mathematical proofs are provided to validate the proposed method.

MS-CFNet: a multi-scale clinical studying-based and feature extraction-guided network for breast fibroadenoma segmentation in ultrasonography.

Segmenting breast tumors in ultrasonography is challenging due to the low image quality and presence of artifacts. Radiologists' studying and diagnosis skills are integrated with artificial intelligence to establish a clinical learning-based deep learning network in order to robustly extract and delineate features of breast fibroadenoma. The spatial local feature contrast (SLFC) module captures overall tumor contours, while the channel recursive gated attention (CRGA) module enhances edge perception through high-dimensional information interaction. Additionally, full-scale feature fusion and enhanced deep supervision are applied to improve model stability and performance. To achieve smoother boundaries, we introduce a new loss function (cosh-smooth) that penalizes and finely tunes tumor edges. Our dataset comprises 1016 clinical ultrasound images of breast fibroadenoma with labeled masks, alongside a publicly available dataset of 246 ones. Segmentation performance is evaluated using the Dice similarity coefficient (DSC) and mean intersection over union (MIOU). Extensive experiments demonstrate that our proposed MS-CFNet outperforms state-of-the-art methods. Compared to TransUNet as a baseline model, MS-CFNet improves by 1.47% in DSC and 2.56% in MIOU. The promising result of MS-CFNet is attributed to the integration of radiologists' clinical diagnosis procedure and the bionic mindset, enhancing the network's ability to recognize and segment breast fibroadenomas effectively.

A Fusion Underwater Salient Object Detection Based on Multi-Scale Saliency and Spatial Optimization

Underwater images contain abundant information, but many challenges remain for underwater object detection tasks. Various salient object detection methods may encounter low detection precision, and the segmented map has an incomplete region of the target object. To deal with blurry underwater scenes and vague detection problems, a novel fusion underwater salient object detection algorithm (FUSOD) is proposed based on multi-scale saliency and spatial optimization. Firstly, an improved underwater color restoration was utilized to restore the color information for afterward color contrast saliency calculation. Secondly, a more accurate multi-scale fusion saliency map was obtained by fully considering both the global and local feature contrast information. Finally, the fusion saliency was optimized by the proposed spatial optimization method to enhance the spatial coherence. The proposed FUSOD algorithm may process turbid and complex underwater scenes and preserve a complete structure of the target object. Experimental results on the USOD dataset show that the proposed FUSOD algorithm can segment the salient object with a comparatively higher detection precision than the other traditional state-of-the-art algorithms. An ablation experiment showed that the proposed spatial optimization method increases the detection precision by 0.0325 scores in the F-Measure.

Which processes dominate visual search: Bottom-up feature contrast, top-down tuning or trial history?

Previous research has identified three mechanisms that guide visual attention: bottom-up feature contrasts, top-down tuning, and the trial history (e.g., priming effects). However, only few studies have simultaneously examined all three mechanisms. Hence, it is currently unclear how they interact or which mechanisms dominate over others. With respect to local feature contrasts, it has been claimed that a pop-out target can only be selected immediately in dense displays when the target has a high local feature contrast, but not when the displays are sparse, which leads to an inverse set-size effect. The present study critically evaluated this view by systematically varying local feature contrasts (i.e., set size), top-down knowledge, and the trial history in pop-out search. We used eye tracking to distinguish between early selection and later identification-related processes. The results revealed that early visual selection was mainly dominated by top-down knowledge and the trial history: When attention was biased to the target feature, either by valid pre-cueing (top-down) or automatic priming, the target could be localised immediately, regardless of display density. Bottom-up feature contrasts only modulated selection when the target was unknown and attention was biased to the non-targets. We also replicated the often-reported finding of reliable feature contrast effects in the mean RTs, but showed that these were due to later, target identification processes (e.g., in the target dwell times). Thus, contrary to the prevalent view, bottom-up feature contrasts in dense displays do not seem to directly guide attention, but only facilitate nontarget rejection, probably by facilitating nontarget grouping.

Method of Infrared Small Moving Target Detection Based on Coarse-to-Fine Structure in Complex Scenes

In the combat system, infrared target detection is an important issue worthy of study. However, due to the small size of the target in the infrared image, the low signal-to-noise ratio of the image and the uncertainty of motion, how to detect the target accurately and quickly is still difficult. Therefore, in this paper, an infrared method of detecting small moving targets based on a coarse-to-fine structure (MCFS) is proposed. The algorithm mainly consists of three modules. The potential target extraction module first smoothes the image through a Laplacian filter and extracts the prior weight of the image by the proposed weighted harmonic method to enhance the target and suppress the background. Then, the local variance feature map and local contrast feature map of the image are calculated through a multiscale three-layer window to obtain the potential target region. Next, a new robust region intensity level (RRIL) algorithm is proposed in the spatial-domain weighting module. Finally, the temporal-domain weighting module is established to enhance the target positions by analyzing the kurtosis features of temporal signals. Experiments are conducted on real infrared datasets. Through scientific analysis, the proposed method can successfully detect the target, at the same time, the ability to suppress the background and the ability to improve the target has reached the maximum, which verifies the effectiveness of the algorithm.

Multiform Ensemble Self-Supervised Learning for Few-Shot Remote Sensing Scene Classification

Self-supervised learning is an effective way to solve model collapse for few-shot remote sensing scene classification (FSRSSC). However, most self-supervised contrastive learning auxiliary tasks perform poorly on the high interclass similarity problem in FSRSSC. Furthermore, it is time-consuming and computationally expensive to obtain the best combination among numerous self-supervised auxiliary tasks. In practical applications, we may encounter difficulties in remote sensing data acquisition and labeling, while most FSRSSC studies only focus on the former. To alleviate the above problems, we propose a multiform ensemble self-supervised learning (MES2L) framework for FSRSSC in this article. Based on the transfer learning-based few-shot scheme, we design a novel global–local contrastive learning auxiliary task to solve the low interclass separability problem. The self-attention mechanism is designed in the local contrast features to investigate the intrinsic associations between different remote sensing scene objectives. We also present a multiform ensemble enhancement (MEE) training method. Ensemble enhancement involves the concatenation of features extracted from different backbones trained by a combination of multiform self-supervised auxiliary tasks. MEE can not only be regarded as a more straightforward alternative to knowledge distillation but also can achieve an effective compromise between expensive computational cost and classification accuracy. In addition, we provide two scene classification schemes of inductive and transductive settings, corresponding to solving the difficulties of remote sensing data acquisition and labeling. The proposed network achieves state-of-the-art results on three benchmark FSRSSC datasets. The potential of the MES2L framework is also demonstrated in combination with classical metalearning-based and metric learning-based few-shot algorithms.

MallesNet: A multi-object assistance based network for brachial plexus segmentation in ultrasound images.

Ultrasound-guided injection is widely used to help anesthesiologists perform anesthesia in peripheral nerve blockade (PNB). However, it is a daunting task to accurately identify nerve structure in ultrasound images even for the experienced anesthesiologists. In this paper, a Multi-object assistance based Brachial Plexus Segmentation Network, named MallesNet, is proposed to improve the nerve segmentation performance in ultrasound image with the assistance of simultaneously segmenting its surrounding anatomical structures (e.g., muscle, vein, and artery). The MallesNet is designed by following the framework of Mask R-CNN to implement the multi object identification and segmentation. Moreover, a spatial local contrast feature (SLCF) extraction module is proposed to compute contrast features at different scales to effectively obtain useful features for small objects. And the self-attention gate (SAG) is also utilized to capture the spatial relationships in different channels and further re-weight the channels in feature maps by following the design of non-local operation and channel attention. Furthermore, the upsampling mechanism in original Feature Pyramid Network (FPN) is improved by adopting the transpose convolution and skip concatenation to fine-tune the feature maps. The Ultrasound Brachial Plexus Dataset (UBPD) is also proposed to support the research on brachial plexus segmentation, which consists of 1055 ultrasound images with four objects (i.e., nerve, artery, vein and muscle) and their corresponding label masks. Extensive experimental results using UBPD dataset demonstrate that MallesNet can achieve a better segmentation performance on nerves structure and also on surrounding structures in comparison to other competing approaches.

Infrared small target detection based on multiscale local contrast learning networks

Recently, model-driven deep networks have achieved excellent detection performance on infrared small targets in cluttered environments. However, its detection performance is sensitive to the hyperparameters in the embedded model-driven module. Therefore, we propose a novel multiscale local contrast learning network (MLCL-Net), which is an end-to-end fully convolutional infrared small target detection network. By constructing a local contrast learning (LCL) structure, it can learn to generate local contrast feature maps during training. Considering the difference in target size, we further build a multiscale local contrast learning (MLCL) module based on LCL. By extracting and fusing local contrast information of different scales from feature maps of the same level, the feature information of targets is fully excavated. At the same time, due to the small size of the target, a slight pixel shift will cause a severe loss of accuracy. We propose a bilinear feature pyramid network (BFPN) based on the feature pyramid network (FPN). Compared to state-of-the-art methods, the proposed MLCL-Net achieves superior performance with an intersection-over-union (IoU) of 0.772 and normalized IoU (nIoU) of 0.755 on the public SIRST dataset.

A Ratio-Difference Local Feature Contrast Method for Infrared Small Target Detection

Local contrast has been proven to be effective in infrared (IR) small target detection, but existing algorithms still have some drawbacks. In this letter, a ratio-difference (RD) local feature contrast (LFC) method is proposed. First, a new nested window is designed, and the local gray contrast is selected as the features of inner and outer windows to investigate the characteristics of a local area. Then, the RD LFC is calculated. At last, a simple but useful weighting function is proposed. Experiments show that the proposed algorithm can achieve better performance in terms of detection rate and false alarm rate and has good robustness against noise.

Infrared dim target detection method based on local feature contrast and energy concentration degree

Infrared dim target detection under complex backgrounds is of great importance in many applications. However, it is difficult for the conventional detection method to enhance dim targets and suppress clutter effectively and simultaneously, which leads to a low detection rate, poor background adaptability, and many false alarms. To address this issue, a simple method based on local feature contrast and energy concentration degree (LFC-ECD) is proposed in this paper to detect infrared dim targets under backgrounds with clutters. The LFC-ECD consists of the LFC model and ECD model; the former suppresses most background and effectively enhances the target, the latter suppresses residual clutter having similar features to the real target. Eventually, the true target is extracted from the background by adaptive threshold segmentation. The experiment is conducted on six infrared sequences that have different backgrounds and a minimum average signal-to-clutter rate (SCR) of 0.9299. The experimental results indicate that, compared with the other 7 methods, LFC-ECD not only achieves background suppression and target enhancement with higher values of the background suppression factor (BSF) and gain of SCR (GSCR) but also has a faster processing speed. In conclusion, LFC-ECD improves the effectiveness of IR dim target detection under complex background.

Noise Resilient Local Gradient Orientation for Content-Based Image Retrieval

In a world of multimedia information, where users seek accurate results against search query and demand relevant multimedia content retrieval, developing an accurate content-based image retrieval (CBIR) system is difficult due to the presence of noise in the image. The performance of the CBIR system is impaired by this noise. To estimate the distance between the query and database images, CBIR systems use image feature representation. The noise or artifacts present within the visual data might confuse the CBIR when retrieving relevant results. Therefore, we propose Noise Resilient Local Gradient Orientation (NRLGO) feature representation that overcomes the noise factor within the visual information and strengthens the CBIR to retrieve accurate and relevant results. The proposed NRLGO consists of three steps: estimation and removal of noise to protect the local visual structure; extraction of color, texture, and local contrast features; and, at the end, generation of microstructure for visual representation. The Manhattan distance between the query image and the database image is used to measure their similarity. The proposed technique was tested using the Corel dataset, which contains 10000 images from 100 different categories. The outcomes of the experiment signify that the proposed NRLGO has higher retrieval performance in comparison with state-of-the-art techniques.

Perceptual visual security assessment by fusing local and global feature similarity

Selective encryption is an effective technique for multimedia big data encryption. A perceptual metric that takes the human visual system (HVS) into account is necessary for evaluating the visual security degrees of selectively encrypted images. Existing metrics are mainly based on the similarity of structural information which is represented by local spatial contrast features. However, visual security is concerned with not only the distortion of structural information but also the leakage of important visual information. The local features cannot exactly express the leakage of important visual information. This paper presents a perceptual visual security assessment metric by fusing local and global feature similarity. Considering the HVS response, the proposed metric measures in three aspects: the distortion of structural information, the leakage of important visual information, and the changes of frequency components. To measure the distortion of structure information, local pattern similarity is calculated based on the normalized Hamming distance between the local binary pattern (LBP) binary codes. The similarity of the global LBP histogram is computed to evaluate the leakage of important visual information. A lowpass weighted discrete cosine transform (DCT) frequency similarity is presented to detect the changes of various frequency components. Experimental results demonstrate that the proposed metric achieves significantly higher performance and stronger robustness than the state-of-the-art metrics.

Infrared Small Target Detection Using Homogeneity-Weighted Local Contrast Measure

Detecting small targets in infrared (IR) image sequences is an important task in IR guidance systems. The clutter of complex backgrounds often submerges small targets, making detection difficult. Achieving high detection and low false alarm rates with complex backgrounds is a primary problem. We propose an IR small target detection method using our new homogeneity-weighted local contrast measure (HWLCM). Inspired by the ability of the human visual system (HVS) to determine saliency characteristics, we implement our method to use the local contrast features of the central and surrounding regions and the weighted homogeneity characteristics of the surrounding regions to enhance the target while suppressing the complex background. Our method divides each image into blocks with a sliding window for which the HWLCM is calculated. The HWLCM enhances the actual target and suppresses interference simultaneously. We apply an adaptive threshold to target region extraction to further refine the results. Our experimental results show that our proposed method is more effective than six comparable methods, especially in terms of the signal-to-clutter gain (SCRG) and background suppression factor (BSF) indicators.

Ghost-free multi exposure image fusion technique using dense SIFT descriptor and guided filter

A ghost-free multi-exposure image fusion technique using the dense SIFT descriptor and the guided filter is proposed in this paper. The results suggest that the presented scheme produces high-quality images using ordinary cameras and that too without the ghosting artifact. To do so, the dense SIFT descriptor is used to extract the local contrast information from source images. Whereas, for the dynamic scenes, the histogram equalization and median filtering are used to calculate the color dissimilarity feature. Three weighting terms: local contrast, brightness, and color dissimilarity feature are used to estimate the initial weights. The estimated initial weights contain discontinuities. Therefore, the guided filter is used to remove the noise and discontinuity in initial weights. Finally, the fusion is performed using a pyramid decomposition method. Experimental results prove the superiority of the proposed technique over existing state-of-the-art methods in terms of both subjective and objective evaluation.

Infrared Small Target Detection via Modified Random Walks

Infrared small target detection under intricate background and heavy noise is one of the crucial tasks in the field of remote sensing. Conventional algorithms can fail in detecting small targets due to the low signal-to-noise ratios of the images. To solve this problem, an effective infrared small target detection algorithm inspired by random walks is presented in this paper. The novelty of our contribution involves the combination of the local contrast feature and the global uniqueness of the small targets. Firstly, the original pixel-wise image is transformed into an multi-dimensional image with respect to the local contrast measure. Secondly, a reconstructed seeds selection map (SSM) is generated based on the multi-dimensional image. Then, an adaptive seeds selection method is proposed to automatically select the foreground seeds potentially placed in the areas of the small targets in the SSM. After that, a confidence map is constructed using a modified random walks (MRW) algorithm to represent the global uniqueness of the small targets. Finally, we segment the targets from the confidence map by utilizing an adaptive threshold. Extensive experimental evaluation results on a real test dataset demonstrate that our algorithm is superior to the state-of-the-art algorithms in both target enhancement and detection performance.