Saliency Features Research Articles

Classification and Identification of Frequency-Hopping Signals Based on Jacobi Salient Map for Adversarial Sample Attack Approach.

Frequency-hopping (FH) communication adversarial research is a key area in modern electronic countermeasures. To address the challenge posed by interfering parties that use deep neural networks (DNNs) to classify and identify multiple intercepted FH signals-enabling targeted interference and degrading communication performance-this paper presents a batch feature point targetless adversarial sample generation method based on the Jacobi saliency map (BPNT-JSMA). This method builds on the traditional JSMA to generate feature saliency maps, selects the top 8% of salient feature points in batches for perturbation, and increases the perturbation limit to restrict the extreme values of single-point perturbations. Experimental results in a white-box environment show that, compared with the traditional JSMA method, BPNT-JSMA not only maintains a high attack success rate but also enhances attack efficiency and improves the stealthiness of the adversarial samples.

Semantic feature salience matters: new values of significance and relative weight for the Spanish feature norms for young and older adults

Abstract To determine the semantic importance of certain features has been a topic of interest for many semantic memory models. Diverse measures have been proposed in the last years. Semantic significance, the latest measure proposed by Montefinese and colleagues, is both sophisticated and comprehensive. Given the cultural and linguistic variability of semantic measures, this study presents values of significance and relative weight for the Spanish-speaking population corresponding to 400 concrete concepts. First, we presented data for both young and older adults. Second, we assessed the effect of significance on response times in two speeded feature verification tasks. Third, we compared significance values with the existing Italian significance norms. To evaluate the effect of significance to predict response times, two speeded verification tasks (Experiment 1 and 2) were carried out, selecting a total of a 130 concepts for analysis. In Experiment 1, subjects were presented with a feature followed by a concept, while in Experiment 2, the order of stimulus presentation was reversed (i.e., the concept was presented before the feature). An independent linear mixed model showed that significance was a good predictor of response latencies in Experiments 1 and 2. Moreover, results revealed a strong positive correlation between the Spanish and Italian significance values. Findings are discussed in terms of recent theories of semantic cognition.

Personalized Model-Driven Interventions for Decisions From Experience.

Cognitive models that represent individuals provide many benefits for understanding the full range of human behavior. One way in which individual differences emerge is through differences in knowledge. In dynamic situations, where decisions are made from experience, models built upon a theory of experiential choice (instance-based learning theory; IBLT) can provide accurate predictions of individual human learning and adaptivity to changing environments. Here, we demonstrate how an instance-based learning (IBL) cognitive model, implemented in a cognitive architecture (Adaptive Control of Thought-Rational), can be used to model an individual's decisions in a cybersecurity defense task, accounting for both population average and individual variances. The same IBL model structure with identical architectural parameters generates the full range of human behavior through stochastic memory retrieval processes operating over and contributing to unique experiences. Recurrence quantification analyses allow us to look beyond average behavior between and within individuals to sequential patterns of trial-to-trial behavior. We show how model-tracing and knowledge-tracing techniques can be used to align the model to an individual in real time to drive adaptive and personalized signaling algorithms for a cybersecurity defense system. We also present a method for introspecting into the cognitive model to gain further insight into the cognitive salience of features factored into individual decisions. The combination of techniques provides a blueprint for personalized modeling of individuals. We discuss the results and implications of this adaptive and personalized method for cybersecurity defense and more generally for intelligent artifacts tailored to individual differences in domains such as human-machine teaming.

Significant feature suppression and cross-feature fusion networks for fine-grained visual classification

The technique of extracting different distinguishing features by locating different part regions to achieve fine-grained visual classification (FGVC) has made significant improvements. Utilizing attention mechanisms for feature extraction has become one of the mainstream methods in computer vision, but these methods have certain limitations. They typically focus on the most discriminative regions and directly combine the features of these parts, neglecting other less prominent yet still discriminative regions. Additionally, these methods may not fully explore the intrinsic connections between higher-order and lower-order features to optimize model classification performance. By considering the potential relationships between different higher-order feature representations in the object image, we can enable the integrated higher-order features to contribute more significantly to the model’s classification decision-making capabilities. To this end, we propose a saliency feature suppression and cross-feature fusion network model (SFSCF-Net) to explore the interaction learning between different higher-order feature representations. These include (1) an object-level image generator (OIG): the intersection of the output feature maps of the last two convolutional blocks of the backbone network is used as an object mask and mapped to the original image for cropping to obtain an object-level image, which can effectively reduce the interference caused by complex backgrounds. (2) A saliency feature suppression module (SFSM): the most distinguishing part of the object image is obtained by a feature extractor, and the part is masked by a two-dimensional suppression method, which improves the accuracy of feature suppression. (3) A cross-feature fusion method (CFM) based on inter-layer interaction: the output feature maps of different network layers are interactively integrated to obtain high-dimensional features, and then the high-dimensional features are channel compressed to obtain the inter-layer interaction feature representation, which enriches the output feature semantic information. The proposed SFSCF-Net can be trained end-to-end and achieves state-of-the-art or competitive results on four FGVC benchmark datasets.

AI driven interpretable deep learning based fetal health classification

In this study, a deep learning model is proposed for the classification of fetal health into 3 categories: Normal, suspect, and pathological. The primary objective is to utilize the power of deep learning to improve the efficiency and effectiveness of diagnostic processes. A deep neural network (DNN) model is proposed for fetal health analysis using data obtained from Cardiotocography (CTG). A dataset containing 21 attributes is used to carry out this work. The model incorporates multiple hidden layers, augmented with batch normalization and dropout layers for improved generalization. This study assesses the model's interpretation ability in fetal health classification using explainable deep learning. This enhances transparency in decision-making of the classifier model by leveraging feature importance and feature saliency analysis, fostering trust and facilitating the clinical adoption of fetal health assessments. Our proposed model demonstrates a remarkable performance with 0.99 accuracy, 0.93 sensitivity, 0.93 specificity, 0.96 AUC, 0.93 precision, and 0.93 F1 scores in classifying fetal health. We also performed comparative analysis with six other models including Logistic Regression, KNN, SVM, Naive Bayes, Random Forest, and Gradient Boosting to assess and compare the effectiveness of our model and the accuracies of 0.89, 0.88, 0.90, 081, 0.93, and 0.93 were achieved respectively by these baseline models. The results revealed that our proposed model outperformed all the baseline models in terms of accuracy. This indicates the potential of deep learning in improving fetal health assessment and contributing to the field of obstetrics by providing a robust tool for early risk detection.

High-Precision Heterogeneous Satellite Image Manipulation Localization: Feature Point Rules and Semantic Similarity Measurement

Misusing image tampering software makes it easier to manipulate satellite images, leading to a crisis of trust and security concerns in society. This study compares the inconsistencies between heterogeneous images to locate tampered areas and proposes a high-precision heterogeneous satellite image manipulation localization (HSIML) framework to distinguish tampered from real landcover changes, such as artificial constructions, and pseudo-changes, such as seasonal variations. The model operates at the patch level and comprises three modules: The heterogeneous image preprocessing module aligns heterogeneous images and filters noisy data. The feature point constraint module mitigates the effects of lighting and seasonal variations in the images by performing feature point matching, applying filtering rules to conduct an initial screening to identify candidate tampered patches. The semantic similarity measurement module designs a classification network to assess RS image feature saliency. It determines image consistency based on the similarity of semantic features and implements IML using predefined classification rules. Additionally, a dataset for IML is constructed based on satellite images. Extensive experiments compared with existing SOTA models demonstrate that our method achieved the highest F1 score in both localization accuracy and robustness tests and demonstrates the capability for handling large-scale areas.

Autoencoder-based image fusion network with enhanced channels and feature saliency

The existing deep learning based infrared and visible image fusion technologies have made significant progress, but there are still many problems need to be solved, such as information loss (targets and texture, etc.) of both infrared and visible images, noise and artifacts existing in fused image. To address these issues in fusion, an infrared and visible image fusion method based on autoencoder network is proposed in this paper. Firstly, novel enhanced channels are designed and input parallelly with source images into the network to enhance the specific features and reduce information loss in feature fusion. Then, the feature maps are obtained by the encoder. Next, a feature fusion method based on feature saliency is proposed, using a pre-trained classifier to measure the saliency of features, and the fused image is obtained by the decoder finally. Experimental results demonstrate that the targets are obvious and the textures are plentiful in the fused images generated by the proposed method. Also, the objective metrics of the proposed method are higher than the state of the art methods, which demonstrate that the proposed method is effective to fuse the infrared and visible images.

Role of inhibition in overcoming interferences of misconception under similar feature saliency: An eye-tracking study of the projectile motion problem

Misconceptions coexisting with scientific understanding pose significant challenges in physics education. Inhibitory control may enable individuals to overcome interference from misconceptions. However, discerning the role of inhibitory control becomes intricate when the saliency of scientific- and misconception-related features varies in a problem. This study investigates the role of inhibitory control in overcoming such misconceptions when scientific and misconception-related features are similarly salient, particularly focusing on the range-time (range determines flight time) misconception. We adopted a negative priming (NP) paradigm with the eye-tracking technique in the study. Thirty-six college physics majors participated in the experiment. In the NP paradigm, the participants first compared the flight times in a neutral (same range) or incongruent (longer range, less flight time) primes and then completed the same task in a congruent (longer range, more flight time) probe. Results revealed longer reaction times for incongruent primes compared to neutral primes and longer reaction times to respond to congruent probes following incongruent primes than when neutral primes were present beforehand, suggesting that the inhibitory control may be involved in overcoming the interference from the range-time misconception. Moreover, the eye-tracking analysis highlighted an early overt attention to the salient misconception-related feature (longer range) during primes, along with additional attentional investment necessary to inhibit the range-time misconception. During probes, an early shift of attention away from the salient misconception-related feature and a greater investment of attention on processing the congruent probes after the incongruent primes than that after the neutral primes were observed, indicating a possible effect of the previous inhibitory control in the primes on the subsequent cognitive process on the probes. These findings contribute to a deeper understanding of the cognitive process underlying physics conceptual change and provide insights into physics education. Published by the American Physical Society 2024

Cooperative thalamocortical circuit mechanism for sensory prediction errors

The brain functions as a prediction machine, utilizing an internal model of the world to anticipate sensations and the outcomes of our actions. Discrepancies between expected and actual events, referred to as prediction errors, are leveraged to update the internal model and guide our attention towards unexpected events1–10. Despite the importance of prediction-error signals for various neural computations across the brain, surprisingly little is known about the neural circuit mechanisms responsible for their implementation. Here we describe a thalamocortical disinhibitory circuit that is required for generating sensory prediction-error signals in mouse primary visual cortex (V1). We show that violating animals’ predictions by an unexpected visual stimulus preferentially boosts responses of the layer 2/3 V1 neurons that are most selective for that stimulus. Prediction errors specifically amplify the unexpected visual input, rather than representing non-specific surprise or difference signals about how the visual input deviates from the animal’s predictions. This selective amplification is implemented by a cooperative mechanism requiring thalamic input from the pulvinar and cortical vasoactive-intestinal-peptide-expressing (VIP) inhibitory interneurons. In response to prediction errors, VIP neurons inhibit a specific subpopulation of somatostatin-expressing inhibitory interneurons that gate excitatory pulvinar input to V1, resulting in specific pulvinar-driven response amplification of the most stimulus-selective neurons in V1. Therefore, the brain prioritizes unpredicted sensory information by selectively increasing the salience of unpredicted sensory features through the synergistic interaction of thalamic input and neocortical disinhibitory circuits.

Depth Matters: Spatial Proximity-based Gaze Cone Generation for Gaze Following in Wild

Gaze following aims to predict where a person is looking in a scene. Existing methods tend to prioritize traditional 2D RGB visual cues or require burdensome prior knowledge and extra expensive datasets annotated in 3D coordinate systems to train specialized modules to enhance scene modeling. In this work, we introduce a novel framework deployed on a simple ResNet backbone, which exclusively uses image and depth maps to mimic human visual preferences and realize 3D-like depth perception. We first leverage depth maps to formulate spatial-based proximity information regarding the objects with the target person. This process sharpens the focus of the gaze cone on the specific region of interest pertaining to the target while diminishing the impact of surrounding distractions. To capture the diverse dependence of scene context on the saliency gaze cone, we then introduce a learnable grid-level regularized attention that anticipates coarse-grained regions of interest, thereby refining the mapping of the saliency feature to pixel-level heatmaps. This allows our model to better account for individual differences when predicting others’ gaze locations. Finally, we employ the KL-divergence loss to super the grid-level regularized attention, which combines the gaze direction, heatmap regression, and in/out classification losses, providing comprehensive supervision for model optimization. Experimental results on two publicly available datasets demonstrate the comparable performance of our model with less help of modal information. Quantitative visualization results further validate the interpretability of our method. The source code will be available at https://github.com/VUT-HFUT/DepthMatters .

Nighttime wildlife object detection based on YOLOv8‐night

AbstractMonitoring nocturnal animals in the field is an important task in ecological research and wildlife conservation, but the complexity of nocturnal images and low light conditions make it difficult to cope with traditional image processing methods. To address this problem, researchers have introduced infrared cameras to improve the accuracy of nocturnal animal behaviour observations. Object detection in nighttime images captured by infrared cameras faces several challenges, including low image quality, animal scale variations, occlusion, and pose changes. This study proposes the YOLOv8‐night model, which effectively overcomes these challenges by introducing a channel attention mechanism in YOLOv8. The model is more focused on capturing animal‐related features by dynamically adjusting the channel weights, which improves the saliency of key features and increases the accuracy rate in complex backgrounds. The main contribution of this study is the introduction of the channel attention mechanism into the YOLOv8 framework to create a YOLOv8‐night model suitable for object detection in nighttime images. When tested on nighttime images, the model performs well with a significantly higher mAP (0.854) than YOLOv8 (0.831), and YOLOv8‐night scores 0.856 on mAP_l, which is obviously better than YOLOv8 (0.833) in terms of processing large objects. The study provides a reliable technical tool for ecological research, wildlife conservation and environmental monitoring, and offers new methods and insights for the study of nocturnal animal behaviour.

Task-Related Saliency for Few-Shot Image Classification.

A weakness of the existing metric-based few-shot classification method is that task-unrelated objects or backgrounds may mislead the model since the small number of samples in the support set is insufficient to reveal the task-related targets. An essential cue of human wisdom in the few-shot classification task is that they can recognize the task-related targets by a glimpse of support images without being distracted by task-unrelated things. Thus, we propose to explicitly learn task-related saliency features and make use of them in the metric-based few-shot learning schema. We divide the tackling of the task into three phases, namely, the modeling, the analyzing, and the matching. In the modeling phase, we introduce a saliency sensitive module (SSM), which is an inexact supervision task jointly trained with a standard multiclass classification task. SSM not only enhances the fine-grained representation of feature embedding but also can locate the task-related saliency features. Meanwhile, we propose a self-training-based task-related saliency network (TRSN) which is a lightweight network to distill task-related salience produced by SSM. In the analyzing phase, we freeze TRSN and use it to handle novel tasks. TRSN extracts task-relevant features while suppressing the disturbing task-unrelated features. We, therefore, can discriminate samples accurately in the matching phase by strengthening the task-related features. We conduct extensive experiments on five-way 1-shot and 5-shot settings to evaluate the proposed method. Results show that our method achieves a consistent performance gain on benchmarks and achieves the state-of-the-art.

Contextual Enhancement–Interaction and Multi-Scale Weighted Fusion Network for Aerial Tracking

Siamese-based trackers have been widely utilized in UAV visual tracking due to their outstanding performance. However, UAV visual tracking encounters numerous challenges, such as similar targets, scale variations, and background clutter. Existing Siamese trackers face two significant issues: firstly, they rely on single-branch features, limiting their ability to achieve long-term and accurate aerial tracking. Secondly, current tracking algorithms treat multi-level similarity responses equally, making it difficult to ensure tracking accuracy in complex airborne environments. To tackle these challenges, we propose a novel UAV tracking Siamese network named the contextual enhancement–interaction and multi-scale weighted fusion network, which is designed to improve aerial tracking performance. Firstly, we designed a contextual enhancement–interaction module to improve feature representation. This module effectively facilitates the interaction between the template and search branches and strengthens the features of each branch in parallel. Specifically, a cross-attention mechanism within the module integrates the branch information effectively. The parallel Transformer-based enhancement structure improves the feature saliency significantly. Additionally, we designed an efficient multi-scale weighted fusion module that adaptively weights the correlation response maps across different feature scales. This module fully utilizes the global similarity response between the template and the search area, enhancing feature distinctiveness and improving tracking results. We conducted experiments using several state-of-the-art trackers on aerial tracking benchmarks, including DTB70, UAV123, UAV20L, and UAV123@10fps, to validate the efficacy of the proposed network. The experimental results demonstrate that our tracker performs effectively in complex aerial tracking scenarios and competes well with state-of-the-art trackers.

Rethinking Feature Mining for Light Field Salient Object Detection

Light field salient object detection (LF SOD) has recently received increasing attention. However, most current works typically rely on an individual focal stack backbone for feature extraction. This manner ignores the characteristic of blurred saliency-related regions and contour within focal slices, resulting in insufficient or even inaccurate saliency responses. Aiming at addressing this issue, we rethink the feature mining (i.e., exploration) within focal slices, and focus on exploiting informative focal slice features and fully leveraging contour information for accurate LF SOD. First, we observe that the geometric relation between different regions within the focal slices is conducive to useful saliency feature mining if utilized properly. In light of this, we propose an implicit graph learning (IGL) approach. The IGL constructs graph structures to propagate informative geometric relations within the focal slices and all-focus features, and promotes crucial and discriminative focal stack feature mining via graph feature distillation. Second, unlike previous works that rarely utilize contour information, we propose a reciprocal refinement fusion (RRF) strategy. This strategy encourages saliency features and object contour cues to effectively complement each other. Furthermore, a contour hint injection mechanism is introduced to refine the feature expressions. Extensive experiments showcase the superiority of our approach over previous state-of-the-art models with an efficient real-time inference speed. Codes are available at: https://github.com/gbliao/IRNet .

TranSalNet+: Distortion-aware saliency prediction

Predicting the saliency of images affected by distortion is a challenging but emerging research problem. Given a distorted image, we wish to accurately predict saliency as perceived by humans. A recent distortion-aware saliency benchmark – the CUDAS database – reveals the inadequacy of existing saliency models in handling distorted images. In this paper, we devise a deep learning Distortion-Aware Saliency Module (DASM) that enables capturing saliency features related to image distortions, and integrates this module into a saliency prediction architecture. To achieve the high expressive capability of DASM using supervised learning, we create a dedicated dataset that draws upon a large-scale saliency dataset and machine-generated image quality assessments. Experimental results demonstrate the superior performance of the proposed model in predicting the saliency of distorted images.

Flexible Conceptual Representations

AbstractA view that has been gaining prevalence over the past decade is that the human conceptual system is malleable, dynamic, context‐dependent, and task‐dependent, that is, flexible. Within the flexible conceptual representation framework, conceptual representations are constructed ad hoc, forming a different, idiosyncratic instantiation upon each occurrence. In this review, we scrutinize the neurocognitive literature to better understand the nature of this flexibility. First, we identify some key characteristics of these representations. Next, we consider how these flexible representations are constructed by addressing some of the open questions in this framework: We review the age‐old question of how to reconcile flexibility with the apparent need for shareable stable definitions to anchor meaning and come to mutual understanding, as well as some newer questions we find critical, namely, the nature of relations among flexible representations, the role of feature saliency in activation, and the viability of all‐or‐none feature activations. We suggest replacing the debate about the existence of a definitional stable core that is obligatorily activated with a question of the degree and probability of activation of the information constituting a conceptual representation. We rely on published works to suggest that (1) prior featural salience matters, (2) feature activation may be graded, and (3) Bayesian updating of prior information according to current demands offers a viable account of how flexible representations are constructed. This proposal provides a theoretical mechanism for incorporating a changing momentary context into a constructed representation, while still preserving some of the concept's constituent meaning.

MFUR-Net: Multimodal feature fusion and unimodal feature refinement for RGB-D salient object detection

RGB-D salient object detection aims to integrate multimodal feature information for accurate salient region localization. Despite the development of several RGB-D salient object detection models, existing methods face challenges in effectively fusing RGB with Depth features to exploit their complementary aspects. To address this challenge, this study introduces MFUR-Net, a network based on multimodal feature fusion and unimodal feature refinement. The contributions of this study are primarily threefold: First, a multimodal multilevel feature fusion module is proposed at the encoder stage to integrate multimodal and multilevel features, generating enhanced RGB-D features; Second, a multi-input feature aggregation module at the decoder stage is introduced, which incorporates the RGB and Depth feature streams into the RGB-D feature streams so that they collaborate with the RGB-D features to learn more discriminative information related to the salient object; Third, a unimodal saliency feature refinement module is introduced to refine saliency feature information across modalities and eliminate redundancy before the integration of feature streams into the decoder; With the gradual refinement of saliency features, MFUR-Net achieves accurate saliency map prediction at the decoder stage. This method has been validated through extensive experiments on seven recognized datasets, demonstrating significant advantages over existing state-of-the-art techniques in key performance metrics. The source code can be found in https://github.com/wangwei678/MFUR-Net.

Crater Triangle Matching Algorithm Based on Fused Geometric and Regional Features

Craters are regarded as significant navigation landmarks during the descent and landing process in small body exploration missions for their universality. Recognizing and matching craters is a crucial prerequisite for visual and LIDAR-based navigation tasks. Compared to traditional algorithms, deep learning-based crater detection algorithms can achieve a higher recognition rate. However, matching crater detection results under various image transformations still poses challenges. To address the problem, a composite feature-matching algorithm that combines geometric descriptors and region descriptors (extracting normalized region pixel gradient features as feature vectors) is proposed. First, the geometric configuration map is constructed based on the crater detection results. Then, geometric descriptors and region descriptors are established within each feature primitive of the map. Subsequently, taking the salience of geometric features into consideration, composite feature descriptors with scale, rotation, and illumination invariance are generated through fusion geometric and region descriptors. Finally, descriptor matching is accomplished by computing the relative distances between descriptors and adhering to the nearest neighbor principle. Experimental results show that the composite feature descriptor proposed in this paper has better matching performance than only using shape descriptors or region descriptors, and can achieve a more than 90% correct matching rate, which can provide technical support for the small body visual navigation task.

PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor Data

Cybersickness, a factor that hinders user immersion in VR, has been the subject of ongoing attempts to predict it using AI. Previous studies have used CNN and LSTM for prediction models and used attention mechanisms and XAI for data analysis, yet none explored a transformer that can better reflect the spatial and temporal characteristics of the data, beneficial for enhancing prediction and feature importance analysis. In this paper, we propose cybersickness prediction models using multimodal time-series sensor data (i.e., eye movement, head movement, and physiological signals) based on a transformer algorithm, considering sensor data pre-processing and multimodal data fusion methods. We constructed the MSCVR dataset consisting of normalized sensor data, spectrogram formatted sensor data, and cybersickness levels collected from 45 participants through a user study. We proposed two methods for embedding multimodal time-series sensor data into the transformer: modality-specific spatial and temporal transformer encoders for normalized sensor data (MS-STTN) and modality-specific spatial-temporal transformer encoder for spectrogram (MS-STTS). MS-STTN yielded the highest performance in the ablation study and the comparison of the existing models. Furthermore, by analyzing the importance of data features, we determined their relevance to cybersickness over time, especially the salience of eye movement features. Our results and insights derived from multimodal time-series sensor data and the transformer model provide a comprehensive understanding of cybersickness and its association with sensor data. Our MSCVR dataset and code are publicly available: https://github.com/dayoung-jeong/PRECYSE.git.

Simultaneous count data feature selection and clustering using Multinomial Nested Dirichlet Mixture

The elevating effect of the curse of dimensionality in count data has made clustering a challenging task. This paper solves this by adopting the concept of feature saliency as a feature selection method in the context of using the Multinomial Nested Dirichlet Mixture (MNDM). The MNDM is a generalization of the Dirichlet Compound Mixture (DCM) that suffers from several limitations. The model learning is accomplished through the expectation-maximization method. The Minimum Message Length criterion is used to simultaneously determine the best number of components in the mixture with the updated selected features. At the price of convergence times, the results show better performance through different metrics, as the model aims to select the salient features and tune away the non-salient anomalistic features.