Saliency Model Research Articles

Development and validation of BVB-UIE: an underwater image-enhancement model with two-layer deep neural network

Due to the absorption and scattering of light in water, underwater visual visibility is poor, and underwater images usually suffer from color distortion and feature loss, which poses a great challenge for underwater vision tasks. To handle these issues, we present an underwater image enhancement (UIE) method. A Gaussian pyramid is constructed for the degraded underwater images based on an improved visual saliency model, with the characteristics of luminance, orientation, texture, and color. By combining channel and spatial attention mechanisms, a deep asymmetric feature enhancement network is built to preserve the high-dimensional features of the image. In addition, a polynomial loss function is used for the depth hopping supervising constraints during the enhancement process, and the gating signals are employed to control the importance of the features at different spatial locations. These innovations maximize the efficiency of the feature extraction while simplifying the network complexity. Experiments on an open benchmark dataset for UIE show that our method effectively eliminates the color bias and contrast distortion problems while preserving the complex image details compared to the existing UIE algorithms. Objective metrics show a significant improvement in the algorithm, with a 15% increment in the peak signal-to-noise ratio metric compared to the closest competing algorithm.

Ecolabels and the economic recession.

We examine the effect of the 2008 economic recession on consumers' observed expenditures for eco-labelled grocery products. Traditional price theory predicts that consumers change their spending during an economic downturn and we would expect the sales share of eco-labelled products to fall since these are relatively more expensive than non-labelled products. We use supermarket loyalty card data from the UK and show that the recession had widely different effects on the expenditure share of different eco-labelled grocery products. We confirm, empirically, that expenditure shares on organic products declined over the time period under study but the expenditures share for fair-trade products increased over the same period. We evaluate alternative models of decision making to explain our results, viz., a salience model and a model of reputation signalling. We find that both of these models give a plausible explanation of our empirical results.

Transformer-Based Multi-Scale Feature Integration Network for Video Saliency Prediction

Most cutting-edge video saliency prediction models rely on spatiotemporal features extracted by 3D convolutions due to its local contextual cues acquirement ability. However, the shortage of 3D convolutions is that it cannot effectively capture long-term spatiotemporal dependencies in videos. To address this limitation, we propose a novel Transformer-based Multi-scale Feature Integration Network (TMFI-Net) for video saliency prediction, where the proposed TMFI-Net consists of a semantic-guided encoder and a hierarchical decoder. Firstly, embarking on the Transformer-based multi-level spatiotemporal features, the semantic-guided encoder enhances the features by inserting the high-level feature into each level feature via a top-down pathway and a longitudinal connection, which endows the multi-level spatiotemporal features with rich contextual information. In this way, the features are steered to give more concerns to saliency regions. Secondly, the hierarchical decoder employs a multi-dimensional attention (MA) module to elevate features along channel, temporal, and spatial dimensions jointly. Successively, the hierarchical decoder deploys a progressive decoding block to conduct an initial saliency prediction, which provides a coarse localization of saliency regions. Lastly, considering the complementarity of different saliency predictions, we integrate all initial saliency prediction results into the final saliency map. Comprehensive experimental results on four video saliency datasets firmly demonstrate that our model achieves superior performance when compared with the state-of-the-art video saliency models. The code is available at https://github.com/wusonghe/TMFI-Net.

Saliency models perform best for women’s and young adults' fixations

Saliency models seek to predict fixation locations in (human) gaze behaviour. These are typically created to generalize across a wide range of visual scenes but validated using only a few participants. Generalizations across individuals are generally implied. We tested this implied generalization across people, not images, with gaze data of 1600 participants. Using a single, feature-rich image, we found shortcomings in the prediction of fixations across this diverse sample. Models performed optimally for women and participants aged 18-29. Furthermore, model predictions differed in performance from earlier to later fixations. Our findings show that gaze behavior towards low-level visual input varies across participants and reflects dynamic underlying processes. We conclude that modeling and understanding gaze behavior will require an approach which incorporates differences in gaze behavior across participants and fixations; validates generalizability; and has a critical eye to potential biases in training- and testing data.

Research on Texture Feature Recognition of Regional Architecture Based on Visual Saliency Model

Architecture is a representative of a city. It is also a spatial carrier of urban culture. Identifying the architectural features in a city can help with urban transformation and promote urban development. The use of visual saliency models in regional architectural texture recognition can effectively enhance the effectiveness of regional architectural texture recognition. In this paper, the improved visual saliency model first enhances the texture images of regional buildings through histogram enhancement technology, and uses visual saliency algorithms to extract the visual saliency of the texture features of regional buildings. Then, combined with the maximum interclass difference method of threshold segmentation, the visual saliency image is segmented to achieve accurate target recognition. Finally, the feature factor iteration of the Bag of Visual Words model and the function classification of support vector machines were used to complete the recognition of regional architectural texture features. Through experimental verification, the constructed regional architectural texture feature recognition method based on visual saliency model can effectively enhance the recognition image. This method performs well in boundary contour separation and visual saliency, with an average recognition rate of 0.814 for texture features in different building scenes, indicating high stability.

Stakeholder Salience Analysis in the Policy Formulation of Anti-Violence Against Women and Children

This article explores stakeholders influence in the policy formulation of anti-violence against women and children by using the stakeholder salience model as a conceptual framework. Stakeholder engagement in policy formulation is essential to achieving optimal policy practise and performance. However, the contestation of interests and imbalance of power between stakeholders in the policy formulation of anti-violence against women and children inhibit the development of policy alternatives based on problem solving, resulting in low policy performance. The research method used is qualitative research with a case study approach, and the stakeholder analysis method is applied to the data analysis. The research findings demonstrate that the policy formulation of anti-violence against women and children is dominated by dependent stakeholders with only power and legitimacy attributes, while the attributes of urgency are not available, reflected in their inattentiveness in providing a quick response to the rising cases of violence and their lack of knowledge on the issue of violence. This study concludes that stakeholders' influence in the policy process is limited. Stakeholders are only involved during public hearings, and the policy is only drafted and discussed by the policy elite, specifically Commission VI of the Aceh Legislative Council, an expert’s team, and the Aceh Women's Empowerment and Child Protection Unit. It indicates that stakeholder engagement is only to gain legitimacy for policy decisions. This research contributes to providing knowledge about stakeholder dynamics in policy formulation contexts and theoretical knowledge about the creation of the most powerful and influential stakeholders for maximising policy formulation and policy performance.

Identity salience: An intersectional approach to understanding multicultural processes and outcomes in psychotherapy.

A growing body of research has demonstrated the importance of therapists' multicultural orientation (MCO), namely, their cultural humility (CH), cultural comfort, and cultural missed opportunities, on treatment processes and outcomes (Davis et al., 2018). However, to date, few research has attempted to identify client factors that may moderate the relationship between therapists' MCO and therapeutic processes and outcomes. Informed by Yakushko et al.'s (2009) identity salience model, this study seeks to advance the MCO literature by examining the saliency of clients' cultural identities, therapists' MCO, and improvement in therapy. Data for this study consisted of 193 individuals who had received at least five sessions of psychotherapy in the last 6 months and responded to an online survey about their experience in therapy. Moderated polynomial regression and response surface analysis was used to examine if the relationship between therapists' MCO and clients' perceived improvement in psychotherapy differed as a function of the salience of clients' first and second most important cultural identities. The results indicated that when clients report only one highly salient cultural identity and perceive their therapist high in cultural humility, they report high levels of improvement. In contrast, when clients reported two highly salient identities, cultural humility and improvement in therapy were not significantly related. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Mechanical fault diagnosis of gas-insulated switchgear based on saliency feature of auditory brainstem response under noise background

The mechanical fault of gas-insulated switchgear (GIS) seriously threatens the security of the power grid. Recently, acoustic-based fault diagnosis methods, which have the advantage of non-contact measurement, have been applied to the GIS mechanical fault diagnosis, but vulnerable to the interference of the background noise. To improve the capacity of the acoustic-based GIS fault diagnosis under noise background, by simulating the sound feature extraction ability and anti-noise ability of human auditory system, a novel GIS mechanical fault diagnosis method based on saliency feature of auditory brainstem response (SFABR) is proposed. First, an auditory saliency model, which considers both the auditory periphery and the auditory nerve center was constructed by combining the deep auditory model and the saliency model. After processing GIS emitted acoustic signal, the auditory brainstem response (ABR) was obtained, and the saliency features of the ABR were extracted to obtain the SFABR. Then, the characteristic frequency distribution of the auditory saliency model was adjusted to make it more suitable for the spectral characteristics of the GIS sound signal. Finally, the SFABR was mapped to a two-dimensional CNN to train a model for GIS mechanical fault diagnosis. This method simulates the process of auditory response extraction and the selection effect of auditory attention on sound elements. The 110 kV three-phase GIS fault simulation experiment shows that for GIS mechanical faults, the diagnosis method based on SFABR can obtain 96.1% fault identification accuracy. In different noise environments, compared with the traditional acoustic-based fault diagnosis methods, this method has stronger anti-noise performance, and can more effectively realize the identification of GIS mechanical failure types. In future research, the method can be further extended to fault diagnosis of more types of power equipment.

Functional Conflicts in Tourist Coastal Resort Cities with Special Spa Status in Poland. The Stakeholder Approach

This study aims to identify and analyse the potential conflicts of tourism, resort and residents’ functions based on coastal resort cities of the highest importance in Poland (Sopot, Ustka, Kołobrzeg, Świnoujście). Coastal health resorts are specific urban units where tourist and resort functions provided under high environmental restrictions dominate the primary urban function typically focused on social and resident matters. Coastal resort cities attract tourists heavily. Hence, such cities must deal with varied stakeholder expectations. This may lead to conflicts in sustainable urban development.
 The analysis considers the stakeholder perspective regarding the occurrence of various conflicts. Different research methods have been used, including case studies and stakeholder analysis based on the salience model and classifying stakeholders into homogenous groups with the attributes assigned.
 The diverse and often divergent needs of stakeholders in coastal cities require a specific approach to developing such urban units, including stakeholder relationship management. A method used for the resort stakeholder analysis is based on classical stakeholder classification proposed by Mitchel, Agle and Wood (1997).
 The nature of functions performed in resort cities naturally lead to internal and developmental conflicts. However, based on an assessment of attributes assigned their evaluation across key stakeholder groups, the study results show that these conflicts occurred dormant in studied cities.
 The research methods used, an approach taken, and results achieved can be successfully used by decision-makers responsible for urban development to identify and manage potential conflicts deriving from diversified expectations of urban stakeholders or conflicting urban functions.
 The article is of an international character and might be used to various aspects of city management, as it combines stakeholder management approach with urban development and potential function conflicts.

Reward prediction-errors weighted by cue salience produces addictive behaviours in simulations, with asymmetrical learning and steeper delay discounting

Dysfunction in learning and motivational systems are thought to contribute to addictive behaviours. Previous models have suggested that dopaminergic roles in learning and motivation could produce addictive behaviours through pharmacological manipulations that provide excess dopaminergic signalling towards these learning and motivational systems. Redish (2004) suggested a role based on dopaminergic signals of value prediction error, while (Zhang et al., 2009) suggested a role based on dopaminergic signals of motivation. However, both models present significant limitations. They do not explain the reduced sensitivity to drug-related costs/negative consequences, the increased impulsivity generally found in people with a substance use disorder, craving behaviours, and non-pharmacological dependence, all of which are key hallmarks of addictive behaviours. Here, we propose a novel mathematical definition of salience, that combines aspects of dopamine’s role in both learning and motivation within the reinforcement learning framework. Using a single parameter regime, we simulated addictive behaviours that the (Zhang et al., 2009; Redish, 2004) models also produce but we went further in simulating the downweighting of drug-related negative prediction-errors, steeper delay discounting of drug rewards, craving behaviours and aspects of behavioural/non-pharmacological addictions. The current salience model builds on our recently proposed conceptual theory that salience modulates internal representation updating and may contribute to addictive behaviours by producing misaligned internal representations (Kalhan et al., 2021). Critically, our current mathematical model of salience argues that the seemingly disparate learning and motivational aspects of dopaminergic functioning may interact through a salience mechanism that modulates internal representation updating.

BIM Product Style Classification and Retrieval Based on Long-Range Style Dependencies

The rapid increase in building components on the building information model (BIM) object database has created new demand for BIM product recommendations to improve design efficiency. Current efforts mainly focus on the shape and contents of the products, instead of stylistic consistency, which is a crucial factor during the practical design process. To tackle such a problem, this paper proposes a novel framework to capture stylistic features based on long-range design dependencies with structural preservation, of which the snapshots of BIM products have been used to extract the stylistic features; core patches with strong style, generated by the pre-trained saliency model, are the root nodes; stylistic correlations are calculated as the hyperedges by tree-based operations; deep features and design features are proposed to represent the low-level and style distribution based on the study of design theory; and an ensemble learning strategy is introduced to solve the unbalanced classifier performance. An ablation study is conducted to validate the effectiveness of the proposed framework, in which comparative experiments with state-of-the-art baselines demonstrate the advantages of the proposed method.

Saliency prediction in 360° architectural scenes: Performance and impact of daylight variations

Saliency models are image-based prediction models that estimate human visual attention. Such models, when applied to architectural spaces, could pave the way for design decisions where visual attention is taken into account. In this study, we tested the performance of eleven commonly used saliency models that combine traditional and deep learning methods on 126 rendered interior scenes with associated head tracking data. The data was extracted from three experiments conducted in virtual reality between 2016 and 2018. Two of these datasets pertain to the perceptual effects of daylight and include variations of daylighting conditions for a limited set of interior spaces, thereby allowing to test the influence of light conditions on human head movement. Ground truth maps were extracted from the collected head tracking logs, and the prediction accuracy of the models was tested via the correlation coefficient between ground truth and prediction maps. To address the possible inflation of results due to the equator bias, we conducted complementary analyses by restricting the area of investigation to the equatorial image regions. Although limited to immersive virtual environments, the promising performance of some traditional models such as GBVS360eq and BMS360eq for colored and textured architectural rendered spaces offers us the prospect of their possible integration into design tools. We also observed a strong correlation in head movements for the same space lit by different types of sky, a finding whose generalization requires further investigations based on datasets more specifically developed to address this question.

No-Reference Light Field Image Quality Assessment Exploiting Saliency

In the near future, the broadcasting scenario will be characterized by immersive content. One of the systems for capturing the 3D content of a scene is the Light Field imaging. The huge amount of data and the specific transmission scenario impose strong constraints on services and applications. Among others, the evaluation of the quality of the received media cannot rely on the original signal but should be based only on the received data. In this direction, we propose a no-reference quality metric for light field images which is based on spatial and angular characteristics. In more details, the estimated saliency and cyclopean maps of light field images are exploited to extract the spatial features. The angular consistency features are, instead, measured with the use of the Global Luminance Distribution knowledge and the Weighted Local Binary Patterns operator on Epipolar Plane Images. The effectiveness of the proposed metric is assessed by comparing its performance with state-of-the-art quality metrics using 4 datasets: SMART, Win5-LID, VALID 10-bit, and VALID 8-bit. Furthermore, the performance is analyzed in cross-datasets, with different distortions, and for different saliency maps. The achieved results show that the performance of the proposed model outperforms state-of-the-art approaches and perform well for different distortion types and with various saliency models.

An integrative framework to evaluate impacts of complex tourism change initiatives

There is a need for holistic frameworks that show why, how, and by whom complex tourism change initiatives should be planned and implemented to achieve social impacts. This study aims to fill this knowledge gap by synthesising the theoretical underpinnings of three models—Theory of Change, Realist Evaluation, and Stakeholder Analysis—and crafting an integrative theory-based framework (ToCRESA) demonstrating pathways to social impacts. This study also advances Mitchell et al. (1997) stakeholder salience model by including a new attribute of ‘sustained perceived value’ to introduce a new classification of ‘determined stakeholder’. Due to the complexities of tourism change initiatives, ToCRESA suggests considering stakeholders' sustained perceived value in order to be engaged throughout the change initiatives. We used the complex social phenomenon of sustainable development of smart tourism to showcase the application of ToCRESA in developing impacts on society.

A deep unsupervised saliency model for lung segmentation in chest X-ray images

Medical image segmentation is a critical and laborious step in the diagnosis workflow by separating regions of organs or anomalies to be analyzed further by a physician, or classified automatically by an intelligent tool. Deep machine learning techniques have achieved impressive results in image segmentation tasks whenever immense amounts of labeled data are provided. However, having big amounts of manually supervised image data is labor intensive, and costly impractical. Unsupervised learning, contrariwise, can provide results using unlabeled images by reaching decisions, clustering, and deriving insights from the data only. We propose a deep unsupervised learning approach to perform lung segmentation in chest X-ray images, which is guided by heuristic saliency maps as a noisy labeling framework, which through a composite loss of reconstruction and regularization reach saliency estimation as final segmentation. Tests were performed using public chest X-ray images from the Japanese Society of Radiological Technology (JSRT), and from Montgomery County (MC) datasets. The proposed model based on saliency scored a dice coefficient of 0.87, against a state-of-the-art unsupervised method, which scored 0.68, on JSRT, and on the MC dataset 0.88, and 0.64, respectively. Qualitatively comparing, the saliency model worked better in reducing background noise. Considering that unsupervised models can be used as robust preparation steps, and as baselines for further different tasks, either guided by constant feedback from physicians, or as input to semi-supervised learning, the results have shown this to be a promise line of research to medical image segmentation problems.

None to rule them all? No generalization of saliency models across age

None to rule them all? No generalization of saliency models across age

Aggregating transformers and CNNs for salient object detection in optical remote sensing images

Salient object detection (SOD) in optical remote sensing images (RSIs) plays a significant role in many areas such as agriculture, environmental protection, and the military. However, since the difference in imaging mode and image complexity between RSIs and natural scene images (NSIs), it is difficult to achieve remarkable results by directly extending the saliency method targeting NSIs to RSIs. Besides, we note that the convolutional neural networks (CNNs) based U-Net cannot effectively acquire the global long-range dependency, and the Transformer doesn’t adequately characterize the spatial local details of each patch. Therefore, to conduct salient object detection in RSIs, we propose a novel two-branch architecture based network for Aggregating the Transformers and CNNs, namely ATC-Net, where the local spatial details and the global semantic information are fused into the final high-quality saliency map. Specifically, our saliency model adopts an encoder-decoder architecture including two parallel encoder branches and a decoder. Firstly, the two parallel encoder branches extract global and local features by using Transformer and CNNs, respectively. Then, the decoder employs a series of feature-enhanced fusion (FF) modules to aggregate multi-level global and local features by interactive guidance and enhance the fused feature via attention mechanism. Finally, the decoder deploys the read out (RO) module to fuse the aggregated feature of FF module and the low-level CNN feature, steering the feature to focus more on spatial local details. Extensive experiments are performed on two public optical RSIs datasets, and the results show that our saliency model consistently outperforms 30 state-of-the-art methods.

An Effective Method of Infrared Maritime Target Enhancement and Detection with Multiple Maritime Scene

Aiming at maritime infrared target detection with low contrast influenced by maritime clutter and illumination, this paper proposes a Modified Histogram Equalization with Edge Fusion (MHEEF) pre-processing algorithm in backlight maritime scenes and establishes Local-Contrast Saliency Models with Double Scale and Modes (LCMDSM) for detecting a target with the properties of positive and negative contrast. We propose a local-contrast saliency mathematical model with double modes in the extension of only one mode. Then, the big scale and small scale are combined into one Target Detection Unit (TDU), which can approach the “from bottom to up” mechanism of the Visual Attention Model (VAM) better and identify the target with a suitable size, approaching the target’s actual shape. In the experimental results and analysis, clutter, foggy, backlight, and dim maritime scenes are chosen to verify the effectiveness of the target detection algorithm. From the enhancement result, the LCMDSM algorithm can achieve a Detection Rate (DR) with a value of 98.26% under each maritime scene on the average level and can be used in real-time detection with low computational cost.

Towards explainable deep visual saliency models

Deep neural networks have shown their profound impact on achieving human-level performance in visual saliency prediction. However, it is still unclear how they learn their task and what it means in terms of understanding human visual system. In this work, we propose a framework to derive explainable saliency models from their corresponding deep architectures. Mainly, we explain a deep saliency model by understanding its four different aspects: (1) intermediate activation maps of deep layers, (2) biologically plausible Log-Gabor (LG) filters for salient region identification, (3) positional biased behavior of Log-Gabor filters and (4) processing of color information by establishing a relevance with human visual system. We consider four state-of-the-art (SOTA) deep saliency models, namely CMRNet, UNISAL, DeepGaze IIE, and MSI-Net for their interpretation using our proposed framework. We observe that explainable models perform way better than the classical SOTA models. We also find that CMRNet transforms the input RGB space to a representation after the input layer, which is very close to YUV space of a color image. Then, we discuss about the biological consideration and relevance of our framework for its possible anatomical substratum of visual attention. We find a good correlation between components of HVS and the base operations of the proposed technique. Hence, we say that this generic explainable framework provides a new perspective to see relationship between classical methods/human visual system and DNN based ones.

Audio-visual aligned saliency model for omnidirectional video with implicit neural representation learning

Audio-visual aligned saliency model for omnidirectional video with implicit neural representation learning