Visual Space Research Articles

Creating an endless visual space: An Isovist analysis of a small traditional Chinese garden

Traditional Chinese private gardens, as a unique spatial typology, create a maze-like quality through the arrangement of sightlines and pathways, offering occupants a sense of perpetual circulation and boundless time. The rich experiential nature cultivated within these confined spaces, alongside the ecological worldview of “harmony between heaven and humanity,” holds significant relevance for urban landscape design and the sustainable development of urban environments. While there is ample research literature on various aspects of garden spaces, studies exploring how private gardens generate visually infinite and cyclic spatial experiences within limited areas often remain confined to intuitive analysis, lacking concrete and rational explanations. This study employs the method of Isovist Analysis to test the validity of the spatial consciousness of “Limitless vision, Endless recurrence” within the spatial layout and tour organization of traditional private gardens, using the analysis of Isovists at eight scenic spots within the Xiaocanglang Water Courtyard, as well as an examination of visual-spatial changes along both primary and secondary pathways in different tour directions. It is hoped that this research method will serve as a tool for urban designers and landscape architects to assess spatial-visual characteristics and experiences during the design phase.

ACMamba: A State Space Model-Based Approach for Multi-Weather Degraded Image Restoration

In computer vision, eliminating the effects of adverse weather conditions such as rain, snow, and fog on images is a key research challenge. Existing studies primarily focus on image restoration for single weather types, while methods addressing image restoration under multiple combined weather conditions remain relatively scarce. Furthermore, current mainstream restoration networks, mostly based on Transformer and CNN architectures, struggle to achieve an effective balance between global receptive field and computational efficiency, limiting their performance in practical applications. This study proposes ACMamba, an end-to-end lightweight network based on selective state space models, aimed at achieving image restoration under multiple weather conditions using a unified set of parameters. Specifically, we design a novel Visual State Space Module (VSSM) and a Spatially Aware Feed-Forward Network (SAFN), which organically combine the local feature extraction capabilities of convolutions with the long-range dependency modeling capabilities of selective state space models (SSMs). This combination significantly improves computational efficiency while maintaining a global receptive field, enabling effective application of the Mamba architecture to multi-weather image restoration tasks. Comprehensive experiments demonstrate that our proposed approach significantly outperforms existing methods for both specific and multi-weather tasks across multiple benchmark datasets, showcasing its efficient long-range modeling potential in multi-weather image restoration tasks.

Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism

In studying the joint object detection and classification problem for facial expression recognition (FER) deploying the YOLOX framework, we introduce a novel feature extractor, called neighborhood coordinate attention Mamba (NCAMamba) to substitute for the original feature extractor in the Feature Pyramid Network (FPN). NCAMamba combines the background information reduction capabilities of Mamba, the local neighborhood relationship understanding of neighborhood attention, and the directional relationship understanding of coordinate attention. The resulting FER-YOLO-NCAMamba model, when applied to two unaligned FER benchmark datasets, RAF-DB and SFEW, obtains significantly improved mean average precision (mAP) scores when compared with those obtained by other state-of-the-art methods. Moreover, in ablation studies, it is found that the NCA module is relatively more important than the Visual State Space (VSS), a version of using Mamba for image processing, and in visualization studies using the grad-CAM method, it reveals that regions around the nose tip are critical to recognizing the expression; if it is too large, it may lead to erroneous prediction, while a small focused region would lead to correct recognition; this may explain why FER of unaligned faces is such a challenging problem.

Mamba- and ResNet-Based Dual-Branch Network for Ultrasound Thyroid Nodule Segmentation.

Accurate segmentation of thyroid nodules in ultrasound images is crucial for the diagnosis of thyroid cancer and preoperative planning. However, the segmentation of thyroid nodules is challenging due to their irregular shape, blurred boundary, and uneven echo texture. To address these challenges, a novel Mamba- and ResNet-based dual-branch network (MRDB) is proposed. Specifically, the visual state space block (VSSB) from Mamba and ResNet-34 are utilized to construct a dual encoder for extracting global semantics and local details, and establishing multi-dimensional feature connections. Meanwhile, an upsampling-convolution strategy is employed in the left decoder focusing on image size and detail reconstruction. A convolution-upsampling strategy is used in the right decoder to emphasize gradual feature refinement and recovery. To facilitate the interaction between local details and global context within the encoder and decoder, cross-skip connection is introduced. Additionally, a novel hybrid loss function is proposed to improve the boundary segmentation performance of thyroid nodules. Experimental results show that MRDB outperforms the state-of-the-art approaches with DSC of 90.02% and 80.6% on two public thyroid nodule datasets, TN3K and TNUI-2021, respectively. Furthermore, experiments on a third external dataset, DDTI, demonstrate that our method improves the DSC by 10.8% compared to baseline and exhibits good generalization to clinical small-scale thyroid nodule datasets. The proposed MRDB can effectively improve thyroid nodule segmentation accuracy and has great potential for clinical applications.

Enhancing bridge damage detection with Mamba-Enhanced HRNet for semantic segmentation.

With the acceleration of urbanization, bridges, as crucial infrastructure, their structural health and stability are paramount to public safety. This paper proposes Mamba-Enhanced HRNet for bridge damage detection. Mamba-Enhanced HRNet integrates the advantages of HRNet's multi-resolution parallel design and VMamba's visual state space model. By replacing the residual convolutional blocks in HRNet with a combination of VSS blocks and convolution, this model enhances the network's capability to capture global contextual information while maintaining computational efficiency. This work builds an extensive dataset with multiple damage kinds and uses Mean Intersection over Union (Mean IoU) as the assessment metric to assess the performance of Mamba-Enhanced HRNet. Experimental results demonstrate that Mamba-Enhanced HRNet achieves significant performance improvements in bridge damage semantic segmentation tasks, with Mean IoU scores of 0.963, outperforming several other semantic segmentation models.

Visual pathways functioning in healthy pre-term adolescents: Sex but not gestational age effect.

Visuo-spatial and visuo-perceptual functioning is widely studied in preterm child and is strongly sex-specific. However, little to no data is available regarding male-female differences in preterm children and adolescents and about the interaction effect between sex and preterm birth. We studied 30 adolescents born preterm with normal cognitive and clinical neurological outcomes and 34 age-matched controls to investigate the interaction between levels of prematurity and sex in predicting the outcome of visual pathways functioning and to explore the relation between psychophysiological perceptive processing and neuropsychological performance. In the presence of prematurity, a greater female vulnerability in central visuo-cognitive processing (Form Coherence Task), but not in neuropsychological accuracy (Street Completion Test and Visual Object and Space Perception battery), seems to be more evident. Moreover, the psychophysical threshold is correlated to neuropsychological accuracy only in preterm females and not in males. These results support the idea that the male vulnerability in cognitive functioning described in prematurity-related developmental conditions is negligible during school age in children-adolescents with normal cognitive and clinical neurological outcomes. Visuo-perceptual functioning is widely studied in prematurity. However, few data are available about the interaction effect between sex and preterm birth in predicting visuo-perceptual functioning. We evidenced that in females born preterm with preserved cognitive abilities, the efficiency of the psychophysical visuo-perceptual threshold is reduced, but not related to the neuropsychological performance. Females may implement compensation strategies to achieve good performance regardless of the perceptual threshold. The present study addresses an important gap in literature, suggesting possible sex-specific outcomes in visuo-perceptual ability among preterm children and adolescents with normal intelligence and neurological outcomes.

Infrared Image Generation Based on Visual State Space and Contrastive Learning

The preparation of infrared reference images is of great significance for improving the accuracy and precision of infrared imaging guidance. However, collecting infrared data on-site is difficult and time-consuming. Fortunately, the infrared images can be obtained from the corresponding visible-light images to enrich the infrared data. To this end, this present work proposes an image translation algorithm that converts visible-light images to infrared images. This algorithm, named V2IGAN, is founded on the visual state space attention module and multi-scale feature contrastive learning loss. Firstly, we introduce a visual state space attention module designed to sharpen the generative network’s focus on critical regions within visible-light images. This enhancement not only improves feature extraction but also bolsters the generator’s capacity to accurately model features, ultimately enhancing the quality of generated images. Furthermore, the method incorporates a multi-scale feature contrastive learning loss function, which serves to bolster the robustness of the model and refine the detail of the generated images. Experimental results show that the V2IGAN method outperforms existing typical infrared image generation techniques in both subjective visual assessments and objective metric evaluations. This suggests that the V2IGAN method is adept at enhancing the feature representation in images, refining the details of the generated infrared images, and yielding reliable, high-quality results.

Preselection of potential target spaces based on partial information by the supplementary eye field

Before selecting a saccadic target, we often acquire partial information about the location of the forthcoming target and preselect a region of visual space even before the target becomes visible. To determine whether the supplementary eye field (SEF) represents information signifying the potential target space, we examined neuronal activity in the SEF of monkeys performing a behavioral task designed to isolate the process of visuospatial preselection under uncertainty from the process of selecting a specified location. Our data showed that the activity of SEF neurons represented information about the potential target space instructed by symbolic cues. Increased activity of visuospatially selective SEF neurons encoded the potential target space, which could be a mechanism facilitating subsequent selection of an appropriate target. Furthermore, electrical stimulation of the SEF during the preselection period disrupted subsequent target selection. These results demonstrate that the SEF contributes to the preselection of potential target spaces based on partial information.

Word vector embedding and self-supplementing network for Generalized Few-shot Semantic Segmentation

Under the condition of sufficient base class samples and a few novel class samples, Generalized Few-shot Semantic Segmentation (GFSS) classifies each pixel in the query image as base class, novel class, or background. A standard GFSS approach involves two training stages: base class learning and novel class updating. However, inter-class interference and information loss which contribute to the poor performance of GFSS, have not been synthetical considered. To address the problem, we propose an Embedded-Self-Supplementing Network (ESSNet), i.e., semantic word embedding and query set self-supplementing information to enhance segmentation accuracy. Specifically, the semantic word embedding module employs distance information between word vectors to assist the model in learning the distance between class prototypes. In order to transform the semantic word vector prototypes from the semantic space to the visual embedding space, we designed a triplet loss function to supervise the word vector embedding module, where the word vector prototype serves as an anchor and positive-negative samples are collected among the general features of the support image. To compensate for the information loss caused by using prototypes to represent classes, we propose a self-supplementing module to mine the information contained in the query image. Specifically, this module first makes a preliminary prediction on the query image, then selects high-confidence area to form pseudo labels, and finally uses pseudo labels to extract query prototypes to supplement the missing information. Extensive experiments on PASCAL-5i and COCO-20i show that ESSNet has superior performance and outperforms state-of-the-art methods in all settings.

DCSS-UNet: UNet based on State Space Model for Polyp Segmentation

Early and accurate segmentation of medical images can provide valuable information for medical treatment. In recent years, the automatic and accurate segmentation of polyps in colonoscopy images has received extensive attention from the research community of artificial intelligence and computer vision. Many researchers have conducted in-depth research on models based on CNN and Transformer. However, CNN have limited ability to model remote dependencies, which makes it challenging to fully utilize semantic information in images. On the other hand, the complexity of the secondary computation poses a challenge to the transformer. Recently, state-space models (SSMS), such as Mamba, have been recognized as a promising approach. They not only show superior performance in remote interaction, but also maintain linear computational complexity. Inspired by Mamba, we propose DCSS-UNet, where we utilize visual state space (VSS) blocks in VMamba to capture a wide range of contextual information. In the Skip connection phase, we propose Skip Connects Feature Attention modules(SFA) to better communicate information from the encoder. In the decoder stage, we innovatively combined the Temporal Fusion Attention Module(TFAM) to effectively fuse the feature information. In addition, we introduced a custom Loss calculation method, Tversky Loss, for the model to achieve faster convergence and improve segmentation along polyp boundaries. Our model was trained on the Kvasir-SEG and CVC-ClinicDB datasets, and validated on datasets Kvasir-SEG, CVC-ColonDB, CVC-300, and ETIS. The results show that the model achieves good segmentation accuracy and generalization performance with a low number of parameters. We are 6.1% ahead in the Kavirs-SEG dataset and 3.1% ahead in the CVC-ClinicDB dataset compared to VM-UNet.

Accessibility and expansion of the narratives’ space: convergent technologies and semiotics of exposition in an art museum

The development of convergent technologies opens up new horizons for expanding the sensory experience of museum visitors, especially the people with disabilities and alternative perception. We argue that augmented reality (AR) technologies, mainly used to develop applications that visually expand physical spaces, may serve not only as a tool of inclusion and immersion, but also an instrument of reviewing the narrative of art history delivered by a classical museum. Although AR technologies are focused mainly on visuality, detecting and localizing objects and images, as well as mapping and augmenting the visual space, they could be employed to expand the museum environment by other sensory modalities, such as sound and tactile sensations. Increasing the level of information accessibility of museum expositions and the independence of the development of exhibition space by people with disabilities is possible, among other things, by providing them access to context-aware information (MoMA, Tate, Garage and other institutions provide examples of such type of curatorship and exposition design).Furthermore, we argue that the issue of contextualization of knowledge received by the museum visitor is considered should be considered in a broader sense: we take the museum as a space of narration, where the main concepts of art history (with its conventional semantic accents and limitations) are shaped, consolidated and delivered. Having analyzed the logic of a classical art museum exposition, we review the alternative models for presenting art history narratives (in curatorial projects of A. Malraux, J.-H. Martin and F. Lyotard) and argue that new media may become a tool not only for inclusiveness and immersion, but also for a special type of subjectivation of the viewer involved in the process of creative rethinking of the history of art. Inclusiveness, immersion and semantic flexibility provided by new media makes museums a space for an open dialogue. Instead of outlined birders and «ready-to-use» senses an exhibition can show how «other things are possible». We argue that curatorial experiments and media applications dedicated to art can turn a museum into a zone of seeking that awakens the viewer to an independent search for subtle meanings and stories that exist on the periphery of the classical history of art. And the new media extensions of museum space, considered in the context of inclusion, may also be viewed as a tool for building new narratives and developing models for a more flexible and complex interpretation of art history in a multipolar world.

RSVP for VPSA : A Meta Design Study on Rapid Suggestive Visualization Prototyping for Visual Parameter Space Analysis.

Visual Parameter Space Analysis (VPSA) enables domain scientists to explore input-output relationships of computational models. Existing VPSA applications often feature multi-view visualizations designed by visualization experts for a specific scenario, making it hard for domain scientists to adapt them to their problems without professional help. We present RSVP, the Rapid Suggestive Visualization Prototyping system encoding VPSA knowledge to enable domain scientists to prototype custom visualization dashboards tailored to their specific needs. The system implements a task-oriented, multi-view visualization recommendation strategy over a visualization design space optimized for VPSA to guide users in meeting their analytical demands. We derived the VPSA knowledge implemented in the system by conducting an extensive meta design study over the body of work on VPSA. We show how this process can be used to perform a data and task abstraction, extract a common visualization design space, and derive a task-oriented VisRec strategy. User studies indicate that the system is user-friendly and can uncover novel insights.

A Novel End-to-End Deep Learning Framework for Chip Packaging Defect Detection.

As semiconductor chip manufacturing technology advances, chip structures are becoming more complex, leading to an increased likelihood of void defects in the solder layer during packaging. However, identifying void defects in packaged chips remains a significant challenge due to the complex chip background, varying defect sizes and shapes, and blurred boundaries between voids and their surroundings. To address these challenges, we present a deep-learning-based framework for void defect segmentation in chip packaging. The framework consists of two main components: a solder region extraction method and a void defect segmentation network. The solder region extraction method includes a lightweight segmentation network and a rotation correction algorithm that eliminates background noise and accurately captures the solder region of the chip. The void defect segmentation network is designed for efficient and accurate defect segmentation. To cope with the variability of void defect shapes and sizes, we propose a Mamba model-based encoder that uses a visual state space module for multi-scale information extraction. In addition, we propose an interactive dual-stream decoder that uses a feature correlation cross gate module to fuse the streams' features to improve their correlation and produce more accurate void defect segmentation maps. The effectiveness of the framework is evaluated through quantitative and qualitative experiments on our custom X-ray chip dataset. Furthermore, the proposed void defect segmentation framework for chip packaging has been applied to a real factory inspection line, achieving an accuracy of 93.3% in chip qualification.

MFAD-RTDETR: A Multi-Frequency Aggregate Diffusion Feature Flow Composite Model for Printed Circuit Board Defect Detection

To address the challenges of excessive model parameters and low detection accuracy in printed circuit board (PCB) defect detection, this paper proposes a novel PCB defect detection model based on the improved RTDETR (Real-Time Detection, Embedding and Tracking) method, named MFAD-RTDETR. Specifically, the proposed model introduces the designed Detail Feature Retainer (DFR) into the original RTDETR backbone to capture and retain local details. Subsequently, based on the Mamba architecture, the Visual State Space (VSS) module is integrated to enhance global attention while reducing the original quadratic complexity to a linear level. Furthermore, by exploiting the deformable attention mechanism, which dynamically adjusts reference points, the model achieves precise localization of target defects and improves the accuracy of the transformer in complex visual tasks. Meanwhile, a receptive field synthesis mechanism is incorporated to enrich multi-scale semantic information and reduce parameter complexity. In addition, the scheme proposes a novel Multi-frequency Aggregation and Diffusion feature composite paradigm (MFAD-feature composite paradigm), which consists of the Aggregation Diffusion Fusion (ADF) module and the Refiner Feature Composition (RFC) module. It aims to strengthen features with fine-grained awareness while preserving a certain level of global attention. Finally, the Wise IoU (WIoU) dynamic nonmonotonic focusing mechanism is used to reduce competition among high-quality anchor boxes and mitigate the effects of the harmful gradients from low-quality examples, thereby concentrating on anchor boxes of average quality to promote the overall performance of the detector. Extensive experiments are conducted on the PCB defect dataset released by Peking University to validate the effectiveness of the proposed model. The experimental results show that our approach achieves the 97.0% and 51.0% performance in mean Average Precision (mAP)@0.5 and mAP@0.5:0.95, respectively, which significantly outperforms the original RTDETR. Moreover, the model reduces the number of parameters by approximately 18.2% compared to the original RTDETR.

Exploring the Impact of Vertical Access Elements on Visual Richness and Space Quality within Shopping Mall Atriums

Exploring the Impact of Vertical Access Elements on Visual Richness and Space Quality within Shopping Mall Atriums

Superclass-aware visual feature disentangling for generalized zero-shot learning

Zero-shot learning (ZSL) aims to learn a model trained on seen samples with the ability to recognize samples from unseen classes, while generalized ZSL (GZSL) takes a step closer to realistic scenarios by recognizing both of seen and unseen samples. The existing methods rely on the semantic descriptions as the side-information and conduct tight alignment between the visual and semantic spaces. However, the tight modality alignment may result in incomplete representations, leading to the loss of originally detailed and discriminative information. In this paper, we propose a simple yet effective superclass-aware visual feature disentangling method termed as SupVFD for GZSL. We use the neighbor relations of the semantic descriptions to define superclass and with the guide of superclass, our method disentangles visual features into discriminative and transferable factors. To this end, the semantic descriptions are used as implicit supervision, which preserves the valuable detailed and discriminative information in the visual features. The extensive experiments in both ZSL and GZSL settings prove our method outperforms the state-of-the-art methods for image object classification as well as video action recognition. Code is available at our github: https://github.com/changniu54/SupVFD-Master.

Semantic-driven dual consistency learning for weakly supervised video anomaly detection

Video anomaly detection presents a significant challenge in computer vision, with the aim of distinguishing various anomaly events from numerous normal ones. Weakly supervised video anomaly detection has recently emerged as a promising solution, enabling the detection of anomaly snippets with only video-level annotations. However, knowledge about anomaly annotation remains underutilized, resulting in a gap between visual space and semantic understanding of anomalies, thus failing to capture the clear boundary between anomalies and normalities. Therefore, we propose a weakly supervised paradigm of cross-modal detection and consistency learning, leveraging dual consistency to provide discriminative representations for anomalies at both the semantic-to-target and target-to-snippet levels. Specifically, we introduce a cross-modal detection network, which detects the targets in each frame according to given semantic rules, to derive semantic-consistent visual embeddings. To depict the clear boundary between anomalies and normalities, a cross-domain alignment module is proposed to enhance the discriminative representation of abnormal targets by learning the contextual consistency between the target and snippet embeddings. Our architecture integrates the detection of semantic-consistent targets based on variable semantic rules, ensuring transferable deployment across scenarios and enabling comprehensive identification, localization, and recognition of abnormal events through a “when-where-which” pipeline. The evaluation of our approach is conducted on four widely used public benchmarks: ShanghaiTech, UCSD Ped2, CUHK Avenue, and UBnormal through extensive qualitative and quantitative analyzes. The results demonstrate the remarkable performance of our approach in dealing with the VAD task.

On Blackness, images and anti-racist work

ABSTRACT In this article, I explore the use, production, and readings of images in antiracist work as processes in constant tension with racist logics. Racist logics are deceptive and entrapping, and the visible world is a particularly dynamic arena in which to analyse this. Images, specifically photographs, incorporate a nuanced and productive contradiction between their ability to illustrate, explain and evoke, on the one hand, while also ensnaring, on the other, partly through alluring and providing pleasure. This contradiction means that anti-racist projects may include the possibility of re-inscribing racist discourse and practice. Drawing on empirical examples from Mexico, this article interrogates how photographic representations of Blackness have been entangled with the racist logics and racial projects that feed them. Racialised images that emphasise the physical body of Black people raise questions about how mestizaje, Latin America’s main racial formation, has coded the visual space of representation in confusing ways.

Top-Down Priors Disambiguate Target and Distractor Features in Simulated Covert Visual Search.

Several models of visual search consider visual attention as part of a perceptual inference process, in which top-down priors disambiguate bottom-up sensory information. Many of these models have focused on gaze behavior, but there are relatively fewer models of covert spatial attention, in which attention is directed to a peripheral location in visual space without a shift in gaze direction. Here, we propose a biologically plausible model of covert attention during visual search that helps to bridge the gap between Bayesian modeling and neurophysiological modeling by using (1) top-down priors over target features that are acquired through Hebbian learning, and (2) spatial resampling of modeled cortical receptive fields to enhance local spatial resolution of image representations for downstream target classification. By training a simple generative model using a Hebbian update rule, top-down priors for target features naturally emerge without the need for hand-tuned or predetermined priors. Furthermore, the implementation of covert spatial attention in our model is based on a known neurobiological mechanism, providing a plausible process through which Bayesian priors could locally enhance the spatial resolution of image representations. We validate this model during simulated visual search for handwritten digits among nondigit distractors, demonstrating that top-down priors improve accuracy for estimation of target location and classification, relative to bottom-up signals alone. Our results support previous reports in the literature that demonstrated beneficial effects of top-down priors on visual search performance, while extending this literature to incorporate known neural mechanisms of covert spatial attention.

Uniform Metric Graphs

We prove that every complete metric space “is” the boundary of a uniform length space whose quasihyperbolization is a geodesic visual Gromov hyperbolic space. There is a natural quasimöbius identification of the original space’s conformal gauge with the canonical gauge on the Gromov boundary. All parameters are absolute constants.