Visual Features Research Articles

Unveiling Urban River Visual Features Through Immersive Virtual Reality: Analyzing Youth Perceptions with UAV Panoramic Imagery

The visual evaluation and characteristic analysis of urban rivers are pivotal for advancing our understanding of urban waterscapes and their surrounding environments. Unmanned aerial vehicles (UAVs) offer significant advantages over traditional satellite remote sensing, including flexible aerial surveying, diverse perspectives, and high-resolution imagery. This study centers on the Haihe River, South Canal, and North Canal in Tianjin China, employing UAVs to capture continuous panoramic image data. Through immersive virtual reality (VR) technology, visual evaluations of these panoramic images were obtained from a cohort of young participants. These evaluations encompassed assessments of scenic beauty, color richness, vitality, and historical sense. Subsequently, computer vision techniques were utilized to quantitatively analyze the proportions of various landscape elements (e.g., trees, grass, buildings) within the images. Clustering analysis of visual evaluation results and semantic segmentation outcomes from different study points facilitated the effective identification and grouping of river visual features. The findings reveal significant differences in scenic beauty, color richness, and vitality among the Haihe River, South Canal, and North Canal, whereas the South and North Canals exhibited a limited sense of history. Six landscape elements—water bodies, buildings, trees, etc.—comprised over 90% of the images, forming the primary visual characteristics of the three rivers. Nonetheless, the uneven spatial distribution of these elements resulted in notable variations in the visual features of the rivers. This study demonstrates that the visual feature analysis method based on UAV panoramic images can achieve a quantitative evaluation of multi-scene urban 3D landscapes, thereby providing a robust scientific foundation for the optimization of urban river environments.

Correcting images for individual differences in color appearance

AbstractIndividual differences are a prominent feature of normal color vision and range from variations in sensitivity to perception and color naming. Corrections for differences in spectral sensitivity are common, and there is growing interest in calibrating displays for the sensitivity of an individual observer. In contrast, few studies have explored calibrations for aspects of color appearance. We developed a technique for adjusting images based directly on an individual's hue percepts and illustrate the principle of the approach using a set of hue scaling functions measured previously for a large sample of color‐normal observers (Emery et al. PNAS 2023). Colors in an image are mapped onto the average scaling function to define the hue perceived by the average “standard observer.” This hue is then mapped back to the chromaticity that would elicit the same response in any individual observer. With this correction, different observers – each looking at physically different images calibrated for their own hue percepts – should in principle agree on the perceived colors. Adjustments of this kind could be easily implemented on standard displays, because they require only measures of hue percepts and not spectral sensitivity, and could lead to greater consistency in the perception and communication about color across individuals with potentially widely different perceptual experiences of color.

Speech recognition using an english multimodal corpus with integrated image and depth information.

Traditional English corpora mainly collect information from a single modality, but lack information from multimodal information, resulting in low quality of corpus information and certain problems with recognition accuracy. To solve the above problems, this paper proposes to introduce depth information into multimodal corpora, and studies the construction method of English multimodal corpora that integrates electronic images and depth information, as well as the speech recognition method of the corpus. The multimodal fusion strategy adopted integrates speech signals and image information, including key visual information such as the speaker's lip movements and facial expressions, and uses deep learning technology to mine acoustic and visual features. The acoustic model in the Kaldi toolkit is used for experimental research.Through experimental research, the following conclusions were drawn: Under 15-dimensional lip features, the accuracy of corpus A under monophone model was 2.4% higher than that of corpus B under monophone model when the SNR (signal-to-noise ratio) was 10dB, and the accuracy of corpus A under the triphone model at the signal-to-noise ratio of 10dB was 1.7% higher than that of corpus B under the triphone model at the signal-to-noise ratio of 10dB. Under the 32-dimensional lip features, the speech recognition effect of corpus A under the monophone model at the SNR of 10dB was 1.4% higher than that of corpus B under the monophone model at the SNR of 10dB, and the accuracy of corpus A under the triphone model at the SNR of 10dB was 2.6% higher than that of corpus B under the triphone model at the SNR of 10dB. The English multimodal corpus with image and depth information has a high accuracy, and the depth information helps to improve the accuracy of the corpus.

Day2Dark: Pseudo-Supervised Activity Recognition Beyond Silent Daylight

AbstractThis paper strives to recognize activities in the dark, as well as in the day. We first establish that state-of-the-art activity recognizers are effective during the day, but not trustworthy in the dark. The main causes are the limited availability of labeled dark videos to learn from, as well as the distribution shift towards the lower color contrast at test-time. To compensate for the lack of labeled dark videos, we introduce a pseudo-supervised learning scheme, which utilizes easy to obtain unlabeled and task-irrelevant dark videos to improve an activity recognizer in low light. As the lower color contrast results in visual information loss, we further propose to incorporate the complementary activity information within audio, which is invariant to illumination. Since the usefulness of audio and visual features differs depending on the amount of illumination, we introduce our ‘darkness-adaptive’ audio-visual recognizer. Experiments on EPIC-Kitchens, Kinetics-Sound, and Charades demonstrate our proposals are superior to image enhancement, domain adaptation and alternative audio-visual fusion methods, and can even improve robustness to local darkness caused by occlusions. Project page: https://xiaobai1217.github.io/Day2Dark/.

Optical aspects of Born-Infeld BTZ black holes in massive gravity

We explore the dynamics of thin accretion disks, the radius of black hole shadows, observed intensities, and the visual characteristics of Born-Infeld BTZ black holes in massive gravity. We find out the relations for angular velocity, specific energy, and angular momentum of particles around the black hole. We observe that intense Born-Infeld electromagnetic effects lead to a reduction in the rotational motion of particles within the accretion disk, and the massive gravity slows down the orbital motion of these particles. We reveal that the influence of massive gravity parameter correlates with a reduction in the black hole’s shadow size, which suggests that massive gravity effects intensify the gravitational fields, thereby reducing the angular diameter of the shadows. On the other hand, a higher Born-Infeld parameter enlarges the black hole’s shadow, which manifests a visual relationship between the black hole’s physical dimensions and its gravitational influence. Moreover, we also uncover the optical characteristics of Born-Infeld BTZ black holes, which show that the Born-Infeld parameter greatly influences the electromagnetic field around the black hole, which affects energy distribution in the space. Finally, we observe that massive gravity significantly influences the spacetime structure near black holes, which is crucial for grasping gravitational lensing and the dynamics of accretion disks under such extreme conditions.

Estimation of horizontal spatial specifications for ideal head-mounted displays in practical conditions

This study presents the horizontal spatial specifications required for developing ideal head-mounted displays (HMDs) that provide visual experiences indistinguishable from those obtained without wearing HMDs. We investigated the minimum specifications for pixel density and field of view (FoV) such that users cannot perceive any degradation in these aspects. Conventional studies have measured visual acuity in the periphery and FoV size without eye movement. However, these results are not sufficient to determine the spatial specifications because users may notice a degradation in the quality of images displayed in the periphery when shifting their gaze away from the front. In this study, we measured visual characteristics under practical conditions, wherein participants moved their eyes naturally in coordination with their head movements, as observed when viewing natural scenes. Using a cylindrical display that covered the participant’s entire horizontal visual field with high resolution images, we asked participants to identify degraded spatial resolution or narrowed FoV. Results showed that ideal HMDs do not have to provide spatial frequency components above 0.5 and 2 cycles per degree outside the visual fields of approximately 210° and 120° horizontally, respectively. These results suggest that a resolution equivalent to one-fifteenth of the frontal area is sufficient for the lateral areas of the head. In addition, we found that ideal HMDs require a FoV of at least 240°. This is considerably smaller than a naive estimate of 310° based on previous studies, obtained by integrating the FoV size during fixation and the movable range of the eyes. We believe that these findings will be helpful in developing ideal HMDs.

An effective Key Frame Extraction technique based on Feature Fusion and Fuzzy-C means clustering with Artificial Hummingbird

Key frame extraction is very important in video summarization and content-based video analysis to address the problem of data redundancy in a video. Key frame extraction enables quick navigation and expert video arrangement in many applications. The visually impaired can benefit from the use of key frame extraction for rapid object recognition and tracking. Most key frame extraction techniques consider only a single visual feature instead of multiple features or full pictorial information of the video. This study proposes a key frame extraction method from a video that (i) first removes insignificant frames by pre-processing, (ii) second, four visual and structural feature differences among the consecutive frames are extracted and aggregated to identify informative frames, (iii) third, to cluster the obtained frames, a hybrid FCM-AHA method is proposed by combining Fuzzy C-means(FCM) with artificial hummingbird optimization algorithm (AHA) to circumvent the local minima trapping problem of FCM, and finally, from each cluster, the two frames having greatest Euclidean distance from all the other frames within a cluster is selected as key frames to remove redundant frames. Experimental results on the Open video and YouTube datasets show that the suggested method outperforms state-of-the-art methods both in terms of subjective qualitative analysis and objective quantitative evaluation, e.g., Precision, Recall, and F-score. Further, results are also taken on real video to demonstrate its applicability in real-life applications.

Investigating the relationship between subjective perception and unconscious feature integration.

Visual features need to be temporally integrated to detect motion signals and solve the many ill-posed problems of vision. It has previously been shown that such integration occurs in windows of unconscious processing of up to 450 milliseconds. However, whether features are integrated should be governed by perceptually meaningful mechanisms. Here, we expand on previous findings suggesting that subjective perception and integration may be linked. Specifically, different observers were found to group elements differently and to exhibit corresponding feature integration behavior. If the former were to influence the latter, perception would appear to not only be the outcome of integration but to potentially also be part of it. To test any such linkages more systematically, we here examined the role of one of the key perceptual grouping cues, color similarity, in the Sequential Metacontrast Paradigm (SQM). In the SQM, participants are presented with two streams of lines that are expanding from the center outwards. If several lines in the attended motion stream are offset, offsets integrate unconsciously and mandatorily for periods of up to 450 milliseconds. Across three experiments, we presented lines of varied colors. Our results reveal that individuals who perceive differently colored lines as "popping out" from the motion stream do not exhibit mandatory integration but that individuals who perceive such lines as part of an integrated motion stream do show offset integration behavior across the entire stream. These results attest to the proposed linkage between subjective perception and integration behavior in the SQM.

Pola Penerapan Implementasi Teknologi Tepat Guna Berbasis Machine Learning dalam Identifikasi Keramik sebagai Infrastruktur Wisata Edukasi di Desa Pagelaran

The application pattern of appropriate technology based on machine learning for ceramic identification as educational tourism infrastructure in Pagelaran Village aims to enhance the effectiveness and efficiency of the ceramic identification process. This program employs machine learning algorithms to classify and recognize various types of ceramics based on visual and textural characteristics. Through this approach, educational tourism in Pagelaran Village can offer an interactive and immersive learning experience for visitors. Initial results indicate increased accuracy in ceramic identification and heightened visitor interest and engagement. This initiative is expected to support the development of technology-based educational tourism in Pagelaran Village.

Embodied Spaces in Digital Times: Exploring the Role of Instagram in Shaping Temporal Dimensions and Perceptions of Architecture

This study explores Instagram’s influence on sensory and experiential engagement with architecture in the digital age. Using a phenomenological approach, we studied the impact of Instagram’s visual features, such as filters, geotagging, and hashtags, on user interactions and perceptions of architectural spaces. The research demonstrates that Instagram transforms traditional architectural experiences into dynamic visual narratives that integrate real and virtual elements, altering our understanding of space and time. While acknowledging that architectural experience encompasses form, function, and historical context, this paper specifically focuses on Instagram’s role in mediating perceptual experiences. By analyzing user engagement patterns and content trends, the study highlights how Instagram shapes architectural design practices and the creation of spaces tailored for digital interaction. This study offers a comprehensive view of the complex relationship between digital media and architectural perception, identifying both the opportunities and challenges presented by the platform in influencing our understanding of architectural spaces.

Event-horizon-scale Imaging of M87* under Different Assumptions via Deep Generative Image Priors

Reconstructing images from the Event Horizon Telescope (EHT) observations of M87*, the supermassive black hole at the center of the galaxy M87, depends on a prior to impose desired image statistics. However, given the impossibility of directly observing black holes, there is no clear choice for a prior. We present a framework for flexibly designing a range of priors, each bringing different biases to the image reconstruction. These priors can be weak (e.g., impose only basic natural-image statistics) or strong (e.g., impose assumptions of black hole structure). Our framework uses Bayesian inference with score-based priors, which are data-driven priors arising from a deep generative model that can learn complicated image distributions. Using our Bayesian imaging approach with sophisticated data-driven priors, we can assess how visual features and uncertainty of reconstructed images change depending on the prior. In addition to simulated data, we image the real EHT M87* data and discuss how recovered features are influenced by the choice of prior.

Scene-aware Foveated Rendering.

We propose a new scene-aware foveated rendering method, which incorporates the scene awareness and characteristics of the human visual system into the mapping-based foveated rendering framework. First, we generate the conservative visual importance map that encodes the visual features of the scene, visual acuity, and gaze motion. Second, we construct the pixel size control map using a convolution kernel method. Third, we utilize the pixel size control map to guide the foveated rendering. At last, a temporal coherent refinement strategy is used to maintain the smooth foveated rendering for the adjacent frames. Compared to the state-of-the-art mapping-based foveated rendering methods using the same compression ratio, our method achieves smaller MSE, higher PSNR, and SSIM in the fovea, periphery, salient regions, and the whole image. We also conducted user studies, and the results proved that the perceptual quality of our method has a high visual similarity with the around truth rendered with the full resolution.

A Picture May Be Worth a Hundred Words for Visual Question Answering

How far can textual representations go in understanding images? In image understanding, effective representations are essential. Deep visual features from object recognition models currently dominate various tasks, especially Visual Question Answering (VQA). However, these conventional features often struggle to capture image details in ways that match human understanding, and their decision processes lack interpretability. Meanwhile, the recent progress in language models suggests that descriptive text could offer a viable alternative. This paper investigated the use of descriptive text as an alternative to deep visual features in VQA. We propose to process description–question pairs rather than visual features, utilizing a language-only Transformer model. We also explored data augmentation strategies to enhance training set diversity and mitigate statistical bias. Extensive evaluation shows that textual representations using approximately a hundred words can effectively compete with deep visual features on both the VQA 2.0 and VQA-CP v2 datasets. Our qualitative experiments further reveal that these textual representations enable clearer investigation of VQA model decision processes, thereby improving interpretability.

Assessment of cervical spine CT by an image quality audit using qualitative and quantitative methods.

To study the feasibility and assess the correlation of qualitative and quantitative methods for an image quality (IQ) audit of a Cervical spine CT. Five radiologists retrospectively performed a blinded visual grading analysis (VGA) on 20 studies (10 from Protocol 1 and 10 from Protocol 2), using the RANZCR CT IQ Self-Audit worksheet. A Visual Grading Analysis Score (VGAS) and Area under the curve using Visual Grading Characteristics (AUCVGC) were the figures of merit. Quantitative metrics for noise and contrast were correlated to the qualitative assessment. No statistically significant difference was observed in the IQ, VGASProtocol1 = 0.65, 95% CI [0.54, 0.75] and VGASProtocol 2 = 0.73, 95% CI [0.67, 0.79] and AUCVGC = 0.548, 95% CI [0.40, 0.69]. Protocol 2 indicated a statistically significant average dose reduction of 35% in CTDIvol (P = 0.020) and a higher noise; however, the difference was statistically insignificant. There was a moderate correlation between the manual noise measurements in soft tissue and air (P = 0.035) and a strong correlation between the manual and automated noise measurements (P < 0.001). The contrast resolution-based quantitative parameter, EdgeGradientSoft, correlated to the qualitative scores (P = 0.031). Validated VGA tools can be used for IQ audits; however, tailoring the image criteria and rating scale to the clinical practice is suggested. The use of contrast-based IQ metrics showed encouraging results, and further larger-scale studies are needed to explore their potential use in quality management.

Modeling the Effect of Greenways’ Multilevel Visual Characteristics on Thermal Perception in Summer Based on Bayesian Network and Computer Vision

The aim of this study is to reveal the effects of multilevel visual characteristics of greenways on thermal perception in hot and humid regions during summer and to explore the potential of visual design to enhance psychological thermal comfort. Data on light (L), color (C), plant richness (PR), space openness (SO), scenic view (SV), thermal sensation (TS), and thermal preference (TP) were collected through questionnaires (n = 546). Computer vision technology was applied to measure the green view index (GVI), sky view index (SVI), paving index (PI), spatial enclosure (SE), and water index (WI). Using the hill climbing algorithm in R to construct a Bayesian network, model validation results indicated prediction accuracies of 0.799 for TS and 0.838 for TP. The results showed that: (1) SE, WI, and SV significantly positively influence TS, while L significantly negatively influences TS (R2 = 0.6805, p-value &lt; 0.05); (2) WI, TS, and SV significantly positively influence TP (R2 = 0.759, p-value &lt; 0.05).

The influence of neighbor selection on self-organized UAV swarm based on finite perception vision.

Recently, vision-based unmanned aerial vehicle (UAV) swarming has emerged as a promising alternative that can overcome the adaptability and scalability limitations of distributed and communication-based UAV swarm systems. While most vision-based control algorithms are predicated on the detection of neighboring objects, they often overlook key perceptual factors such as visual occlusion and the impact of visual sensor limitations on swarm performance. To address the interaction problem of neighbor selection at the core of self-organizing UAV swarm control, a perceptually realistic finite perception visual (FPV) neighbor selection model is proposed, which is based on the lateral visual characteristics of birds, incorporates adjustable lateral visual field widths and orientations, and is able to ignore occluded agents. Based on the FPV model, a neighbor selection method based on the Acute Angle Test (AAT) is proposed,&#xD;which overcomes the limitation that the traditional neighbor selection mechanism can only interact with the nearest neighboring agents. A large number of Monte Carlo simulation comparison experiments show that the proposed FPV+AAT neighborselection mechanism can reduce the redundant communication burden between large scale self-organized UAV swarms, and outperforms the traditional neighbor selection method in terms of order, safety, union, connectivity, and noise resistance.

A Study on the Visual Characteristics of Children's Picture Book Covers from the Perspective of Peircean Semiotics -Focusing on Picture Books Themed on Death-

A Study on the Visual Characteristics of Children's Picture Book Covers from the Perspective of Peircean Semiotics -Focusing on Picture Books Themed on Death-

User emotion recognition and indoor space interaction design: a CNN model optimized by multimodal weighted networks

In interior interaction design, achieving intelligent user-interior interaction is contingent upon understanding the user’s emotional responses. Precise identification of the user’s visual emotions holds paramount importance. Current visual emotion recognition methods rely solely on singular features, predominantly facial expressions, resulting in inadequate coverage of visual characteristics and low recognition rates. This study introduces a deep learning-based multimodal weighting network model to address this challenge. The model initiates with a convolutional attention module, employing a self-attention mechanism within a convolutional neural network (CNN). As a result, the multimodal weighting network model is integrated to optimize weights during training. Finally, a weight network classifier is derived from these optimized weights to facilitate visual emotion recognition. Experimental outcomes reveal a 77.057% correctness rate and a 74.75% accuracy rate in visual emotion recognition. Comparative analysis against existing models demonstrates the superiority of the multimodal weight network model, showcasing its potential to enhance human-centric and intelligent indoor interaction design.

Using Deep Learning and Cbir To Address Copyright Concerns of AI-Generated Art: A Systematic Literature Review

This systematic literature review explores the intersection of deep learning and content-based image retrieval (CBIR) in addressing copyright concerns related to AI-generated art. As artificial intelligence rapidly transforms various artistic domains, it raises critical questions regarding authorship, ownership, and the ethical implications of machine-generated creativity. The review examines the capabilities of CBIR systems in identifying AI-generated images by analyzing visual features such as color, texture, and shape. Additionally, it highlights the role of deep learning models in enhancing the accuracy of these systems through the detection of distinctive patterns characteristic of AI artworks. The findings underscore the importance of developing robust methodologies that leverage AI and CBIR technologies to protect intellectual property rights while fostering innovation in the creative industries. This research contributes to the broader discourse on the legal and ethical challenges posed by AI in art, providing insights for policymakers, artists, and technologists in navigating the evolving landscape of AI-generated content.

Early neural development of social interaction perception: evidence from voxel-wise encoding in young children and adults.

From a young age, children have advanced social perceptual and reasoning abilities. However, the neural development of these abilities is still poorly understood. To address this gap, we used fMRI data collected 122 3-12-year-old children (64 females) and 33 adults (20 females) watched an engaging and socially rich movie to investigate how the cortical basis of social processing changes throughout development. We labeled the movie with visual and social features, including motion energy, presence of a face, presence of a social interaction, theory of mind (ToM) events, valence and arousal. Using a voxel-wise encoding model trained on these features, we find that models based on visual (motion energy) and social (faces, social interaction, ToM, valence, and arousal) features can both predict brain activity in children as young as three years old across the cortex, with particularly high predictivity in motion selective middle temporal region (MT) and the superior temporal sulcus (STS). Furthermore, models based on individual social features showed that while there may be some development throughout childhood, social interaction information in the STS is present in children as young as three years old and appears adult-like by age seven. The current study, for the first time, links neural activity in children to pre-defined social features in a narrative movie and suggests social interaction perception is supported by early developing neural responses in the STS.Significance Statement This study investigates the neural basis for social scene perception ability in children using fMRI data collected while participants watch a short, animated movie. Unlike most prior studies with movies, we labeled a range of visual and social features in the movie and used machine learning analyses to link each feature to fMRI responses in adults and children ages 3-12. Notably, our results demonstrate strong evidence that children as young as three years old show significant responses to most visual and social features in the movie, including social interaction responses in the superior temporal sulcus (STS), a region in the brain that is well known to be important in social interaction processing in adults.