Visual Ambiguity Research Articles

Close-range photogrammetry for analysis of rock relief details: An investigation of symbols purported to be Jewish Menorahs in Rough Cilicia

Close-range photogrammetry of certain rock reliefs in Rough Cilicia was used to investigate published claims that symbols in them represent Jewish menorahs, but with five branches instead of the usual seven. Details of ancient rock reliefs are difficult to assess because of mottled rock texture, color variation, wear or damage, and conditions of lighting. Thus, published photos of the rock reliefs in question are insufficient for evaluation of claims about them. Simple photogrammetry, however, produces 3D and digital elevation models that eliminate visual ambiguity and provide certainty of detail for analysis. This study describes the simple, non-invasive, and inexpensive data collection technique and the photogrammetry software processing workflow for creation of such models. Resulting models in various presentation formats provide factual data for reevaluation of the claimed menorahs themselves and their relationship to other elements of the reliefs. These findings contribute to informed discussion on interpretation of the symbols and their meaning. Adoption of the process described here is encouraged for subsequent publication, analysis, and interpretation of rock relief details in the region.

Long-short diffeomorphism memory network for weakly-supervised ultrasound landmark tracking

Ultrasound is a promising medical imaging modality benefiting from low-cost and real-time acquisition. Accurate tracking of an anatomical landmark has been of high interest for various clinical workflows such as minimally invasive surgery and ultrasound-guided radiation therapy. However, tracking an anatomical landmark accurately in ultrasound video is very challenging, due to landmark deformation, visual ambiguity and partial observation. In this paper, we propose a long-short diffeomorphism memory network (LSDM), which is a multi-task framework with an auxiliary learnable deformation prior to supporting accurate landmark tracking. Specifically, we design a novel diffeomorphic representation, which contains both long and short temporal information stored in separate memory banks for delineating motion margins and reducing cumulative errors. We further propose an expectation maximization memory alignment (EMMA) algorithm to iteratively optimize both the long and short deformation memory, updating the memory queue for mitigating local anatomical ambiguity. The proposed multi-task system can be trained in a weakly-supervised manner, which only requires few landmark annotations for tracking and zero annotation for deformation learning. We conduct extensive experiments on both public and private ultrasound landmark tracking datasets. Experimental results show that LSDM can achieve better or competitive landmark tracking performance with a strong generalization capability across different scanner types and different ultrasound modalities, compared with other state-of-the-art methods.

Integrated visual transformer and flash attention for lip-to-speech generation GAN

Lip-to-Speech (LTS) generation is an emerging technology that is highly visible, widely supported, and rapidly evolving. LTS has a wide range of promising applications, including assisting speech impairment and improving speech interaction in virtual assistants and robots. However, the technique faces the following challenges: (1) Chinese lip-to-speech generation is poorly recognized. (2) The wide range of variation in lip-speaking is poorly aligned with lip movements. Addressing these challenges will contribute to advancing Lip-to-Speech (LTS) technology, enhancing the communication abilities, and improving the quality of life for individuals with disabilities. Currently, lip-to-speech generation techniques usually employ the GAN architecture but suffer from the following problems: The primary issue lies in the insufficient joint modeling of local and global lip movements, resulting in visual ambiguities and inadequate image representations. To solve these problems, we design Flash Attention GAN (FA-GAN) with the following features: (1) Vision and audio are separately coded, and lip motion is jointly modelled to improve speech recognition accuracy. (2) A multilevel Swin-transformer is introduced to improve image representation. (3) A hierarchical iterative generator is introduced to improve speech generation. (4) A flash attention mechanism is introduced to improve computational efficiency. Many experiments have indicated that FA-GAN can recognize Chinese and English datasets better than existing architectures, especially the recognition error rate of Chinese, which is only 43.19%, the lowest among the same type.

A Loadability Study of 30 kV Misurata Distribution Network

Due to the expansion of power distribution networks, the power delivered to the loads might get through corridors which limits their capability of transmitting the power. This concept is termed as the loadability. An unplanned increase in power consumption in the Libyan network, especially after 2011, led to major problems at different levels of the power network. For instance, these issues are addressed at the generation level as power shortages and load shedding while at the distribution level presented as under voltages and loadability issues. In this paper, a study of the loadability of 30 kV Misurata electrical distribution network (MDN) is carried out using the NEPLAN software. Even though the MDN is considered one of the largest distribution networks in Libya, a lack of data is found which makes an ambiguous vision about this network and the necessary developments required in the future as well as adds some difficulties to making studies. The work presents the steady state operation of MDN under the existence of the current and estimated loads up to the year 2026. The results show significant loading problems in Misurata distribution network which should be considered to avoid the anticipated future power transfer limitations. Some visible solutions are proposed toensure the optimal operation of the distribution network.

TroublingGAN: generated visual ambiguity as a speculative alternative to photojournalism

This exposition documents artistic research that engages with generative neural networks and artificial intelligence-driven visual synthesis, the goal being to challenge the limits of the research and question the value of the generated visual outcomes. We present here our experiment with a customised StyleGAN model. In contrast to its utilisation by computer scientists, it has been trained on a heterogeneous dataset, voluntarily exposing the generative neural network to failure while focusing on the unexpected moments of surprise that arise from such a process. Led by the critical thoughts of The Nooscope Manifested (Pasquinelli and Joler 2020), this experiment questions what kind of knowledge generative neural networks produce, whether they could change one’s perspective on studied objects (the dataset) and whether they could function as a viable tool for artistic research. The observed object in this project is the concept of ‘troubling times’ (inspired by ‘Designing in Troubling Times’, the theme of the 2021 Uroboros Festival) — rapid socio-ecological changes caused by shifts in global economic, political and technological power and the subsequent series of troubling events, including the COVID-19 pandemic, violent conflicts and environmental catastrophes. StyleGAN is used as a pattern-recognition and knowledge-production tool to create an intuitive understanding of what this ambiguous term ‘troubling times’ actually means today. During the experimentation process, multiple unforeseen moments and twists happened, offering valuable insights into the nature of synthetic media, generating ethical questions concerning the use of generative neural networks and spawning new propositions for further research.

Hybrid-MVS: Robust Multi-View Reconstruction With Hybrid Optimization of Visual and Depth Cues

Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Distance-based Weighted Transformer Network for image completion

The challenge of image generation has been effectively modeled as a problem of structure priors or transformation. However, existing models have unsatisfactory performance in understanding the global input image structures because of particular inherent features (for example, local inductive prior). Recent studies have shown that self-attention is an efficient modeling technique for image completion problems. In this paper, we propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image’s components. In our model, we leverage the strengths of both Convolutional Neural Networks (CNNs) and DWT blocks to enhance the image completion process. Specifically, CNNs are used to augment the local texture information of coarse priors and DWT blocks are used to recover certain coarse textures and coherent visual structures. Unlike current approaches that generally use CNNs to create feature maps, we use the DWT to encode global dependencies and compute distance-based weighted feature maps, which substantially minimizes the problem of visual ambiguities. Meanwhile, to better produce repeated textures, we introduce Residual Fast Fourier Convolution (Res-FFC) blocks to combine the encoder’s skip features with the coarse features provided by our generator. Furthermore, a simple yet effective technique is proposed to normalize the non-zero values of convolutions, and fine-tune the network layers for regularization of the gradient norms to provide an efficient training stabilizer. Extensive quantitative and qualitative experiments on three challenging datasets demonstrate the superiority of our proposed model compared to existing approaches.

Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation

Medical image segmentation modeling is a high-stakes task where understanding of uncertainty is crucial for addressing visual ambiguity. Prior work has developed segmentation models utilizing probabilistic or generative mechanisms to infer uncertainty from labels where annotators draw a singular boundary. However, as these annotations cannot represent an individual annotator's uncertainty, models trained on them produce uncertainty maps that are difficult to interpret. We propose a novel segmentation representation, Confidence Contours, which uses high- and low-confidence ``contours’’ to capture uncertainty directly, and develop a novel annotation system for collecting contours. We conduct an evaluation on the Lung Image Dataset Consortium (LIDC) and a synthetic dataset. From an annotation study with 30 participants, results show that Confidence Contours provide high representative capacity without considerably higher annotator effort. We also find that general-purpose segmentation models can learn Confidence Contours at the same performance level as standard singular annotations. Finally, from interviews with 5 medical experts, we find that Confidence Contour maps are more interpretable than Bayesian maps due to representation of structural uncertainty.

Challenges Affecting Effective Application of Strategic Management Techniques in the Process of Implementing Organisational Changes in Zambia Correctional Service

Strategic management is a management tool that both public and private sector entities adopt to improve organizational performance especially in the process of implementing organizational changes. Zambia Correctional Service in particular has been undergoing a paradigm shift in order to implement new security sector reforms. Like many other public entities world over has continuously recorded failure in achieving strategic objectives, as noted by cabinet Office’s 2020 performance assessment. Therefore, this study sought to identify challenges that affect effective application of strategic management techniques in the process of implementing organisational changes. This research is a literature review that gathered facts from literature findings of a scientific publications and existing data in the study area of the topic under review. Research findings indicate that the Service has been experiencing a number of challenges and these include; having an ambiguous vision statement, operating on an outdated organisational structure, inadequate end-user engagement and ineffective examination and modification of work processes. The findings show that the Service needs to develop a model for application of strategic management techniques that will support implementation of organisational changes. The model should include element such as establishing a clear end goal (vision), revise the structure to suit the demands of the current strategic plan, conduct program impact assessments to effectively measure performance and examine and modify work processes for effective and efficient public resource management.

SelectAugment: Hierarchical Deterministic Sample Selection for Data Augmentation

Data augmentation (DA) has been extensively studied to facilitate model optimization in many tasks. Prior DA works focus on designing augmentation operations themselves, while leaving selecting suitable samples for augmentation out of consideration. This might incur visual ambiguities and further induce training biases. In this paper, we propose an effective approach, dubbed SelectAugment, to select samples for augmentation in a deterministic and online manner based on the sample contents and the network training status. To facilitate the policy learning, in each batch, we exploit the hierarchy of this task by first determining the augmentation ratio and then deciding whether to augment each training sample under this ratio. We model this process as two-step decision-making and adopt Hierarchical Reinforcement Learning (HRL) to learn the selection policy. In this way, the negative effects of the randomness in selecting samples to augment can be effectively alleviated and the effectiveness of DA is improved. Extensive experiments demonstrate that our proposed SelectAugment significantly improves various off-the-shelf DA methods on image classification and fine-grained image recognition.

Semantic segmentation of mobile mapping point clouds via multi-view label transfer

We study how to learn semantic segmentation of 3D point clouds from small training sets. The problem arises because annotating 3D point clouds is a lot more time-consuming and error-prone than annotating 2D images. On the one hand this means that one cannot afford to create a large enough training dataset for each new project. On the other hand it also means that there is not nearly as much public data available as there is for images, which one could use to pretrain a generic feature extractor that could then, with only little dedicated training data, be adapted (“fine-tuned”) to the task at hand. To address this bottleneck we explore the possibility to transfer knowledge from the 2D image domain to 3D point clouds. That strategy is of particular interest for mobile mapping systems that capture both point clouds and images, in a fully calibrated setting that makes it easy to connect the two domains. We find that, as expected, naively segmenting in image space and mapping the resulting labels onto the point cloud is not sufficient, as visual ambiguities, residual calibration errors, etc. affect the result. Instead, we propose a system that learns to merge image evidence from a varying number viewpoint, and 3D geometry information, into a common representation that encodes point-wise 3D semantics. To validate our approach we make use of a new mobile mapping dataset with 88M annotated 3D points and 2205 oriented multi-view images. In a series of experiments, we show how much label noise is caused by simplistic label transfer, and how well existing semantic segmentation architectures can correct it. Finally, we demonstrate that adding our learned 2D-to-3D multi-view label transfer significantly improves the performance of different segmentation backbones.

Comparing Speaker Adaptation Methods for Visual Speech Recognition for Continuous Spanish

Visual speech recognition (VSR) is a challenging task that aims to interpret speech based solely on lip movements. However, although remarkable results have recently been reached in the field, this task remains an open research problem due to different challenges, such as visual ambiguities, the intra-personal variability among speakers, and the complex modeling of silence. Nonetheless, these challenges can be alleviated when the task is approached from a speaker-dependent perspective. Our work focuses on the adaptation of end-to-end VSR systems to a specific speaker. Hence, we propose two different adaptation methods based on the conventional fine-tuning technique, the so-called Adapters. We conduct a comparative study in terms of performance while considering different deployment aspects such as training time and storage cost. Results on the Spanish LIP-RTVE database show that both methods are able to obtain recognition rates comparable to the state of the art, even when only a limited amount of training data is available. Although it incurs a deterioration in performance, the Adapters-based method presents a more scalable and efficient solution, significantly reducing the training time and storage cost by up to 80%.

The McGurk Illusion: A Default Mechanism of the Auditory System

Recent studies have questioned past conclusions regarding the mechanisms of the McGurk illusion, especially how McGurk susceptibility might inform our understanding of audiovisual (AV) integration. We previously proposed that the McGurk illusion is likely attributable to a default mechanism, whereby either the visual system, auditory system, or both default to specific phonemes-those implicated in the McGurk illusion. We hypothesized that the default mechanism occurs because visual stimuli with an indiscernible place of articulation (like those traditionally used in the McGurk illusion) lead to an ambiguous perceptual environment and thus a failure in AV integration. In the current study, we tested the default hypothesis as it pertains to the auditory system. Participants performed two tasks. One task was a typical McGurk illusion task, in which individuals listened to auditory-/ba/ paired with visual-/ga/ and judged what they heard. The second task was an auditory-only task, in which individuals transcribed trisyllabic words with a phoneme replaced by silence. We found that individuals' transcription of missing phonemes often defaulted to '/d/t/th/', the same phonemes often experienced during the McGurk illusion. Importantly, individuals' default rate was positively correlated with their McGurk rate. We conclude that the McGurk illusion arises when people fail to integrate visual percepts with auditory percepts, due to visual ambiguity, thus leading the auditory system to default to phonemes often implicated in the McGurk illusion.

Review on deep learning based techniques for person re-identification

In-depth study has recently been concentrated on human re-identification, which is a crucial component of automated video surveillance. Re-identification is the act of identifying someone in photos or videos acquired from other cameras after they have already been recognized in an image or video from one camera. Re-identification, which involves generating consistent labelling between several cameras, or even just one camera, is required to reconnect missing or interrupted tracks. In addition to surveillance, it may be used in forensics, multimedia, and robotics.Re-identification of the person is a difficult problem since their look fluctuates across many cameras with visual ambiguity and spatiotemporal uncertainty. These issues can be largely caused by inadequate video feeds or lowresolution photos that are full of unnecessary facts and prevent re-identification. The geographical or temporal restrictions of the challenge are difficult to capture. The computer vision research community has given the problem a lot of attention because of how widely used and valuable it is. In this article, we look at the issue of human re-identification and discuss some viable approaches.

Will she or won’t she? The ambivalence of female musicianship in two paintings by Bernardino Licinio (1489–1565)

Abstract This article examines the depiction of sexual propo­sition, the corruption of female virtue and the ambivalent allurements of secular music in a pair of 16th-century Concert paintings by the Venetian artist Bernardino Licinio. It argues that Licinio’s multi-figure concerts, centred on the music-making of young women, both parody and invert the classical elegiac theme of the pauper amans, the impoverished poet-lover who condemns the power of wealth and bestows poetry instead of gold on his beloved, only to be rejected due to her vanity and materialism. The pauper amans theme appears in several 16th-century Venetian strambotti and early madrigals, which are decidedly satiric in tone. In his paintings, Licinio reverses the familiar roles of poet and beloved, and the refined music of the virtuous-seeming young woman is challenged by the dissonant sound of her older male admirer shaking a purse full of coins or by his bold physical advances, which impede her playing. The dissonant and crude gesture of offering her coin payment equates the young woman’s art of music with her sexuality. It alleges not only her corruptibility and low status as a piece of merchandise for sale, but also debases the spiritually elevating art of music, with its noble and abstract figurations of love. The motif of the shaken purse with its ‘music of gold’ first appears in the Venetian dialect comedy La buelsca (c.1514), and is repeated in a series of later songs and poems centred on the subject of the grasping, mercenary prostitute in the 1520s and 30s. Licinio’s paintings play with such popular literary and musical themes but ultimately prefer visual ambiguity and unresolved narrative tension to their frankness and specificity.

WaSR-A Water Segmentation and Refinement Maritime Obstacle Detection Network.

Obstacle detection using semantic segmentation has become an established approach in autonomous vehicles. However, existing segmentation methods, primarily developed for ground vehicles, are inadequate in an aquatic environment as they produce many false positive (FP) detections in the presence of water reflections and wakes. We propose a novel deep encoder-decoder architecture, a water segmentation and refinement (WaSR) network, specifically designed for the marine environment to address these issues. A deep encoder based on ResNet101 with atrous convolutions enables the extraction of rich visual features, while a novel decoder gradually fuses them with inertial information from the inertial measurement unit (IMU). The inertial information greatly improves the segmentation accuracy of the water component in the presence of visual ambiguities, such as fog on the horizon. Furthermore, a novel loss function for semantic separation is proposed to enforce the separation of different semantic components to increase the robustness of the segmentation. We investigate different loss variants and observe a significant reduction in FPs and an increase in true positives (TPs). Experimental results show that WaSR outperforms the current state of the art by approximately 4% in F1 score on a challenging unmanned surface vehicle dataset. WaSR shows remarkable generalization capabilities and outperforms the state of the art by over 24% in F1 score on a strict domain generalization experiment.

Effectiveness of 2020 Airborne Lidar for Identifying Archaeological Sites and Features on Guåhan (Guam)

Guåhan (Guam), part of the Mariana Islands in the western Pacific, has an archaeological record spanning almost four millennia. In this seminal academic paper on the use of lidar for archaeological investigation on Guåhan, we determined which known sites can be visually detected to explore the effectiveness of lidar in this context. Several archaeological site types and features from the Latte, Spanish, and Modern Periods were examined using high-resolution 2020 airborne lidar. We generated the most commonly used digital elevation models (DEMs), which are digital terrain models (DTMs) and digital surface models (DSMs), and visualised them as a hillshade. Lidar proved effective for identifying all the Modern Period sites and most of the Latte and Spanish Period sites, although some features remained obscure. Major challenges and limitations of using lidar on Guåhan were vegetation, site size, and visual ambiguities. While this study used conventional lidar derivatives, it exposed the potential to find more archaeological sites by applying more complex lidar processing techniques in the future. The benefits of lidar are of great interest to the local community, especially the Indigenous CHamoru, interested in non-destructive ways to assist their cultural heritage management.

Processing visual ambiguity in fractal patterns: Pareidolia as a sign of creativity

SummaryCreativity is a highly valued and beneficial skill that empirical research typically probes using “divergent thinking” (DT) tasks such as problem solving and novel idea generation. Here, in contrast, we examine the perceptual aspect of creativity by asking whether creative individuals are more likely to perceive recognizable forms in ambiguous stimuli –a phenomenon known as pareidolia. To this end, we designed a visual task in which participants were asked to identify as many recognizable forms as possible in cloud-like fractal images. We found that pareidolic perceptions arise more often and more rapidly in highly creative individuals. Furthermore, high-creatives report pareidolia across a broader range of image contrasts and fractal dimensions than do low creatives. These results extend the established body of work on DT by introducing divergent perception as a complementary manifestation of the creative mind, thus clarifying the perception-creation link while opening new paths for studying creative behavior in humans.

Automatic vocabulary and graph verification for accurate loop closure detection

AbstractLocalizing previously visited places during long‐term localization and mapping, that is, loop closure detection (LCD), is a crucial technique to correct accumulated inconsistencies. In common bag‐of‐words (BoW) model, a visual vocabulary is built to associate features for detecting loops. Currently, methods that build vocabularies off‐line determine scales of the vocabulary by trial‐and‐error, which results in unreasonable feature association. Moreover, the detection precision of the algorithm declines due to perceptual aliasing given that the BoW‐based method ignores the positions of visual features. To build the optimal vocabulary automatically and eliminate human heuristics, we propose a natural convergence criterion based on the comparison between the radii of nodes and the drifts of feature descriptors in vocabulary construction. Furthermore, a novel topological graph verification method is proposed for validating loop candidates, which can effectively distinguish visual ambiguities by involving geometrical position of features and thus improve the precision of LCD. Experiments on various public datasets verify the effectiveness of our proposed approach.

A sparse graph wavelet convolution neural network for video-based person re-identification

Video-based person re-identification (Re-ID) aims to match identical person sequences captured across non-overlapping surveillance areas. It is an essential yet challenging task to effectively embed spatial and temporal information into the video feature representation. For one thing, we observe that different frames in the video can provide complementary information for each other. Also, local features which is lost due to target occlusion or visual ambiguity in one frame can be supplemented by the same pedestrian part in other frames. For another thing, graph neural network enables the contextual interactions between relevant regional features. Therefore, we propose a novel sparse graph wavelet convolution neural network (SGWCNN) for video-based person Re-ID. Distinct from previous graph-based Re-ID methods, we exploit the weighted sparse graph to model the semantic relation among the local patches of pedestrians in the video. Each local patch in one frame can extract supplementary information from highly related patches in other frames. Moreover, to effectively solve the problems of short time occlusion and pedestrian misalignment, the graph wavelet convolution neural network is adopted for feature propagation to refine regional features iteratively. Experiments and evaluation on three challenging benchmarks, that is, MARS, DukeMTMC-VideoReID, and iLIDS-VID, show that the proposed SGWCNN effectively improves the performance of video-based person re-identification.