Vision Augmentation Research Articles

Perception-Driven Soft-Edge Occlusion for Optical See-Through Head-Mounted Displays.

Systems with occlusion capabilities, such as those used in vision augmentation, image processing, and optical see-through head-mounted display (OST-HMD), have gained popularity. Achieving precise (hard-edge) occlusion in these systems is challenging, often requiring complex optical designs and bulky volumes. On the other hand, utilizing a single transparent liquid crystal display (LCD) is a simple approach to create occlusion masks. However, the generated mask will appear defocused (soft-edge) resulting in insufficient blocking or occlusion leakage. In our work, we delve into the perception of soft-edge occlusion by the human visual system and present a preference-based optimal expansion method that minimizes perceived occlusion leakage. In a user study involving 20 participants, we made a noteworthy observation that the human eye perceives a sharper edge blur of the occlusion mask when individuals see through it and gaze at a far distance, in contrast to the camera system's observation. Moreover, our study revealed significant individual differences in the perception of soft-edge masks in human vision when focusing. These differences may lead to varying degrees of demand for mask size among individuals. Our evaluation demonstrates that our method successfully accounts for individual differences and achieves optimal masking effects at arbitrary distances and pupil sizes.

CutFreq: Cut-and-Swap Frequency Components for Low-Level Vision Augmentation

Low-level vision plays a crucial role in a wide range of imaging quality and image recognition applications. However, the limited size, quality, and diversity of datasets often pose significant challenges for low-level tasks. Data augmentation is the most effective and practical way of sample expansion, but the commonly used augmentation methods in high-level tasks have limited improvement in the low-level due to the boundary effects or the non-realistic context information. In this paper, we propose the Cut-and-Swap Frequency Components (CutFreq) method for low-level vision, which aims to preserve high-level representations with directionality and improve image synthesis quality. Observing the significant frequency domain differences between reconstructed images and real ones, in CutFreq, we propose to transform the input and real images separately in the frequency domain, then define two stages for the model training process, and finally swap the specified frequency bands respectively and inversely transform to generate augmented samples. The experimental results show the superior performance of CutFreq on five low-level vision tasks. Moreover, we demonstrate the effectiveness of CutFreq in the low-data regime. Code is available at https://github.com/DreamerCCC/CutFreq.

MP52-01 ADOPTING MACHINE VISION AUGMENTATION TO DETECT DETRUSOR INSTABILITY IN OVERACTIVE BLADDER: A FRONTIER OF ARTIFICIAL INTELLIGENCE APPLICATION IN FUNCTIONAL UROLOGY

You have accessJournal of UrologyCME1 Apr 2023MP52-01 ADOPTING MACHINE VISION AUGMENTATION TO DETECT DETRUSOR INSTABILITY IN OVERACTIVE BLADDER: A FRONTIER OF ARTIFICIAL INTELLIGENCE APPLICATION IN FUNCTIONAL UROLOGY Yu Guang Tan, Shauna Woo, Jin Yong, and Lay Guat Ng Yu Guang TanYu Guang Tan More articles by this author , Shauna WooShauna Woo More articles by this author , Jin YongJin Yong More articles by this author , and Lay Guat NgLay Guat Ng More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003300.01AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Overactive bladder (OAB) is an increasingly prevalent urological condition. Routine urodynamics study (UDS) is invasive, inconsistent and simply impractical in every outpatient setting. We propose a novel machine vision augmentative model on cystoscopic images to identify segments of detrusor overactivity (DO) based on motion differences of the vascular network across time. METHODS: We prospectively collected 158 cystoscopy videos [38 previously UDS confirmed DO, 120 non-OAB] and extracted 30 seconds clip from each video, with an image capture rate of 24 frames/sec, amounting to >100,000 frames. The recordings were first denoised to remove artefacts and account for the cystoscopic movements, and have the resolutions enhanced with a set of Deep Learning techniques. Following which, the vascular network is segmented out and multiple key points are determined along the visualised blood vessels. An average of ∼300 key points were identified on each frame captured. Next, mosaic stitching is performed to reconstruct the extracted 2D images into a 3D bladder map to account for the geometric distortions. Finally, the motion differences of the key points across timeframes are tracked as a surrogate for areas of detrusor microcontraction. RESULTS: The structure-from-motion pipeline demonstrated satisfactory 3D reconstructions of the processed cystoscopy videos. The AI-based semantic image segmentation was trained to differentiate UDS confirmed DO from non-OAB patients with a coefficient score of 0.613. The motion differences could be tracked both quantitatively (number of key points) and qualitatively (extent of displacement of key points over time). Areas of exaggerated motion differences detected would represent increased focal detrusor contractions. CONCLUSIONS: This novel machine vision augmentation model yields promising results in identifying OAB patients based on cystoscopy. Potential identification of detrusor microcontraction points can aid targeted drug delivery to enhance treatment efficacy. Source of Funding: Not applicable © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e702 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Yu Guang Tan More articles by this author Shauna Woo More articles by this author Jin Yong More articles by this author Lay Guat Ng More articles by this author Expand All Advertisement PDF downloadLoading ...

Adopting machine vision augmentation to detect detrusor instability in overactive bladder: A frontier of artificial intelligence application in functional urology

Adopting machine vision augmentation to detect detrusor instability in overactive bladder: A frontier of artificial intelligence application in functional urology

A Survey of Deep Learning-Based Image Restoration Methods for Enhancing Situational Awareness at Disaster Sites: The Cases of Rain, Snow and Haze.

This survey article is concerned with the emergence of vision augmentation AI tools for enhancing the situational awareness of first responders (FRs) in rescue operations. More specifically, the article surveys three families of image restoration methods serving the purpose of vision augmentation under adverse weather conditions. These image restoration methods are: (a) deraining; (b) desnowing; (c) dehazing ones. The contribution of this article is a survey of the recent literature on these three problem families, focusing on the utilization of deep learning (DL) models and meeting the requirements of their application in rescue operations. A faceted taxonomy is introduced in past and recent literature including various DL architectures, loss functions and datasets. Although there are multiple surveys on recovering images degraded by natural phenomena, the literature lacks a comprehensive survey focused explicitly on assisting FRs. This paper aims to fill this gap by presenting existing methods in the literature, assessing their suitability for FR applications, and providing insights for future research directions.

VGTC Virtual Reality Technical Achievement Award

The 2022 IEEE VGTC Virtual Reality Technical Achievement Award goes to Kiyoshi Kiyokawa of the Nara Institute of Science and Technology, in recognition of his pioneering research contributions in the development of advanced head mounted display systems, vision augmentation and assistive interfaces, immersive modeling, collaborative virtual and augmented reality, seamless transitional interfaces, and multimodal interfaces.

Laparoscopic image enhancement based on distributed retinex optimization with refined information fusion

With intrinsic non-uniform illumination variations in laparoscopic images, such images often suffer from insufficient lighting and low visibility. This in turn may cause incorrect targeting, surgical risk, and extended operating time during laparoscopic surgery. Although various methods for nature image enhancement have been proposed in past decades, surgical laparoscopic image enhancement still needs improving due to issues including naturalness, blurred texture, color cast, and extensive computational time. To address these problems, this paper proposes a laparoscopic image enhancement method, based on distributed retinex optimization with refined information fusion. The proposed method minimizes a distributed optimization model with l2-norm regularization terms, resulting in a low computational complexity. By integrating the refined image information into the optimization process, our method significantly enhances dark regions while preserving naturalness and texture structures. Experimental results show that our optimization algorithm is more effective than conventional enhancement algorithms in terms of vision augmentation, performance index, and computation time.

Illuminated Focus: Vision Augmentation using Spatial Defocusing via Focal Sweep Eyeglasses and High-Speed Projector.

Aiming at realizing novel vision augmentation experiences, this paper proposes the IlluminatedFocus technique, which spatially defocuses real-world appearances regardless of the distance from the user's eyes to observed real objects. With the proposed technique, a part of a real object in an image appears blurred, while the fine details of the other part at the same distance remain visible. We apply Electrically Focus-Tunable Lenses (ETL) as eyeglasses and a synchronized high-speed projector as illumination for a real scene. We periodically modulate the focal lengths of the glasses (focal sweep) at more than 60 Hz so that a wearer cannot perceive the modulation. A part of the scene to appear focused is illuminated by the projector when it is in focus of the user's eyes, while another part to appear blurred is illuminated when it is out of the focus. As the basis of our spatial focus control, we build mathematical models to predict the range of distance from the ETL within which real objects become blurred on the retina of a user. Based on the blur range, we discuss a design guideline for effective illumination timing and focal sweep range. We also model the apparent size of a real scene altered by the focal length modulation. This leads to an undesirable visible seam between focused and blurred areas. We solve this unique problem by gradually blending the two areas. Finally, we demonstrate the feasibility of our proposal by implementing various vision augmentation applications.

Image Quality in Augmented Binocular Vision: QoE Approximations with QoS and EEG

Augmented reality (AR) applications become increasingly popular, however, little is known about how multimedia consumption interplays with the Quality of Experience (QoE) in these settings. We experimentally evaluate binocular vision augmentation with optical see-through devices by contrasting QoE, expressed by Mean Opinion Scores (MOS), with a ground truth reference data set. We find that the QoE in AR settings (i) is higher for small media impairments and lower for high impairment levels compared to opaque settings, (ii) exhibits an emerging quantifiable relationship with the QoE in traditional setups, (iii) can be approached with common objective image quality metrics as Quality of Service (QoS) factors, and (iv) exhibits a relationship between the Low Gamma frequency band levels determined with consumer-grade electroencephalograms (EEG) and image quality levels as rated by the participants.