Using direct visual augmentation to provide people with low vision equal access to information

Low vision is a visual impairment that falls short of blindness but cannot be corrected by eyeglasses or contact lenses. While current low vision aids (e.g., magnifier, CCTV) support basic vision enhancements, such as magnification and contrast enhancement, these enhancements often arbitrarily alter a user's full field of view without considering the user's context, such as their visual abilities, tasks, and environmental factors. As a result, these low vision aids are not sufficient or preferred by low vision users in many important tasks. Augmented reality (AR) technology presents a unique opportunity to enhance low vision people's visual experience by automatically recognizing the surrounding environment and presenting tailored visual augmentations. In this talk, I will talk about how we design and build intelligent AR systems to support low vision people in visual tasks, such as a head-mounted AR system that presents visual cues to orient users' attention in a visual search task, as well as a projection-based AR system that projects visual highlights on the stair edges to support safe stair navigation. I will conclude my talk by discussing our future research direction on AR for low vision accessibility.

While our community has many active projects involving blind people, low vision is rarely addressed. People with low vision have functional vision, but their visual impairment adversely affects their daily life and it cannot be corrected with glasses or contact lenses. Over the last few years, we have been conducting research with this understudied demographic: understanding low vision people's needs and designing applications to address the challenges they face. In this article, we discuss our ongoing research in this area, focusing on designing augmented reality applications for low vision users. We begin this article by describing low vision and motivating our focus on augmented reality applications on smartglasses for low vision people. We then provide overviews of three research projects that exemplify our research agenda: a study where we observed low vision people conducting a navigation and shopping task, a study where we examined low vision people's perception of virtual text and shapes on smartglasses, and the design of a smartglasses application that facilitates a visual search task.

People with low vision have a visual impairment that affects their ability to perform daily activities. Unlike blind people, low vision people have functional vision and can potentially benefit from smart glasses that provide dynamic, always-available visual information. We sought to determine what low vision people could see on mainstream commercial augmented reality (AR) glasses, despite their visual limitations and the device's constraints. We conducted a study with 20 low vision participants and 18 sighted controls, asking them to identify virtual shapes and text in different sizes, colors, and thicknesses. We also evaluated their ability to see the virtual elements while walking. We found that low vision participants were able to identify basic shapes and read short phrases on the glasses while sitting and walking. Identifying virtual elements had a similar effect on low vision and sighted people's walking speed, slowing it down slightly. Our study yielded preliminary evidence that mainstream AR glasses can be powerful accessibility tools. We derive guidelines for presenting visual output for low vision people and discuss opportunities for accessibility applications on this platform.

Cooking is a vital yet challenging activity for blind and low vision (BLV) people, which involves many visual tasks that can be difficult and dangerous. BLV training services, such as vision rehabilitation, can effectively improve BLV people’s independence and quality of life in daily tasks, such as cooking. However, there is a lack of understanding on the practices employed by the training professionals and the barriers faced by BLV people in such training. To fill the gap, we interviewed six professionals to explore their training strategies and technology recommendations for BLV clients in cooking activities. Our findings revealed the fundamental principles, practices, and barriers in current BLV training services, identifying the gaps between training and reality.

Blind and low vision people use visual description services (VDS) to gain visual interpretation and build access in a world that privileges sight. Despite their many benefits, VDS have many harmful privacy and security implications. As a result, researchers are suggesting, exploring, and building obfuscation systems that detect and obscure private or sensitive materials. However, as obfuscation depends largely on sight to interpret outcomes, it is unknown whether Blind and low vision people would find such approaches useful. Our work aims to center the perspectives and opinions of Blind and low vision people on the potential of obfuscation to address privacy concerns in VDS. By reporting on interviews with 20 Blind and low vision people who use VDS, our findings reveal that popular research trends in obfuscation fail to capture the needs of Blind and low vision people. While obfuscation might be helpful in gaining more control, tensions around obfuscation misrecognition and confirmation are prominent. We turn to the framework of interdependence to unpack and understand obfuscation in VDS, enabling us to complicate privacy concerns, uncover the labor of Blind and low vision people, and emphasize the importance of safeguards. We provide design directions to move the trajectory of obfuscation research forward.

Visual impairments encompass a range of visual abilities. People with low vision have functional vision and thus their experiences are likely to be different from people with no vision. We sought to answer two research questions: (1) what challenges do low vision people face when performing daily activities and (2) what aids (high- and low-tech) do low vision people use to alleviate these challenges? Our goal was to reveal gaps in current technologies that can be addressed by the UbiComp community. Using contextual inquiry, we observed 11 low vision people perform a wayfinding and shopping task in an unfamiliar environment. The task involved wayfinding and searching and purchasing a product. We found that, although there are low vision aids on the market, participants mostly used their smartphones, despite interface accessibility challenges. While smartphones helped them outdoors, participants were overwhelmed and frustrated when shopping in a store. We discuss the inadequacies of existing aids and highlight the need for systems that enhance visual information, rather than convert it to audio or tactile.

Most low vision people have functional vision and would likely prefer to use their vision to access information. Recently, there have been advances in head-mounted displays, cameras, and image processing technology that create opportunities to improve the visual experience for low vision people. In this paper, we present ForeSee, a head-mounted vision enhancement system with five enhancement methods: Magnification, Contrast Enhancement, Edge Enhancement, Black/White Reversal, and Text Extraction; in two display modes: Full and Window. ForeSee enables users to customize their visual experience by selecting, adjusting, and combining different enhancement methods and display modes in real time. We evaluated ForeSee by conducting a study with 19 low vision participants who performed near- and far-distance viewing tasks. We found that participants had different preferences for enhancement methods and display modes when performing different tasks. The Magnification Enhancement Method and the Window Display Mode were popular choices, but most participants felt that combining several methods produced the best results. The ability to customize the system was key to enabling people with a variety of different vision abilities to improve their visual experience.

Background: Approximately 250,000 visually impaired people were registered in Korea in 2021, and eye diseases are often accompanied by various systemic diseases. This study aims to investigate the epidemiology, causative disease, and rehabilitation treatment for visually impaired and low-vision patients.Current Concepts: Visual impairment ranges from blindness in one eye to total blindness. Low vision refers to the visual acuity of 0.3 to 0.05 in the better eye, and the visual field is narrowed to less than 10 degrees. The common causes of visual impairment eye disorders include macular degeneration, glaucoma, diabetic maculopathy, retinitis pigmentosa, and congenital diseases. Among them, macular degeneration has been increasing significantly in recent years. If the disability is not changed, it is difficult to improve vision through treatment. The aim of treatment should be to retain the remaining visual function through the low vision aids for the visually impaired person.Discussion and Conclusion: Although various visual aids are being distributed, medical insurance coverage does not align with the need for improvement. Like other disabilities, multidisciplinary approaches to the visually impaired people are necessary. In addition, awareness of low vision should be improved so that there are no visually impaired and low vision people who are placed in a blind spot and suffer inconvenience.

One-year follow-up is recommended for patients with macular diseases to assess functional changes associated with disease progression and to modify low-vision (LV) treatment plans, if indicated. The purpose of this study was to observe 255 patients with macular diseases who received LV rehabilitation (rehabilitation with a therapist) or basic LV services (LV devices dispensed without therapy) during Veterans Affairs Low-vision Intervention Trial II after the trial ended at 4 months until 1-year follow-up. The primary outcome measure was visual ability measured with the 48-item Veterans Affairs Low-vision Visual Functioning Questionnaire. Mean visual ability scores for the treatment groups were compared from baseline to 4 months, 4 months to 1 year, and baseline to 1 year. Changes from baseline to 1 year were compared between the two groups. Predictors of changes in visual ability from 4 months to 1 year were assessed using linear regression. Both groups experienced significant improvement in all measures of visual ability from baseline to 1 year but lost visual reading ability during the observation period (LV rehabilitation group, -0.64 [1.2] logit; 95% confidence interval [CI], -0.84 to -0.44 logit; basic LV group, -0.63 [1.4] logit; 95% CI, -0.88 to -0.38 logit), and overall visual ability was lost in the LV rehabilitation group (-0.20 [0.8] logit; 95% CI, -0.34 to -0.06 logit). Loss of visual reading ability in both groups from 4 months to 1 year was predicted by reading ability scores at 4 months, loss of near visual acuity from 4 months to 1 year, and lower EuroQol-5D utility index scores; loss of overall visual ability in the LV rehabilitation group during the same time period was predicted by lower overall ability scores at 4 months. Visual ability significantly improved in all groups from baseline to 1 year. However, the loss of visual reading ability experienced by both groups from 4 months to 1 year reduced the benefit of the services provided.

Gait is an individual's walking pattern, and it is a significant part of daily living activities. Quantitative gait assessments, like spatiotemporal parameters (STPs), are related to the functional conditions to provide useful information. This study reviewed the comprehensive differences in spatiotemporal gait variability measures between visually impaired people and the sighted. The search strategy was performed in three databases (PubMed/MEDLINE, Web of Science, and Scopus) from the start date to October 2022, and the utilized keywords for this search are related to gait and blindness. This review considered only those studies that evaluated gait parameters in people with visual impairment and blind people without any limitations in age and gender. In this review, studies without a control group (sighted people) were excluded. The Newcastle-Ottawa Scale (NOS) was applied for critical appraisal. Six full manuscripts were included. The sample size ranged from 19 to 91. The mean modified NOS critical appraisal scores for cross-sectional studies were 6.0. In these studies, among nine STPs: stride length, walking speed, stance and swing phase, step width, cadence, step length, double support, and single support, at least five and at most seven factors were examined. The gait pattern of blind and low-vision people is characterized by a slower walking speed, shorter stride length, increased step width, decreased cadence, prolonged duration of double support, and reduced single support compared to the controls.

Visuomotor integration deficits precede clinical onset in Huntington's disease

Self-service terminals (SSTs) are almost everywhere in our daily life and increasingly use capacitive and infrared touchscreens as the interface. Most of the current solutions to help blind and low vision (BLV) people access existing touchscreens mostly are only suitable for capacitive touchscreens and not for infrared touchscreens. In this paper, we proposed a voice-based interactive method using a conductive folding stand with the phone camera to allow BLV people to access both touchscreens of SSTs. Voice feedback was provided to guide users to move the phone close to the button and touch it with the end of the unfolded stand. Using a portable accessory, this method directly guided users to touch the target and effectively avoids false triggering. A preliminary evaluation indicated that our approach enabled users to access the target buttons on the touchscreen with high accuracy and a short completion time.

Visual search is a major challenge for low vision people. Conventional vision enhancements like magnification help low vision people see more details, but cannot indicate the location of a target in a visual search task. In this paper, we explore visual cues---a new approach to facilitate visual search tasks for low vision people. We focus on product search and present CueSee, an augmented reality application on a head-mounted display (HMD) that facilitates product search by recognizing the product automatically and using visual cues to direct the user's attention to the product. We designed five visual cues that users can combine to suit their visual condition. We evaluated the visual cues with 12 low vision participants and found that participants preferred using our cues to conventional enhancements for product search. We also found that CueSee outperformed participants' best-corrected vision in both time and accuracy.

Most patients with low vision are elderly and have functional limitations from other health problems that could add to the functional limitations caused by their visual impairments. To identify factors that contribute to visual ability measures in patients who present for outpatient low vision rehabilitation (LVR) services. As part of a prospective, observational study of new patients seeking outpatient LVR, 779 patients from 28 clinical centers in the United States were enrolled in the Low Vision Rehabilitation Outcomes Study (LVROS) from April 25, 2008, through May 2, 2011. The Activity Inventory (AI), an adaptive visual function questionnaire, was administered to measure overall visual ability and visual ability in 4 functional domains (reading, mobility, visual motor function, and visual information processing) at baseline before LVR. The Geriatric Depression Scale, Telephone Interview for Cognitive Status, and Medical Outcomes Study 36-Item Short-Form Health Survey physical functioning questionnaires were also administered to measure patients' psychological, cognitive, and physical health states, respectively. Predictors of visual ability and functional domains as measured by the AI. Among the 779 patients in the LVROS sample, the mean age was 76.4 years, 33% were male, and the median logMAR visual acuity score was 0.60 (0.40-0.90 interquartile range). Correlations were observed between logMAR visual acuity and baseline visual ability overall (r = -0.42) and for all functional domains. Visual acuity was the strongest predictor of visual ability (P < .001) and reading ability (P < .001) and had a significant independent effect on the other functional domains. Physical ability was independently associated with (P < .001) overall visual ability as well as mobility and visual motor function. Depression had a consistent independent effect (P < .001) on overall visual ability and on all functional domains, whereas cognition had an effect on only reading and mobility (P < .001). Visual ability is a multidimensional construct, with visual acuity, depression, physical ability, and cognition explaining more than one-third of the variance in visual ability as measured by the AI. The significant contributions of the nonvisual factors to visual ability measures and the rehabilitation potential (ie, ceiling) effects they may impose on LVR are important considerations when measuring baseline visual ability and ultimately LVR outcomes in ongoing clinical research.

How will optometry practice change during the 21st century? No doubt, this will be influenced by many elusive factors, such as the emergence of new diagnostic and treatment modalities, changes in relevant health legislation, and economic developments. One vital factor that is very easy to anticipate is the changing demography of the people who require optometric services. In advancing their premise that demographics can be used to predict future directions in society, Foot and Stoffman suggested that “demographics explains two thirds or everything. 1 The remaining one-third may be the decisive factor in shaping events because counting bodies will never predict the arrival of a wonder drug, electoral outcomes or the emergence of gifted leaders.” Discussion of demographics at the present time is dominated by the aging of the population. According to the U.S. Census Bureau’s “middle series” projections, the elderly population (those aged over 65 years) will more than double between now and the year 2050, to 80 million. By that year, as many as 1 in 5 Americans could be elderly. Most of this growth should occur between 2010 and 2030, when the “baby boom” generation enters their elderly years. The “oldest old,” those aged 85 and over, is the most rapidly growing elderly age group. It is expected the oldest old will number 19 million in 2050. That would make them 24% of elderly Americans and 5% of all Americans. What is not as widely appreciated is the fact that population aging has occurred and is occurring in less industrialized countries as well. Uruguay, for example, currently has a higher percentage of older (60+) population than does Canada and the United States. Outside of Europe and North America, the Caribbean is the “oldest” region of the world, with 10% of its aggregate population aged 60 years and over. Optometric Practice Issues These demographic changes will have a significant impact on optometric practice. Indeed, some practitioners have seen an increase in the number of seniors reporting for optometric examination during the early 1990s 2 and this will continue to increase, particularly between 2010 and 2030. A practical first point is that offices/practices should be appropriately designed for the elderly. In this respect, in addition to considering the reduced vision of some older patients, it is important to remember that these patients may also have functional problems with mobility, hearing, and agility among other issues. A future major issue is likely to be the mushrooming of the low vision population and how these people can be adequately cared for. 3 Assuming that no miracle cures are imminent for any of the conditions that cause low vision, the most powerful intervention to assist people with low vision is proficient vision rehabilitation. Even to meet the needs of the current population, the optometric profession needs to increase its capacity to provide primary low vision care. 3 Several studies confirm that more than 75% of all low vision patients can benefit significantly when they are assessed with a modest selection of basic low vision aids. 4,5 The optometric profession will also need to increase it’s outreach services, as many seniors with low vision live in sheltered retirement or nursing homes. Key Research Areas Research into vision and aging is vital for several reasons: • We do not fully understand how and why even the most common age-related changes, such as presbyopia, occur. • We do not fully understand how and why vision deteriorates in people with healthy eyes. In addition, our understanding of human vision is based largely on the study of university-age subjects and it is not clear how relevant the conclusions reached are to the elderly person’s vision. • Several age-related conditions are as yet untreatable and cause considerable reductions in quality of life. • Although it is well-known how various age-related diseases affect visual function, we still have much to learn about the impact of visual impairment on disability and quality of life. 6,7 Similarly, ophthalmic intervention and rehabilitation should be more directed toward improving functional vision and quality of life rather than Snellen visual acuity. Presbyopia The burgeoning presbyopic patient population represents a huge market for practitioners prescribing spectacle lenses and/or contact lenses or for surgeons offering anterior segment surgeries. Just a few of the more widely accepted nonsurgical options available for treating presbyopes include bifocal or multifocal spectacle lenses, and monovision or multifocal contact lenses. There are still many questions regarding performance, tolerance, and comfort for elderly, first-time contact lens wearers. Current and future research efforts will provide answers to these questions. 8 Surgical options include monovision or multifocal corneal refractive surgical procedures, and for cataract patients monovision or multifocal intraocular lenses (IOLs). Several controversial and unproven scleral expansion surgical procedures, accomplished with PMMA segments inserted in the sclera or through radial slits in the sclera (anterior ciliary sclerotomy or ACS) are claimed to restore accommodation. These procedures are based on a revisionist theory of accommodation and presbyopia 9,10 that has been refuted 11,12 and for which no independent supporting studies exist. Furthermore, zero postoperative accommodation has been reported using independent, objective assessments. 13 Less controversial, although equally poorly understood, is the efficacy of artificial accommodative IOLs. Numerous patents exist for such devices. Research is underway to understand the efficacy of these devices and how they might be improved. 14 Some of the accommodative IOLs are undergoing clinical trials inside and outside of the United States. It is toward these developing surgical procedures that research effort is and will be directed. Although this research effort may establish if accommodation can be restored, there is much that remains unclear about the accommodative mechanism, aging of the accommodative apparatus, and the development of presbyopia. Research directed at understanding how the lens ages 15,16 and at the accommodative apparatus 17 continues to provide new insights, but there is much still to be learned. This research may help to understand how best accommodation may be restored, but it may also provide new information on how and why presbyopia develops. Although it is unlikely that presbyopia can be prevented from occurring, certainly a better understanding of why it occurs could provide important information as to how best to relieve presbyopia. Aging Changes in Normal, Healthy Eyes It is well-known that even in eyes free of eye disease, there is a reduction in many aspects of visual function with age. 18 These changes may have a significant impact on daily visual activities such as reading. 19 It is thought that the majority of these reductions are due to changes in the neural system rather than the optical system. 20 Reductions in vision due to reduced retinal illuminance caused by age-related miosis appear to be offset by the reduction in aberrations caused by the smaller pupil. 21 Because of the amount that is known about the visual system and the underlying neural mechanisms, plus the fact that visual stimuli can be produced with great precision and the responses recorded quantitatively and accurately, visual system aging has been proposed as an excellent model of aging for the neural system as a whole. 20 Age-Related Diseases The majority of low vision patients have age-related macular degeneration (AMD). Indeed, 75% of visually impaired seniors have AMD. Given the tremendous growth that is forecast to occur in the elderly population, the only development that could curb a correspondingly meteoric increase in the size of the low vision population is a safe and efficacious treatment for dry AMD. Many different treatment modalities are being researched, including laser and radiation therapies, retinal transplants, gene therapies, ozone and hyperbaric therapies, antioxidant and zinc therapies, thalidomide therapy, strontium 90 therapy, photochemical therapy, and microcurrent stimulation therapy. 22,23 No breakthrough developments have been reported to date, but fundamental and clinical research in this area should be strongly supported. A related area of study is whether cataract surgery could improve the vision of patients with both cataract and co-morbid eye disease, particularly AMD. There is growing evidence that some patients with both cataract and AMD benefit considerably from cataract surgery, 24 although other evidence suggests that cataract surgery may lead to the development of exudative macular degeneration in some patients with preoperative macular changes. 25 Quality of Life Issues Vision plays an important role in most everyday activities, so that elderly people with visual impairment are daily faced with problems such as recognizing objects and people, preparing meals, getting around (mobility), driving, reading, managing finances, socializing, and working. 26,27 These visual disabilities can lead to serious reductions in quality of life, including depression and social isolation. Research is required to provide an understanding of how different types of visual impairment affect functional vision and quality of life. 28 An important example is the lack of knowledge of the visual requirements for driving 29 and this is particularly pertinent given the importance of driving to independence in the U.S. Research is also required to determine whether these reductions in functional vision and quality of life could be improved with ophthalmic intervention. 30 Evidence from the UK suggests that despite being the most likely group of people to have obsolete prescriptions and some form of chronic eye disease, elderly people make insufficient use of eye care facilities. 31 Indeed, surveys suggest that over half the visual impairment in the over 65 age group could potentially be reduced with treatment, notably by refractive correction or cataract surgery. 32 Rehabilitation of the Visually Impaired Rehabilitation involves the assessment of visual impairment and the evaluation of functional performance, such as reading, writing, and mobility, within the context of lifestyle (e.g., employment, family activities), attitudes, and psychological well-being. Rehabilitation goals are defined in terms of what matters most in a person’s life, and attempts are made to solve functional problems through adaptive options (e.g., vision enhancement and substitution devices, environmental modifications) and coping strategies. The research challenge is to develop rehabilitation mechanisms proven to be effective in enhancing quality of life, to determine which approaches are most successful with different types of individuals, and to improve the delivery of these services to those in need. The area of technology and assistive devices is a critically important research area because of its central role in the rehabilitation process. This research focus includes the development of new devices, application of advanced technologies to visual or sensory substitution aids, and the continuous development and exploitation of new technologies, including communication, information, and computer technology. 33,34 It is also essential that research in this area addresses how to optimize training in the effective use of devices, particularly for the elderly. Research is also required that addresses the effects of vision impairment on accessing the environment in the home, the workplace, and while traveling.