TapNav: Adaptive Spatiotactile Screen Readers for Tactually Guided Touchscreen Interactions for Blind and Low Vision People

Blind and low vision people use visual description services (VDS) to gain visual interpretation and build access in a world that privileges sight. Despite their many benefits, VDS have many harmful privacy and security implications. As a result, researchers are suggesting, exploring, and building obfuscation systems that detect and obscure private or sensitive materials. However, as obfuscation depends largely on sight to interpret outcomes, it is unknown whether Blind and low vision people would find such approaches useful. Our work aims to center the perspectives and opinions of Blind and low vision people on the potential of obfuscation to address privacy concerns in VDS. By reporting on interviews with 20 Blind and low vision people who use VDS, our findings reveal that popular research trends in obfuscation fail to capture the needs of Blind and low vision people. While obfuscation might be helpful in gaining more control, tensions around obfuscation misrecognition and confirmation are prominent. We turn to the framework of interdependence to unpack and understand obfuscation in VDS, enabling us to complicate privacy concerns, uncover the labor of Blind and low vision people, and emphasize the importance of safeguards. We provide design directions to move the trajectory of obfuscation research forward.

We design and build A11yBits, a tangible toolkit that empowers blind and low vision (BLV) people to easily create personalized do-it-yourself assistive technologies (DIY-ATs). A11yBits includes (1) a series of Sensing modules to detect both environmental information and user commands, (2) a set of Feedback modules to send multi-modal feedback, and (3) two Base modules (Sensing Base and Feedback Base) to power and connect the sensing and feedback modules. The toolkit enables accessible and easy assembly via a “plug-and-play” mechanism. BLV users can select and assemble their preferred modules to create personalized DIY-ATs.

Self-service terminals (SSTs) are almost everywhere in our daily life and increasingly use capacitive and infrared touchscreens as the interface. Most of the current solutions to help blind and low vision (BLV) people access existing touchscreens mostly are only suitable for capacitive touchscreens and not for infrared touchscreens. In this paper, we proposed a voice-based interactive method using a conductive folding stand with the phone camera to allow BLV people to access both touchscreens of SSTs. Voice feedback was provided to guide users to move the phone close to the button and touch it with the end of the unfolded stand. Using a portable accessory, this method directly guided users to touch the target and effectively avoids false triggering. A preliminary evaluation indicated that our approach enabled users to access the target buttons on the touchscreen with high accuracy and a short completion time.

While audio description (AD) is the standard approach for making videos accessible to blind and low vision (BLV) people, existing AD guidelines do not consider BLV users’ varied preferences across viewing scenarios. These scenarios range from how-to videos on YouTube, where users seek to learn new skills, to historical dramas on Netflix, where a user’s goal is entertainment. Additionally, the increase in video watching on mobile devices provides an opportunity to integrate nonverbal output modalities (e.g., audio cues, tactile elements, and visual enhancements). Through a formative survey and 15 semi-structured interviews, we identified BLV people’s video accessibility preferences across diverse scenarios. For example, participants valued action and equipment details for how-to videos, tactile graphics for learning scenarios, and 3D models for fantastical content. We define a six-dimensional video accessibility design space to guide future innovation and discuss how to move from “one-size-fits-all” paradigms to scenario-specific approaches.

Cooking is a vital yet challenging activity for blind and low vision (BLV) people, which involves many visual tasks that can be difficult and dangerous. BLV training services, such as vision rehabilitation, can effectively improve BLV people’s independence and quality of life in daily tasks, such as cooking. However, there is a lack of understanding on the practices employed by the training professionals and the barriers faced by BLV people in such training. To fill the gap, we interviewed six professionals to explore their training strategies and technology recommendations for BLV clients in cooking activities. Our findings revealed the fundamental principles, practices, and barriers in current BLV training services, identifying the gaps between training and reality.

While our community has many active projects involving blind people, low vision is rarely addressed. People with low vision have functional vision, but their visual impairment adversely affects their daily life and it cannot be corrected with glasses or contact lenses. Over the last few years, we have been conducting research with this understudied demographic: understanding low vision people's needs and designing applications to address the challenges they face. In this article, we discuss our ongoing research in this area, focusing on designing augmented reality applications for low vision users. We begin this article by describing low vision and motivating our focus on augmented reality applications on smartglasses for low vision people. We then provide overviews of three research projects that exemplify our research agenda: a study where we observed low vision people conducting a navigation and shopping task, a study where we examined low vision people's perception of virtual text and shapes on smartglasses, and the design of a smartglasses application that facilitates a visual search task.

People with low vision have a visual impairment that affects their ability to perform daily activities. Unlike blind people, low vision people have functional vision and can potentially benefit from smart glasses that provide dynamic, always-available visual information. We sought to determine what low vision people could see on mainstream commercial augmented reality (AR) glasses, despite their visual limitations and the device's constraints. We conducted a study with 20 low vision participants and 18 sighted controls, asking them to identify virtual shapes and text in different sizes, colors, and thicknesses. We also evaluated their ability to see the virtual elements while walking. We found that low vision participants were able to identify basic shapes and read short phrases on the glasses while sitting and walking. Identifying virtual elements had a similar effect on low vision and sighted people's walking speed, slowing it down slightly. Our study yielded preliminary evidence that mainstream AR glasses can be powerful accessibility tools. We derive guidelines for presenting visual output for low vision people and discuss opportunities for accessibility applications on this platform.

Modern mobile devices provide several functionalities and new ones are being added at a breakneck pace. Unfortunately browsing the menu and accessing the functions of a mobile phone is not a trivial task for visual impaired users. Low vision people typically rely on screen readers and voice commands. However, depending on the situations, screen readers are not ideal because blind people may need their hearing for safety, and automatic recognition of voice commands is challenging in noisy environments. Novel smart watches technologies provides an interesting opportunity to design new forms of user interaction with mobile phones. We present our first works towards the realization of a system, based on the combination of a mobile phone and a smart watch for gesture control, for assisting low vision people during daily life activities. More specifically we propose a novel approach for gesture recognition which is based on global alignment kernels and is shown to be effective in the challenging scenario of user independent recognition. This method is used to build a gesture-based user interaction module and is embedded into a system targeted to visually impaired which will also integrate several other modules. We present two of them: one for identifying wet floor signs, the other for automatic recognition of predefined logos.

Low vision people who are more than 88% of visually impaired people want to use their residual vision and don't want to look like disabled. However, many assistive devices for low vision are suitable for use indoors and people with disabled are exposed using assistive device so that they are reluctant to use that. So, many low vision people want to use smart phone to solve problem but now functions of smartphone are not enough. In this study, we want to suggest smart assistive software and device for low vision people to use with residual vision as much as possible without being self-conscious. For that, we interviewed expert of low vision and low vision people with qualitative research methods. Based on the results, we present solution and suggest EYESEE, assistive device and application for low vision people.

While being able to read with screen magnifiers, low vision people have slow and unpleasant reading experiences. Eye tracking has the potential to improve their experience by recognizing fine-grained gaze behaviors and providing more targeted enhancements. To inspire gaze-based low vision technology, we investigate the suitable method to collect low vision users’ gaze data via commercial eye trackers and thoroughly explore their challenges in reading based on their gaze behaviors. With an improved calibration interface, we collected the gaze data of 20 low vision participants and 20 sighted controls who performed reading tasks on a computer screen; low vision participants were also asked to read with different screen magnifiers. We found that, with an accessible calibration interface and data collection method, commercial eye trackers can collect gaze data of comparable quality from low vision and sighted people. Our study identified low vision people’s unique gaze patterns during reading, building upon which, we propose design implications for gaze-based low vision technology.

The blind or low vision people should exercise, in equal conditions, the rights and duties, which assure them the citizenship. To do this, they can use differentiated resources, that facilitate or promote the development of functional skills and, consequently, the social inclusion, between which stand out the Assistive Technologies (TA). Thus, this article presents the results of a literature review from the years 2007 to 2015, which aimed to analyze which are the TA that can be used by blind or low vision people in the social context, and how they influence in the inclusion. The methodology used was a bibliographical and documentary research about TA, blindness and low vision, realized in the virtual library Scielo and in the Portal Periodicos CAPES. It was selected brazilian articles that mentioned the theme, which were, later, analyzed and categorized, according common elements perceived, and served as a basis for the considerations and discussions presented here. It was mainly perceived, that the resources in question are used in different social contexts and with varied purposes, and that they allow greater independence for the users. The final considerations pointed to the importance of the TA study related withblindness or low vision people, to the awareness of society against prejudice, and to the implementation of public policies to support the use of these resources with the public in question.

Two-factor Authentication (also known as 2FA or two-step verification) is an authentication method that provides an extra layer of protection to ensure online account security. 2FA methods are used along with other primary authentication methods like PINs and Passwords to verify that the person trying to access any digital account is the person they are claiming to be. However, 2FA methods can be inaccessible for blind and low vision (BLV) users due to the requirement of multiple steps, apps, and/or devices for authentication. In addition, it can be a security risk as screen readers may read out the verification codes to bystanders. To address this, we present Haptic2FA, a haptic-based authentication method to improve 2FA accessibility for BLV users. Here, as a part of the 2FA process, the users are sent a 'haptic pattern' (similar to a one-time passcode in traditional 2FA methods) that they are required to enter or select for verification. Through a usability study with 10 BLV participants, we evaluated haptic patterns and input methods for the haptic patterns in the Haptic2FA method. Through the findings, we discuss the accessibility and usability of the Haptic2FA method.

Blind and low-vision (BLV) people experience difficulty accessing graphical information, particularly regarding travel and education. Tactile diagrams and 3D printed models can improve access to graphical information for BLV people; however, these formats only allow limited detailed and contextual information. Interactive 3D printed models (I3Ms) exist, but many rely on passive audio labels that don't particularly empower BLV people in independent knowledge building and interpretation. This project investigates the creation of I3Ms that offer more engaging experiences with a focus on facilitating independent exploration and knowledge discovery. Specifically, this project explores how BLV people want to interact with I3Ms, interactive functionalities and behaviours that I3Ms should support, such as conversational interfaces and model agency, and to understand the relationship between I3Ms and conventional accessible graphics.

Low vision is a visual impairment that cannot be corrected with eyeglasses or contact lenses. Low vision people have functional vision and prefer using that vision instead of relying on audition and touch. Existing approaches to low vision accessibility enhance people's vision using simple "signal-to-signal" techniques that do not take into account the user's context. There is thus a major gap between low vision people's needs and existing low vision technologies. My doctorial research aims to address this gap by augmenting low vision people's visual experience with direct and optimal visual feedback based on the user's context. I will design and study novel methods for visual augmentation , which involves visual feedback beyond simple enhancements. My research considers two dimensions: visual condition and task. By understanding the visual perception of people with different visual abilities and exploring their needs in different visual tasks, I will design applications with visual feedback that is optimal for specific context to maximize people's access to information. My research will yield design insights and novel applications for people with all visual abilities.

This article introduces the “Mobile Bio-Eye-Tronic System” which is an artificial vision system for the impaired (blind) people and also low vision (sight loss) people. “Mobile Bio-Eye-Tronic System” is a completely original project and unique to the author of this article. There are 45 million visually impaired (blind) people in the world and 135 million low vision (sight loss) people. 60% of blindness in the world is treatable and 20% is preventable. 25 million people are blind in Europe, 12 million in America, 9 million in India, 6 million in China, and 7 million in Africa. In Turkey, this number is approximately 300 thousand. Based on these statistics, the main aim of this article is to appeal to hundreds of thousands of people, to help them fulfill their daily activities, even partially, to improve the quality of life of these visually impaired people and to restore their health. In addition, the scientific and technical studies to be carried out on this subject will contribute to the enrichment of the literature on the subject and will also be beneficial for scientific and technical progress. When the current studies on bionic eyes are examined, there is no other system in the literature that obtains results using a mobile phone camera and software. The bionic eye stated in this article will be a first in this respect.