Non-visual Cues Research Articles

A paradoxical misperception of relative motion.

Motion perception is considered a hyperacuity. The presence of a visual frame of reference to compute relative motion is necessary to achieve this sensitivity [Legge, Gordon E., and F. W. Campbell. "Displacement detection in human vision." Vision Research 21.2 (1981): 205-213.]. However, there is a special condition where humans are unable to accurately detect relative motion: images moving in a direction consistent with retinal slip where the motion is unnaturally amplified can, under some conditions, appear stable [Arathorn, David W., et al. "How the unstable eye sees a stable and moving world." Journal of Vision 13.10.22 (2013)]. In this study, we asked: Is world-fixed retinal image background content necessary for the visual system to compute the direction of eye motion to render in the percept images moving with amplified slip as stable? Or, are non-visual cues sufficient? Subjects adjusted the parameters of a stimulus moving in a random trajectory to match the perceived motion of images moving contingent to the retina. Experiments were done with and without retinal image background content. The perceived motion of stimuli moving with amplified retinal slip was suppressed in the presence of visual content; however, higher magnitudes of motion were perceived under conditions with no visual cues. Our results demonstrate that the presence of retinal image background content is essential for the visual system to compute its direction of motion. The visual content that might be thought to provide a strong frame of reference to detect amplified retinal slips, instead paradoxically drives the misperception of relative motion.

Smart Assistive System for Visually Impaired using PI

Visually impaired individuals face significant challenges when navigating and engaging with their surroundings independently. Our solution, "Smart Assistive System for visually Impaired using pi" employs a Raspberry Pi and camera for real-time image capture, precise object classification (with over 90% accuracy), and auditory feedback. The project addresses a pressing need for greater inclusion and accessibility for the visually impaired, offering a cost-effective and innovative solution that converts visual information into non-visual cues. The "Caption-Speak" system holds the potential to significantly enhance the independence, mobility, and overall quality of life for visually impaired individuals

Object Detection for the Visually Impaired

This paper presents the design and development of a mobile application, built using Flutter, that leverages object detection to enhance the lives of visually impaired individuals. The application addresses a crucial challenge faced by this community – the lack of real-time information about their surroundings. We propose a solution that utilizes pre-trained machine learning models, potentially through TensorFlow Lite for on-device processing, to identify objects within the user's field of view captured by the smartphone camera. The application goes beyond simple object recognition. Detected objects are translated into natural language descriptions through text-to-speech functionality, providing crucial auditory cues about the environment. This real-time information stream empowers users to navigate their surroundings with greater confidence and independence. Accessibility is a core principle of this project. The user interface will be designed with compatibility for screen readers, ensuring seamless interaction for users who rely on assistive technologies. Haptic feedback mechanisms will be incorporated to provide non-visual cues and enhance the user experience. The ultimate goal of this project is to create a user-friendly and informative application that empowers visually impaired people to gain greater independence in their daily lives. The application has the potential to improve spatial awareness, foster a sense of security, and promote overall inclusion within society

The effects of long-term exposure to microgravity and body orientation relative to gravity on perceived traveled distance

Self-motion perception is a multi-sensory process that involves visual, vestibular, and other cues. When perception of self-motion is induced using only visual motion, vestibular cues indicate that the body remains stationary, which may bias an observer’s perception. When lowering the precision of the vestibular cue by for example, lying down or by adapting to microgravity, these biases may decrease, accompanied by a decrease in precision. To test this hypothesis, we used a move-to-target task in virtual reality. Astronauts and Earth-based controls were shown a target at a range of simulated distances. After the target disappeared, forward self-motion was induced by optic flow. Participants indicated when they thought they had arrived at the target’s previously seen location. Astronauts completed the task on Earth (supine and sitting upright) prior to space travel, early and late in space, and early and late after landing. Controls completed the experiment on Earth using a similar regime with a supine posture used to simulate being in space. While variability was similar across all conditions, the supine posture led to significantly higher gains (target distance/perceived travel distance) than the sitting posture for the astronauts pre-flight and early post-flight but not late post-flight. No difference was detected between the astronauts’ performance on Earth and onboard the ISS, indicating that judgments of traveled distance were largely unaffected by long-term exposure to microgravity. Overall, this constitutes mixed evidence as to whether non-visual cues to travel distance are integrated with relevant visual cues when self-motion is simulated using optic flow alone.

Smart Assistive System for Visually Impaired using PI

Visually impaired individuals face significant challenges when navigating and engaging with their surroundings independently. Our solution, "Smart Assistive System for visually Impaired using pi" employs a Raspberry Pi and camera for real-time image capture, precise object classification (with over 90% accuracy), and auditory feedback. The project addresses a pressing need for greater inclusion and accessibility for the visually impaired, offering a cost- effective and innovative solution that converts visual information into non-visual cues. The "Caption-Speak" system holds the potential to significantly enhance the independence, mobility, and overall quality of life for visually impaired individuals.

Normalized stability time analysis within the boundaries between adults with and without fear of falling

BackgroundThe unilateral stance test, measured by the center of pressure (COP), has been widely used to identify balance deficits. However, there is a critical gap in understanding the specific COP thresholds on postural stability in adults with a fear of falling (FOF).AimsTo investigate the normalized stability time, which was defined as the ratio of time spent within stability boundaries to the total test duration, under different visual conditions and specific thresholds between adults with and without FOF.MethodsTwenty-one older adults with FOF and 22 control subjects completed the unilateral limb standing test in eyes-open and eyes-closed conditions. Normalized stability times were computed based on five pre-determined COP sway range thresholds: 10 mm, 15 mm, 20 mm, 25 mm, and 30 mm.ResultsReceiver operating characteristic analysis determined the diagnostic accuracy of FOF. There were significant differences in the effects of both visual conditions (F = 46.88, p = 0.001) and threshold settings (F = 119.38, p = 0.001) on stability time between groups. The FOF group significantly reduced normalized stability time at the 10 mm COP threshold under eyes-closed conditions (t = – 1.95, p = 0.03).DiscussionThe findings highlight the heightened sensitivity of the 10 mm COP threshold in identifying group variances in postural stability when eyes are closed. Moreover, the FOF group displayed a marked reduction in stability duration based on visual scenarios and normalized thresholds.ConclusionThe study highlights the need to account for both COP boundaries and visual conditions in adults with FOF. When assessing postural control during unilateral stances, clinicians must also give attention to non-visual cues.

Beyond the Eye: Multisensory Contributions to the Sensation of Illusory Self-Motion (Vection)

Abstract Vection is typically defined as the embodied illusion of self-motion in the absence of real physical movement through space. Vection can occur in real-life situations (e.g., ‘train illusion’) and in virtual environments and simulators. The vast majority of vection research focuses on vection caused by visual stimulation. Even though visually induced vection is arguably the most compelling type of vection, the role of nonvisual sensory inputs, such as auditory, biomechanical, tactile, and vestibular cues, have recently gained more attention. Non-visual cues can play an important role in inducing vection in two ways. First, nonvisual cues can affect the occurrence and strength of vection when added to corresponding visual information. Second, nonvisual cues can also elicit vection in the absence of visual information, for instance when observers are blindfolded or tested in darkness. The present paper provides a narrative review of the literature on multimodal contributions to vection. We will discuss both the theoretical and applied relevance of multisensory processing as related to the experience of vection and provide design considerations on how to enhance vection in various contexts.

Behavioural adjustments enable the colonization of subterranean environments

Abstract Specialized subterranean species are iconic examples of convergent evolution driven by environmental constraints, representing an ideal model system for eco-evolutionary studies. However, scientific research on the behavioural adaptations of subterranean organisms has lagged and is biased mostly towards a few model species. Through a systematic literature review, we aimed to assess quantitatively whether a congruent evolution of behavioural traits among subterranean species exists. We considered four types of behaviours (exploratory, feeding, social and anti-predator) and tested predictions about their occurrence among species with different degrees of adaptation to subterranean environments. We argue that these behaviours constitute the earliest adaptations to three main ecological factors characterizing subterranean habitats, namely the constant darkness, the limited food resources and the reduced predator pressure. Darkness selects for individuals that rely on non-visual cues to explore their surroundings, track food resources and locate potential sexual partners. The reduced availability of food promotes high foraging plasticity and deters species from adopting aggressive behaviours towards conspecifics. The reduced predator pressure allows species to abandon specific anti-predator behaviours and permits safer execution of their biological activities. Our analyses show that these four behaviours are likely to represent the result of an early convergent adaptation to the subterranean realm.

Clinical assessment of the TechArm system on visually impaired and blind children during uni- and multi-sensory perception tasks.

We developed the TechArm system as a novel technological tool intended for visual rehabilitation settings. The system is designed to provide a quantitative assessment of the stage of development of perceptual and functional skills that are normally vision-dependent, and to be integrated in customized training protocols. Indeed, the system can provide uni- and multisensory stimulation, allowing visually impaired people to train their capability of correctly interpreting non-visual cues from the environment. Importantly, the TechArm is suitable to be used by very young children, when the rehabilitative potential is maximal. In the present work, we validated the TechArm system on a pediatric population of low-vision, blind, and sighted children. In particular, four TechArm units were used to deliver uni- (audio or tactile) or multi-sensory stimulation (audio-tactile) on the participant's arm, and subject was asked to evaluate the number of active units. Results showed no significant difference among groups (normal or impaired vision). Overall, we observed the best performance in tactile condition, while auditory accuracy was around chance level. Also, we found that the audio-tactile condition is better than the audio condition alone, suggesting that multisensory stimulation is beneficial when perceptual accuracy and precision are low. Interestingly, we observed that for low-vision children the accuracy in audio condition improved proportionally to the severity of the visual impairment. Our findings confirmed the TechArm system's effectiveness in assessing perceptual competencies in sighted and visually impaired children, and its potential to be used to develop personalized rehabilitation programs for people with visual and sensory impairments.

Enhancement of visual cues to self-motion during a visual/vestibular conflict.

Perceiving our orientation and motion requires sensory information provided by vision, our body and acceleration. Normally, these cues are redundant however in some situations they can conflict. Here, we created a visual-vestibular conflict by simulating a body-upright virtual world while participants were either standing (no conflict), supine or prone (conflict) and assessed the perception of "forward" distance travelled induced by visual motion. Some participants felt they were standing upright even when lying, indicating a visual reorientation illusion (VRI). We previously showed that when experiencing a VRI, visually induced self-motion is enhanced. Here, we determined if there was a relationship between VRI vulnerability and sensory weighting. Confirming our previous findings, the VRI-vulnerable group showed enhanced self-motion perception. We then assessed the relative weightings of visual and non-visual cues in VRI-vulnerable and VRI-resistant individuals using the Oriented Character Recognition Test. Surprisingly, VRI-vulnerable individuals weighted visual cues less and gravity cues more compared to VRI-resistant individuals. These findings are in line with robust integration where, when the difference between two cues is large, the discrepant cue (here gravity) is ignored. Ignoring the gravity cue then leads to relatively more emphasis being placed on visual information and thus a higher gain.

Surface newt Calotriton asper acclimation to cave conditions improved their foraging ability in darkness

Caves in temperate regions are characterized by food scarcity compared to surface habitats. Therefore, hypotheses on adaptation to cave life suggest that many characteristics, such as resistance to starvation, reduced energy demand, and increased food-finding ability, have evolved among cave dwellers to cope with food frugality. To test the hypothesis involving increased food-finding ability, the prey detection performances of three surface and three subterranean populations of the Pyrenean newt, Calotriton asper, were compared. First, the rapidity of surface individuals in capturing prey at short distances inside a restricted aquarium under dark and light conditions and their score for capturing prey in a larger aquarium under similar conditions were measured. Simultaneously the scores obtained from individuals in cave populations in darkness were compared. Individuals from one of the surface population were maintained in darkness for approximately five years and retested. The surface C. asper individuals captured prey faster at short distances under light conditions than in total darkness; however, the differences were not significant. The scores of the surface C. asper individuals in the large aquarium were significantly better in darkness than in light conditions. In addition, the scores of cave populations at short distances in darkness were better than those of epigean populations in light conditions; however, the differences were not significant. Finally, the scores of surface individuals maintained in darkness for five years improved. To explain these results, it should be noted that surface C. asper populations forage at night using non-visual cues (e.g., chemical and mechanical cues), further suggesting that vision may somewhat inhibit other senses. This ability to forage at night may be favorable for cave colonization. Moreover, some hypogean populations, when the use of prey-detecting non-visual senses has developed, may start to exhibit adaptivity.

V2ReID: Vision-Outlooker-Based Vehicle Re-Identification.

With the increase of large camera networks around us, it is becoming more difficult to manually identify vehicles. Computer vision enables us to automate this task. More specifically, vehicle re-identification (ReID) aims to identify cars in a camera network with non-overlapping views. Images captured of vehicles can undergo intense variations of appearance due to illumination, pose, or viewpoint. Furthermore, due to small inter-class similarities and large intra-class differences, feature learning is often enhanced with non-visual cues, such as the topology of camera networks and temporal information. These are, however, not always available or can be resource intensive for the model. Following the success of Transformer baselines in ReID, we propose for the first time an outlook-attention-based vehicle ReID framework using the Vision Outlooker as its backbone, which is able to encode finer-level features. We show that, without embedding any additional side information and using only the visual cues, we can achieve an 80.31% mAP and 97.13% R-1 on the VeRi-776 dataset. Besides documenting our research, this paper also aims to provide a comprehensive walkthrough of vehicle ReID. We aim to provide a starting point for individuals and organisations, as it is difficult to navigate through the myriad of complex research in this field.

How Face Masks Affect the Use of Echolocation by Individuals With Visual Impairments During COVID-19: International Cross-sectional Online Survey.

BackgroundAlthough a critical safety measure, preliminary studies have suggested that the use of a face mask may pose a problem for some users with disabilities. To date, little is known about how the wearing of a traditional face mask may pose a barrier to individuals with visual impairments who draw on auditory cues and echolocation techniques during independent travel.ObjectiveThe goal of this study was to document the difficulties, if any, encountered during orientation and mobility due to the use of a face mask during the COVID-19 pandemic and the strategies used to address these barriers.MethodsIn total, 135 individuals aged 18 years and older who self-identified as being blind, being deafblind, or having low vision and who could communicate in either English or French completed an anonymous cross-sectional online survey between March 29 and August 23, 2021.ResultsIn total, 135 respondents (n=52, 38.5%, men; n=83, 61.5%, women) between the ages of 18 and 79 (mean 48.22, SD 14.48) years participated. Overall, 78 (57.7%) self-identified as blind and 57 (42.3%) as having low vision. In addition, 13 (9.6%) identified as having a combined vision and hearing loss and 3 (2.2%) as deafblind. The most common face coverings used were cloth (n=119, 88.1%) and surgical masks (n=74, 54.8%). Among the barriers raised, participants highlighted that face masks made it more difficult to locate people (n=86, 63.7%), communicate with others (n=101, 74.8%), and locate landmarks (n=82, 60.7%). Although the percentage of those who used a white cane before the pandemic did not substantially change, 6 (14.6%) of the 41 participants who were guide dog users prior to the pandemic reported no longer working with a guide dog at the time of the survey. Moreover, although guide dog users reported the highest level of confidence with independent travel before the pandemic, they indicated the lowest level of confidence a year after the pandemic began.ConclusionsThese results suggest that participants were less able to draw on nonvisual cues during independent travel and social interactions due to the use of a facemask, contributing to a reduction in perceived self-confidence and independence. Findings inform the development of evidence-based recommendations to address identified barriers.

Perception of object motion during self-motion: Correlated biases in judgments of heading direction and object motion.

This study investigated the relationship between perceived heading direction and perceived motion of an independently moving object during self-motion. Using a dual task paradigm, we tested whether object motion judgments showed biases consistent with heading perception, both across conditions and from trial to trial. Subjects viewed simulated self-motion and estimated their heading direction (Experiment 1), or walked toward a target in virtual reality with conflicting physical and visual cues (Experiment 2). During self-motion, an independently moving object briefly appeared, with varied horizontal velocity, and observers judged whether the object was moving leftward or rightward. In Experiment 1, heading estimates showed an expected center bias, and object motion judgments showed corresponding biases. Trial-to-trial variations were also correlated: on trials with a more rightward heading bias, object motion judgments were consistent with a more rightward heading, and vice versa. In Experiment 2, we estimated the relative weighting of visual and physical cues in control of walking and object motion judgments. Both were strongly influenced by nonvisual cues, with less weighting for object motion (86% vs. 63%). There were also trial-to-trial correlations between biases in walking direction and object motion judgments. The results provide evidence that shared mechanisms contribute to heading perception and perception of object motion.

Visual self-motion information contributes to passable width perception during a bike riding situation.

Previous studies have shown that space perception around the body is altered by self-motion, and that several self-motion cues from different modalities, including vision, proprioception, the vestibular system, and the motor system (motor commands) contribute to it. However, studies on how online self-motion information affects the perception of a passable width of a narrow aperture is largely overlooked by existing literature. Therefore, this study investigated this issue during virtual bike riding. Participants observed a narrow doorway aperture with varied widths in a virtual environment through a head-mounted display while riding a stationary bike. Visual self-motion information was presented by optical flow, while motor commands and proprioceptive feedback (non-visual information) was provided by having participants pedal the bike. The participants were then required to judge whether the presented aperture was passable. Experiment 1, where both visual and non-visual cues were provided, confirmed that the perceived passable width significantly increased with increasing self-motion speed, as previously shown during walking. Experiment 2, where self-motion cues were manipulated, showed that expansion of the perceived passable width was mainly induced by visual self-motion information. These results suggest that online self-motion information can affect passable width perception during bike riding and that visual self-motion information plays a significant role in this perception.

Integration of artificial vision with non-visual peripheral cues to guide mobility.

Visual prostheses can improve vision for people with severe vision loss, but low image resolution and lack of peripheral vision limit their effectiveness. To address both problems, we developed a prototype advanced video processing system with a headworn depth camera and feature detection capabilities. We used computer vision algorithms to detect landmarks representing a goal and plan a path towards the goal, while removing unnecessary distractors from the video. If the landmark fell outside the visual prosthesis's field-of-view (20 degrees central vision) but within the camera's field-of-view (70 degrees), we provided vibrational cues to the left or right temple to guide the user in pointing the camera. We evaluated an Argus II retinal prosthesis participant with significant vision loss who could not complete the task (finding a door in a large room) with either his remaining vision or his retinal prosthesis. His success rate improved to 57%, 37.5%, and 100% while requiring 52.3, 83.0, and 58.8 seconds to reach the door using only vibration feedback, retinal prosthesis with modified video, and retinal prosthesis with modified video and vibration feedback, respectively. This case study demonstrates a possible means of augmenting artificial vision. Clinical Relevance- Retinal prostheses can be enhanced by adding computer vision and non-visual cues.

Postural and visual aftereffects to a slanted floor in lying and sitting positions

After lying on a slanted floor for a while with the eyes closed, we may perceive it to be less slanted than at the beginning. After viewing a slanted floor while lying on a flat base, we may perceive it to be more horizontal. We investigated these postural and visual adaptations and their interactions with participants lying and sitting on the floor. The participants were adapted to a floor that was posturally, visually, or jointly slanted, and were asked to estimate the test slants around the adapting slant. The estimates were described as a linear function of the test slant with a high goodness-of-fit over the adapting slant. This supported normalization, not satiation, view. Second, the slope of the function, i.e., sensitivity to slant, in the lying position was low in the postural and visual conditions but high in the joint condition, whereas the sensitivity in the sitting position was equally high in all conditions. This was explained by an increase in visual and non-visual cues to the gravitational vertical in the sitting position, and by an abnormal pattern of intracorporeal hydrostatic pressure in the lying position. Third, in both body positions, the angle at which the slant appeared horizontal, i.e., the subjective horizontal (SH), was larger in the postural condition than in the visual condition. Finally, when the postural and visual adaptations were joint, the SH in the lying position was somewhere between the postural- and visual-alone SHs, whereas the SH in the sitting position approximated the visual-alone SH.

Olfactory behavioural and neural responses of planktivorous lacustrine sockeye salmon (Oncorhynchus nerka) to prey odours.

Fish use a variety of sensory systems when foraging. Salmonids are generally considered visual feeders. However, some species feed on zooplanktons under dark conditions, suggesting they also detect prey using nonvisual cues. Under experimental conditions, hatchery-reared rainbow trout (Oncorhynchus mykiss) have been shown to use olfaction when searching for food pellets, but olfactory foraging has not been documented in wild salmonids. In the present study, to examine their behavioural response and neural activity in the olfactory nervous system using c-fos expression as a neural molecular marker, immature wild-caught lacustrine sockeye salmon (Oncorhynchus nerka) in a flow-through aquarium were exposed to zooplanktons (Daphnia spp.) extract including zooplanktons odorant and to dimethyl sulfide. The salmon exposed to zooplanktons odour increased their total swimming distance and time, numbers of turns and ascents, and c-fos expression in the olfactory bulb, suggesting that they can detect zooplanktons extract to locate prey in the laboratory experiments. However, no response was seen in those exposed to dimethyl sulfide. The results of this study suggest that prey odour may serve as a chemosensory cue for wild immature salmonids.

A Computational Look at Oral History Archives

Computational technologies have revolutionized the archival sciences field, prompting new approaches to process the extensive data in these collections. Automatic speech recognition and natural language processing create unique possibilities for analysis of oral history (OH) interviews, where otherwise the transcription and analysis of the full recording would be too time consuming. However, many oral historians note the loss of aural information when converting the speech into text, pointing out the relevance of subjective cues for a full understanding of the interviewee narrative. In this article, we explore various computational technologies for social signal processing and their potential application space in OH archives, as well as neighboring domains where qualitative studies is a frequently used method. We also highlight the latest developments in key technologies for multimedia archiving practices such as natural language processing and automatic speech recognition. We discuss the analysis of both visual (body language and facial expressions), and non-visual cues (paralinguistics, breathing, and heart rate), stating the specific challenges introduced by the characteristics of OH collections. We argue that applying social signal processing to OH archives will have a wider influence than solely OH practices, bringing benefits for various fields from humanities to computer sciences, as well as to archival sciences. Looking at human emotions and somatic reactions on extensive interview collections would give scholars from multiple fields the opportunity to focus on feelings, mood, culture, and subjective experiences expressed in these interviews on a larger scale.

Panoramic Uncertainty in Vertical Perception.

Judgments of the orientation of a visual line with respect to earth vertical are affected by panoramic visual cues. This is illustrated by the rod-and-frame effect (RFE), the finding that the perceived orientation of a luminous rod is biased by the orientation of a surrounding squared frame. In this study, we tested how the uncertainty of frame orientation affects the RFE by asking upright or tilted participants to psychometrically judge the orientation of a briefly flashed rod contained within either a circular frame, a squared frame, or either of two intermediate frame forms, called squircles, presented in various orientations. Results showed a cyclical modulation of frame-induced bias across the range of the square and squircular frame orientations. The magnitude of this bias increased with increasing squaredness of the frame, as if the more unequivocal the orientation cues of the frame, the larger the reliance on them for rod orientation judgments. These findings are explained with a Bayesian optimal integration model in which participants flexibly weigh visual panoramic cues, depending on their orientation reliability, and non-visual cues in the perception of vertical.