Research on Multimodal Interaction Design Patterns for Visually Impaired People Under Sensory Compensation Theory

Abstract

Based on the theory of sensory compensation, this paper mainly applies the multimodal interaction design method to solve the difficulties and challenges faced by the visually impaired people watching the game as an application scenario, so as to provide a reference for the accessibility design. In terms of research ideas, the technical principles and applications of visual enhancement, auditory compensation, tactile compensation and olfactory compensation are investigated by combing the development of sensory compensation and the three realization methods of visual enhancement, visual substitution and visual replacement for visually impaired people. By dividing the game-viewing scene into five sub-scenes to gain insight into the specific needs of visually impaired people, combining the needs with the three forms of application of sensory compensation mentioned above, discussing their adaptive relationship with different scenes, and exploring the multimodal interaction design mode under the different needs of visually impaired people, in summary, this research come up with the design of the multimodal interaction system for the visually impaired people used to watch the game. The design example verifies that the multimodal interaction design mode based on sensory compensation design with intersecting and complementary senses of vision, audio, and tactility can help to improve the quality of spectator services and daily travel and life of the visually impaired people.