Development of an Online Home Appliance Control System Using Augmented Reality and an SSVEP-Based Brain–Computer Interface

Seonghun Park,Chang-Hwan Im,Ho-Seung Cha

doi:10.1109/access.2019.2952613

Seonghun Park, Chang-Hwan Im + Show 1 more

Open Access

https://doi.org/10.1109/access.2019.2952613

Copy DOI

Abstract

In this study, we implemented a new home appliance control system by combining electroencephalography (EEG)-based brain-computer interface (BCI), augmented reality (AR), and internet of things (IoT) technologies. We adopted a steady-state visual evoked potential (SSVEP)-based BCI paradigm for the implementation of a fast and robust BCI system. In the offline experiment, we compared the performances of three BCIs adopting different types of visual stimuli in an AR environment to determine the optimal visual stimulus. In the online experiment, we evaluated the feasibility of the proposed smart home system using the optimal stimulus by controlling three home appliances in real time. The visual stimuli were presented on a see-through head-mounted display (HMD), while the recorded brain activity was analyzed to classify the control command, and the home appliances were controlled through IoT. In the offline experiment, a grow/shrink stimulus (GSS) consisting of a star-shaped flickering object of varying size was selected as the optimal stimulus, eliciting SSVEP responses more effectively than the other options. In the online experiment, all users could turn the BCI-based control system on/off whenever they wanted using the eye-blinking-based electrooculogram (EOG) switch, and could successfully perform all the designated control tasks without difficulty. The average classification accuracy of the SSVEP-BCI-based control system was 92.8%, with an information transfer rate (ITR) of 37.4 bits/min. The proposed system exhibited an excellent performance, surpassing the best results reported in previous studies regarding external device control based on BCI using an HMD as rendering device.

Highlights

Brain-computer interface (BCI) is a technology that provides a direct communication channel between a user and the external environment using the user’s brain activity [1]–[4]
We investigated whether an SSVEPbased home appliance control system with augmented reality (AR)-head-mounted display (HMD) exhibits a sufficiently strong performance for utilization in practical scenarios
To develop a smart home system for home appliance control that can be utilized in real-world scenarios, we combined AR, internet of things (IoT), and BCI technology

Summary

Introduction

Brain-computer interface (BCI) is a technology that provides a direct communication channel between a user and the external environment using the user’s brain activity [1]–[4]. As the number of people with disability and elderly people who require special care in daily life activities have rapidly increased [15], [16], the necessity of developing ‘‘smart home’’ systems tailored for such people has arisen. In line with this trend, the implementation of home automation systems using BCI technology has drawn increasing interest [17].

Methods

Results

Conclusion