Abstract
This paper presents a new approach to benchmarking brain-computer interfaces (BCIs) outside the lab. A computer game was created that mimics a real-world application of assistive BCIs, with the main outcome metric being the time needed to complete the game. This approach was used at the Cybathlon 2016, a competition for people with disabilities who use assistive technology to achieve tasks. The paper summarizes the technical challenges of BCIs, describes the design of the benchmarking game, then describes the rules for acceptable hardware, software and inclusion of human pilots in the BCI competition at the Cybathlon. The 11 participating teams, their approaches, and their results at the Cybathlon are presented. Though the benchmarking procedure has some limitations (for instance, we were unable to identify any factors that clearly contribute to BCI performance), it can be successfully used to analyze BCI performance in realistic, less structured conditions. In the future, the parameters of the benchmarking game could be modified to better mimic different applications (e.g., the need to use some commands more frequently than others). Furthermore, the Cybathlon has the potential to showcase such devices to the general public.
Highlights
Noninvasive brain-computer interfaces (BCIs), which measure a human’s brain activity and use it to control machines, have the potential to improve human-machine interaction in numerous ways
Each race lasted approximately 3 min, though moving the teams to and from the competition stage took approximately 15 min per race. We acknowledge that such short races were likely influenced by random factors and may not have captured the best performance of each BCI technology; the need for short and attractive races was dictated by the live audience of laypersons
We successfully developed a benchmarking game that allowed BCI performance to be measured via task completion time, and the teams agreed that the game was an appropriate stand-in for actual assistive technologies without the danger present in actual assistive devices
Summary
Noninvasive brain-computer interfaces (BCIs), which measure a human’s brain activity and use it to control machines, have the potential to improve human-machine interaction in numerous ways. As assistive devices, they can be used by people with disabilities to control wheelchairs (Carlson and Millán, 2013), orthoses (Ortner et al, 2011; Do et al, 2013), neuroprostheses (Rohm et al, 2013) and robots (Leeb et al, 2015) as well as to write messages (Sellers et al, 2014). The question remains: just how accurate and reliable are BCIs?
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
