Smart Home Appliance Control via Hand Gesture Recognition Using a Depth Camera

Abstract

The user-friendly and -intuitive interface for household appliances is considered as one of the highly promising fields for researches in the area of smart home and environment. Instead of traditional interface methodologies such as keyboard, mouse, touchscreen, or remote control, users in smart home or environment can control smart appliances via their hand gestures. This chapter presents a novel hand gesture interface system via a single depth imaging sensor to control smart appliances in smart home and environment. To control the appliances with hand gestures, our system recognizes the hand parts from depth hand silhouettes and generates control commands. In our methodology, we first create a database of synthetic hand depth silhouettes and their corresponding hand parts-labelled maps, and then train a random forests (RFs) classifier with the database. Via the trained RFs, our system recognizes the hand parts from depth silhouettes. Finally based on the information of the recognized hand parts, control commands are generated according to our predefined logics. With our interface system, users can control smart appliances which could be TV, radio, air conditioner, or robots with their hand gestures.