P3‐314: Pictorial education of safe medication for the elderly with cognitive impairment

Abstract

Background: In this work, we developed a video processing framework, which simultaneously analyzes hands motion of multiple-people to understand the type of action, to measure the reaction time and accuracy of the motion during physical exercise or social activity. The location of the hands, head and center of the body are captured to encode the motion characteristic at each frame from a video sequence. The system can detect multiple people and analyze their hands motion. In the experiments, analysis of various hands motion of two elderly patients is presented. Methods: Recently released Kinect sensor is used to capture depth data and skeleton data along with an RGB camera. The positions of the hands, head, shoulders, elbows and body center are extracted and stored as key points in a 2-dimensional matrix, which is called a location map. Location maps are generated from skeleton views. Later, each location map is compared with a previously defined matrix for each activity and the most similar one is chosen to recognize the type of the activity. The accuracy of the motion is given by the value of the similarity; and the reaction time is given by the time between voice order and realization of the action. Extracting depth data helps us avoid occlusions and easily distinguish people and body parts in the scene. Various activities (Hands Up, Hands Down, Hands on Head, Hand Shaking, Hands Up Arms Straight, Hands Up Arms Bended, Hands Reunited Front, etc.) are encoded in real-time by using the positions of key points in the 2D location map. Results: In the supplementary figure, five different activities (normal position, hands clapping, hands on shoulder, hands up, hand shaking) are investigated for two patients by using the proposed framework. Skeleton views for each frame are extracted and the position of nine key points (head, neck, shoulders, elbows, hands, center) from each skeleton are calculated to compose location maps. Location maps are used to categorize the activity. Conclusions: In this work, a video processing framework to simultaneously capture and categorize hands motion of multiple-people is presented. Experimental results show the potential usage and successful application of the system to analyze five different types of activity. P3-314 PICTORIAL EDUCATION OF SAFE MEDICATION FOR THE ELDERLY WITH COGNITIVE IMPAIRMENT