CONSTRUCTION OF A SOMATOSENSORY INTERACTIVE SYSTEM BASED ON COMPUTER VISION AND AUGMENTED REALITY TECHNIQUES USING THE KINECT DEVICE FOR FOOD AND AGRICULTURAL EDUCATION

Chao-Ming Wang,Yu-Hui Lin

doi:10.29121/ijoest.v5.i2.2021.162

Abstract

A somatosensory interactive system based on computer vision and augmented reality (AR) techniques using the Kinect device is proposed, on which a game of harvesting three kinds of fruit can be played for food and agricultural education. The Kinect is used to capture users’ motion images, the Unity3D is used as the game engine, and the Kinect SDK is used for developing programs, to implement the tasks of face detection and tracking, hand-gesture recognition, and body-model matching and tracking involved in fruit-harvesting activities. AR-based photos of the harvest result can be taken and downloaded as souvenirs. The system was exhibited and observations of the users’ performances as well as interviews with experts and the users were conducted. The collected opinions were used to evaluate the effectiveness of the system, reaching the following conclusions: 1) the interactive experience of using this system is simple and intuitive; 2) the use of body movements for man-machine interaction is given positive reviews; 3) the introduction of somatosensory interactive education can arouse participants’ interest, achieving the effect of edutainment; and 4) the experience of taking commemorative photos can achieve the publicity and promotion effect of food and agricultural education through sharing on social media.

Highlights

AND MOTIVATIONThe issue of agricultural development is always one of the main concerns of human beings in the world
A somatosensory interactive system based on computer vision and augmented reality (AR) techniques using the Kinect device is proposed, on which a game of harvesting three kinds of fruit can be played for food and agricultural education
The aim of this study is to utilize computer vision and augmented reality (AR) techniques to design an interactive system for food and agricultural education, on which a game of fruit harvesting can be played via the use of the Kinect device with the “human” as the interface

Summary

Introduction

BACKGROUND AND MOTIVATIONThe issue of agricultural development is always one of the main concerns of human beings in the world. Construction of a Somatosensory Interactive System Based on Computer Vision and Augmented Reality Techniques Using the Kinect Device for Food and Agricultural Education is via food and agricultural education [4], [5]. With the progress of technology, many digital media and related devices have been developed as tools for assisting educations in various knowledge fields. These devices can be used to simulate the experiencing situation related to the educational content, and bring interactive experiences into the learning process [8], [9], [10], [11].

Objectives

Methods

Results

Conclusion