Development of Visual Data Acquisition Systems of Household Objects

Abstract

What is most needed in the development of intelligent robots is visual information. Especially, Domestic Service Robots (DSR) in carrying out their duties in a complex and dynamic environment such as a household. To accomplish the task given in such environment, various kinds of recognition including household object recognition are necessary. The first step in object recognition is object learning, which one of the processes needs visual information. This information is provided by a visual sensor such as a second version of Microsoft Kinect (Kinect V2). Kinect V2 provides data such as, color information, depth information, and near infrared information. To make the object learning process simple and trackable, the development of object data acquisition systems is needed. In general, the captured data will be incorporated with labels such as the name of household object and the corresponding pose. To facilitate the object labeling, we develop smartphone applications to enable simple user interaction. To capture visual information in several poses, we make a turntable that can rotate synchronously during the data acquisition process. To extract the visual information of the object autonomously, an object extraction is inevitable. To this end, we propose an object extraction based on combination of information captured from Kinect V2. Our proposed method using a probabilistic method which integrates several Gaussian Mixture Models (GMM). Evaluation of proposed systems has been done through several experiments. Based on conducted experiments, our systems can capture the household object with size specifications greater than 3 cm × 3 cm × 2 cm, smaller than 28 cm × 28 cm × 30 cm, and with a weight not exceeding 800 g. In addition, the proposed systems can extract 40 objects (each of which covered 40 poses) with the F1-score of 76.43%.