Object Tracking Test Automation Using a Robotic Arm

Abstract

Touch-to-track is a software feature that is used to track objects in embedded systems, such as mobile phones. The research question that is discussed in the paper is as follows: how can one design an automated test method for accurately testing object tracking algorithms? The challenge for the verification team was to design a method of test automation for testing this object tracking feature. The solution to the problem was to develop test automation algorithms using a robotic arm to accurately test this software feature. The robotic arm is used for executing test cases for tracking single or multiple objects in motion. To use the touch-to-track feature, the user selects an area in the camera preview by drawing a bounding box. Hough transformation and color segmentation are applied to accurately detect the bounding box drawn on every frame in which the object is detected. The area inside the bounding box is considered as the template. Template matching algorithms based on normalized cross-correlation, phase correlation, and speeded up robust features (SURF)-based feature extraction and matching are then applied to compare the template and original images. The tracking efficiency is calculated by dividing the total number of frames that contain the object being tracked by the total number of frames being tested. The tracking efficiencies achieved with the Hough-based bounding box detection algorithm followed by template matching algorithms based on normalized cross-correlation, phase correlation, and SURF-based feature extraction and matching are compared. These tracking efficiencies are calculated by comparing the ground truth to the tracking results for each frame. The tracking efficiency calculated for three objects is 87.7% with no colored background. The proposed object tracking solution with hardware acceleration is found to perform better than the third-party solution with respect to the latency in re-establishing tracking and the hand jitter scenario.