FPGA-Based Real-Time Color Tracking for Robotic Formation Control

Abstract

In construction automation, the tracking of worker location or a moving object is important for labor monitoring, resource management, and machine coordination. For object tracking, a camera is often utilized to obtain information of construction vehicle motion which can then be employed in coordination controls. Real-time tracking of autonomous vehicles, particularly for the control of multiple targets in formations, still suffers from constraints imposed in computation resources. Here the field programmable gate array (FPGA) technology is applied in a prototypical tracking system for vehicles by using a CMOS camera to detect their color-tags. The raw image from the Bayer color pattern is used to indicate the 2-dimensional position of vehicles and encrypted infrared commands are issued to deploy them in a leader-follower formation. It is shown that the novel system-on-programmable-chip with parallel control cores design can efficiently handle color recognition and multi-vehicle control while significantly reduces memory requirement and computation time. Introduction The digital camera has become a popular sensor for monitoring and surveillance systems. By improvement of the semiconductor technology, modern digital cameras with high pixel number can provide greater image detail in various applications. A digital image sensor usually utilizes an array consists of the charged-couple device (CCD) or complementary metal oxide semiconductor (CMOS). By passing light through the lens and color filter array (CFA), the real image is transformed into the mosaic-like RGB image projected on the digital image sensor array (Lukac and Plataniotis, 2007). The raw RGB image then is reconstructed as a meaningful image for human perception by de-mosaicking (Lee, 2005) and color correction (Gonzalez and Woods, 2002). For sensing applications, the additional image processing to extract object’s color strength or contour (Nixon and Aguado, 2008) is required for every image pixel. Moreover, it should be noted that imaging processing consumes a lot of memory space and most of computation resources in a computer system. The digital camera is also a popular sensor in a multi-robot system (Fierro et al., 2002) for obstacle detection and environment learning. Such a system is a very valuable tool in the development of an automated coordination system for construction vehicles. In a real-time object tracking and detection system, particularly for moving targets, the picture acquisition and image processing speed critically determines the system performance. Low computation speed will cause the loss of precise driving control for the multirobot system and results in collisions. When the robot system needs a high resolution camera to improve imaging details of the object, the coherent time delay with massive computation for image processing will occur. On the other hand, even the powerful microprocessor can mitigate time delay, the higher power consumption is also unsuitable for a mini robot system with battery powered operation (Braunl, 2006). Thus a compromise scheme for the multi-vehicle system in image processing issues is sharing the image processing and strategy making tasks in an external server (Fierro et al., 2002). Therefore, the vehicle carries only a simple embedded system which captures and sends image data to the server by wireless in order to get a balance between power consumption and execution speed. Based on the same idea, we propose a new colour-based tracking system for vehicle coordination using color-tag recognition for identification with an external server. This scheme will achieve higher performance of power management and real-time control speed. In our external se when runn propose u designed b of low pow designed e digital cam et al., 2000 burden in leader-foll The pa prediction resource u Some disc Processi Image in comput better cho prediction Bayer Patt The m The co the circled need to re define the G ⇔ B ⇔ where Color Pred In real reflection 26th Internatio design, each v rver by an en ing the whol se the Field P y using an FP er, low cost, xternal server era equipped ), and the dyn the FPGA ch owing strateg per is organiz , and noise fil sage, and the ussion of the ng of Raw I processing wi ation speed. T ice for lower and noise filt ern osaic-like RG