ColorCam: a color-based object recognition camera (Invited Paper)

Abstract

A 128(H) x 64(V) x RGB CMOS imager is integrated with region-of-interest selection, RGB-to-HSI transformation, HSI-based pixel segmentation, 36-bins x 12bits HSI histogramming and sum-of-absolute-difference (SAD) template matching. 32 learned color templates are stored and compared to each image. The chip captures the R, G and B images using in-pixel storage before passing the pixel content to a multiplying digital-to-analog converter (DAC) for white balancing. The DAC can also be used to pipe in images for a PC. The color processing uses a biologically inspired color opponent representation and an analog look-up table to determine the Hue (H) of each pixel. Saturation (S) is computed using a loser-take-all circuit. Intensity (I) is given by the sum color components. A histogram of the segments of the image, constructed by counting the number of pixels falling into 36 Hue intervals of 10 degrees, is stored on chip and compared against the histograms of new segments using SAD comparisons. We demonstrate color-based image segmentation and object recognition with this chip. Running at 30fps, it uses 1mW. To our knowledge, this is the first chip that integrates imaging, color segmentation and color-based object recognition at the focal plane.