Color-based Object Recognition Research Articles

Quaternion fast and accurate polar harmonic Fourier moments for color image analysis and object recognition

Image moments, as a kind of global feature descriptor of images, have become a valuable tool for pattern recognition and image analysis. However, traditional methods are mainly used to deal with grayscale images. In this paper, we apply quaternions to fast and accurate polar harmonic Fourier moments, proposing a kind of quaternion fast and accurate polar harmonic Fourier moment (QFAPHFM) capable of handling color images. Furthermore, this paper provides a detailed analysis of the invariance of QFAPHFMs under rotation, scaling, and translation transformations. The experimental results show that QFAPHFMs exhibit excellent performance in both image reconstruction and object recognition tasks. QFAPHFMs achieve accurate image reconstruction under noiseless and noisy conditions, and demonstrate excellent recognition performance in the color-based object recognition tasks.

Color Homography: Theory and Applications

Images of co-planar points in 3-dimensional space taken from different camera positions are a homography apart. Homographies are at the heart of geometric methods in computer vision and are used in geometric camera calibration, 3D reconstruction, stereo vision and image mosaicking among other tasks. In this paper we show the surprising result that homographies are the apposite tool for relating image colors of the same scene when the capture conditions-illumination color, shading and device-change. Three applications of color homographies are investigated. First, we show that color calibration is correctly formulated as a homography problem. Second, we compare the chromaticity distributions of an image of colorful objects to a database of object chromaticity distributions using homography matching. In the color transfer problem, the colors in one image are mapped so that the resulting image color style matches that of a target image. We show that natural image color transfer can be re-interpreted as a color homography mapping. Experiments demonstrate that solving the color homography problem leads to more accurate calibration, improved color-based object recognition, and we present a new direction for developing natural color transfer algorithms.

Field Programmable Gate Arrays Based Real Time Robot Arm Inverse Kinematic Calculations and Visual Servoing

DOI: 10.26650/electrica.2018.49877 Reliability and precision are very important in space, medical, and industrial robot control applications. Recently, researchers have tried to increase the reliability and precision of the robot control implementations. High precision calculation of inverse kinematic, color based object recognition, and parallel robot control based on field programmable gate arrays (FPGA) are combined in the proposed system. The precision of the inverse kinematic solution is improved using the coordinate rotation digital computer (CORDIC) algorithm based on double precision floating point number format. Red, green, and blue (RGB) color space is converted to hue saturation value (HSV) color space, which is more convenient for recognizing the object in different illuminations. Moreover, to realize a smooth operation of the robot arm, a parallel pulse width modulation (PWM) generator is designed. All applications are simulated, synthesized, and loaded in a single FPGA chip, so that the reliability requirement is met. The proposed method was tested with different objects, and the results prove that the proposed inverse kinematic calculations have high precision and the color based object recognition is quite successful in finding coordinates of the objects.

Quaternion angular radial transform and properties transformation for color-based object recognition

Nowadays, with the increased use of digital images, almost all of which are in color format. Conventional methods process color images by converting them into gray scale, which is definitely not effective in representing and which may lose some significant color information. Recently, a novel method of the Color Angular Radial Transform (CART) is presented. This transform combines the information by considering the shape information inherent in the color. However, ART is adapted on the MPEG-7 standard is only limited to binary images and gray-scale images has many properties: invariant to rotation, Translation and scaling, ability to describe complex objects, so it cannot handle color images directly. To solve this problem we proposed in this article to generalize ART from complex domain to hypercomplex domain using quaternion algebras to achieve the Quaternion Angular Radial Transform (QART) to describe finally two dimensional color images and to insure these properties robustness to all possible rotations and translation and scaling. The performance of QART is then evaluated with large database of color image as an example. We first provide a general formula of ART from which we derive a set of quaternion-valued QART properties transformations by eliminating the influence of transformation parameters. The experimental results show that the QART performs better than the commonly used Quaternion form Zernike Moment (QZM) in terms of image representation capability and numerical stability.

A Computational Model for Color Perception

ABSTRACT Color is not a physical quantity of an object. It cannot be measured. We can only measure reflectance, i.e. the amount of light reflected for each wavelength. Nevertheless, we attach colors to the objects around us. A human observer perceives colors as being approximately constant irrespective of the illuminant which is used to illuminate the scene. Colors are a very important cue in everyday life. They can be used to recognize or distinguish different objects. Currently, we do not yet know how the brain arrives at a color constant or approximately color constant descriptor, i.e. what computational processing is actually performed by the brain. What we need is a computational description of color perception in particular and color vision in general. Only if we are able to write down a full computational theory of the visual system then we have understood how the visual system works. With this contribution, a computational model of color perception is presented. This model is much simpler compared to previous theories. It is able to compute a color constant descriptor even in the presence of spatially varying illuminants. According to this model, the cones respond approximately logarithmic to the irradiance entering the eye. Cells in V1 perform a change of the coordinate system such that colors are represented along a red-green, a blue-yellow and a black-white axis. Cells in V4 compute local space average color using a resistive grid. The resistive grid is formed by cells in V4. The left and right hemispheres are connected via the corpus callosum. A color constant descriptor which is presumably used for color based object recognition is computed by subtracting local space average color from the cone response within a rotated coordinate system.

Chromaticity Space for Illuminant Invariant Recognition

In this paper an algorithm is proposed to extract two illuminant invariant chromaticity features from three image sensor responses. The algorithm extracts these chromaticity features at pixel level and therefore can perform well in scenes illuminated with non-uniform illuminant. An approach is proposed to use the algorithm with cameras of unknown sensitivity. The algorithm was tested for separability of perceptually similar colours under the International Commission on Illumination (CIE) standard illuminants and obtained a good performance. It was also tested for colour based object recognition by illuminating objects with typical indoor illuminants and obtained a better performance compared to other existing algorithms investigated in this paper. Finally, the algorithm was tested for skin detection invariant to illuminant, ethnic background and imaging device. In this investigation, daylight scenes under different weather conditions and scenes illuminated by typical indoor illuminants were used. The proposed algorithm gives a better skin detection performance compared to widely used standard colour spaces. Based on the results presented, the proposed illuminant invariant chromaticity space can be used for machine vision applications including illuminant invariant colour based object recognition and skin detection.

A Fast Vision System for Soccer Robot

This paper proposes a fast colour-based object recognition and localization for soccer robots. The traditional HSL colour model is modified for better colour segmentation and edge detection in a colour coded environment. The object recognition is based on only the edge pixels to speed up the computation. The edge pixels are detected by intelligently scanning a small part of whole image pixels which is distributed over the image. A fast method for line and circle centre detection is also discussed. For object localization, 26 key points are defined on the soccer field. While two or more key points can be seen from the robot camera view, the three rotation angles are adjusted to achieve a precise localization of robots and other objects. If no key point is detected, the robot position is estimated according to the history of robot movement and the feedback from the motors and sensors. The experiments on NAO and RoboErectus teen-size humanoid robots show that the proposed vision system is robust and accurate under different lighting conditions and can effectively and precisely locate robots and other objects.

A fast vision system for soccer robot

This paper proposes a fast colour-based object recognition and localization for soccer robots. The traditional HSL colour model is modified for better colour segmentation and edge detection in a colour coded environment. The object recognition is based on only the edge pixels to speed up the computation. The edge pixels are detected by intelligently scanning a small part of whole image pixels which is distributed over the image. A fast method for line and circle centre detection is also discussed. For object localization, 26 key points are defined on the soccer field. While two or more key points can be seen from the robot camera view, the three rotation angles are adjusted to achieve a precise localization of robots and other objects. If no key point is detected, the robot position is estimated according to the history of robot movement and the feedback from the motors and sensors. The experiments on NAO and RoboErectus teen-size humanoid robots show that the proposed vision system is robust and accurate under different lighting conditions and can effectively and precisely locate robots and other objects.

모듈형 로봇의 자가 결합을 위한 퍼지 주행 제어 및 장애물 회피 제어

자기 자신의 형태를 변형하거나 물리적인 결합으로 재구성하여 새로운 환경에 적응하는 모듈형 자가 결합 로봇은 많은 연구가 필요한 분야이다. 본 논문에서는 물리적으로 결합 가능한 모듈형 로봇을 위한 영상기반의 자가 결합 제어기를 제안한다. 먼저 실시간 영상처리가 가능한 모듈형 로봇 플랫폼을 설계하고, 컬러기반 물체 인식 방법을 구현하였다. 모듈형 로봇은 자가 결합을 위해 결합될 로봇 근처의 부목표점까지 장애물들을 회피하면서 주행해 가야 한다. 본 논문에서는 부 목표점의 추적을 위하여 영상처리를 통해 얻은 거리와 방향각 정보들을 사용한 퍼지 주행 제어기와 장애물 회피를 위한 퍼지 제어기를 제안하고, 제안된 퍼지 제어기들과 로봇의 절대 거리 및 방향각 정보를 사용하여 모듈형 로봇을 위한 자가 결합제어기를 구현하였다. 실제 제작된 두 대의 모듈형 로봇을 사용하여 다양한 환경에서 로봇간 거리와 방향각이 다른 상황에서 실험을 수행하여 제안된 자가 결합 제어 방법의 성능을 검증하였다. Modular reconfigurable robots with physical docking capability easily adapt to a new environment and many studies are necessary for the modular robots. In this paper, we propose a vision-based fuzzy autonomous docking controller for the modular docking robots. A modular docking robot platform which performs real-time image processing is designed and color-based object recognition method is implemented on the embedded system. The docking robot can navigate to a subgoal near a target robot while avoiding obstacles. Both a fuzzy obstacle avoidance controller and a fuzzy navigation controller for subgoal tracking are designed. We propose an autonomous docking controller using the fuzzy obstacle avoidance and navigation controllers, absolute distance information and direction informations of robots from PSD sensors and a compass sensor. We verify the proposed docking control method by docking experiments of the developed modular robots in the various environments with different distances and directions between robots.

Adaptive supervision of moving objects for mobile robotics applications

One of the main tasks of mobile robotics is vision. Lighting independence, adaptivity and automated learning are still the main issues when it comes to applications. In this article, we present an image understanding system and its methods targeting automatic, lighting-independent and reliable color-based object recognition under real time conditions. Its application test bed is global vision robot soccer (i.e. FIRA MiroSot und RoboCup Small Size leagues) but it has many other applications in color-based supervision of moving objects. Under typical conditions, it learns the objects of recognition automatically, has zero setup time and tolerates environmental changes during run-time.

Integrating Color Constancy Into JPEG2000

The human visual system is able to perceive colors as approximately constant. This ability is known as color constancy. In contrast, the colors measured by a sensor vary with the type of illuminant used. Color constancy is very important for digital photography and automatic color-based object recognition. In digital photography, this ability is known under the name automatic white balance. A number of algorithms have been developed for color constancy. We review two well-known color constancy algorithms, the gray world assumption and the Retinex algorithm and show how a color constancy algorithm may be integrated into the JPEG2000 framework. Since computer images are usually stored in compressed form anyway, little overhead is required to add color constancy into the processing pipeline.

Spectral gradients for color-based object recognition and indexing

In the past decade color has been developed as an effective cue for object recognition and image indexing. The initial draw of color was how easily object signatures could be captured and compared. Simple histograms of colors radiated from an object proved to be a stable representation of many objects without requiring complex geometric processing to address object pose. While recognition/indexing should depend only on colors intrinsic to object reflectance, a color signature computed from raw image colors is highly sensitive to the color and pose (direction) of incident illumination. Several methods have explored illumination-color invariants, and most of the recent invariants to illumination color and pose employ some form of chromacity representations based on the ratios of image color values or their combinations. This paper presents a comprehensive framework for logarithmic chromacities for the interpretation of image color change due to illumination pose and color. A log-chromacity space is developed from spectral (color) derivatives of logarithmic image irradiance, called spectral gradients, to achieve invariance to global illumination color and pose. In this space, the multiplicative influence of illumination color and pose on image irradiance become additive effects, and this facilitates the analysis of illumination color and pose. A representation from spatial derivatives of spectral gradients, called spectral–spatial gradients is also developed to additionally provide invariance to local variation of illumination color within an image. Experimental results are presented to demonstrate the efficacy of the proposed descriptors.

A Vision Chip for Color Segmentation and Pattern Matching

A 128(H) 64(V) RGB CMOS imager is integrated with region-of-interest selection, RGB-to-HSI transformation, HSI-based pixel segmentation, (36bins 12bits)-HSI histogramming, and sum-of-absolute-difference (SAD) template matching. Thirty-two learned color templates are stored and compared to each image. The chip captures the R, G, and B images using in-pixel storage before passing the pixel content to a multiplying digital-to-analog converter (DAC) for white balancing. The DAC can also be used to pipe in images for a PC. The color processing uses a biologically inspired color opponent representation and an analog lookup table to determine the Hue (H) of each pixel. Saturation (S) is computed using a loser-take-all circuit. Intensity (I) is given by the sum of the color components. A histogram of the segments of the image, constructed by counting the number of pixels falling into 36 Hue intervals of 10 degrees, is stored on a chip and compared against the histograms of new segments using SAD comparisons. We demonstrate color-based image segmentation and object recognition with this chip. Running at 30 fps, it uses 1 mW. To our knowledge, this is the first chip that integrates imaging, color segmentation, and color-based object recognition at the focal plane.

Illuminant and gamma comprehensive normalisation in logRGB space

The light reflected from an object depends not only on object colours but also on lighting geometry and illuminant colour. As a consequence the raw colour recorded by a camera is not a reliable cue for object based tasks such as recognition and tracking. One solution to this problem is to find functions of image colours that cancel out dependencies due to illumination. While many invariant functions cancel out either dependency due to geometry or illuminant colour, only the comprehensive normalisation has been shown (theoretically and experimentally) to cancel both. However, this invariance is bought at the price of an iterative procedure.The first contribution of this paper is to present a non-iterative comprehensive normalisation procedure. Iteration is avoided by working with logarithms of RGB images rather than the RGBs themselves. We show that under certain simplifying assumptions, in log colour space two simple projection operators lead to invariance to geometry and light colour in a single step.Although both comprehensive normalisation and the non-iterative normalisation work well in the context of colour based object recognition, neither of them accounts for all dependencies that might realistically be present in images. For example, a power (gamma) function is typically applied to image data as part of the coding process and this function can be device and even image dependent. Thus in a second part of the paper we ask whether we can also remove colour dependency due to gamma? We show that we can and furthermore, that invariance can be achieved by adding a single further step to the non-iterative normalisation procedure.Finally, we demonstrate the efficacy of these new normalisation procedures by conducting a series of object recognition experiments on sets of linear and non-linear images.

Color by correlation: a simple, unifying framework for color constancy

The paper considers the problem of illuminant estimation: how, given an image of a scene, recorded under an unknown light, we can recover an estimate of that light. Obtaining such an estimate is a central part of solving the color constancy problem. Thus, the work presented will have applications in fields such as color-based object recognition and digital photography. Rather than attempting to recover a single estimate of the illuminant, we instead set out to recover a measure of the likelihood that each of a set of possible illuminants was the scene illuminant. We begin by determining which image colors can occur (and how these colors are distributed) under each of a set of possible lights. We discuss how, for a given camera, we can obtain this knowledge. We then correlate this information with the colors in a particular image to obtain a measure of the likelihood that each of the possible lights was the scene illuminant. Finally, we use this likelihood information to choose a single light as an estimate of the scene illuminant. Computation is expressed and performed in a generic correlation framework which we develop. We propose a new probabilistic instantiation of this correlation framework and show that it delivers very good color constancy on both synthetic and real images. We further show that the proposed framework is rich enough to allow many existing algorithms to be expressed within it: the gray-world and gamut-mapping algorithms are presented in this framework and we also explore the relationship of these algorithms to other probabilistic and neural network approaches to color constancy.

First steps in robot automation of sampling and sample management during cultivation of mammalian cells in pilot scale.

The robot automation of sampling and the subsequent treatment and storage of aliquots during mammalian cell cultivations was investigated. The complete setup, the development and testing of the sampling device, the robot arm, and the cell imaging system are described. The developed sampling device is directly coupled to a pilot bioreactor. It allows the computerized sterile filling of cell broth into 50 mL sample tubes. After each sampling the whole tubing system is steam sterilized. For further off-line treatment a robot takes the sample to the different devices. This robot is equipped with a camera and a force/torque sensor. A color-based object recognition guides the arm in a complex surrounding with different illumination situations, enabling the robot to load the sampling device with tubes and take the sample to further devices. For necessary pipetting and refilling we developed a computerized device. Cells are automatically stained and counted using an imaging system. The cell number and viability is automatically saved in a process control system together with the on-line parameters. During several cultivations in 20 and 100 L scale these main components of the automation strategy were successfully tested.

Color-based object recognition

The purpose is to arrive at recognition of multicolored objects invariant to a substantial change in viewpoint, object geometry and illumination. Assuming dichromatic reflectance and white illumination, it is shown that normalized color rgb, saturation S and hue H, and the newly proposed color models c 1c 2c 3 and l 1l 2l 3 are all invariant to a change in viewing direction, object geometry and illumination. Further, it is shown that hue H and l 1 l 2 l 3 are also invariant to highlights. Finally, a change in spectral power distribution of the illumination is considered to propose a new color constant color model m 1 m 2 m 3. To evaluate the recognition accuracy differentiated for the various color models, experiments have been carried out on a database consisting of 500 images taken from 3-D multicolored man-made objects. The experimental results show that highest object recognition accuracy is achieved by l 1 l 2 l 3 and hue H followed by c 1 c 2 c 3, normalized color rgb and m 1 m 2 m 3 under the constraint of white illumination. Also, it is demonstrated that recognition accuracy degrades substantially for all color features other than m 1 m 2 m 3 with a change in illumination color. The recognition scheme and images are available within the PicToSeek and Pic2Seek systems on-line at: http: //www.wins.uva.nl/research/isis/zomax/.

Colour-based object recognition under spectrally non-uniform illumination

This paper describes a colour-based recognition system with three novel features. Firstly, the proposed system can operate in environments where spectral characteristics of illumination change in both space and time. Secondly, benefits in terms of speed and quality of output are gained by focusing processing to areas of salient colour. Finally, an automatic model acquisition procedure allows rapid creation of the model database.

Color space analysis of mutual illumination

Mutual illumination occurs when light reflected from one surface impinges on a second one. The resulting additional illumination incident on the second surface affects both the color and intensity of the light reflected from it. As a consequence, the image of a surface in the presence of mutual illumination differs from what it otherwise would have been in the absence of mutual illumination. Unaccounted for mutual illumination can easily confuse methods that rely on intensity or color such as shape-from-shading or color-based object recognition. In this correspondence, we introduce an algorithm that removes mutual illumination effects from images. The domain is that of previously-segmented images of convex surfaces of uniform color and diffuse reflectance where for each surface the interreflection occurs mainly from one other surface and can be accurately accounted for within a one-bounce model. The algorithm is based on a singular value decomposition of the colors coming from each surface. Geometrical information about where on the surface the colors emanate from is not required. The RGB triples from a single convex surface experiencing interreflection fall in a plane; intersecting the planes generated from two interreflecting surfaces results in a unique interreflection color. Each pixel can then be factored into its interreflection and no-interreflection components so that a complete no-interreflection image is produced. >