Pixel-by-pixel VIS/NIR and LIR sensor fusion system

Abstract

Visible (VIS) camera (such as CCD) or Near Infrared (NIR) camera (such as low light level CCD or image intensifier) has high resolution and is easy to distinguish enemy and foe, but it cannot see through thin fog/cloud, heavy smoke/dust, foliage, camouflage, and darkness. The Long Infrared (LIR) imager can overcome above problems, but the resolution is too low and it cannot see the NIR aiming light from enemy. The best solution is to fuse the VIS/NIR and LIR sensors to overcome their shortcomings and take advantages of both sensors. In order to see the same target without parallax, the fusio system must have a common optical aperature. In this paper, three common optical apertures are designed: common reflective objective lens, common beam splitter, and common transmissive objective lens. The first one has very small field of view and the second one needs two heads, so the best choice is the third one, but we must find suitable optical materials and correct the color aberrations from 0.6 to 12 μ. It is a tough job. By choosing ZnSe as the first common piece of the objective lens and using glass for NIR and Ge (or IR glass) for LIR as rest pieces, we only need to and are able to correct the aberrations from 0.6 to 1.0 μ for NIR and from 8 to 12 μ for LIR. Finally, a common reflective objective lens and the common beam splitter are also successfully designed. Five application examples are given. In the digital signal processing, we use only one Altera chip. After inserting data, scaling the image size, and adjusting the signal level, the LIR will have the same format and same pixel number of the VIS/NIR, so real-time pixel-by-pixel sensor fusion is realized. The digital output can be used for further image processing and automatic target recognition, such as if we overlap the LIR image on the VIS/NIR image for missile guidance or rifle sight we don't need to worry about the time and the environment again. A gum-size wireless transmitter is also designed that is able to send the VIS/NIR image, the LIR image, and the fused image to the rear commander 1 km away. The cost of the fusion system is less than $12 K, and a patent is granted to Zybron in April of 2002.