Real-Time Full Color Multiband Night Vision

Abstract

Night vision cameras are widely used for military and law enforcement applications related to surveillance, reconnaissance, intelligence gathering, and security. The two most common night-time imaging systems are low-light-level (e.g., image-intensified) cameras, which amplify the reflected visible to near infrared (VNIR) light, and thermal infrared (IR) cameras, which convert thermal energy from the midwave (3 to 5 microns) or the long wave (8 to 12 microns) part of the spectrum into a visible image. These systems create images with a single (one-dimensional) output per pixel. As a result their ability to discriminate different materials is limited. This can be improved by combining systems that are sensitive to different parts of the electromagnetic spectrum, resulting in multiband or hyperspectral imagers. The number of different outputs increases dramatically by combining multiple sensors (e.g. up to N2 for two sensors, when the number of different outputs for each sensor is N), which in turn leads to a significant increase in the number of materials that can be discriminated. The combination of multiple bands allows for meaningful color representation of the system output. It is therefore not surprising that the increasing availability of fused and multiband infrared and visual nightvision systems (e.g. Bandara et al., 2003; Breiter et al., 2002; Cho et al., 2003; Cohen et al., 2005; Goldberg et al., 2003) has led to a growing interest in the (false) color display of night vision imagery (Li & Wang, 2007; Shi et al., 2005a; Shi et al., 2005b; Tsagaris & Anastasopoulos, 2006; Zheng et al., 2005). In principle, color imagery has several benefits over monochrome imagery for surveillance, reconnaissance, and security applications. The human eye can only distinguish about 100 shades of gray at any instant. As a result, grayscale nightvision images are sometimes hard to interpret and may give rise to visual illusions and loss of situational awareness. Since people can discriminate several thousands of colors defined by varying hue, saturation, and brightness, a false color representation may facilitate nightvision image recognition and interpretation. For instance, color may improve feature contrast, thus enabling better scene segmentation and object detection (Walls, 2006). This may allow an observer to construct a more complete mental representation of the perceived scene, resulting in better situational awareness. It has indeed been found that scene understanding and recognition, reaction time, and object identification are faster and more accurate with color imagery than with monochrome imagery (Cavanillas, 1999; Gegenfurtner & Rieger, 2000; Goffaux et al., 2005; Oliva & Schyns, 2000; Rousselet et al., 2005; Sampson, 1996; Spence et al., 2006; Wichmann et al., 2002). Also, observers are able to selectively attend to task-relevant color targets and to