Multimodal image feature detection with ROI-based optimization for image registration

Abstract

Image registration plays an imperative part of multimodal video analysis system. In video surveillance applications, change in the environmental conditions makes the registration process hard. Use of multiple sensors makes the system more robust to environmental changes as compared to single sensor imaging system. Using multiple modalities such as infrared(IR)/thermal sensors and CMOS image sensors augment the sturdiness of the surveillance system. Here we propose hardware implementation of feature detection on Genesys 2 Kintex-7 FPGA for a multimodal surveillance system, which is robust in poor lighting conditions and affine changes. To reduce the processing time, a region of interest (ROI) is identified and feature extraction is performed in this region. Design optimization in hardware architecture resulted in achieving the real-time performance of image registration on HD 720p video.