Abstract
Single sensor systems and standard optical—usually RGB CCTV video cameras—fail to provide adequate observations, or the amount of spectral information required to build rich, expressive, discriminative features for object detection and tracking tasks in challenging outdoor and indoor scenes under various environmental/illumination conditions. Towards this direction, we have designed a multisensor system based on thermal, shortwave infrared, and hyperspectral video sensors and propose a processing pipeline able to perform in real-time object detection tasks despite the huge amount of the concurrently acquired video streams. In particular, in order to avoid the computationally intensive coregistration of the hyperspectral data with other imaging modalities, the initially detected targets are projected through a local coordinate system on the hypercube image plane. Regarding the object detection, a detector-agnostic procedure has been developed, integrating both unsupervised (background subtraction) and supervised (deep learning convolutional neural networks) techniques for validation purposes. The detected and verified targets are extracted through the fusion and data association steps based on temporal spectral signatures of both target and background. The quite promising experimental results in challenging indoor and outdoor scenes indicated the robust and efficient performance of the developed methodology under different conditions like fog, smoke, and illumination changes.
Highlights
Numerous monitoring and surveillance imaging systems for outdoor and indoor environments have been developed during the past decades based mainly on standard RGB optical, usually CCTV, cameras
Hyperspectral video systems have been employed for developing object tracking solutions through hierarchical decomposition for chemical gas plume tracking [3]
Multiple object tracking based on background estimation in hyperspectral video sequences as well as multispectral change detection through joint dictionary data have been addressed [4,5]
Summary
Numerous monitoring and surveillance imaging systems for outdoor and indoor environments have been developed during the past decades based mainly on standard RGB optical, usually CCTV, cameras. Most algorithms are based on learning robust background models from standard optical RGB cameras [10,11,12,13,14] and more recently from other infrared sensors and deep learning architectures [15]. Recent advances in machine learning have provided robust and efficient tools for object detection (i.e., point out a bounding box around the object of interest in order to locate it within the image plane) based on deep neural network architectures. Towards a similar direction and aiming at exploiting multisensor imaging systems for challenging indoor and outdoor scenes, in this paper, we propose a fusion strategy along with an object/target detection and verification processing pipeline for monitoring and surveillance tasks. Figure F1i.gIunrech1a. llIenngchinalgleinngdinogorinodrooourtodroour tednovoirroenmvireonntms wenittshwdiythnadmynicaamlliycaclhlyanchgaingincogncdointidoitniosnliske differenlitkesmdioffkeere,nftosgm, ohkue,mfoidg,ithyu, meitdc.ityl,evetecl.s, ,letvheels,stthaendstaarnddamrdomvionvginogbojbejcetctddeetteeccttiioonn aannddtrtarcakciknigng algorithamlgsorfiathilmtos fdaeiltteoctdmeteocvtimngovtianrggetatrsgbeatssbedaseodnojunsjtusatsainsignlgeleimimaagginingg((uussuuaallllyyRRGGBBCCCCTTVV) s)osuorucer.ce
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
