Adequation and hardware implementation of the color structure descriptor for real-time temporal video segmentation

Abstract

In this paper, we focused on the study of the MPEG-7 color structure descriptor (CSD) for real-time temporal video segmentation. A new hardware architecture of the CSD was proposed. In the aim of optimizing this implementation in terms of hardware resources and execution time, different algorithm transformations have been tested for the considered application. The CSD was applied for different quantization levels in the HMMD color space and for different grey levels, with and without a frame skip. We have demonstrated that the use of a low number of quantization levels and a frame skip can significantly reduce the complexity and assure a better computing performance while preserving a satisfactory level of accuracy in terms of shot boundary detection rate. This is useful for implementation on resource-constrained hardware platform and multiprocessing applications. The performances of the proposed architecture were evaluated for different quantization levels to show the effect on occupied hardware resources and execution time. The comparative study demonstrates the effectiveness of the proposed architecture, which can operate in real-time video and without restriction on image size. In this work we have also designed a system on chip (SOC) for real-time video summarization based on the CSD. The proposed SOC integrating the CSD module was developed using a platform based on a Xilinx Virtex5 FPGA. A complete demonstration, including CSD extraction, shot boundary detection and key-frames visualization, was realized.