Video Coding for Machine

Abstract

Video coding systems, started for TV broadcasting services over satellite and cable networks with limited bandwidth, later on used for surveillance video and internet video, those target on higher compression ratio with lower quality lose, under the trade-off of RDO (rate distortion optimization) model, judged by human experts. In other word, current video coding standards are good for people, for human visual perception, not design for machine intelligence. However, today more and more applications from industry require video coding for machine, which targets to compress image and video for machine usage, object detection and or tracking, image classification, event analysis, and so on, those target on higher compression ratio with higher recognition accuracy, under the trade-off of RAO (rate accuracy optimization) model, judged by system. In this case, video coding needs to do feature compression, which preserves and transmits the most critical information for computer vision and pattern recognition, not for human visual perception. So it is quite different between video coding for human and video coding for machine, even if the two systems will coexist for a long time. In this talk, I will introduce the history of VCM, list some early works on pattern analysis based on compressed data domain, some efforts from ISO/IEC MPEG group on MPEG-7 CDVS (compact descriptor for visual search) and CDVA (compact descriptors for visual analysis), some ongoing projects on AVS working group and MPEG working group, give the key techniques and challenges on VCM, and overview its future.