MPEG Reconfigurable Video Coding Research Articles

Energy-aware decoder management: a case study on RVC-CAL specification based on just-in-time adaptive decoder engine

Motivated by the pervasive use of multimedia applications in battery-powered portable devices, the latest video coding standards have been developed to enable higher data compression rates and decoding efficiencies. An MPEG Reconfigurable Video Coding (RVC) framework has been proposed to efficiently support multiple codec specifications and facilitate innovation in codec design. At the same time, an energy-aware design is expected to be included in video coding standards for the purpose of energy optimization of multimedia applications, which are known to create high computational loads. This paper proposes an energy-aware decoder manager based jointly on a Just-In-Time Adaptive Decoder Engine (Jade) and an energy estimation model. The model is driven by Performance Monitoring Counters (PMCs) and is fitted by the multivariate adaptive regression splines (MARS) method. The estimation results achieve average relative errors of less than 10%. In addition, the maximal model computation overhead is 4.58%. By combining the estimation model into the RVC framework, the experimental results show good potential for energy efficiency improvement with increases of up to 21.8% based on energy considerations.

MPEG Reconfigurable Video Coding: From specification to a reconfigurable implementation

This paper demonstrates that it is possible to produce automatic, reconfigurable, and portable implementations of multimedia decoders onto platforms with the help of the MPEG Reconfigurable Video Coding (RVC) standard. MPEG RVC is a new formalism standardized by the MPEG consortium used to specify multimedia decoders. It produces visual representations of decoder reference software, with the help of graphs that connect several coding tools from MPEG standards. The approach developed in this paper draws on Dataflow Process Networks to produce a Minimal and Canonical Representation (MCR) of MPEG RVC specifications. The MCR makes it possible to form automatic and reconfigurable implementations of decoders which can match any actual platforms. The contribution is demonstrated on one case study where a generic decoder needs to process a multimedia content with the help of the RVC specification of the decoder required to process it. The overall approach is tested on two decoders from MPEG, namely MPEG-4 part 2 Simple Profile and MPEG-4 part 10 Constrained Baseline Profile. The results validate the following benefits on the MCR of decoders: compact representation, low overhead induced by its compilation, reconfiguration and multi-core abilities.

Automatic Generation of Optimized and Synthesizable Hardware Implementation from High‐Level Dataflow Programs

In this paper, we introduce the Reconfigurable Video Coding (RVC) standard based on the idea that video processing algorithms can be defined as a library of components that can be updated and standardized separately. MPEG RVC framework aims at providing a unified high‐level specification of current MPEG coding technologies using a dataflow language called Cal Actor Language (CAL). CAL is associated with a set of tools to design dataflow applications and to generate hardware and software implementations. Before this work, the existing CAL hardware compilers did not support high‐level features of the CAL. After presenting the main notions of the RVC standard, this paper introduces an automatic transformation process that analyses the non‐compliant features and makes the required changes in the intermediate representation of the compiler while keeping the same behavior. Finally, the implementation results of the transformation on video and still image decoders are summarized. We show that the obtained results can largely satisfy the real time constraints for an embedded design on FPGA as we obtain a throughput of 73 FPS for MPEG 4 decoder and 34 FPS for coding and decoding process of the LAR coder using a video of CIF image size. This work resolves the main limitation of hardware generation from CAL designs.

LLVM-based and scalable MPEG-RVC decoder

MPEG reconfigurable video coding (RVC) is a new platform-independent specification methodology chosen by the MPEG community for describing coding standards. This methodology aims at producing abstract decoder models (ADMs) of MPEG decoders as programs described in a dataflow language namely Actor Language (RVC-CAL). RVC-CAL naturally expresses potential parallelism between tasks of an application, which makes an ADM description suitable for implementation to a wide variety of platforms, from uniprocessor systems to FPGAs. MPEG RVC eases the development process of decoders by building decoders at a library-component level instead of using monolithic algorithms, and by providing a library of coding tools standardized in MPEG. This paper presents new mechanisms based on the low level virtual machine that allow the conception of a decoder able to dynamically instantiate several RVC decoder descriptions. This decoder, unlike static decoders generated by RVC tools, keeps de facto the features of an RVC description namely portability, scalability and reconfigurability.

Dynamic replacement of video coding elements

The long timescale between the development of new technologies for video coding and their adoption into standards results in a slow improvement in compression efficiency despite the scale of ongoing research into new compression techniques. Standards-based codecs have limited capabilities to adapt to changes in video content, delivery environments, or platforms. There is a growing recognition, for example, with the MPEG Reconfigurable Video Coding initiative, that increased codec flexibility is needed. However, we anticipate that even further developments are required to address these stumbling blocks to video coder advancement. To this end, we present a new approach to video coding which enables flexible and dynamic re-configuration of video coding functions. This adaptability is achieved by sending configuration information to the decoder during a communications session as part of the compressed video signal. The decoder responds to this information by reconfiguring itself to adapt the video decoding process as prompted by the encoder. In this paper we describe a particular example of how dynamic re-configuration may be implemented in a simple video coding scenario, namely, a video coder is reconfigured dynamically by sending descriptions of new transforms during coding. We evaluate five approaches to re-configuration and show that all demonstrate rate-distortion gains over baseline coders, despite the rate increase due to sending configuration information.

Modeling DV/DVCPRO Standards on Reconfigurable Video Coding Framework

After more than 20 years, several video coding standards and technologies have been delivered. Less consideration is taken on their commonalities and interoperations. Specification and reference code of case by case is time consuming. The MPEG reconfigurable video coding (RVC) framework is a new standard under development by MPEG. It aims to provide a unified high‐level specification of current MPEG video coding technologies. In this framework, the decoder is built as a configuration of video coding tools taken from MPEG toolbox library. Up to now, MPEG‐4 simple profile and China audio video coding standard (AVS) decoders have been successfully modeled with RVC framework. In this paper, we examine another video standard, that is, DV/DVCPRO, and model it with RVC‐CAL. The flexibility and ease of RVC‐CAL is demonstrated as well as the validation of RVC modeling.

Overview of the MPEG Reconfigurable Video Coding Framework

Video coding technology in the last 20 years has evolved producing a variety of different and complex algorithms and coding standards. So far the specification of such standards, and of the algorithms that build them, has been done case by case providing monolithic textual and reference software specifications in different forms and programming languages. However, very little attention has been given to provide a specification formalism that explicitly presents common components between standards, and the incremental modifications of such monolithic standards. The MPEG Reconfigurable Video Coding (RVC) framework is a new ISO standard currently under its final stage of standardization, aiming at providing video codec specifications at the level of library components instead of monolithic algorithms. The new concept is to be able to specify a decoder of an existing standard or a completely new configuration that may better satisfy application-specific constraints by selecting standard components from a library of standard coding algorithms. The possibility of dynamic configuration and reconfiguration of codecs also requires new methodologies and new tools for describing the new bitstream syntaxes and the parsers of such new codecs. The RVC framework is based on the usage of a new actor/ dataflow oriented language called CAL for the specification of the standard library and instantiation of the RVC decoder model. This language has been specifically designed for modeling complex signal processing systems. CAL dataflow models expose the intrinsic concurrency of the algorithms by employing the notions of actor programming and dataflow. The paper gives an overview of the concepts and technologies building the standard RVC framework and the non standard tools supporting the RVC model from the instantiation and simulation of the CAL model to software and/or hardware code synthesis.

Synthesizing Hardware from Dataflow Programs

The MPEG Reconfigurable Video Coding working group is developing a new library-based process for building the reference codecs of future MPEG standards, which is based on dataflow and uses an actor language called Cal. The paper presents a code generator producing RTL targeting FPGAs for Cal, outlines its structure, and demonstrates its performance on an MPEG-4 Simple Profile decoder. The resulting implementation is smaller and faster than a comparable RTL reference design, and the second half of the paper discusses some of the reasons for this counter-intuitive result.

Automatic Synthesis of Parsers and Validation of Bitstreams Within the MPEG Reconfigurable Video Coding Framework

Video coding technology has evolved in the past years into a variety of different and complex algorithms. So far the specifications of such standard algorithms have been done case by case, providing monolithic textual and reference software specifications, but without paying any attention to the possibility of further improvements of such monolithic standards. The MPEG Reconfigurable Video Coding (RVC) framework is a new ISO/IEC standard, currently under its final stage of development aiming at providing video codec specifications at the level of coding tools instead of monolithic descriptions. The possibility to select a subset of standard video coding algorithms to specify a decoder that satisfies application specific constraints is very attractive. However, such possibility to reconfigure codecs requires systematic procedures and tools capable of describing the new bitstream syntaxes of such new codecs. Moreover, it becomes also necessary to generate the associated parsers, capable of parsing the new bitstreams. This paper further explains the problem and describes the technologies used to describe new bitstream syntaxes. Additionally, the paper describes the methodologies and the tools for the validation of bitstream syntaxes descriptions as well as a systematic procedure for automatically synthesizing parsers from the bitstream descriptions.

Software Code Generation for the RVC-CAL Language

The MPEG Reconfigurable Video Coding (RVC) framework is a new standard under development by MPEG that aims at providing a unified high-level specification of current and future MPEG video coding technologies using dataflow models. In this framework, a decoder is built as a configuration of video coding modules taken from the standard MPEG toolbox library or proprietary libraries. The elements of the library are specified by a textual description that expresses the I/O behavior of each module and by a reference software written using a subset of the CAL Actor Language named RVC-CAL. A decoder configuration is written in an XML dialect by connecting a set of CAL modules. Code generators are fundamental supports that enable the direct transformation of a high level specification to efficient hardware and software implementations. This paper presents a synthesis tool that from a CAL dataflow program generates C code and an associated SystemC model. The generated code is validated against the original CAL description simulated using the Open Dataflow environment. Experimental results of the translation of two descriptions of an MPEG-4 Simple Profile decoder with different granularities are shown and discussed.