Reconfigurable Video Research Articles

A Model-Driven Platform for Dynamic Partially Reconfigurable Architectures: A Case Study of a Watermarking System

The reconfigurable feature of FPGAs (Field-Programmable Gate Arrays) has made them a very attractive solution for implementing adaptive systems-on-chip. However, this implies additional design tasks to handle system reconfiguration and control, which increases design complexity. To address this issue, this paper proposes a model-driven design flow that guides the designer through the description of the different elements of a reconfigurable system. It is based on high-level modeling using an extended version of the MARTE (Modeling and Analysis of Real-Time and Embedded systems) UML (Unified Modeling Language) profile. Both centralized and decentralized reconfiguration decision-making solutions are possible with the proposed flow, allowing it to adapt to various reconfigurable systems constraints. It also integrates the IP-XACT standard (standard for the description of electronic Intellectual Properties), allowing the designer to easily target different technologies and commercial FPGAs by reusing both high-level models and actual IP-XACT hardware components. At the end of the flow, the implementation code is generated automatically from the high-level models. The proposed design flow was validated through a reconfigurable video watermarking application as a case study. Experimental results showed that the generated system allowed a good trade-off between resource usage, power consumption, execution time, and image quality compared to static implementations. This hardware efficiency was achieved in a very short time thanks to the design acceleration and automation offered by model-driven engineering.

Scenario-based Validation & Verification

Automated systems can be found on many current vehicles, either land, air or maritime. The reliability, safety and robustness of these systems is extremely important, hence validation approaches need to adapt to the ever-evolving necessities of the industry. The ENABLE-S3 architecture addresses the problem of extensive testing by introducing a set of tools and methodologies that can be used to build up a testing environment for different domains. In this manuscript, a special focus is given to the solution developed for the Reconfigurable Video Processor from the aerospace domain.

Enabling Dynamic System Integration on Maxeler HLS Platforms

High Level Synthesis (HLS) tools enable application domain experts to implement applications and algorithms on FPGAs. The majority of present FPGA applications is following a stream processing model which is almost entirely implemented statically and not exploiting the benefits enabled by partial reconfiguration. In this paper, we propose a generic approach for implementing and using partial reconfiguration through an HLS design flow for Maxeler platforms. Our flow extracts HLS generated HDL code from the Maxeler compilation process in order to implement a static FPGA infrastructure as well as run-time reconfigurable stream processing modules. As a distinct feature, our infrastructure can accommodate multiple partial modules in a pipeline daisy-chained manner, which aligns directly to Maxeler’s dataflow programming paradigm. The benefits of the proposed flow are demonstrated by a case study of a dynamically reconfigurable video processing pipeline delivering 6.4GB/s throughput.

An FPGA-friendly CABAC-encoding architecture with dataflow modelling programming

ABSTRACTVideo compression standards play an important role in video encoding, transmitting and decoding. To exploit the similarities or commonality among standards, a Reconfigurable Video Coding framework is developed in MPEG by employing a dataflow modelling method to modulate the basic configuration components of encoders or decoders. However, the entropy coding for bitstream generating and parsing during the configuration process is very complex, especially when employing the Context Adaptive Based Arithmetic Coding (CABAC). This paper proposes an optimized ‘Producer–Consumer’ architecture for CABAC by dataflow modelling. To achieve high-throughput and low-resource consumption, the buffer accessing speed and buffer size in the architecture is analysed and refined. The proposed CABAC is implemented by dataflow language Cal and is synthesized to FPGA. Results show that it can process 3.5 bins/cycle with a 10-byte buffer consumption at a 120 MHz working frequency. It is sufficient for real-time encoding of H.265/HEVC at level 6.2 main tier.

Reconfigurable Parser Architecture Design with Microprogrammed Controller for Multiple Purposes

This paper utilizes the flexibility of microprogrammed controller with reloadable microcodes for developing reconfigurable parser for multiple purposes. Based on control-dataflow, with microprogrammed controller taken into consideration due to the nature of feedback control in parser, this paper proposes a reconfigurable parser through extracting control commonalities to form shared microinstructions so that the current architecture alleviates the cost in switching control signals for distinct purposes. We employ reconfigurable video coding as a case study to justify the advantages in reconfigurable parser with microprogrammed controller in comparison with finite state machine based controller. Using TSMC 0.18 μm CMOS technology at 108 MHz operating frequency, we reduce 8.93 % gate counts and increase throughput rate twice in comparison with individually implemented finite state machine based controller. We have demonstrated that microprogrammed controller is the trend of flexible architecture design for multiple purposes. Owing to high proportion of shared microinstruction, the higher saving ratio could be envision when multiple purposes are involved in the proposed reconfigurable parser, e.g., more video coding standards.

A multicore DSP HEVC decoder using an actorbased dataflow model and OpenMP

Video coding is one of the most demanding applications, in terms of computational cost, for portable multimedia terminals. In the last years, the new video coding standards, like High Efficiency Video Coding (HEVC), and the increasing resolutions of video codecs have overtaken the capacities of the single core processors in embedded systems. In consequence, multicore architectures are used in current multimedia systems. Besides, new methodologies and frameworks are arising to speed-up the design cycle. In this paper, a methodology based on the Reconfigurable Video Coding CAL Actor Language (RVC-CAL) and the OpenMP API has been used to implement an HEVC decoder based on a multicore DSP. A RVC-CAL description of the HEVC decoder has been used as starting point. The Open RVC-CAL compiler framework (Orcc) has been used to generate C-code from the RVC-CAL specification. This code and the OpenMP library have been ported to the multicore DSP environment. Decoders running on 1, 2, 3 and 4 cores have been tested. Also, the multi DSP based HEVC decoder has been compared with other implementations based on multicore GPPs.

Early Stage Automatic Strategy for Power-Aware Signal Processing Systems Design

Complexity management, portability and long term adaptivity are common challenges in different fields of embedded systems, normally colliding with the needs of efficient resource utilization and power balance. Image/signal processing systems, though required to offer a large variety of complex functions, have also to deal with battery-life limitations. Wearable signal processing systems, for example, should provide high performance and support new generation standards without compromising their portability and their long-term usability. These constraints challenge hardware designers: early stage trade-off analysis and power management automated techniques are helpful to guarantee a reasonable time-to-market. In the field of video codec specifications, the MPEG standard known as Reconfigurable Video Coding (RVC) framework addresses functional complexity and adaptivity leveraging on the intrinsic modularity of the dataflow model of computation, but it still lacks in offering power management support. The main contribution of this work is providing an automatic early-stage power management methodology to be adopted within the MPEG-RVC context. Starting from different high-level specifications, our mapping methodology identifies directly on the high-level models disjointed homogeneous logic clock regions, where the platform resources can be enabled/disabled together without affecting the overall system performance. To extend its usability to the RVC community, we have integrated this methodology within the Multi-Dataflow Composer (MDC) tool. MDC is a tool for on-the-fly reconfigurable signal processing platforms deployment. In this paper, we extended MDC to address power-aware multi-context systems. To prove the effectiveness of our work, a coprocessor for image and video processing acceleration has been assembled. This latter has been synthesized on a 90 nm ASIC technology, where demonstrated up to 90 % of reduction in the dynamic power consumption on different dataflow-intensive applications. The coprocessor has been implemented also on FPGA, confirming, partially, the benefits of adopting the proposed methodology.

Synthesis of Reconfigurable Video Compression Modules in Virtex FPGAs for Multiple Fault Repair Mechanism

This paper proposes a self-repairable video compression module that can be used for space applications. A brief idea of the self-repair technique that is capable of handling multiple faults and the various steps involved in designing the video compression modules is discussed in this paper. The different steps involved in compression are designed using Verilog HDL. The design of compression steps as separate reconfigurable modules is simulated and synthesized using Xilinx ISE 14.2. The design is also synthesized using various Virtex FPGA devices to find out which device will save more logic resources and time on implementing the technique.

Energy estimation models for video decoders: reconfigurable video coding‐CAL case‐study

In this study, a platform-independent energy estimation methodology is proposed to estimate the energy consumption of reconfigurable video coding (RVC)-CAL video codec specifications. This methodology is based on the performance monitoring counters (PMCs) of embedded platforms and demonstrates its portability, simplicity and accuracy for on-line estimation. It has two off-line procedure stages: the former, which automatically identifies the most appropriate PMCs with no specific detailed knowledge of the employed platform, and the latter, which trains the model using either a linear regression or a multivariable adaptive regression splines (MARS) method. Experimenting on an RVC-CAL decoder, the proposed PMC-driven model can achieve an average estimation error <10%. In addition, the maximal model computation overhead is 4.04%. The results show that the training video sequence has significant influence on the model accuracy. An experimental metric is introduced to achieve more stable accurate models based on a combination of training sequences. Furthermore, a comparison demonstrates better predictive ability of MARS techniques in scenarios with multi-core platforms. Finally, the experimental results show a good potential of energy efficiency improvement when the estimation model is combined into the RVC framework. In two different scenarios, the battery lifetime is increased 5.16% and 20.9%, respectively.

Coarse‐grained reconfiguration: dataflow‐based power management

Power reduction in modern embedded systems design is a challenging issue exacerbated by the complexity and heterogeneity of their architecture. In the field of Reconfigurable Video Coding (RVC), to challenge these issues and cut-down time to market, dataflow-based techniques have been adopted. In particular, to master management and composability of dynamically reconfigurable systems, the authors have developed the multi-dataflow composer. Nevertheless, despite the RVC offers several different tools, in its reference design framework power management is still an open issue. To make some steps forward towards filling this gap, in this study, they address power management for coarse-grained reconfigurable systems combining structural and dynamic strategies, both to be applied at the dataflow level.

Energy-aware decoder management: a case study on RVC-CAL specification based on just-in-time adaptive decoder engine

Motivated by the pervasive use of multimedia applications in battery-powered portable devices, the latest video coding standards have been developed to enable higher data compression rates and decoding efficiencies. An MPEG Reconfigurable Video Coding (RVC) framework has been proposed to efficiently support multiple codec specifications and facilitate innovation in codec design. At the same time, an energy-aware design is expected to be included in video coding standards for the purpose of energy optimization of multimedia applications, which are known to create high computational loads. This paper proposes an energy-aware decoder manager based jointly on a Just-In-Time Adaptive Decoder Engine (Jade) and an energy estimation model. The model is driven by Performance Monitoring Counters (PMCs) and is fitted by the multivariate adaptive regression splines (MARS) method. The estimation results achieve average relative errors of less than 10%. In addition, the maximal model computation overhead is 4.58%. By combining the estimation model into the RVC framework, the experimental results show good potential for energy efficiency improvement with increases of up to 21.8% based on energy considerations.

Improved 8-Point Approximate DCT for Image and Video Compression Requiring Only 14 Additions

Video processing systems such as HEVC requiring low energy consumption needed for the multimedia market has lead to extensive development in fast algorithms for the efficient approximation of 2-D DCT transforms. The DCT is employed in a multitude of compression standards due to its remarkable energy compaction properties. Multiplier-free approximate DCT transforms have been proposed that offer superior compression performance at very low circuit complexity. Such approximations can be realized in digital VLSI hardware using additions and subtractions only, leading to significant reductions in chip area and power consumption compared to conventional DCTs and integer transforms. In this paper, we introduce a novel 8-point DCT approximation that requires only 14 addition operations and no multiplications. The proposed transform possesses low computational complexity and is compared to state-of-the-art DCT approximations in terms of both algorithm complexity and peak signal-to-noise ratio. The proposed DCT approximation is a candidate for reconfigurable video standards such as HEVC. The proposed transform and several other DCT approximations are mapped to systolic-array digital architectures and physically realized as digital prototype circuits using FPGA technology and mapped to 45 nm CMOS technology.

Reconfigurable media coding: An overview

This paper provides an overview of the rationale of the Reconfigurable Media Coding framework developed by MPEG standardization committee to overcome the limits of traditional ways of providing decoder specifications. Such framework is an extension of the Reconfigurable Video coding framework now encompassing also 3D Graphics coding standard. The idea of this approach is to specify decoders using an actor dataflow based representation consisting of self-contained processing units (coding tools) connected altogether and communicating by explicitly exchanging data. Such representation provides a specification for which several properties of the algorithms interesting for codec implementations are explicitly exposed and can be used for exploring different implementation objectives.

Automated design of networks of transport-triggered architecture processors using dynamic dataflow programs

Modern embedded systems show a clear trend towards the use of Multiprocessor System-on-Chip (MPSoC) architectures in order to handle the performance and power consumption constraints. However, the design and validation of dedicated MPSoCs is an extremely hard and expensive task due to their complexity. Thus, the development of automated design processes is of highest importance to satisfy the time-to-market pressure of embedded systems.This paper proposes an automated co-design flow based on the high-level language-based approach of the Reconfigurable Video Coding framework. The designer provides the application description in the RVC-CAL dataflow language, after which the presented co-design flow automatically generates a network of heterogeneous processors that can be synthesized on FPGA chips. The synthesized processors are Very Long Instruction Word-style processors. Such a methodology permits the rapid design of a many-core signal processing system which can take advantage of all levels of parallelism.The toolchain functionality has been demonstrated by synthesizing an MPEG-4 Simple Profile video decoder to two different FPGA boards. The decoder is realized into 18 processors that decode QCIF resolution video at 45 frames per second on a 50MHz FPGA clock frequency. The results show that the given application can take advantage of every level of parallelism.

MPEG Reconfigurable Video Coding: From specification to a reconfigurable implementation

This paper demonstrates that it is possible to produce automatic, reconfigurable, and portable implementations of multimedia decoders onto platforms with the help of the MPEG Reconfigurable Video Coding (RVC) standard. MPEG RVC is a new formalism standardized by the MPEG consortium used to specify multimedia decoders. It produces visual representations of decoder reference software, with the help of graphs that connect several coding tools from MPEG standards. The approach developed in this paper draws on Dataflow Process Networks to produce a Minimal and Canonical Representation (MCR) of MPEG RVC specifications. The MCR makes it possible to form automatic and reconfigurable implementations of decoders which can match any actual platforms. The contribution is demonstrated on one case study where a generic decoder needs to process a multimedia content with the help of the RVC specification of the decoder required to process it. The overall approach is tested on two decoders from MPEG, namely MPEG-4 part 2 Simple Profile and MPEG-4 part 10 Constrained Baseline Profile. The results validate the following benefits on the MCR of decoders: compact representation, low overhead induced by its compilation, reconfiguration and multi-core abilities.

Process Antenna Effect Elimination in Ultra Deep Submicron

This paper analyzes the mechanism of process antenna effect in ultra deep submicron IC physical design and provides the antenna ratio calculation method. A new elimination method of process antenna effect combined with clock tree synthesis is proposed. The elimination method minimizes the impact to the clock latency and clock skew by setting up reasonable constraint for clock tree synthesize. Finally, the elimination method is used during place and route of the physical design of a reconfigurable video decoder chip, which is based on TSMC 65nm low power technology. The proposed method eliminates the process antenna effect of the design effectively, also minimizes the impact to clock tree and chip timing to the least.

Design Automation Framework for Reconfigurable Interconnection Networks

A reconfigurable interconnection network (RIN) is a custom-designed on-chip switching network yielding routing solutions for a pre-given set of applications. Like field programmable gate array (FPGA) routing networks, the RIN is used to make reconfigurable interconnections among functional blocks. Unlike FPGAs, the network topology of a RIN is irregular as it is designed for a given set of routing requirements and optimized for the area cost subject to given delay constraints. In this paper, we propose an automatic design scheme for RINs, including routing specification formulation, graph modelings, network topology designs, routing algorithms and multiplexer-based network circuit implementation. The choice of the design scheme is based on the existing routing network design practices and research, which give feasible solutions. Our scheme is to optimize the designs with the choice of design parameters. A computer-aided design (CAD) tool is developed based on the design scheme, which takes a set of routing requirements as input and produces the corresponding RIN network topology and network circuit in hardware description language format. We present the area costs of various RINs generated by the CAD tool subject to delay constraints, and illustrate the RIN design scheme with a reconfigurable multistream video system.

Dynamically reconfigurable entropy coder for multi-standard video adaptation using FaRM

Dynamic and Partial Reconfiguration (DPR) is a feature present in modern Xilinx FPGAs, bringing flexibility to a whole new level. However, it is not yet wide spread in the industry because of poor performance and a lack of a cost model to estimate a solution early in the design process. In this paper, we present our methodology for developing systems capable of dynamic and partial reconfiguration with strict real-time constraints. Our approach is based on FaRM (Fast Reconfiguration Manager), a high-speed controller reaching the configuration port theoretical throughput in Xilinx FPGAs. FaRM performance is estimated using a cost model, which allows us to determine the optimum FIFO size to satisfy timing constraints with the best resources trade-off. We validate our approach with a video application that should be able to encode an H.264 (HD) and an MPEG-2 (SD) stream at the same time. For this we used two entropy encoders on the same reconfigurable zone, while satisfying constraints determined by the video streams. This is the first step of a fully reconfigurable video adaptation system. We also present our unified reconfigurable zone interfaces, specific to video adaptation.

Hybrid Decoder Reconfiguration of AVS-P7 and MPEG-4 /AVC in the Reconfigurable Video Coding Framework

With the rapid development of video coding technology, all kinds of video coding standards have been advanced in recent years with a variety of different and complex algorithms. They share common and/or similar coding tools, yet there is currently no explicit way to exploit such commonalities at the level of specifications or implementations. Reconfigurable video coding (RVC) is to develop a video coding standard that overcomes many shortcomings of current standardization and specification process by updating and progressively incrementing a modular library of components. In this paper, a hybrid decoder reconfiguration is instantiated in the RVC framework by grouping the coding tools from AVS-P7 and MPEG- 4/AVC. Experimental results show that compared with MPEG-4/AVC baseline profile, the reconfigurable coding system reduces the computational complexity and guarantees the coding performance at low bit rate. Moreover, it enriches the RVC video tool library (VTL) by introducing the coding tools of AVS-P7, and also verifies the flexibility and re-configurability of RVC framework to meet the needs of different applications.

Automatic Generation of Optimized and Synthesizable Hardware Implementation from High‐Level Dataflow Programs

In this paper, we introduce the Reconfigurable Video Coding (RVC) standard based on the idea that video processing algorithms can be defined as a library of components that can be updated and standardized separately. MPEG RVC framework aims at providing a unified high‐level specification of current MPEG coding technologies using a dataflow language called Cal Actor Language (CAL). CAL is associated with a set of tools to design dataflow applications and to generate hardware and software implementations. Before this work, the existing CAL hardware compilers did not support high‐level features of the CAL. After presenting the main notions of the RVC standard, this paper introduces an automatic transformation process that analyses the non‐compliant features and makes the required changes in the intermediate representation of the compiler while keeping the same behavior. Finally, the implementation results of the transformation on video and still image decoders are summarized. We show that the obtained results can largely satisfy the real time constraints for an embedded design on FPGA as we obtain a throughput of 73 FPS for MPEG 4 decoder and 34 FPS for coding and decoding process of the LAR coder using a video of CIF image size. This work resolves the main limitation of hardware generation from CAL designs.