Video Transcoding Research Articles

Moonwalk

Cloud services are becoming increasingly globalized and data-center workloads are expanding exponentially. GPU and FPGA-based clouds have illustrated improvements in power and performance by accelerating compute-intensive workloads. ASIC-based clouds are a promising way to optimize the Total Cost of Ownership (TCO) of a given datacenter computation (e.g. YouTube transcoding) by reducing both energy consumption and marginal computation cost. The feasibility of an ASIC Cloud for a particular application is directly gated by the ability to manage the Non-Recurring Engineering (NRE) costs of designing and fabricating the ASIC, so that it is significantly lower (e.g. 2X) than the TCO of the best available alternative. In this paper, we show that technology node selection is a major tool for managing ASIC Cloud NRE, and allows the designer to trade off an accelerator's excess energy efficiency and cost performance for lower total cost. We explore NRE and cross-technology optimization of ASIC Clouds for four different applications: Bitcoin mining, YouTube-style video transcoding, Litecoin, and Deep Learning. We address these challenges and show large reductions in the NRE, potentially enabling ASIC Clouds to address a wider variety of datacenter workloads. Our results suggest that advanced nodes like 16nm will lead to sub-optimal TCO for many workloads, and that use of older nodes like 65nm can enable a greater diversity of ASIC Clouds.

Moonwalk

Cloud services are becoming increasingly globalized and data-center workloads are expanding exponentially. GPU and FPGA-based clouds have illustrated improvements in power and performance by accelerating compute-intensive workloads. ASIC-based clouds are a promising way to optimize the Total Cost of Ownership (TCO) of a given datacenter computation (e.g. YouTube transcoding) by reducing both energy consumption and marginal computation cost. The feasibility of an ASIC Cloud for a particular application is directly gated by the ability to manage the Non-Recurring Engineering (NRE) costs of designing and fabricating the ASIC, so that it is significantly lower (e.g. 2X) than the TCO of the best available alternative. In this paper, we show that technology node selection is a major tool for managing ASIC Cloud NRE, and allows the designer to trade off an accelerator's excess energy efficiency and cost performance for lower total cost. We explore NRE and cross-technology optimization of ASIC Clouds for four different applications: Bitcoin mining, YouTube-style video transcoding, Litecoin, and Deep Learning. We address these challenges and show large reductions in the NRE, potentially enabling ASIC Clouds to address a wider variety of datacenter workloads. Our results suggest that advanced nodes like 16nm will lead to sub-optimal TCO for many workloads, and that use of older nodes like 65nm can enable a greater diversity of ASIC Clouds.

QoS-Aware Resource Allocation for Video Transcoding in Clouds

As the biggest big data, video data streaming in the network contributes the largest portion of global traffic nowadays and in the future. Due to heterogeneous mobile devices, networks, and user preferences, the demands of transcoding source videos into different versions have increased significantly. However, video transcoding is a time-consuming task, and how to guarantee quality-of-service (QoS) for large video data is very challenging, particularly for those real-time applications that hold strict delay requirement such as live TV. In this paper, we propose a cloud-based online video transcoding (COVT) system aiming to offer economical and QoS guaranteed solution for online large-volume video transcoding. COVT utilizes the performance profiling technique to obtain the different performances of transcoding tasks in different infrastructures. Based on the profiles, we model the cloud-based transcoding system as a queue and derive the QoS values of the system based on the queuing theory. With the analytically derived relationship between QoS values and the number of CPU cores required for transcoding workloads, COVT is able to solve the optimization problem and obtain the minimum resource reservation for specific QoS constraints. A task scheduling algorithm is further developed to dynamically adjust the resource reservation and schedule the tasks so as to guarantee the QoS in runtime. We implement a prototype system of COVT and experimentally study the performance on real-world workloads. Experimental results show that the COVT effectively provisions a minimum number of resources for predefined QoS. To validate the effectiveness of our proposed method under large-scale video data, we further perform simulation evaluation, which again shows that the COVT is capable of achieving economical and QoS-aware video transcoding in cloud environment.

HVTS: Hadoop-Based Video Transcoding System for Media Services

HVTS: Hadoop-Based Video Transcoding System for Media Services

Performance Analysis of Rate Control Scheme in H.264 Video Coding

Background/Objectives: Digital Video streams represent huge amount of data at high definition resolution. The video in its original size needs more time and large storage space, which necessitates video compression. To surpass the challenges in video compression those appear in terms of latency, the rate control scheme is the best method. To provide an efficient video coding based on H.264/AVC at substantially low bit rate. This ensures higher performance in terms of compression ratio, lower complexity and video reconstruction. Methods/Statistical Analysis: The cost estimation technique is complex and takes considerable time for computation. In an attempt to make it simple, rate distortion baseline profile encoder is parallelized, which makes the R-D cost calculation feasible. Findings: In this work, the performance analysis of Rate control scheme with optimized Quantization parameter value is carried out. This rate-control scheme in the macro-block layer of H.264 baseline encoder with bit-stream calculation and distortion evaluate can potentially contribute to efficient video transcoding systems. In this proposed work, the rate control for the prediction frame done after encoding the I-frames. Here QP estimation happens between the encoder interface and the user interface. The rate distortion model is attached to the P-frame to perform the rate control for prediction and encoding the P-frames by estimating the QP values, thus making the algorithm less complex. This method achieved the better quality with the optimized quantization parameter. Applications/Improvements: The Quantization parameter is varied to regulate the coded bit streams to achieve good perceptual quality that is suitable for surveillance applications.

ASIC clouds

GPU and FPGA-based clouds have already demonstrated the promise of accelerating computing-intensive workloads with greatly improved power and performance. In this paper, we examine the design of ASIC Clouds, which are purpose-built datacenters comprised of large arrays of ASIC accelerators, whose purpose is to optimize the total cost of ownership (TCO) of large, high-volume chronic computations, which are becoming increasingly common as more and more services are built around the Cloud model. On the surface, the creation of ASIC clouds may seem highly improbable due to high NREs and the inflexibility of ASICs. Surprisingly, however, large-scale ASIC Clouds have already been deployed by a large number of commercial entities, to implement the distributed Bitcoin cryptocurrency system. We begin with a case study of Bitcoin mining ASIC Clouds, which are perhaps the largest ASIC Clouds to date. From there, we design three more ASIC Clouds, including a YouTube-style video transcoding ASIC Cloud, a Litecoin ASIC Cloud, and a Convolutional Neural Network ASIC Cloud and show 2-3 orders of magnitude better TCO versus CPU and GPU. Among our contributions, we present a methodology that given an accelerator design, derives Pareto-optimal ASIC Cloud Servers, by extracting data from place-and-routed circuits and computational fluid dynamic simulations, and then employing clever but brute-force search to find the best jointly-optimized ASIC, DRAM subsystem, motherboard, power delivery system, cooling system, operating voltage, and case design. Moreover, we show how data center parameters determine which of the many Pareto-optimal points is TCO-optimal. Finally we examine when it makes sense to build an ASIC Cloud, and examine the impact of ASIC NRE.

Machine learning based fast H.264/AVC to HEVC transcoding exploiting block partition similarity

Video transcoding is to convert one compressed video stream to another. In this paper, a fast H.264/AVC to High Efficiency Video Coding (HEVC) transcoding method based on machine learning is proposed by considering the similarity between compressed streams, especially the block partition correlations, to reduce the computational complexity. This becomes possible by constructing three-level binary classifiers to predict quad-tree Coding Unit (CU) partition in HEVC. Then, we propose a feature selection algorithm to get representative features to improve predication accuracy of the classification. In addition, we propose an adaptive probability threshold determination scheme to achieve a good trade-off between low coding complexity and high compression efficiency during the CU depth prediction in HEVC. Extensive experimental results demonstrate the proposed transcoder achieves complexity reduction of 50.2% and 49.2% on average under lowdelay P main and random access configurations while the rate-distortion degradation is negligible. The proposed scheme is proved more effective as comparing with the state-of-the-art benchmarks.

Tackling Complexity in High Performance Computing Applications

We present a software framework that supports the specification of user-definable configuration options in HPC applications independently of the application code itself. Such options include model parameter values, the selection of numerical algorithm, target platform etc. and additional constraints that prevent invalid combinations of options from being made. Such constraints, which are capable of describing complex cross-domain dependencies, are often crucial to the correct functioning of the application and are typically either completely absent from the code or a hard to recover from it. The framework uses a combination of functional workflows and constraint solvers. Application workflows are built from a combination of functional components: higher-order co-ordination forms and first-order data processing components which can be either concrete or abstract, i.e. without a specified implementation at the outset. A repository provides alternative implementations for these abstract components. A constraint solver, written in Prolog, guides a user in making valid choices of parameters, implementations, machines etc. for any given context. Partial designs can be stored and shared providing a systematic means of handling application use and maintenance. We describe our methodology and illustrate its application in two classes of application: a data intensive commercial video transcoding example and a numerically intensive incompressible Navier–Stokes solver.

Energy-Efficient Graphics and Multimedia in 28-nm Carrizo Accelerated Processing Unit

Carrizo is AMD's sixth-generation accelerated processing unit (APU), optimized to deliver performance uplifts and battery-life improvements in thinner and lighter notebook platforms. Its primary goals are to deliver a differentiated gaming and multimedia experience to the end user and to make progress toward AMD's goal of improving energy efficiency in its APUs by 25 times by year 2020. This article describes the new features, low-power architecture, and power management algorithms used to achieve these goals. Carrizo delivered a 63 percent increase in the 3DMark11 benchmark score, a 350 percent increase in video transcoding throughput, a 60 percent reduction in full-screen HD video playback power, and a 50 percent reduction in Windows idle power compared to its predecessor, the Kaveri APU.

Parallel video transcoding using Hadoop MapReduce

Video transcoding has become a key technology for video content distribution network service. In this paper, we propose a novel MapReduce-based parallel video transcoding method. In our method, video files are stored on a shared file system to reduce the overhead of disks I/O and networks in the Hadoop MapReduce. FFmpeg is used to compute the splitting point of the video and the actual video transcoding. Experimental results show that our method can significantly reduce the time of transcoding.

A framework for video transcoding and management in cloud manufacturing

Cloud-based manufacturing has been defined as CMfg. It is a system built on cloud computing technology that can integrate a large number of manufacturing resources. In a CMfg system, there are huge amounts of data and one of them is the video used to monitor manufacturing processes and learn knowledge. In cloud platform, the video data generate fast and have diverse formats, so the need for unified transcoding and management is necessary. In this article, an elastic cloud framework for fast video transcoding process and management is proposed. The core of this framework is a distributed system based on Hadoop that is designed for massive data storage and parallel computing.

Energy-Reduction Offloading Technique for Streaming Media Servers

Recent growth in popularity of mobile video services raises a demand for one of the most popular and convenient methods of delivering multimedia data, video streaming. However, heterogeneity of currently existing mobile devices involves an issue of separate video transcoding for each type of mobile devices such as smartphones, tablet PCs, and smart TVs. As a result additional burden comes to media servers, which pretranscode multimedia data for number of clients. Regarding even higher increase of video data in the Internet in the future, the problem of media servers overload is impending. To struggle against the problem an offloading method is introduced in this paper. By the use of SorTube offloading framework video transcoding process is shifted from the centralized media server to the local offloading server. Thus, clients can receive personally customized video stream; meanwhile the overload of centralized servers is reduced.

Method of Video Transcoding from MPEG-2 to HEVC with Picture Structure Information

Method of Video Transcoding from MPEG-2 to HEVC with Picture Structure Information

Routing Algorithms for Network Function Virtualization Enabled Multicast Topology on SDN

Many multicast services such as live multimedia distribution and real-time event monitoring require multicast mechanisms that involve network functions (e.g., firewall and video transcoding). Network function virtualization (NFV) is a concept that proposes using virtualization to implement network functions on infrastructure building block (such as high volume servers and virtual machines), where software provides the functionality of existing purpose-built network equipment. We present an approach for building the multicast mechanism whereby multicast flows are processed by NFV before reaching their end users. We propose a routing algorithm and a method for building an appropriate multicast topology.

QoS-adaptive service configuration framework for cloud-assisted video surveillance systems

Quality of service (QoS)-adaptive service configuration is crucial for seamless access to video services in cloud-assisted video surveillance systems. To maintain seamless access to video on a user's preferred device, suitable video transcoding services are needed. It is a challenging task to choose and configure these services for various devices to ensure QoS-adaptive user experiences. To configure these services for the desired user devices, a suitable configuration algorithm is needed. Therefore, this paper describes a QoS-adaptive service configuration approach to choose the optimal configuration for the preferred user devices in varied contexts so that the user can access the services ubiquitously. We implemented a cloud-assisted video surveillance prototype to show how the proposed method can handle ubiquitous access to target video for possible QoS-adaptive and video processing requirements in terms of bandwidth, delay, and frame rates. The results show that the proposed configuration method outperforms the other comparable approaches.

Towards Cost-Efficient Video Transcoding in Media Cloud: Insights Learned From User Viewing Patterns

Video transcoding in an adaptive bitrate streaming (ABR) system is demanded to support video streaming over heterogenous devices and varying networks. However, it could incur a tremendous cost. Meanwhile, most viewers terminate viewing sessions within 20% of their durations; only a small fraction of each video is consumed. Built upon this user viewing pattern, we propose a Partial Transcoding Scheme for content management in media clouds. Particularly, each content is encoded into different bitrates and split into segments. Some of the segments are stored in cache, resulting in storage cost; others are transcoded online in the case of cache miss, resulting in computing cost. We aim to minimize the long-term overall cost by determining whether a segment should be cached or transcoded online. We formulate it as a constrained stochastic optimization problem. Leveraging Lyapunov optimization framework and Lagrangian relaxation, we design an online algorithm which can achieve the optimal solution within provable upper bounds. Experiments demonstrate that our proposed method can reduce 30% of operational cost, compared with the scheme of caching all the segments.

A Joint Online Transcoding and Delivery Approach for Dynamic Adaptive Streaming

Dynamic adaptive streaming has emerged as a popular approach for video services in today’s Internet. To date, the two important components in dynamic adaptive streaming, video transcoding that generates the adaptive bitrates of a video and video delivery that streams the videos to users, have been separately studied, resulting in a huge waste of computation and storage resource due to producing and caching different versions of videos regardless of their demands. We conduct extensive measurement studies of video sharing systems, including an IPTV service which streams regular, professionally made videos and an instant video clip sharing service which provides extremely short user-generated videos, as well as the availability of computation resource in conventional content delivery networks (CDNs). Based on the measurement insights, we propose an online joint transcoding and delivery approach for adaptive video streaming. We formulate optimization problems to enable high streaming quality for the users, and low computation and replication costs for the system. In particular, our strategy connects video transcoding and video delivery based on users’ preferences of CDN regions and regional preferences of video versions. We analyze hardness of these problems and design distributed solutions. Extensive trace-driven experiments further demonstrate the superiority of our design.

English

In this paper, we address the problem of selecting and composing video transcoding services in a distributed cloud environment. One of the challenging issues for video transcoding service composition is how to find the best transcoding path to route the data flow through while satisfying the viewer requirements and specifications. In a cloud environment, video transcoding service providers provide different video transcoding services that have similar functionality (i.e., format conversion), but with different Quality of Services (QoS) specifications. Since the combination of the QoS specifications, such as frame size, frame rate, video bit rate, and transcoding delay might affect the end user’s experience in non-intuitive and subjective way and also might affect the delivering of a high quality video content over any type of network, we propose a QoS-aware model to select and compose the best video transcoding services to satisfy hard constraints on the input and output video formats and comes as close as possible to satisfying soft constraints on the QoS. This model uses an aggregate function to evaluate the QoS for each transcoding service and for each viewer request to explore the best composition path. In this paper, we adapt the Simulated Annealing (SA) algorithm and the Genetic Algorithm (GA) as candidate solutions to help in the composition process. The SA/GA algorithms provide multi-constraints QoS assurance for video transcoding service composition. They also support directed acyclic graph composition topology. We have implemented a prototype of the proposed algorithms and conducted experiments using small-, medium-, and large-scale graphs of video transcoders and sample viewer requests to measure the performance and the quality of the results. The experimental results show that the SA outperforms the GA in terms of performance and success ratio for small-scale graph, while GA outperforms the SA algorithm in terms of performance for medium- and large-scale graphs. The success ratio for the SA and GA algorithms are close to each other for medium- and large-scale graphs. At the end, we provide several directions and suggestions for future work.

Low complexity semi-fragile watermarking scheme for H.264/AVC authentication

This paper presents a low complexity configurable semi-fragile watermarking scheme for content-based H.264/AVC authentication, which allows content-preserving manipulations such as video transcoding techniques, while it is very sensitive to content-changing and frame manipulations. A low cost spatial analysis is exploited to maximize robustness and lessen the incurred distortion. Invariant features are extracted from intra macroblocks, and frame index timing information constituting the content-based authentication codes are embedded/extracted into Quantized DCT (QDCT) coefficients in a Group of Pictures GOP-based fashion. The embedded watermark can be detected and verified by partially decoding intra and inter macroblocks without the prior knowledge of the original video or complete video decoding. Spatial and temporal tampering can be characterized and localized by comparing the extracted and re-generated feature-based watermarks. Experimental results demonstrate that the performance of the proposed scheme is indeed superb in terms of bit-rate and perceptual quality. Moreover, various types of attacks have been simulated and characterized.

Scalable Video Transcoding with Hadoop MapReduce in Openstack Juno Platform

Scalable Video Transcoding with Hadoop MapReduce in Openstack Juno Platform