Video Transcoding Research Articles

Edge caching and computing of video chunks in multi-tier wireless networks

This paper proposes a video caching and transcoding strategy for delay-constrained content delivery in multi-tier wireless networks. Particularly for multimedia services whose content can be encoded into multiple quality versions and consists of multiple chunks, we present an approach of caching chunks of an identical file separately in different network layers with different qualities. Given this caching policy, a complete file can be retrieved by receiving its chunks from different network tiers; therefore, a video transcoding and delivery method is jointly designed with caching to maximize the expected content quality with a strict delay constraint. Motivated by that caching is proactively conducted in advance of many user requests, the joint optimization problem is decomposed into caching and transcoding subproblems. The optimal transcoding policy and its transcoded qualities at different network tiers are analytically found by linear programming for a given cache state. Using the closed-form transcoded qualities, the caching policy is optimized to maximize the expected content quality with respect to randomness of user locations and channel gains. Due to the difficulty of expectation, the caching policy of each network layer is independently obtained by water-filling first, and an alternating optimization method is used to find the joint optimal caching policy satisfying the cache size constraints. Simulation results show that the proposed scheme succeeds in delivering content of 5% to 10% higher expected quality to users than comparison schemes which do not use partitioning for caching. Simultaneously, our approach meets the delay requirement and effectively manages the trade-offs arising from the varying storage and computational capacities within hierarchies, as well as video diversity.

Enabling lightweight immersive user interaction in smart buildings through learning-based mobile panorama streaming

Smart buildings integrate users’ digitized wearables with their physical surroundings, creating a seamless and interactive user experience. This is achieved through the utilization of multiple sensors, video streaming, artificial intelligence, and edge computing. These technologies gather extensive data and provide users with a wide range of applications, such as 3D audio/video in AR/VR, localization, virtual tours, and vigilant monitoring. Nevertheless, the current AR/VR devices face limitations due to the bulkiness and discomfort of the hardware used for on-body sensing, such as headsets and specialized glasses. These components often become uncomfortable during prolonged usage, posing a challenge for creating an immersive system that combines lightweight interaction with high-quality presentation.This paper presents a comprehensive system designed to enable immersive interaction in smart buildings with a focus on lightweight solutions. The system consists of the following components: (1). A lightweight panoramic imaging framework to address the challenges related to hardware size and functionality. (2). A learning-based video transcoding cost prediction framework for efficient load balancing. (3). A layered networking architecture designed to facilitate high-quality mobile panorama live streaming. Collectively, these components offer lightweight interaction paired with enhanced presentation quality. Our experimental results demonstrate the effectiveness of the system design, showcasing its seamless operation across different times, geographical locations, and heterogeneous wireless networks.

Adaptive Video Streaming in Multi-Tier Computing Networks: Joint Edge Transcoding and Client Enhancement

With the advancement of multimedia technology and wireless networks, there is a growing demand for high-quality video streaming. Delivering stable video streaming in extremely dynamic wireless networks, nevertheless, is still an open problem. Recent developments in client computing and mobile edge computing (MEC) technologies have both shown promise in enhancing the adaptive bitrate (ABR) streaming services. In this paper, we consider a video streaming system in multi-tier computing networks, enabled by joint edge-side video transcoding and client-side video enhancement. By “enhancement,” we mean that the client improves the video chunk quality via client-side image processing modules. In particular, we aim to design a joint bitrate adaptation, edge transcoding, and client image-processing algorithm, maximizing the quality of experience (QoE) of streaming services. The majority of the prior art has concentrated on super-resolution-enabled video streaming. Contrarily, we show that the video enhancement method outperforms the super-resolution approach in terms of signal-to-noise ratio and frames per second, implying a superior alternative for client-side processing in ABR streaming. We formulate the problem as an event-triggered Markov decision process (E-MDP), and propose a deep reinforcement learning (DRL)-based framework, named EDTEA. To deal with the delayed feedback induced by multi-tier computing, the entropy and the expected re-buffering terms are introduced to the objective and the reward, respectively. Extensive simulations based on real-world videos and bandwidth traces manifest that compared with state-of-the-art approaches, EDTEA provides <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$10.4\%\sim 78.4\%$</tex-math></inline-formula> extra QoE while reducing re-buffering time by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$85.5\%\sim 91.7\%$</tex-math></inline-formula> .

A systematic literature review on video transcoding acceleration: challenges, solutions, and trends

A systematic literature review on video transcoding acceleration: challenges, solutions, and trends

A Multi-User Cost-Efficient Crowd-Assisted VR Content Delivery Solution in 5G-and-Beyond Heterogeneous Networks

The latest evolution of wireless communications enables user access rich Virtual Reality (VR) services via the Internet, including while on the move. However, providing a premium immersive experience for massive number of concurrent users with various device configurations is a significant challenge due to the ultra-high data rate and ultra-low delay requirements of live VR services. This paper introduces an innovative multi-user cost-efficient crowd-assisted delivery and computing (MEC-DC) framework, which leverages mobile edge computing and end-user resources to support high performance VR content delivery over 5G-and-beyond heterogeneous networks (5G-HetNets). The proposed MEC-DC framework is based on three main solutions. First is a novel buffer-nadir-based multicast (BNM) mechanism for VR transmissions over 5G-HetNets. BNM ensures smooth and synchronized user viewing experience by maximizing the average playback buffer-nadir of all participants with stochastic optimization. Second and third are practical distributed algorithms: the cost-efficient multicast-aware transcoding offloading (MATO) and crowd-assisted delivery algorithm (CAD) which optimize jointly multicast delivery and video transcoding. The algorithms optimality and complexity were investigated. The proposed MATO-CAD solution was evaluated with real datasets, trace-driven numerical simulations, and prototype-based experiments. The trace-driven experimental results showed how the proposed solution provides 18% throughput improvement, lowest delay and best playback freeze ratio in comparison with three other state-of-the-art solutions.

Cost-Effective, Quality-Oriented Transcoding of Live-Streamed Video on Edge-Servers

Cost-Effective, Quality-Oriented Transcoding of Live-Streamed Video on Edge-Servers

A fast full partitioning algorithm for HEVC-to-VVC video transcoding using Bayesian classifiers

The Versatile Video Coding (VVC) standard was released in 2020 to replace the High Efficiency Video Coding (HEVC) standard, making it necessary to convert HEVC encoded content to VCC to exploit its compression performance, which was achieved by using a larger block size of 128 × 128 pixels, among other new coding tools. However, 80.93% of the encoding time is spent on finding a suitable block partitioning. To reduce this time, this proposal presents an HEVC-to-VVC transcoding algorithm focused on accelerating the CTU partitioning decisions. The transcoder takes different information from the input bitstream of HEVC, and feeds it to two Bayes-based models. Experimental results show a time saving in the transcoding process of 45.40%, compared with the traditional cascade transcoder. This time gain has been obtained on average for all test sequences in the Random Access scenario, at the expense of only 1.50% BD-rate.

Evaluating digital video transcoding for forensic derivative results

Video data received for analysis often come in a variety of file formats and compression schemes. These data are often transcoded to a consistent file format for forensic examination and/or ingesting into a video analytic system. The file format often requested is an MP4 file format. The MP4 file format is a very common and a universally accepted file format. The practical application of this transcoding process, across the analytical community, has generated differences in video quality. This study sought to explore possible origins of the differences and assist the practitioner by defining minimum recommendations to ensure that quality of the video data is maintained through the transcoding process. This study sought to generate real world data by asking participants to transcode provided video files to an MP4 file format using programs they would typically utilize to perform this task. The transcoded results were evaluated based on measurable metrics of quality. As the results were analyzed, determining why these differences might have occurred became less about a particular software application and more about the settings employed by the practitioner or of the capabilities of the program. This study supports the need for any video examiner who is transcoding video data to be cognizant of the settings utilized by the programs employed for transcoding video data, as loss of video quality can affect analytics as well as further analysis.

Risk-Aware Contextual Learning for Edge-Assisted Crowdsourced Live Streaming

This paper proposes an edge-assisted crowdsourced live video transcoding approach where the transcoding capabilities of the edge transcoders are unknown and dynamic. The resilience and trustworthiness of highly unstable transcoders in decision making are characterized with mean-variance-based measures to avoid making highly risky decisions. The risk level of each device’s situation is assessed and two upper confidence bounds of the variance of transcoding performance are presented. Based on the derived bounds and by leveraging the contextual information of devices, two risk-aware contextual learning schemes are developed to efficiently estimate the transcoding capabilities of the edge devices. Combining context awareness and risk sensitivity, a novel transcoding task assignment and viewer association algorithm is proposed. Simulation results demonstrate that the proposed algorithm achieves robust task offloading with superior network utility performance as compared to the linear upper confidence bound and the risk-aware mean-variance upper confidence bound-based algorithms. In particular, an epoch-based task assignment strategy is designed to reduce the task switching costs incurred in assigning the same transcoding task to different transcoders over time. This strategy also reduces the computational time needed. Numerical results confirm that this strategy achieves up to 86.8% switching costs reduction and 92.3% computational time reduction.

A Novel Strategy to Achieve Video Transcoding Using Cloud Computing

One of the fundamental challenges faced in deploying multimedia systems is delivering smooth and uninterrupted audio-visual information anywhere and anytime. In such systems, multimedia content is compressed within a certain format, this requires format conversion for various devices. Thus, a transcoding mechanism is required to make the content adaptive for various devices in the network. Video transcoding converts one digitally encoded format into another, this involves translating any file format containing video and audio at the same time. This is an essential feature for devices that do not support a specific format of media or have limited storage that requires a reduced file size. Through this paper, we provide a novel way of transcoding the block-based video coding schemes using cloud architecture by establishing a video pipelining architecture. The solution discussed in this paper would enable the end users to extract videos in any format and resolution seamlessly, combined with the scalability, reliability, and cost-effectiveness of the cloud. The proposed idea would be lucrative for all the video streaming applications that are currently relying on their legacy infrastructure for video transcoding.

YouTube Clone by Using Ethereum Block Chain

Decentralized video sharing structures are much like video sharing structures wherein creators post content material and customers view it. However, the primary distinction lies withinside the community in the back of the decentralized video sharing platform. A peer-to-peer (P2P) community of decentralized video sharing structures helps the steady switch of files. To make sure speedy facts switch, facts is break up into smaller blocks for less complicated switch and download, making sure quicker downloads and browsing. The decentralized video sharing platform transfers facts over a P2P community, however with an extra layer of blockchain era encryption. Less operational value, higher fault tolerance, much less consider necessities among garage carriers and facts owners, and much less vulnerability to attacks. An occasion in blockchain era has delivered a decentralized garage mode to the public. Video transcoding is extensively carried out in video streaming commerce, changing films into a couple of codecs for extraordinary audiences.

Degraded Reference Image Quality Assessment.

In practical media distribution systems, visual content usually undergoes multiple stages of quality degradation along the delivery chain, but the pristine source content is rarely available at most quality monitoring points along the chain to serve as a reference for quality assessment. As a result, full-reference (FR) and reduced-reference (RR) image quality assessment (IQA) methods are generally infeasible. Although no-reference (NR) methods are readily applicable, their performance is often not reliable. On the other hand, intermediate references of degraded quality are often available, e.g., at the input of video transcoders, but how to make the best use of them in proper ways has not been deeply investigated. Here we make one of the first attempts to establish a new paradigm named degraded-reference IQA (DR IQA). Specifically, by using a two-stage distortion pipeline we lay out the architectures of DR IQA and introduce a 6-bit code to denote the choices of configurations. We construct the first large-scale databases dedicated to DR IQA and will make them publicly available. We make novel observations on distortion behavior in multi-stage distortion pipelines by comprehensively analyzing five multiple distortion combinations. Based on these observations, we develop novel DR IQA models and make extensive comparisons with a series of baseline models derived from top-performing FR and NR models. The results suggest that DR IQA may offer significant performance improvement in multiple distortion environments, thereby establishing DR IQA as a valid IQA paradigm that is worth further exploration.

Detection and Localization of Video Transcoding From AVC to HEVC Based on Deep Representations of Decoded Frames and PU Maps

Detection and Localization of Video Transcoding From AVC to HEVC Based on Deep Representations of Decoded Frames and PU Maps

User-Generated Video Quality Assessment: A Subjective and Objective Study

Recently, we have observed an exponential increase of user-generated content (UGC) videos. The distinguished characteristic of UGC videos originates from the video production and delivery chain, as they are usually acquired and processed by non-professional users before uploading to the hosting platforms for sharing. As such, these videos usually undergo multiple distortion stages that may affect visual quality before ultimately being viewed. Inspired by the increasing consensus that the optimization of the video coding and processing shall be fully driven by the perceptual quality, in this paper, we propose to study the quality of the UGC videos from both objective and subjective perspectives. We first construct a UGC video quality assessment (VQA) database, aiming to provide useful guidance for the UGC video coding and processing in the hosting platform. The database contains source UGC videos uploaded to the platform and their transcoded versions that are ultimately enjoyed by end-users, along with their subjective scores. Furthermore, we develop an objective quality assessment algorithm that automatically evaluates the quality of the transcoded videos based on the corrupted reference, which is in accordance with the application scenarios of UGC video sharing in the hosting platforms. The information from the corrupted reference is well leveraged and the quality is predicted based on the inferred quality maps with deep neural networks (DNN). Experimental results show that the proposed method yields superior performance. Both subjective and objective evaluations of the UGC videos also shed lights on the design of perceptual UGC video coding.

Online Orchestration of Collaborative Caching for Multi-Bitrate Videos in Edge Computing

In the traditional video streaming service provisioning paradigm, users typically request video contents through nearby Content Delivery Network (CDN) server(s). However, because of the uncertain wide area networks delays, the (remote) users usually suffer from long video streaming delay, which affects the quality of experience. Multi-Access Edge Computing (MEC) offers caching infrastructures in closer proximity to end users than conventional Content Delivery Networks (CDNs). Yet, for video caching, MEC's potential has not been fully unleashed as it overlooks the opportunities of collaborative caching and multi-bitrate video transcoding. In this paper, we model and formulate an Integer Linear Program (ILP) to capture the long-term cost minimization problem for caching videos at MEC, allowing joint exploitation of MEC with CDN and real-time video transcoding to satisfy arbitrary user demands. While this problem is intractable and couples the caching decisions for adjacent time slots, we design a polynomial-time online orchestration framework which first relaxes and carefully decomposes the problem into a series of subproblems solvable in each individual time slot and then converts the fractional solutions into integers without violating constraints. We have formally proved a parameterized-constant competitive ratio as the performance guarantee for our approach, and also conducted extensive evaluations to confirm its superior practical performance. Simulation results demonstrate that our proposed algorithm outperforms the state-of-the-art algorithms, with 13.6% improvement on average in terms of total cost.

Artificial intelligence and big data driven digital media design

Traditional digital media system can not complete the format conversion and video transcoding of massive image and video information at the same time, which leads to long time of information processing and loss of data storage. Therefore, a digital media system driven by artificial intelligence and big data is designed. Using FastDFS design of digital media data management module. Design digital media image, video conversion module and digital media resource data dictionary library. Develop image plug-ins based on the MapInfo platform. Design video plug-in, introduce virtual reality technology to retrieve image information, call video source, create CvCapture object. Design system software functions and digital media information acquisition algorithm. Intelligent artificial pixel feature acquisition technology is used to collect 3D visual information of digital media and design its pseudo-code. Compared with the traditional system, the information processing time of the designed system is shorter, and it takes 11.555 ms when there are more information objects. The experimental results show that the designed system can complete more complete storage of data.

FPGA HLS Today: Successes, Challenges, and Opportunities

The year 2011 marked an important transition for FPGA high-level synthesis (HLS), as it went from prototyping to deployment. A decade later, in this article, we assess the progress of the deployment of HLS technology and highlight the successes in several application domains, including deep learning, video transcoding, graph processing, and genome sequencing. We also discuss the challenges faced by today’s HLS technology and the opportunities for further research and development, especially in the areas of achieving high clock frequency, coping with complex pragmas and system integration, legacy code transformation, building on open source HLS infrastructures, supporting domain-specific languages, and standardization. It is our hope that this article will inspire more research on FPGA HLS and bring it to a new height.

Warehouse-Scale Video Acceleration

Video processing is foundational to a spectrum of important workloads: video sharing, video conferencing, photos/video archival, virtual/augmented reality, cloud gaming, and live streaming. The exponential growth in these workloads, coincident with a slowing Moore’s law, poses significant challenges to deliver more computing at higher efficiencies. In this article, we present the design of a new accelerator—the video coding unit—targeted at warehouse-scale (cloud) video transcoding. Our deployment at scale in Google, the first of its kind, has demonstrated significant benefits on several products (YouTube, Meet, Stadia, Photos, etc.) serving billions of users.

Transcoded Video Restoration by Temporal Spatial Auxiliary Network

In most video platforms, such as Youtube, Kwai, and TikTok, the played videos usually have undergone multiple video encodings such as hardware encoding by recording devices, software encoding by video editing apps, and single/multiple video transcoding by video application servers. Previous works in compressed video restoration typically assume the compression artifacts are caused by one-time encoding. Thus, the derived solution usually does not work very well in practice. In this paper, we propose a new method, temporal spatial auxiliary network (TSAN), for transcoded video restoration. Our method considers the unique traits between video encoding and transcoding, and we consider the initial shallow encoded videos as the intermediate labels to assist the network to conduct self-supervised attention training. In addition, we employ adjacent multi-frame information and propose the temporal deformable alignment and pyramidal spatial fusion for transcoded video restoration. The experimental results demonstrate that the performance of the proposed method is superior to that of the previous techniques. The code is available at https://github.com/icecherylXuli/TSAN.

Robust video steganography for social media sharing based on principal component analysis

Most social media channels are lossy where videos are transcoded to reduce transmission bandwidth or storage space, such as social networking sites and video sharing platforms. Video transcoding makes most video steganographic schemes unusable for hidden communication based on social media. This paper proposes robust video steganography against video transcoding to construct reliable hidden communication on social media channels. A new strategy based on principal component analysis is provided to select robust embedding regions. Besides, side information is generated to label these selected regions. Side information compression is designed to reduce the transmission bandwidth cost. Then, one luminance component and one chrominance component are joined to embed secret messages and side information, notifying the receiver of correct extraction positions. Video preprocessing is conducted to improve the applicability of our proposed method to various video transcoding mechanisms. Experimental results have shown that our proposed method provides stronger robustness against video transcoding than other methods and achieves satisfactory security performance against steganalysis. Compared with some existing methods, our proposed method is more robust and reliable to realize hidden communication over social media channels, such as YouTube and Vimeo.