High Resolution Video Processing Research Articles

Guided Linear Upsampling

Guided upsampling is an effective approach for accelerating high-resolution image processing. In this paper, we propose a simple yet effective guided upsampling method. Each pixel in the high-resolution image is represented as a linear interpolation of two low-resolution pixels, whose indices and weights are optimized to minimize the upsampling error. The downsampling can be jointly optimized in order to prevent missing small isolated regions. Our method can be derived from the color line model and local color transformations. Compared to previous methods, our method can better preserve detail effects while suppressing artifacts such as bleeding and blurring. It is efficient, easy to implement, and free of sensitive parameters. We evaluate the proposed method with a wide range of image operators, and show its advantages through quantitative and qualitative analysis. We demonstrate the advantages of our method for both interactive image editing and real-time high-resolution video processing. In particular, for interactive editing, the joint optimization can be precomputed, thus allowing for instant feedback without hardware acceleration.

APPLICATION OF DISTRIBUTED COMPUTING SYSTEMS FOR IMAGE PROCESSING IN ORDER TO SEARCH FOR UNMANNED AERIAL VEHICLES

This article describes an approach to solving the problem of searching, identifying and tracking UAVs (Unmanned Aerial Vehicles) using a distributed computing system for processing images from multiple surveillance cameras. Today, the problem of finding UAVs is becoming especially relevant due to their widespread distribution and low cost, which gives a wide scope for illegal use: the implementation of terrorist attacks in crowded places and critical infrastructure, as well as unauthorized tracking of specially protected areas. At the same time, modern radars have low efficiency for searching for UAVs, so today visual detection tools are used, for which effective work requires complex calculations. In this article, it is proposed to use distributed computing systems to solve these complex problems of processing a video stream for the purpose of searching, identifying and tracking objects (UAVs). For this, the author of the article, proceeding from the potential areas of application of such systems, decided to apply a multiagent approach, which makes it possible to create fault-tolerant and scalable systems. In the course of work on the article, softwarefor a distributed computing system for image processing in order to search for unmanned aerial vehicleswas created, a hardware stand was assembled to test it. While performing tests, it was concluded that the proposed method can be applied to implement high-resolution video processing and frame rate in a distributed computing system.

Deep learning based HEVC in-loop filter and noise reduction

The lossy compression techniques at low bit rate often create ringing and contouring effects on the output images and introduce various blurring and distortion at block bounders. To overcome those compression artifacts different neural network based post-processing techniques have been experimented with over the last few years. The traditional loop-filter methods in the HEVC frame-work support two post-processing operations namely a de-blocking filter followed by a sample adaptive offset (SAO) filter. These operations usually introduce extra signaling bits and become overhead to the network with high-resolution video processing. In this study, we came up with a new deep learning-based algorithm for SAO filtering operations and substantiated the merits of the proposed method. We introduced a variable filter size sub-layered dense CNN (SDCNN) to improve the denoising operation and incorporated large stride deconvolution layers for further computation improvement. We demonstrate that our deconvolution model can effectively be trained by leveraging the high-frequency edge features learned in a shallow network using residual learning and data augmentation techniques. Extensive experiments show that our approach outperformed other state-of-the-art approaches in terms of SSIM, Bjøntegaard delta bit-rate (BD-BR), BD-PSNR measurements on the standard video test set and achieves an average of 8.73 % bit rate saving compared to HEVC baseline.

Optimizing Ultra High-resolution Video Processing on Mobile Architecture with Massively Parallel Processing

This paper introduces an optimized video frame pre-processing scheme for UHD video with up to 8K resolution using OpenCL for mobile architectures, particularly for convolution. The introduced scheme can fully utilize the maximum computational resources of the mobile architecture, with an adaptive work-group size adjustment. As a prototype, a simple video player with a Sobel kernel was implemented as an example of a convolution kernel. The prototype implementation showed a better video frame processing time than the de-facto image-processing library, OpenCV. On the other hand, the decoding time of the video increased because the OpenCL kernel utilizes GPU (Graphics Processing Unit) resources to almost its maximum. In the future, the processing workload will be distributed between the CPU (Central Processing Unit) and GPU to achieve higher performance.

고해상도 영상의 효과적인 처리를 위한 블록 버퍼 기반의 저 복잡도 무손실 프레임 메모리 압축 방법

본 논문에서는 고해상도 영상의 효과적인 처리를 위한 블록 버퍼 기반의 저 복잡도 무손실 프레임 메모리 (frame memory) 압축 방법을 제안한다. 제안하는 압축 방법은 공간적 상관도를 제거하기 위하여 블록단위 MHT (modified Hadamard transform)를 사용하고, 엔트로피 부호화를 위하여 AGR (adaptive Golomb-Rice) 부호화 기법을 적용하여 저 복잡도 무손실 압축 및 효과적인 하드웨어 구현을 달성한다. MHT는 가산기와 1비트 오른쪽 시프트(1-bit right shift) 연산만으로 구성되어 있고, AGR은 별도의 메모리 공간 및 메모리 접근 동작(memory access operation)을 포함하지 않아 저 복잡도 구현이 용이하다. 기존의 저 복잡도 무손실 압축 방법과 비교하여 제안한 알고리즘은 압축률 측면에서 우수한 성능을 나타내고, 기존 코덱(codec)의 구조를 크게 수정하지 않으면서 화질의 열화없이 하드웨어 장치에 적용될 수 있음을 다양한 영상에 대한 실험 및 복잡도 분석을 통해 보인다. 또한 제안한 방법은 메모리 접근 동작을 필요로 하지 않아 하드웨어 구현을 위한 비용을 최소화 할 수 있어, Fill HD급 이상의 고해상도 영상을 효과적으로 처리하는데 유용하다. This study addresses a low complexity and lossless frame memory compression algorithm based on block-buffer structure for efficient high resolution video processing. Our study utilizes the block-based MHT (modified Hadamard transform) for spatial decorrelation and AGR (adaptive Golomb-Rice) coding as an entropy encoding stage to achieve lossless image compression with low complexity and efficient hardware implementation. The MHT contains only adders and 1-bit shift operators. As a result of AGR not requiring additional memory space and memory access operations, AGR is effective for low complexity development. Comprehensive experiments and computational complexity analysis demonstrate that the proposed algorithm accomplishes superior compression performance relative to existing methods, and can be applied to hardware devices without image quality degradation as well as negligible modification of the existing codec structure. Moreover, the proposed method does not require the memory access operation, and thus it can reduce costs for hardware implementation and can be useful for processing high resolution video over Full HD.

Compressed dynamic mode decomposition for background modeling

We introduce the method of compressed dynamic mode decomposition (cDMD) for background modeling. The dynamic mode decomposition is a regression technique that integrates two of the leading data analysis methods in use today: Fourier transforms and singular value decomposition. Borrowing ideas from compressed sensing and matrix sketching, cDMD eases the computational workload of high-resolution video processing. The key principal of cDMD is to obtain the decomposition on a (small) compressed matrix representation of the video feed. Hence, the cDMD algorithm scales with the intrinsic rank of the matrix, rather than the size of the actual video (data) matrix. Selection of the optimal modes characterizing the background is formulated as a sparsity-constrained sparse coding problem. Our results show that the quality of the resulting background model is competitive, quantified by the F-measure, recall and precision. A graphics processing unit accelerated implementation is also presented which further boosts the computational performance of the algorithm.

A Parallelization Technique with Integrated Multi-Threading for Video Decoding on Multi-core Systems

Increasing demand for Full High-Definition (FHD) video and Ultra High-Definition (UHD) video services has led to active research on high speed video processing. Widespread deployment of multi-core systems has accelerated studies on high resolution video processing based on parallelization of multimedia software. Even if parallelization of a specific decoding step may improve decoding performance partially, such partial parallelization may not result in sufficient performance improvement. Particularly, entropy decoding has often been considered separately from other decoding steps since the entropy decoding step could not be parallelized easily. In this paper, we propose a parallelization technique called Integrated Multi-Threaded Parallelization (IMTP) which takes parallelization of the entropy decoding step, with other decoding steps, into consideration in an integrated fashion. We used the Simultaneous Multi-Threading (SMT) technique with appropriate thread scheduling techniques to achieve the best performance for the entire decoding step. The speedup of the proposed IMTP method is up to 3.35 times faster with respect to the entire decoding time over a conventional decoding technique for H.264/AVC videos.

Multi-threaded syntax element partitioning for parallel entropy decoding

Strong demand for high resolution video services leads to active studies on high speed video processing. Especially, widespread deployment of multi-core systems accelerates researches on high resolution video processing based on parallelization of multimedia software. Even if parallelization of other decoding steps on a multi-core platform may improve performance, entropy decoding often becomes a performance bottleneck since it should be processed sequentially. To resolve this concern, parallel entropy coding algorithms have been proposed. Syntax element partitioning is an algorithm for parallelization of Context Adaptive Binary Arithmetic Coding (CABAC). In this paper, we propose Multi-Threaded Syntax Element Partitioning (MT-SEP) for parallel entropy decoding. One major advantage of software parallel video decoding over hardware implementations will be that versatile video codecs can be implemented flexibly. We parallelized the KTA 2.7 decoder with the proposed technique on an Intel Quad-Core platform. We achieved up to 56% performance improvement using the proposed version of syntax element partitioning.

Stage-based frame-partitioned parallelization of H.264/AVC decoding

Strong demands for high resolution video services lead to active studies on high speed video processing. Especially, widespread deployment of multi-core systems accelerates researches on high resolution video processing based on parallelization of multimedia software. In this paper, we propose a novel parallel H.264/AVC decoding scheme on a homogeneous multi-core platform. Parallelization of H.264/AVC decoding is challenging not only because parallelization may incur significant synchronization overhead but also because software may have complicated dependencies. To overcome such issues, we propose a novel approach called Stage-based Frame-Partitioned Parallelization (SFPP). In SFPP, we divide a frame into multiple partitions, and execute them in a pipelined fashion. To reduce synchronization overhead, a separate thread is allocated to each stage in the pipeline. In addition, an efficient memory reuse technique is used to reduce the memory requirement. To verify the effectiveness of the proposed approach, we parallelized FFmpeg H.264/AVC decoder with the proposed technique using OpenMP, and carried out experiments on an Intel Quad-Core platform. The proposed design performs better than FFmpeg H.264/AVC decoder before parallelization by 53%. We also reduced the amount of memory usage by 65% and 81% for a high-definition (HD) and a full high-definition (FHD) video, respectively compared with that of a popular existing method.