Ultra High Definition Research Articles

A Full-Featured FPGA-Based Pipelined Architecture for SIFT Extraction

Image feature detection is a key task in computer vision. Scale Invariant Feature Transform (SIFT) is a prevalent and well known algorithm for robust feature detection. However, it is computationally demanding and software implementations are not applicable for real-time performance. In this paper, a versatile and pipelined hardware implementation is proposed, that is capable of computing keypoints and rotation invariant descriptors on-chip. All computations are performed in single precision floating-point format which makes it possible to implement the original algorithm with little alteration. Various rotation resolutions and filter kernel sizes are supported for images of any resolution up to ultra-high definition. For full high definition images, 84 fps can be processed. Ultra high definition images can be processed at 21 fps.

Hybrid Ultra-Wide Viewing Broadcasting Platform With Reduced Keypoint Redundancy Matching and Robust Blending for Parallax Errors

Advanced Television Systems Committee (ATSC) 3.0 is a more innovative standardization including Ultra High Definition (UHD) broadcasting and mobile High-Definition (HD) broadcasting. It also includes the deployment of a disaster information alarm function as well as the adoption of All-Internet Protocol (IP) based transmission in terrestrial broadcasting networks. Thus, ATSC enables additional technologies such as hybrid broadcasting services by interworking terrestrial broadcasting and internet networks for bidirectional customized services. In addition, there is a growing interest in wider viewing contents, which provide more immersive contents than existing UHD broadcasting mediums. Wider viewing contents can be acquired by using large field of view (FoV) lenses such as fish-eye lenses or by stitching several sub-images. The images captured by large FoV lenses have heavily curved surface distortions, while the distortion of the stitching-based wider viewing contents caused by the synthesizing process on overlapping regions is lower. Provision of wider viewing contents over the traditional broadcasting service has limitations in terms of bandwidth. However, they can be overcome by using hybrid broadcasting platforms such as ATSC 3.0, which transmits a video sequence over an RF channel and other viewing-angle video sequences through the Internet network, where signalling among those video sequences is required. Therefore, this paper proposes a hybrid broadcasting platform for transmitting multiple video sequences over broadcasting and Internet networks, and stitching them for wider viewing.

VVC In-Loop Filtering Based on Deep Convolutional Neural Network.

With the rapid advancement in many multimedia applications, such as video gaming, computer vision applications, and video streaming and surveillance, video quality remains an open challenge. Despite the existence of the standardized video quality as well as high definition (HD) and ultrahigh definition (UHD), enhancing the quality for the video compression standard will improve the video streaming resolution and satisfy end user's quality of service (QoS). Versatile video coding (VVC) is the latest video coding standard that achieves significant coding efficiency. VVC will help spread high-quality video services and emerging applications, such as high dynamic range (HDR), high frame rate (HFR), and omnidirectional 360-degree multimedia compared to its predecessor high efficiency video coding (HEVC). Given its valuable results, the emerging field of deep learning is attracting the attention of scientists and prompts them to solve many contributions. In this study, we investigate the deep learning efficiency to the new VVC standard in order to improve video quality. However, in this work, we propose a wide-activated squeeze-and-excitation deep convolutional neural network (WSE-DCNN) technique-based video quality enhancement for VVC. Thus, the VVC conventional in-loop filtering will be replaced by the suggested WSE-DCNN technique that is expected to eliminate the compression artifacts in order to improve visual quality. Numerical results demonstrate the efficacy of the proposed model achieving approximately −2.85%, −8.89%, and −10.05% BD-rate reduction of the luma (Y) and both chroma (U, V) components, respectively, under random access profile.

Graph Learning Based Head Movement Prediction for Interactive 360 Video Streaming.

Ultra-high definition (UHD) 360 videos encoded in fine quality are typically too large to stream in its entirety over bandwidth (BW)-constrained networks. One popular approach is to interactively extract and send a spatial sub-region corresponding to a viewer's current field-of-view (FoV) in a head-mounted display (HMD) for more BW-efficient streaming. Due to the non-negligible round-trip-time (RTT) delay between server and client, accurate head movement prediction foretelling a viewer's future FoVs is essential. In this paper, we cast the head movement prediction task as a sparse directed graph learning problem: three sources of relevant information-collected viewers' head movement traces, a 360 image saliency map, and a biological human head model-are distilled into a view transition Markov model. Specifically, we formulate a constrained maximum a posteriori (MAP) problem with likelihood and prior terms defined using the three information sources. We solve the MAP problem alternately using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe (FW) optimization strategy. In each FW iteration, a linear program (LP) is solved, whose runtime is reduced thanks to warm start initialization. Having estimated a Markov model from data, we employ it to optimize a tile-based 360 video streaming system. Extensive experiments show that our head movement prediction scheme noticeably outperformed existing proposals, and our optimized tile-based streaming scheme outperformed competitors in rate-distortion performance.

A New Method for the Simultaneous Creation of Live HDR and SDR Video

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deploying separate high dynamic range (HDR) and standard dynamic range (SDR) production facilities to cover the same event simultaneously can be economically prohibitive. In this case study, we present a new method for creating both ultrahigh-definition (UHD) HDR and HD SDR streams from a single UHD HDR production workflow, developed through a series of real-world UHD HDR production trials in 2018 and 2019, concluding with the 2019 Football Association Challenge Cup (FA Cup), a major U.K. sporting event. We describe the findings from those trials and the lessons learned and include descriptions of what worked well and what worked less well. The workflow solution presented allows for the integration of both HDR camera sources and SDR camera sources (e.g., super slow-motion and specialist cameras), prerecorded or archive SDR content and SDR graphics into a single HDR production environment. It can exploit the backward-compatible nature of the hybrid log-gamma (HLG) HDR solution and recent advances in color volume management. We show how a production’s two outputs, HDR and SDR, can be made using current SDR shading/racking monitoring, with only a single HDR vision guarantee monitor</i> .

Radio Propagation in Terrestrial Broadcasting Television Systems: A Comprehensive Survey

During the last two decades, terrestrial TV broadcasting has evolved from analog to digital technology, permitting a better spectrum efficiency, being more resistant to noise and interference as well as improving signal quality. High Definition TeleVison (HDTV) has been introduced to enhance the viewer's experience. HDTV and Ultra High Definition TeleVision bring the challenge of having a quasi-error free performance at a bit error rate as low as 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-12</sup> ; i.e. less than one uncorrectable error during one hour's continuous transmission of 5 Mbps data stream. Such low error rates require robust standards, careful network planning and optimized service operation. Propagation phenomena have a direct (and possibly critical) impact on those three pillars. This paper fills the current gap of a thorough survey on propagation methods and models for broadcasting. We contribute to this field with a description and analysis of propagation phenomena concerning different aspects of the broadcast network planning, broadcast standard evaluation, and broadcast service operation. The paper provides also a technical perspective of state-of-the-art TV broadcast standards and discusses the relevancy of propagation studies with future and development and regulation challenges.

A Conformance Testing Methodology and System for Cognitive Radios

The fifth generation (5G) of mobile networks has started its operation in some countries and is aimed at meeting demands beyond the current system capabilities such as the huge amount of connected devices from IoT applications (e.g., smart cities), explosive growth of high‐speed mobile data traffic (e.g., ultrahigh definition video streaming), and ultrareliable and low latency communication (e.g., autonomous vehicle). To attend to these needs, the electromagnetic spectrum must be made available, but the static spectrum allocation policy has caused a spectrum shortage and impaired the employment/expansion of the wireless systems. To overcome this issue, the dynamic spectrum access (DSA) has been promoted in 5G/6G networks, which is enabled by the cognitive radio (CR) technology. Although diverse mechanisms have been developed to tackle the challenges that emerge in different CR layers/functionalities, a standardized testing methodology and system for CR is still immature. Existing standards or methodologies and systems for CR only focus on the definition of network technologies (e.g., IEEE 802.22 and IEEE 802.11af), performance evaluation of CR algorithms/mechanisms, or definition of the device cognition level via performance results or psychometric approaches, not covering systems/methodologies to verify if the device meets the CR capabilities and regulatory policies, neglecting the conformance testing. In this respect, this paper proposes a flexible methodology and system for CR conformance testing under two perspectives, functionalities and limits. We instantiate it by using the Universal Software Radio Peripheral (USRP) software‐defined radio platform and present a proof‐of‐concept with a conformance metric. The results show the feasibility of our proposal.

Ultra high definition ultra-high resolution large size LED display

Ultra high definition ultra-high resolution large size LED display

Comparative Rate-Distortion-Complexity Analysis of VVC and HEVC Video Codecs

Versatile Video Coding (VVC/H.266) is the next-generation international video coding standard and a successor to the widespread High Efficiency Video Coding (HEVC/H.265). This paper analyzes the rate-distortion-complexity characteristics of the VVC reference software (VTM10.0) by using HEVC reference software (HM16.22) as an anchor. In this independent study, the rate-distortion performance of VTM was benchmarked against HM with the objective PSNR, SSIM, and VMAF quality metrics and the associated encoder and decoder complexities were profiled at function level using Intel VTune Profiler on Intel Xeon E5-2699 v4 22-core processors. For a fair comparison, all our experiments were conducted under the VTM common test conditions (CTC) that define 10-bit configurations of the VTM codec for the addressed All Intra (AI), Random Access (RA), and Low Delay B (LB) conditions. The VTM CTC test set was also extended with complementary 4K UHD sequences to elaborate RD characteristics with higher resolutions. According to our evaluations, VTM improves the average coding efficiency over HM, depending on quality metric, by 23.0-23.9% under the AI condition, 33.1-36.6% under the RA condition, and 26.7-29.5% under the LB condition. However, the coding gain of VTM comes with $34.0\times $ , $8.8\times $ , and $7.5\times $ encoding complexity over that of HM under the AI, RA, and LB conditions, respectively. The corresponding overhead of the VTM decoder stays steady at $1.8\times $ across all conditions. This study also pinpoints the most complex parts of the VTM codec and discusses practical implementation aspects of prospective real-time VVC encoders and decoders.

Maneuvering Control of Hexrotor UAV Equipped With a Cable-Driven Gripper

This study proposes a control method for the maneuvering control of an aircraft system equipped with a cable-driven gripper. The UAV chosen for the study is a hex-rotor due to its durability and redundancy. The reason for opting for the cable-driven robotic arm is because of its extended range as compared to a fixed robotic arm. The aircraft system also contains an Ultra HD camera which helps to locate and identify the target object. Since the UAV grabs the object, the extra weight introduces some instability in the system. To help stabilize the dynamic behavior of the gripper and reduce external wind disturbance, we need an effective and robust control mechanism. To tackle these issues, this paper designs a control method based on the Model Reference Adaptive Control with an integrator (MRACI) with reference dynamics like an observer. On one hand, first, verify the performance of the proposed controller using computer-based simulations. On the other hand, it also applies the designed method to hardware-based results. The simulations and experimental results show that the designed controller works as intended and the errors are within an acceptable range.

멀티스크린의 발전과 초고화질 콘텐츠 응용에 대한 연구

멀티스크린 영상시스템은 하나의 스크린에 상영되는 영상을 용도에 맞게 여러 화면으로 구성하여 상영할 수 있다. 관객들은 특별한 장치 없이 폭 넓은 실감 영상을 감상할 수 있어 강한 몰입감을 얻을 수 있다. 최근 멀티스크린을 이용한 실감 영상은 Screen X나 Escape와 같은 멀티 프로젝션 기술을 바탕으로 스토리와 정보 전달이 가능한 영화로 제작되고 있다. 또한, 디스플레이의 크기는 점점 대형화 추세이고 화질도 고해상도로 향상되고 있어 HD나 UHD급 디스플레이 여러 대를 입체적으로 구성한 디지털 사이니지 형태로 발전을 가속하고 있으나 촬영된 영상을 사용하는 데는 화질열화로 인해 한계가 있다. 이번 연구는 1950년부터 현재까지 멀티스크린 영상의 발전사와 기술적 분석, 제작 방법에 대해 분석하고 멀티스크린을 이용한 영상 콘텐츠 상영 시 발생되는 화질 열화 최소화 방법에 대해 연구한다. 연구 결과를 바탕으로 촬영된 실사 영상은 디지털 사이니지와 같은 고해상도 영상 상영이 가능한 플랫폼에서 실감영상 구현이 가능할 것이라 기대한다.

DASH Live Broadcast Traffic Model: A Time-Bound Delay Model for IP-Based Digital Terrestrial Broadcasting Systems

This paper proposes a live broadcast traffic model for an internet protocol (IP)-based terrestrial digital broadcasting system to transmit dynamic adaptive streaming over hypertext transfer protocol (DASH) media. The IP-based terrestrial digital broadcasting systems such as Advanced Television Systems Committee (ATSC) 3.0 transmit media content (e.g., full high definition and ultra-high definition) in units of DASH segment files. Although the DASH segment file has the same quality and playback time, the size of each DASH segment file can vary according to the media composition. The transmission resource of the terrestrial broadcasting system has increased the transmission capacity of broadcasting with new technologies. However, the transmission capacity is still limited and fixed compared to wired broadcasting networks. Therefore, a problem occurs with the efficiency of broadcasting resources and transmission delay when transmitting a variable segment file to a terrestrial digital broadcasting network. In this paper, the resource efficiency and transmission delay results that occur when transmitting the actual DASH segment file are simulated through the live broadcast traffic model, and the maximum delay time that a viewer accessing the terrestrial broadcast can experience is presented.

High Throughput CBAC Hardware Encoder With Bin Merging for AVS 2.0 Video Coding

Context-based binary arithmetic coding (CBAC) is an entropy coder of AVS 2.0, the latest Chinese video compression standard for 8K ultra-high-definition (UHD) video, which employs a logarithmic number system to avoid multiplication operations for range updates for binary arithmetic coding (BAE). In this paper, we propose a high-throughput CBAC hardware encoder for 8K UHD. To enhance its throughput, we introduce a bin-merging technique that converts a subsequence of bins into a merged bin without affecting the functionality of the CBAC encoder. Moreover, we also employ a 4-stage pipeline architecture for CBAC and lookup tables for context updates. Consequently, the proposed CBAC encoder can process about 3.94 bins per cycle and achieve a throughput of 2.9 Gbins per second. When synthesized in a TSMC 65nm LP process, it operates up to 735 MHz and the total gate count is 182.5 K. To the best of our knowledge, this is the highest throughput for a CBAC encoder.

Quality-Driven Variable Frame-Rate for Green Video Coding in Broadcast Applications

The Digital Video Broadcasting (DVB) has proposed to introduce the Ultra-High Definition services in three phases: UHD-1 phase 1, UHD-1 phase 2 and UHD-2. The UHD-1 phase 2 specification includes several new features such as High Dynamic Range (HDR) and High Frame-Rate (HFR). It has been shown in several studies that HFR (+100 fps) enhances the perceptual quality and that this quality enhancement is content-dependent. On the other hand, HFR brings several challenges to the transmission chain including codec complexity increase and bit-rate overhead, which may delay or even prevent its deployment in the broadcast echo-system. In this paper, we propose a Variable Frame Rate (VFR) solution to determine the minimum (critical) frame-rate that preserves the perceived video quality of HFR video. The frame-rate determination is modeled as a 3-class classification problem which consists in dynamically and locally selecting one frame-rate among three: 30, 60 and 120 frames per second. Two random forests classifiers are trained with a ground truth carefully built by experts for this purpose. The subjective results conducted on ten HFR video contents, not included in the training set, clearly show the efficiency of the proposed solution enabling to locally determine the lowest possible frame-rate while preserving the quality of the HFR content. Moreover, our VFR solution enables significant bit-rate savings and complexity reductions at both encoder and decoder sides.

Low-Cost and High-Throughput Hardware Design for the HEVC 16x16 2-D DCT Transform

This article presents the hardware design of the 16x16 2-D DCT used in the new video coding standard, the HEVC – High Efficiency Video Coding. The transforms stage is one of the innovations proposed by HEVC, since a variable size transforms stage is available (from 4x4 to 32x32), allowing the use of transforms with larger dimensions than used in previous standards. The presented design explores the 2-D DCT separability property, using two instances of the one-dimension DCT. The architecture focuses on low hardware cost and high throughput, thus the HEVC 16-points DCT algorithm was simplified targeting a more efficient hardware implementation. Operations and hardware minimization strategies were used in order to achieve such simplifications: operation reordering, factoring, multiplications to shift-adds conversion, and sharing of common sub-expressions. The 1-D DCT architectures were designed in a fully combinational way in order to reduce control overhead. A transposition buffer is used to connect the two 1-D DCT architectures. The synthesis was directed to Stratix III FPGA and TSMC 65nm standard cells technologies. The complete 2-D DCT architecture is able to achieve real-time processing for high and ultra-high definition videos, such as Full HD, QFHD and UHD 8K. When compared with related works, the architectures designed in this work reached the highest throughput and the lowest hardware resources consumption.

Hardware Implementation for the HEVC Fractional Motion Estimation Targeting Real-Time and Low-Energy

This paper presents an energy-aware and high-throughput hardware design for the Fractional Motion Estimation (FME) compliant with the High Efficiency Video Coding (HEVC) standard. An extensive software evaluation was performed to guide the hardware design. The adopted strategy mainly consists in using only the four squareshaped Prediction Unit (PU) sizes rather than using all 24 possible PU sizes in the Motion Estimation (ME). This approach reduces about 59% the total encoding time and, as a penalty, it leads to an increase of only 4% in the bit rate for the same image quality. Together with this simplification, a multiplierless approach, algebraic optimizations and low-power techniques were applied to the hardware design to reduce the hardware-resource usage and the energy consumption, maintaining a high processing rate. The architecture was described in VHDL and the synthesis results for ASIC 45nm Nangate standard cells demonstrate that the developed architecture is able to process Ultra-High Definition (UHD) 2160p videos at 60 frames per second (fps), with the lowest power consumption and the lowest hardware-resource usage among the related works.

R/G/B Micro-LEDs for In-Pixel Integrated Arrays and Temperature Sensing

In this paper, we demonstrate that nitride-based R/G/B micro-LEDs allow temperature sensing through a forward voltage response on both temperature and display devices. In temperature sensing, micro-LEDs operate in current constant mode, exhibiting an excellent performance with a sensitivity of 2.6/2.7/2.2 mV/K for blue/green InGaN micro-LEDs and red AlGaInP micro-LED under an injection current density of 1 mA/cm2, which is about two times lower sensitivity than the sensitivity limit obtained from the Shockley ideal diode equation. It is speculated that the lower sensitivity originated from parasitic resistances. It is also found that the band gaps of ternary InGaN and quaternary AlGaInP obtained from temperature-dependent electroluminescence spectra of red micro-LED agrees with the calculated values by using the semiempirical Varshni relationship. The EL of micro-LEDs is characterized by narrow emission widths of 18–41 nm, leading to wide color gamut covering up to 170/97% of the DCI P3/BT 2020, which are the standard color space of ultrahigh definition television system.

Design of Convolutional Neural Network with Cuckoo Search Algorithm for Super-Resolution UHD Systems on FPGA

Ultra high definition (UHD) video is considered as a technology that generates a video image with a resolution 16 times as many pixels as high definition television. Extensive and complex computational difficulties are there to generate a video in super-resolution (SR). Some extensive floating-point computations are needed for the direct implementation of area-pixel scaling, which is a factor for calculating one target pixel. In this paper, a hybrid Convolutional Neural Network and Cuckoo Search (CNN-CS) algorithm are proposed, which can replace the large scale of complex training and time-consuming machine learning technique. Cuckoo Search (CS) algorithm is based on bird’s levy-flight approach that can optimize the CNN. CS forecasting model isolates the redundant or irrelevant image patches and, therefore, it reduces the required memory space. The SR is performed effectively in hardware with less hardware utilization and memory space, where LR input data are processed line-by-line using the proposed hybrid CNN-CS algorithm. To optimize the design constraints, CS forecasting model estimates the required patches. Parallel sub pipeline (PSP) technique improves the parallelism process thereby reducing the computational complexity and improving the throughput rate and speed of execution. Finally, the proposed hybrid CNN-CS SR design is coded in Verilog HDL and is implemented with the XCKU040 FPGA device. To validate the proposed CNN-CS SR algorithm, standard datasets, such as Set-5, Set-14, BS-100 and Urban-100, are used. The visual quality of the proposed CNN-CS SR design is evaluated by the standard evaluation metrics such as peak signal-to-noise ratio (PSNR) and SSIM. The proposed hybrid CNN-CS SR design is compared with the existing state-of-the-art techniques both qualitatively and quantitatively. Experimental results demonstrate that the proposed method improved the throughput of 42.83%, memory space and power consumption are reduced to 16.16%, 63.81%, respectively, over the existing techniques.

Ways of Seeing Animals

This article looks at contemporary visual material aimed at documenting animal life. Focusing essentially on two television series produced by the BBC Natural History Unit, Planet Dinosaur (2011) and Spy in the Wild (2017), I explore the stories that such recent visual productions tell about our capacity to apprehend and represent otherness. A connection immediately appears between the documentary ambition that underpins the series and the latest forms of visual technological innovation. From the use of high and ultra-high definition, to the production of animatronic creatures sent out to “spy” on animals in their habitat, the series highlight the new modalities of seeing produced by the digital era, and the opportunities that such modalities open up for the exploration of natural history. By staging the encounter between two figures of the Other—the animal qua “natural” creature on the one hand, and the machine as a product of culture on the other hand—those productions uniquely question the ways in which we apprehend and produce images of alterity. In doing so, they immediately remind us that in imaging and imagining otherness we always run the risk of obfuscating or overlooking it. This is especially the case when the documentary manifests its own anthropocentrism—whether it tends to assimilate the Other and bypass its singularity or chooses to focus on the modalities of human knowledge more than on the animals themselves. In contradiction to this appropriative logic, however, the series do seem to allow for the emergence of visual forms of otherness. They do so precisely when their images contribute to destabilising the sovereign, inquisitive gaze that initially appeared as crucial to their documentary ambition. Alterity then finds its place in our world, in so far as the gaze lets itself be altered—by the uncanny quality of digitally generated images, which our sensory system struggles to process, but also by the reciprocal contamination which those images evidence between documentary and imaginative representation, science and fiction.

Industrial Applications of Ultrahigh Definition Video Coding With an Optimized Supersample Adaptive Offset Framework

This article presents an efficient superblock-based sample adaptive offset (superSAO) that jointly exploits block-wise filter partition and probabilistic band interval segmentation to improve quality-of-experience of industrial video applications. Specifically, we investigate the partition flexibility of a superSAO block whose root size is up to 256 × 256, and propose to optimize the block-wise SAO filter by considering computation complexity and compression efficiency. Furthermore, we segment the band interval by equal probability of the sample intensity distributions, which facilitates the computation of better band offsets to attenuate ringing artifact due to quantization errors or encoded motion vectors. Experimental results show that the proposed superSAO method outperforms state-of-the-art approaches by obtaining 4.6% bandwidth reduction on average for the low delay and high-compression video applications.