Video Encoder Research Articles

Real-Time Compressed Video Encryption: Based on Quasigroup on System on Chip (SOC)

Real-Time Compressed Video Encryption: Based on Quasigroup on System on Chip (SOC)

RETRACTED ARTICLE: Big data VR system oriented to cloud services in Spanish cultural video simulation application

The data volume of cloud service big data is huge, usually more than 10 TB, and it has even jumped to the petabyte level. Big data VR has a variety of data, not only structured, but also semi-structured, unstructured and interactive. In the Internet multimedia era, the output of images, sounds, videos and other data is amazing, so semi-structured and unstructured data are very important to a large extent. However, the main data of cloud services is not only a huge amount of data, but also has its diversity. The speed of big data VR is not only the speed of big data generation, but also the speed of processing. According to data, the amount of global data is growing at a rate of 50% per year, far exceeding the development speed of computer design. Therefore, processing and analyzing large data should be faster. Another way of spreading Spanish culture is non-governmental cultural exchanges. For example, some Chinese and Western art festivals held between China and Spain. China will also have some non-governmental cultural organizations about Spain, which can spread Spanish culture. Video simulation is a new type of video carrier, which has the characteristics of sound and image alignment, giving users a sense of in-depth immersion. Compared with traditional video images, virtual reality requires a higher bit rate, and also takes up more network bandwidth and storage space. While giving users a high-quality experience, how to optimize video quality and low bit rate video encoding is video simulation, one of the keys to rapid development, which is also the focus of this article.

Content Dependent Representation Selection Model for Systems Based on MPEG DASH

A standard called MPEG Dynamic Adaptive Streaming over HTTP (MPEG DASH) ensures the interoperability between different streaming services and the highest possible video quality in changing network conditions. The solutions described in the available literature that focus on video segmentation are mostly proprietary, use a high amount of computational power, lack the methodology, model notation, information needed for reproduction, or do not consider the spatial and temporal activity of video sequences. This paper presents a new model for selecting optimal parameters and number of representations for video encoding and segmentation, based on a measure of the spatial and temporal activity of the video content. The model was developed for the H.264 encoder, using Structural Similarity Index Measure (SSIM) objective metrics as well as Spatial Information (SI) and Temporal Information (TI) as measures of video spatial and temporal activity. The methodology that we used to develop the mathematical model is also presented in detail so that it can be applied to adapt the mathematical model to another type of an encoder or a set of encoding parameters. The efficiency of the segmentation made by the proposed model was tested using the Basic Adaptation algorithm (BAA) and Segment Aware Rate Adaptation (SARA) algorithm as well as two different network scenarios. In comparison to the segmentation available in the relevant literature, the segmentation based on the proposed model obtains better SSIM values in 92% of cases and subjective testing showed that it achieves better results in 83.3% of cases.

Bidirectional Temporal Context Fusion with Bi-Modal Semantic Features using a gating mechanism for Dense Video Captioning

Dense video captioning involves detecting interesting events and generating textual descriptions for each event in an untrimmed video. Many machine intelligent applications such as video summarization, search and retrieval, automatic video subtitling for supporting blind disabled people, benefit from automated dense captions generator. Most recent works attempted to make use of an encoder-decoder neural network framework which employs a 3D-CNN as an encoder for representing a detected event frames, and an RNN as a decoder for caption generation. They follow an attention based mechanism to learn where to focus in the encoded video frames during caption generation. Although the attention-based approaches have achieved excellent results, they directly link visual features to textual captions and ignore the rich intermediate/high-level video concepts such as people, objects, scenes, and actions. In this paper, we firstly propose to obtain a better event representation that discriminates between events nearly ending at the same time by applying an attention based fusion. Where hidden states from a bi-directional LSTM sequence video encoder, which encodes past and future surrounding context information of a detected event are fused along with its visual (R3D) features. Secondly, we propose to explicitly extract bi-modal semantic concepts (nouns and verbs) from a detected event segment and equilibrate the contributions from the proposed event representation and the semantic concepts dynamically using a gating mechanism while captioning. Experimental results demonstrates that our proposed attention based fusion is better in representing an event for captioning. Also involving semantic concepts improves captioning performance.

Gem5-X

The increasing adoption of smart systems in our daily life has led to the development of new applications with varying performance and energy constraints, and suitable computing architectures need to be developed for these new applications. In this article, we present gem5-X, a system-level simulation framework, based on gem-5, for architectural exploration of heterogeneous many-core systems. To demonstrate the capabilities of gem5-X, real-time video analytics is used as a case-study. It is composed of two kernels, namely, video encoding and image classification using convolutional neural networks (CNNs). First, we explore through gem5-X the benefits of latest 3D high bandwidth memory (HBM2) in different architectural configurations. Then, using a two-step exploration methodology, we develop a new optimized clustered-heterogeneous architecture with HBM2 in gem5-X for video analytics application. In this proposed clustered-heterogeneous architecture, ARMv8 in-order cluster with in-cache computing engine executes the video encoding kernel, giving 20% performance and 54% energy benefits compared to baseline ARM in-order and Out-of-Order systems, respectively. Furthermore, thanks to gem5-X, we conclude that ARM Out-of-Order clusters with HBM2 are the best choice to run visual recognition using CNNs, as they outperform DDR4-based system by up to 30% both in terms of performance and energy savings.

Secure Video Surveillance Framework in Smart City.

In the construction process of smart cities, more and more video surveillance systems have been deployed for traffic, office buildings, shopping malls, and families. Thus, the security of video surveillance systems has attracted more attention. At present, many researchers focus on how to select the region of interest (RoI) accurately and then realize privacy protection in videos by selective encryption. However, relatively few researchers focus on building a security framework by analyzing the security of a video surveillance system from the system and data life cycle. By analyzing the surveillance video protection and the attack surface of a video surveillance system in a smart city, we constructed a secure surveillance framework in this manuscript. In the secure framework, a secure video surveillance model is proposed, and a secure authentication protocol that can resist man-in-the-middle attacks (MITM) and replay attacks is implemented. For the management of the video encryption key, we introduced the Chinese remainder theorem (CRT) on the basis of group key management to provide an efficient and secure key update. In addition, we built a decryption suite based on transparent encryption to ensure the security of the decryption environment. The security analysis proved that our system can guarantee the forward and backward security of the key update. In the experiment environment, the average decryption speed of our system can reach 91.47 Mb/s, which can meet the real-time requirement of practical applications.

Sparse representation based compressive video encryption using hyper-chaos and DNA coding

This work proposes a new and efficient sparse representation-based spatial compression of video signals integrated with a hyper-chaotic DNA coding-based encryption module that provides higher security. The betterment of compression efficiency is achieved by introducing sparse coding on the video frames, and higher security is obtained by applying 5D hyper-chaotic DNA coding on the sparse coded frames. This novel technique is applied to a large set of video signals for testing and compares its performance with some recently proposed video coding and encryption systems. To test and measure the efficiency and reliability of different encoding and encryption schemes, we consider a set of well-defined quality metrics, e.g., compression ratio, bit rate in compression, PSNR, NPCR, UACI, correlation coefficients, and histogram analysis, and measure the values of those metrics for different encoding and encryption modules. In addition, an ablation analysis is done. It shows improved quality matrices when we utilize DNA coding along with the 5D hyper-chaotic system-based encryption mechanism. We also perform the differential attack analysis for testing the security of the encryption modules. The quantitative measurements of these quality metrics for different algorithms demonstrate the superiority of the proposed algorithm to the other algorithms in efficiency, quality, and security.

Low-energy motion estimation memory system with dynamic management.

The digital video coding process imposes severe pressure on memory traffic, leading to considerable power consumption related to frequent DRAM accesses. External off-chip memory demand needs to be minimized by clever architecture/algorithm co-design, thus saving energy and extending battery lifetime during video encoding. To exploit temporal redundancies among neighboring frames, the motion estimation (ME) algorithm searches for good matching between the current block and blocks within reference frames stored in external memory. To save energy during ME, this work performs memory accesses distribution analysis of the test zone search (TZS) ME algorithm and, based on this analysis, proposes both a multi-sector scratchpad memory design and dynamic management for the TZS memory access. Our dynamic memory management, called neighbor management, reduces both static consumption—by employing sector-level power gating—and dynamic consumption—by reducing the number of accesses for ME execution. Additionally, our dynamic management was integrated with two previously proposed solutions: a hardware reference frame compressor and the Level C data reuse scheme (using a scratchpad memory). This system achieves a memory energy consumption savings of 99.8% and, when compared to the baseline solution composed of a reference frame compressor and data reuse scheme, the memory energy consumption was reduced by 44.1% at a cost of just 0.35% loss in coding efficiency, on average. When compared with related works, our system presents better memory bandwidth/energy savings and coding efficiency results.

Medicolegal Sidebar: Telemedicine-New Opportunities and New Risks.

Medicolegal Sidebar: Telemedicine-New Opportunities and New Risks.

Development of an 8K UHDTV Wireless Camera Using Millimeter-Wave Bands

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Wireless cameras enable dynamic shooting and have become indispensable in television program production for sporting events and music concerts. Japan Broadcasting Corporation (NHK) has researched and developed an 8K ultrahigh-definition television (UHDTV) wireless camera using millimeter waves that can transmit an 8K signal with low latency and high quality. We have prototyped a compact transmitter using the 42-GHz band and developed an 8K UHDTV wireless camera by combining this transmitter with a portable 8K UHDTV video encoder. In this study, we introduce a prototype 8K UHDTV wireless camera system using millimeter waves and report the results of laboratory and field wireless transmission experiments</i> .

Tele-proctoring in vascular surgery implementing percutaneous creation of arteriovenous fistula

ObjectivePercutaneous ultrasound-guided creation of an arteriovenous fistula (pAVF) for hemodialysis access is a relatively novel procedure with promising technical success and patency rates. The vascular surgery departments of two collaborating Swiss hospitals had decided to introduce the technique to their services. A surgeon experienced in the technique (AM) was supposed to come to Switzerland and proctor the first four procedures. Due to the SARS-CoV-2 pandemic and travel restrictions, this was not possible and we decided to perform the first four pAVF procedures tele-proctored via a video conference system. We present the setup and our experience with tele-proctoring.MethodsThe setup relied on an all-in-one live video production device, video encoder, video streamer and video recorder (Pearl-2, Epiphan), which made it possible to simultaneously transmit the live image from the ultrasound device (GE Logiq S8, linear probe 9L-D) and a live image from a video camera (JVC Camcorder G/-HM440E, Japan), both connected via HDMI (Figure). The live stream was shared with the proctor in France and the device support team in the US, using an encrypted Swiss video client (www.vitimway.ch). The setup was tested with all parties three days in advance.ResultsAll 4 procedures started with a verbal briefing and a live ultrasound scan. All steps of the procedures were taken under the proctor’s instruction and supervision. The proctor gave on average 21 instructions per procedure. An average 4 were device-related but these became less frequent as we proceeded. The operators consulted the proctor on average 5 times per procedure. The average duration of the procedure was 34 min. The pAVF creation was successful in all 4 patients, with an average fistula flow measured at the end of the procedure of 600 ml/min.ConclusionOur experience showed us that pAVF creation, which is a procedure performed entirely under sonographic guidance lends itself particularly well to tele-proctoring. The simultaneous transmission of the live sonographic image and the live image of the operators' hands allowed the proctor to supervise and correct the key steps of the procedures. The simplicity of the set-up and the quality of proctor-operator interaction was such a positive experience that we can well envisage a much wider use of tele-proctoring in the future.

Video Slice: Image Compression and Transmission for Agricultural Systems

When agricultural automation systems are required to send cultivation field images to the cloud for field monitoring, pay-as-you-go mobile communication leads to high operation costs. To minimize cost, one can exploit a characteristic of cultivation field images wherein the landscape does not change considerably besides the appearance of the plants. Therefore, this paper presents a method that transmits only the difference data between the past and current images to minimize the amount of transmitted data. This method is easy to implement because the difference data are generated using an existing video encoder. Further, the difference data are generated based on an image at a specific time instead of the images at adjacent times, and thus the subsequent images can be reproduced even if the previous difference data are lost because of unstable mobile communication. A prototype of the proposed method was implemented with a MPEG-4 Visual video encoder. The amount of transmitted and received data on the medium access control layer was decreased to approximately 1/4 of that when using the secure copy protocol. The transmission time for one image was 5.6 s; thus, the proposed method achieved a reasonable processing time and a reduction of transmitted data.

SnakeBinning: Efficient Temporally Coherent Triangle Packing for Shading Streaming

AbstractStreaming rendering, e.g., rendering in the cloud and streaming via a mobile connection, suffers from increased latency and unreliable connections. High quality framerate upsampling can hide these issues, especially when capturing shading into an atlas and transmitting it alongside geometric information. The captured shading information must consider triangle footprints and temporal stability to ensure efficient video encoding. Previous approaches only consider either temporal stability or sample distributions, but none focuses on both. With SnakeBinning, we present an efficient triangle packing approach that adjusts sample distributions and caters for temporal coherence. Using a multi‐dimensional binning approach, we enforce tight packing among triangles while creating optimal sample distributions. Our binning is built on top of hardware supported real‐time rendering where bins are mapped to individual pixels in a virtual framebuffer. Fragment shader interlock and atomic operations enforce global ordering of triangles within each bin, and thus temporal coherence according to the primitive order is achieved. Resampling the bin distribution guarantees high occupancy among all bins and a dense atlas packing. Shading samples are directly captured into the atlas using a rasterization pass, adjusting samples for perspective effects and creating a tight packing. Comparison to previous atlas packing approaches shows that our approach is faster than previous work and achieves the best sample distributions while maintaining temporal coherence. In this way, SnakeBinning achieves the highest rendering quality under equal atlas memory requirements. At the same time, its temporal coherence ensures that we require equal or less bandwidth than previous state‐of‐the‐art. As SnakeBinning outperforms previous approach in all relevant aspects, it is the preferred choice for texture‐based streaming rendering.

Secure and Efficient Video Transmission in VANET

Currently, vehicular communications have become a reality used by various applications, especially applications that broadcast video in real time. However, the video quality received is penalized by the poor characteristics of the transmission channel (availability, non-stationarity, the ration of signal-to-noise, etc.). To improve and ensure minimum video quality at reception, we propose in this work a mechanism entitled “Secure and Efficient Transmission of Videos in VANET (SETV)”. It's based on the Quality of Experience (QoE) and using hierarchical packet management. This last is based on the importance of the images of the stream video. To this end, the use of transmission error correction with uneven error protection has proven to be effective in delivering high quality videos with low network overhead. This is done based on the specific details of video encoding and actual network conditions such as signal to noise ratio, network density, vehicle position and current packet loss rate (PLR) not to mention the prediction of the future DPP. Machine learning models were developed on our work to estimate perceived audio-visual quality. The protocol previously gathers information about its neighbouring vehicles to perform distributed jump reinforcement learning. The simulation results obtained for several types of realistic vehicular scenarios show that our proposed mechanism offers significant improvements in terms of video quality on reception and end-to-end delay compared to conventional schemes. The results prove that the proposed mechanism has showed 11% to 18% improvement in video quality and 9% load gain compared to ShieldHEVC.

VLSI Implementation of a Cost-Efficient Loeffler DCT Algorithm with Recursive CORDIC for DCT-Based Encoder

This paper presents a low-cost and high-quality, hardware-oriented, two-dimensional discrete cosine transform (2-D DCT) signal analyzer for image and video encoders. In order to reduce memory requirement and improve image quality, a novel Loeffler DCT based on a coordinate rotation digital computer (CORDIC) technique is proposed. In addition, the proposed algorithm is realized by a recursive CORDIC architecture instead of an unfolded CORDIC architecture with approximated scale factors. In the proposed design, a fully pipelined architecture is developed to efficiently increase operating frequency and throughput, and scale factors are implemented by using four hardware-sharing machines for complexity reduction. Thus, the computational complexity can be decreased significantly with only 0.01 dB loss deviated from the optimal image quality of the Loeffler DCT. Experimental results show that the proposed 2-D DCT spectral analyzer not only achieved a superior average peak signal–noise ratio (PSNR) compared to the previous CORDIC-DCT algorithms but also designed cost-efficient architecture for very large scale integration (VLSI) implementation. The proposed design was realized using a UMC 0.18-μm CMOS process with a synthesized gate count of 8.04 k and core area of 75,100 μm2. Its operating frequency was 100 MHz and power consumption was 4.17 mW. Moreover, this work had at least a 64.1% gate count reduction and saved at least 22.5% in power consumption compared to previous designs.

Machine learning-based energy consumption modeling and comparison of H.264/AVC and google VP8 encoders

Advancement of the prediction models used in a variety of fields is a result of the contribution of machine learning approaches. Utilizing such modeling in feature engineering is exceptionally imperative and required. In this research, we show how to utilize machine learning to save time in research experiments, where we save more than five thousand hours of measuring the energy consumption of encoding recordings. Since measuring the energy consumption has got to be done by humans and since we require more than eleven thousand experiments to cover all the combinations of video sequences, video bit_rate, and video encoding settings, we utilize machine learning to model the energy consumption utilizing linear regression. VP8 codec has been offered by Google as an open video encoder in an effort to replace the popular MPEG-4 Part 10, known as H.264/AVC video encoder standard. This research model energy consumption and describes the major differences between H.264/AVC and VP8 encoders in terms of energy consumption and performance through experiments that are based on machine learning modeling. Twenty-nine raw video sequences are used, offering a wide range of resolutions and contents, with the frame sizes ranging from QCIF(176x144) to 2160p(3840x2160). For fairness in comparison analysis, we use seven settings in VP8 encoder and fifteen types of tuning in H.264/AVC. The settings cover various video qualities. The performance metrics include video qualities, encoding time, and encoding energy consumption.

Global motion estimation with iterative optimization-based independent univariate model for action recognition

Motion information used in the existed video action recognition schemes is mixing of global motion(GM) and local motion(LM). In fact, GM & LM have their respective semantic concepts. Thus, it is promising to decouple GM and LM from the mixed motions. Numerous efforts have been made on the design of global motion models for video encoding, video dejittering, video denoising, and so on. Nevertheless, some of the models are too basic to cover the camera motions in action recognition while others are over-complicated. In this paper, we focus on the characteristic of the action recognition and propose a novel independent univariate GM model. It ignores camera rotation, which appears rarely in action recognition videos, and represents the GM in x and y direction respectively. Furthermore, GM is position invariant because it is from the universal camera motion. Pixels with global motions are subjected to the same parametric model and pixels with mixed motion can be seen as outliers. Motivated by this, we develop an iterative optimization scheme for GM estimation which removes the outlier points step by step and estimates global motions in a coarse-to-fine manner. Finally, the LM is estimated through a Spatio-temporal threshold-based method. Experimental results demonstrate that the proposed GM model makes a better trade-off between the model complexity and the robustness. And the iterative optimization scheme is more effective than the existed algorithms. The compared experiments using four popular action recognition models on UCF-101 (for action recognition) and NCAA (for group activity recognition) demonstrate that local motions are more effective than the mixed motions.

Gaussian-Wiener Representation and Hierarchical Coding Scheme for Focal Stack Images

Focal stack images (FoSIs) are a set of 2D images that captured one scene with serial focal depths. The redundancies of FoSIs mainly come from gradual focused depths changes rather than motion of objects. Conventional coding schemes cannot fully exploit such redundancies, leading to coding inefficiency. In this paper, we propose a new Gaussian-Wiener representation to model the gradual focused depths changes among FoSIs. In the representation, image degradation-restoration relations are utilized to describe the focus-defocus changing characteristics of FoSIs. Based on this representation, we propose a new hierarchical coding scheme for fully exploiting the inter-frame redundancies of FoSIs. In the scheme, a Gaussian-Wiener representation based inter prediction (GWR-IP) is presented by embedding Gaussian convolution and Wiener deconvolution into normal video encoder. Block-wise focus-defocus changing of FoSIs can be predicted in bi-directional manner by solving optimization problem. For higher coding efficiency, a Gaussian-Wiener representation based hierarchical prediction structure (GWR-HPS) is also designed and applied in the coding scheme. The proposed coding scheme is performed on 10 test sequences, including 5 synthetic scenes and 5 realistic scenes. Experimental results show that proposed coding scheme can obtain 2.640 dB PSNR gains and 51.830% bitrate savings on average of all test sequences in Low Delay P configuration, 2.123 dB PSNR gains and 43.975% bitrate savings in Low Delay B configuration, and 1.044 dB PSNR gains and 26.078% bitrate savings in Random Access configuration. Particularly, it achieves up to 65.544% bit rate savings and 3.901 dB PSNR increments for test sequence I09 in Low Delay P configuration. Furthermore, ablation test demonstrates that Gaussian representation contributes more on coding performance than Wiener representation and GWR-HPS.

Design and Smartphone Implementation of Chaotic Duplex H.264-Codec Video Communications

In this paper, a chaotic duplex H.264-codec-based secure video communication scheme is designed and its smartphone implementation is also carried out. First, an improved self-synchronous chaotic stream cipher algorithm equipped with a sinusoidal modulation, a multiplication, a modulo operation and a round down operation (SCSCA-SMMR) is developed. Using the sinusoidal modulation and multiplication, the improved algorithm can resist the divide-and-conquer attack by traversing multiple nonzero component initial conditions (DCA-TMNCIC). Meanwhile, also by means of the round down operation and modulo operation, on the premise that the DCA-TMNCIC does not work, the original keys cannot be further deciphered only by the known-plaintext attack, the chosen-plaintext attack and the chosen-ciphertext attack, respectively. Then, the Android low-level multimedia support infrastructure MediaCodec class is used to access low-level media encoder/decoder components and the H.264 hardware encoding/decoding is performed on real-time videos, so the chaotic video encryption and decryption can be realized in real-time by smartphones. Security analysis and smartphone experimental results verify the effectiveness of the proposed method.

Hybrid Distortion-Based Rate-Distortion Optimization and Rate Control for H.265/HEVC

Mean squared error (MSE) is adopted as the distortion metric for the rate-distortion optimization (RDO) and rate control (RC) in most of the existing video coding standards, e.g., H.265/High-Efficiency Video Coding (HEVC) and H.264/Advanced Video Coding (AVC), because the pixel-wise MSE can represent the image fidelity efficiently. As videos are usually used for visual enjoyment, recently, researchers found that the MSE is not always consistent with perceptual quality, and proposed some perceptual quality-based video coding methods by exchanging the MSE with other perceptual quality metrics completely, like the structure similarity index (SSIM), just noticeable distortion (JND), etc. We believe that a sophisticated video encoder should guarantee both the subject quality and the data fidelity. By investigating the properties of the human visual system (HVS), hybrid distortion-based RDO and RC methods are proposed to improve subjective quality while avoiding fidelity degradation. Experimental results demonstrate that the proposed methods can improve subjective quality efficiently, while preserving the image fidelity of the reconstructed videos as much as possible.