Video Distortion Research Articles

Enhancing No Reference Laparoscopic Video Quality Assessment with Evolutionary ANFIS

Abstract Distortions in laparoscopic videos affect surgeon visibility and surgical precision, underscoring the need for sustained high video quality. This study presents a real-time laparoscopic video quality assessment algorithm independent of reference content availability. Statistical parameters derived from luminance, local binary pattern and motion-vector maps of video frames are observed to effectively discern distortion types and severities. These parameters are used to train an evolutionary adaptive neuro-fuzzy inference system (ANFIS) end-to-end with subjective score labels. Training and validation loss curves saturate at the 85th epoch, demonstrating the model’s efficient data fitting capability. Performance comparison with other state-of-the-art methods reveals superior results, with high correlation scores of 0.9989 and 0.9446 for experts and 0.9956 and 0.9847 for non-experts, alongside low root mean square errors of 0.0828 and 0.1685 for expert and non-experts, respectively. The model accurately replicates the expert and non-expert perceptual opinions, encouraging future research in stereoscopic, augmented, and virtual reality data.

TRRHA: A two-stream re-parameterized refocusing hybrid attention network for synthesized view quality enhancement

In multi-view video systems, the decoded texture video and its corresponding depth video are utilized to synthesize virtual views from different perspectives using the depth-image-based rendering (DIBR) technology in 3D-high efficiency video coding (3D-HEVC). However, the distortion of the compressed multi-view video and the disocclusion problem in DIBR can easily cause obvious holes and cracks in the synthesized views, degrading the visual quality of the synthesized views. To address this problem, a novel two-stream re-parameterized refocusing hybrid attention (TRRHA) network is proposed to significantly improve the quality of synthesized views. Firstly, a global multi-scale residual information stream is applied to extract the global context information by using refocusing attention module (RAM), and the RAM can detect the contextual feature and adaptively learn channel and spatial attention feature to selectively focus on different areas. Secondly, a local feature pyramid attention information stream is used to fully capture complex local texture details by using re-parameterized refocusing attention module (RRAM). The RRAM can effectively capture multi-scale texture details with different receptive fields, and adaptively adjust channel and spatial weights to adapt to information transformation at different sizes and levels. Finally, an efficient feature fusion module is proposed to effectively fuse the extracted global and local information streams. Extensive experimental results show that the proposed TRRHA achieves significantly better performance than the state-of-the-art methods. The source code will be available at https://github.com/647-bei/TRRHA.

Spatio-temporal feature learning for enhancing video quality based on screen content characteristics

With the rising demands for remote desktops and online meetings, screen content videos have drawn significant attention. Different from natural videos, screen content videos often exhibit scene switches where the content abruptly changes from one frame to the next. These scene switches result in obvious distortions in compressed videos. Besides, frame freezing, where the content remains unchanged for a certain duration, is also very common in screen content videos. Existing alignment-based models struggle to effectively enhance scene switch frames and lack efficiency when dealing with frame freezing situations. Therefore, we propose a novel alignment-free method that effectively handles both scene switches and frame freezing. In our approach, we develop a spatial and temporal feature extraction module that compresses and extracts spatio-temporal information from three groups of frame inputs. This enables efficient handling of scene switches. In addition, an edge aware block is proposed for extracting edge information, which guides the model to focus on restoring the high-frequency components in frame freezing situations. The fusion module is then designed to adaptively fuse the features from three groups, considering different positions of video frames, to enhance frames during scene switch and frame freezing scenarios. Experimental results demonstrate the significant advancements achieved by the proposed edge aware with spatio-temporal information fusion network (EAST) in enhancing the quality of compressed videos, surpassing the current state-of-the-art methods.

Comparison of Subjective Methodologies For Local Perception of Distortion in Videos and Impact on Objective Metrics Resolving Power

Comparison of Subjective Methodologies For Local Perception of Distortion in Videos and Impact on Objective Metrics Resolving Power

3D Convolutional Neural Network-Based Multi-Parameter Video Quality Assessment Model on Cloud Platforms

In light of the rapid advancements in big data and artificial intelligence technologies, the trend of uploading local files to cloud servers to mitigate local storage limitations is growing. However, the surge of duplicate files, especially images and videos, results in significant network bandwidth wastage and complicates server management. To tackle these issues, we have developed a multi-parameter video quality assessment model utilizing a 3D convolutional neural network within a video deduplication framework. Our method, inspired by the analytic hierarchy process, thoroughly evaluates the effects of packet loss rate, codec, frame rate, bit rate, and resolution on video quality. The model employs a two-stream 3D convolutional neural network to integrate spatial and temporal streams for capturing video distortion details, with a coding layer configured to remove redundant distortion information. We validated our approach using the LIVE and CSIQ datasets, comparing its performance against the V-BLIINDS and VIDEO schemes across different packet loss rates. Furthermore, we simulated the client-server interaction using a subset of the dataset and assessed the scheme's time efficiency. Our results indicate that the proposed scheme offers a highly efficient solution for video quality assessment.

Self-supervised video distortion correction algorithm based on iterative optimization

Wide-angle video frames typically contain more information, but they also exhibit distortion that degrades the visual quality, especially at the edges. To eliminate this distortion from videos, we propose a self-supervised iterative optimization method in this paper. Specifically, we construct a motion parameter estimation model utilizing two consecutive distorted frames, where motion parameters comprise affine transform and distortion parameters. We apply the Gauss–Newton algorithm to minimize the sum-of-squares error between frames and update parameters. Treating inter-frame motion as undistort-affine-distort transformations, frame alignment is achieved by continuously adjusting transform parameters. Ultimately, frames are corrected using the converged parameters. We generated a synthetic dataset with various distortion parameters for evaluation. Experiments demonstrate superior performance versus state-of-the-art methods on synthetic and real wide-angle videos. Our algorithm also achieves higher parameter estimation accuracy without sacrificing efficiency.

Building a Statistical Model to Detect Foreground Objects and using it in Video Steganography

Video steganography has become a popular option for protecting secret data from hacking attempts and common attacks on the internet. However, when the whole video frame(s) are used to embed secret data, this may lead to visual distortion. This work is an attempt to hide sensitive secret image inside the moving objects in a video based on separating the object from the background of the frame, selecting and arranging them according to object's size for embedding secret image. The XOR technique is used with reverse bits between the secret image bits and the detected moving object bits for embedding. The proposed method provides more security and imperceptibility as the moving objects are used for embedding, so it is difficult to notice the changes in the moving objects instead of using background area for embedding in the video. The experimental results showed the better visual quality of the stego video with PSNR values exceeding 58 dB, this indicates that the proposed method works without causing much distortion in the original video and transmitted secret message.

Learning Spatiotemporal Interactions for User-Generated Video Quality Assessment

Distortions from spatial and temporal domains have been identified as the dominant factors that govern the visual quality. Though both have been studied independently in deep learning-based user-generated content (UGC) video quality assessment (VQA) by frame-wise distortion estimation and temporal quality aggregation, much less work has been dedicated to the integration of them with deep representations. In this paper, we propose a SpatioTemporal Interactive VQA (STI-VQA) model based upon the philosophy that video distortion can be inferred from the integration of both spatial characteristics and temporal motion, along with the flow of time. In particular, for each timestamp, both the spatial distortion explored by the feature statistics and local motion captured by feature difference are extracted and fed to a transformer network for the motion aware interaction learning. Meanwhile, the information flow of spatial distortion from the shallow layer to the deep layer is constructed adaptively during the temporal aggregation. The transformer network enjoys an advanced advantage for long-range dependencies modeling, leading to superior performance on UGC videos. Experimental results on five UGC video benchmarks demonstrate the effectiveness and efficiency of our STI-VQA model, and the source code will be available online at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/h4nwei/STI-VQA</uri> .

OVQE: Omniscient Network for Compressed Video Quality Enhancement

How to use information from temporal, spatial, and frequency domain dimensions is crucial for the quality enhancement of compressed video. The state-of-the-art methods generally design powerful networks to fuse the spatiotemporal information of the videos. But the spatiotemporal information of the entire video is not fully utilized and effectively fused, resulting in the learned context information that is not closely related to the target frame. In addition, various compressed videos have varying degrees of frequency domain information loss. The previous methods ignored the non-uniform distortion of compressed video in different frequency domains and did not design unique algorithms for different frequency domains, so the real texture details of the video could not be restored. In this paper, we propose an omniscient network, which learns video spatiotemporal and omni-frequency information more effectively. The omniscient network consists of two novel components: a Spatio-Temporal Feature Fusion (STFF) module and an Omni-Frequency Adaptive Enhancement (OFAE) block. The former aims to capture spatiotemporal information in adjacent frames, while the latter aims to adaptively recover different frequency domains of compressed video. The information is designed to be bidirectionally propagated in a grid manner such that the omni-enhanced results can be applied. Extensive experiments show that our method outperforms the state-of-the-art method in terms of objective metrics, subjective visual effects, and model complexity.

Dynamic Interference Suppression for TV White Space: the Case of Thailand

In this work, we study the problem of co-existence between the LTE (Long Term Evolution) and the Digital Terrestrial Television Broadcasting (DTTB) channel. There are three scenarios: co-channel, upper and lower adjacent channels. We use the broadcasted signal from channel 3 called ThaiPBS, the actual case of Thailand for our study where the standard of digital terrestrial television broadcasting is DVB-T2 adopting 8 MHz bandwidth, while the 5 MHz bandwidth of LTE is considered as the interference. We propose a dynamic interference suppression method for increasing spectrum usage by optimizing TV white space utilization and minimizing interference. This method adopts the protection ratio concept to suppress the LTE interference on TV receiver. We implement our proposed algorithm as an adaptive interference controller using a Radio Frequency (RF) attenuator and a Raspberry Pi board for our testbed hardware. We illustrate the effectiveness of our proposed algorithm by doing experiment using our testbed and assessing the quality of the received TV signal by adopting the Quality of Experience (QoE) assessment. In our testbed hardware, a Log Periodic antenna is used for receiving the DTTB signal, while an RF digital transmitter is used for generating a 5MHz bandwidth of LTE signal, an ultra-high frequency (UHF) mixer is used to combine both signals, then a field strength meter is used to monitor video picture quality and to analyze the spectrum. According to the experiment, our proposed method can reduce the perceived video distortion by at least 62.5% for co-channel and 87.5% for adjacent channel, while the spectrum usage is increased by 100%.

(Retracted) Simulating human visual perception behaviors for video quality assessment

Multimedia video quality evaluation system plays an important role in modern Internet applications, such as WebRTC-based video conference. However, video quality may be affected during transmission because of noise or network congestion. Constructing an end-to-end intelligent video transmission monitoring system is of great significance. We design a system based on no-reference quality assessment, which can be used to evaluate the video-based object quality evaluation. More specifically, we first leverage Gabor filter for video segmentation. Then we conduct image smoothness estimation to measure the correlation between two adjacent frames. Subsequently, we consider two states of video distortion: compression and video transmission. We model these two states, respectively. We compare our VQA algorithm with a set of well-known NR-IQA. The reported VQA accuracies have clearly shown the superiority of our proposed video quality predictor.

Underwater Adaptive Video Transmissions Using MIMO-Based Software-Defined Acoustic Modems

Achieving reliable acoustic wireless video transmissionsin the extreme and uncertain underwater environment is a challenge due to the limited bandwidth and the error-prone nature of the channel. Aiming at optimizing the received video quality and the user’s experience, an adaptive solution for underwater video transmissions is proposed that is specifically designed for Multi-Input Multi-Output (MIMO)-based Software-Defined Acoustic Modems (SDAMs). To keep the video distortion under an acceptable threshold and to keep the Physical-Layer Throughput (PLT) high, cross-layer techniques utilizing diversity-spatial multiplexing and Unequal Error Protection (UEP) are presented along with the scalable video compression at the application layer. Specifically, the scalability of the utilized SDAM with high processing capabilities is exploited in the proposed structure along with the temporal, spatial, and quality scalability of the Scalable Video Coding (SVC) H.264/MPEG-4 AVC compression standard. The transmitter broadcasts one video stream and realizes multicasting at different users. Experimental results at the Sonny Werblin Recreation Center, Rutgers University-NJ, are presented. Several scenarios for unknown channels at the transmitter are experimentally considered when the hydrophones are placed in different locations in the pool to achieve the required SVC-based video Quality of Service (QoS) and Quality of Experience (QoE) given the channel state information and the robustness of different SVC scalability. The video quality level is determined by the best communication link while the transmission scheme is decided based on the worst communication link, which guarantees that each user is able to receive the video with appropriate quality.

No-Reference Video Quality Assessment Using the Temporal Statistics of Global and Local Image Features.

During acquisition, storage, and transmission, the quality of digital videos degrades significantly. Low-quality videos lead to the failure of many computer vision applications, such as object tracking or detection, intelligent surveillance, etc. Over the years, many different features have been developed to resolve the problem of no-reference video quality assessment (NR-VQA). In this paper, we propose a novel NR-VQA algorithm that integrates the fusion of temporal statistics of local and global image features with an ensemble learning framework in a single architecture. Namely, the temporal statistics of global features reflect all parts of the video frames, while the temporal statistics of local features reflect the details. Specifically, we apply a broad spectrum of statistics of local and global features to characterize the variety of possible video distortions. In order to study the effectiveness of the method introduced in this paper, we conducted experiments on two large benchmark databases, i.e., KoNViD-1k and LIVE VQC, which contain authentic distortions, and we compared it to 14 other well-known NR-VQA algorithms. The experimental results show that the proposed method is able to achieve greatly improved results on the considered benchmark datasets. Namely, the proposed method exhibits significant progress in performance over other recent NR-VQA approaches.

IV-PSNR—The Objective Quality Metric for Immersive Video Applications

This paper presents a new objective quality metric that was adapted to the complex characteristics of immersive video (IV) which is prone to errors caused by processing and compression of multiple input views and virtual view synthesis. The proposed metric, IV-PSNR, contains two techniques that allow for the evaluation of quality loss for typical immersive video distortions: corresponding pixel shift and global component difference. The performed experiments compared the proposal with 31 state-of-the-art quality metrics, showing their performance in the assessment of quality in immersive video coding and processing, and in other applications, using commonly used image quality assessment databases– TID2013 and CVIQ. As presented, IV-PSNR outperforms other metrics in immersive video applications and still can be efficiently used in the evaluation of different images and videos. Moreover, basing the metric on the calculation of PSNR allowed the computational complexity to remain low. Publicly available, efficient implementation of IV-PSNR software was provided by the authors of this paper and is used by ISO/IEC MPEG for evaluation and research on the upcoming MPEG Immersive video (MIV) coding standard.

Cost-Effective Design of a Miniaturized Zoom Lens for a Capsule Endoscope.

This paper presents a miniaturized design of a 2× zoom lens for application to a one-megapixel image sensor in a capsule endoscope. The zoom lens is composed of four lenses, including three plastic aspheric lenses and one glass spherical lens, and adopts a three-lens group design. This capsule endoscope is mainly for observation of the small intestine, which has a radius of about 12.5 mm. The height of the object is thus set to 12.5 mm. The object surface is designed to be curved surface with a radius of curvature of 15 mm. The focal length of the zoom lens ranges from 1.064 mm to 2.039 mm, and the full angle of view ranges from 60° to 143°, the f-number is F/2.8-F/3.5, the zoom lens is 11.6 mm in length, and the maximum effective diameter of the zoom lens is 6 mm. The zoom lens design is divided into six segments, corresponding to the different magnifications from Zoom 1 to Zoom 6. The magnification ratios are -0.0845, -0.0984, -0.1150, -0.1317, -0.1482, and -0.1690, respectively. Comparing the positions from Zoom 1 to Zoom 6, the maximum optical distortion is -14.89% for the Zoom 1 and 1.45% for the Zoom 6. The maximum vertical video distortion is 8.19% for Zoom 1 and 1.00% for Zoom 6. At a 1.0 field of view, the minimum relative illuminance is 71.8% at a magnification of M = -0.1317. Finally, we perform the tolerance analysis and lens resolution analysis at different zooming positions. Our design can obtain high-quality images for capsule endoscope.

Robust video steganography framework based on channel-space attention

In robust video steganography, a message is embedded into a video such that video distortions are avoided while producing a stego video of imperceptible difference from the cover video. Traditional techniques achieved robustness against particular distortions but are complicated in computation and design, and rely on different compression standards. Nowadays, deep-learning-based methods can achieve impressive visual quality and robustness to attacks. We propose a framework with a channel-space attention mechanism for robust video steganography. The framework is composed of depthwise separable convolution layers that can learn channel-space segments for embedding and extraction. The secret messages are distributed across channel-space scales to increase imperceptibility and robustness to distortions. This end-to-end solution is trained with the 3-player game approach to conducting robust steganography, where three networks compete. Two of these handle embedding and extraction operations, while the third network simulates attacks and detection from a steganalyst as an adversarial network. Comparative results versus recent research show that our method is more robust against compression and video distortion attacks. Peak signal-to-noise ratio and structural similarity index were used for evaluating visual quality and demonstrate the imperceptibility of our method.

Film and Video Quality Optimization Using Attention Mechanism-Embedded Lightweight Neural Network Model.

In filming, the collected video may be blurred due to camera shake and object movement, causing the target edge to be unclear or deforming the targets. In order to solve these problems and deeply optimize the quality of movie videos, this work proposes a video deblurring (VD) algorithm based on neural network (NN) model and attention mechanism (AM). Based on the scale recurrent network, Haar planar wavelet transform (WT) is introduced to preprocess the video image and to deblur the video image in the wavelet domain. Additionally, the spatial and channel AMs are fused into the overall network framework to improve the feature expression ability. Further, the residual inception spatial-channel attention (RISCA) mechanism is introduced to extract the multiscale feature information from video images. Meanwhile, skip spatial-channel attention (SSCA) accelerates the network training time to achieve a better VD effect. Finally, relevant experiments are designed, factoring in peak signal-to-noise ratio (PSNR) and structural similarity (SSI). The experimental findings corroborate that the proposed Haar and attention video deblurring (HAVD) outperforms multisize network Haar (MSNH) in PSNR and structural similarity (SSIM), improved by 0.10 dB and 0.005, respectively. Therefore, embedding the dual AMs can improve the model performance and optimize the video quality. This work provides technical support for solving the video distortion problems.

No-Reference Video Quality Assessment Using Multi-Pooled, Saliency Weighted Deep Features and Decision Fusion.

With the constantly growing popularity of video-based services and applications, no-reference video quality assessment (NR-VQA) has become a very hot research topic. Over the years, many different approaches have been introduced in the literature to evaluate the perceptual quality of digital videos. Due to the advent of large benchmark video quality assessment databases, deep learning has attracted a significant amount of attention in this field in recent years. This paper presents a novel, innovative deep learning-based approach for NR-VQA that relies on a set of in parallel pre-trained convolutional neural networks (CNN) to characterize versatitely the potential image and video distortions. Specifically, temporally pooled and saliency weighted video-level deep features are extracted with the help of a set of pre-trained CNNs and mapped onto perceptual quality scores independently from each other. Finally, the quality scores coming from the different regressors are fused together to obtain the perceptual quality of a given video sequence. Extensive experiments demonstrate that the proposed method sets a new state-of-the-art on two large benchmark video quality assessment databases with authentic distortions. Moreover, the presented results underline that the decision fusion of multiple deep architectures can significantly benefit NR-VQA.

An enhanced algorithm for improving real-time video transmission for tele-training education.

Tele-training in surgical education has not been effectively implemented. There is a stringent need for a high transmission rate, reliability, throughput, and reduced distortion for high-quality video transmission in the real-time network. This work aims to propose a system that improves video quality during real-time surgical tele-training. The proposed approach aims to minimise the video frame’s total distortion, ensuring better flow rate allocation and enhancing the video frames’ reliability. The proposed system consists of a proposed algorithm for Enhancing Video Quality, Distorting Minimization, Bandwidth efficiency, and Reliability Maximization called (EVQDMBRM) algorithm. The proposed algorithm reduces the video frame’s total distortion. In addition, it enhances the video quality in a real-time network by dynamically allocating the flow rate at the video source and maximizing the transmission reliability of the video frames. The result shows that the proposed EVQDMBRM algorithm improves the video quality with the minimized total distortion. Therefore, it improves the Peak Signal to Noise Ratio (PSNR) average by 51.13 dB against 47.28 dB in the existing systems. Furthermore, it reduces the video frames processing time average by 58.2 milliseconds (ms) against 76.1, and the end-to-end delay average by 114.57 ms against 133.58 ms comparing to the traditional methods. The proposed system concentrates on minimizing video distortion and improving the surgical video transmission quality by using an EVQDMBRM algorithm. It provides the mechanism to allocate the video rate at the source dynamically. Besides that, it minimizes the packet loss ratio and probing status, which estimates the available bandwidth.

Study of the Subjective and Objective Quality of High Motion Live Streaming Videos.

Video livestreaming is gaining prevalence among video streaming service s, especially for the delivery of live, high motion content such as sport ing events. The quality of the se livestreaming videos can be adversely affected by any of a wide variety of events, including capture artifacts, and distortions incurred during coding and transmission. High motion content can cause or exacerbate many kinds of distortion, such as motion blur and stutter. Because of this, the development of objective Video Quality Assessment (VQA) algorithms that can predict the perceptual quality of high motion, live streamed videos is greatly desired. Important resources for developing these algorithms are appropriate databases that exemplify the kinds of live streaming video distortions encountered in practice. Towards making progress in this direction, we built a video quality database specifically designed for live streaming VQA research. The new video database is called the Laboratory for Image and Video Engineering (LIVE) Livestream Database. The LIVE Livestream Database includes 315 videos of 45 source sequences from 33 original contents impaired by 6 types of distortions. We also performed a subjective quality study using the new database, whereby more than 12,000 human opinions were gathered from 40 subjects. We demonstrate the usefulness of the new resource by performing a holistic evaluation of the performance of current state-of-the-art (SOTA) VQA models. We envision that researchers will find the dataset to be useful for the development, testing, and comparison of future VQA models. The LIVE Livestream database is being made publicly available for these purposes at https://live.ece. utexas.edu/research/LIVE_APV_Study/apv_index.html.