Video Block Research Articles

Blind Video Quality Assessment With Weakly Supervised Learning and Resampling Strategy

Due to the 3D spatiotemporal regularities of natural videos and small-scale video quality databases, effective objective video quality assessment (VQA) metrics are difficult to obtain but highly desirable. In this paper, we propose a general-purpose no-reference VQA framework that is based on weakly supervised learning with a convolutional neural network (CNN) and a resampling strategy. First, an eight-layer CNN is trained by weakly supervised learning to construct the relationship between the deformations of the 3D discrete cosine transform of video blocks and the corresponding weak labels judged by a full-reference (FR) VQA metric. Thus, the CNN obtains the quality assessment capacity converted from the FR-VQA metric, and the effective features of the distorted videos can be extracted through the trained network. Then, we map the frequency histogram calculated from the quality score vectors predicted by the trained network onto the perceptual quality. Especially, to improve the performance of the mapping function, we transfer the frequency histogram of the distorted images and videos to resample the training set. The experiments are carried out on several widely used VQA databases. The experimental results demonstrate that the proposed method is on a par with some state-of-the-art VQA metrics and has promising robustness.

Motion Estimation-Assisted Denoising for an Efficient Combination with an HEVC Encoder.

Noise, which is commonly generated in low-light environments or by low-performance cameras, is a major cause of the degradation of compression efficiency. In previous studies that attempted to combine a denoise algorithm and a video encoder, denoising was used independently of the code for pre-processing or post-processing. However, this process must be tightly coupled with encoding because noise affects the compression efficiency greatly. In addition, this represents a major opportunity to reduce the computational complexity, because the encoding process and a denoise algorithm have many similarities. In this paper, a simple, add-on denoising scheme is proposed through a combination of high-efficiency video coding (HEVC) and block matching three-dimensional collaborative filtering (BM3D) algorithms. It is known that BM3D has excellent denoise performance but that it is limited in its use due to its high computational complexity. This paper employs motion estimation in HEVC to replace the block matching of BM3D so that most of the time-consuming functions are shared. To overcome the challenging algorithmic differences, the hierarchical structure in HEVC is uniquely utilized. As a result, the computational complexity is drastically reduced while the competitive performance capabilities in terms of coding efficiency and denoising quality are maintained.

Objective Video Quality Assessment Combining Transfer Learning With CNN.

Nowadays, video quality assessment (VQA) is essential to video compression technology applied to video transmission and storage. However, small-scale video quality databases with imbalanced samples and low-level feature representations for distorted videos impede the development of VQA methods. In this paper, we propose a full-reference (FR) VQA metric integrating transfer learning with a convolutional neural network (CNN). First, we imitate the feature-based transfer learning framework to transfer the distorted images as the related domain, which enriches the distorted samples. Second, to extract high-level spatiotemporal features of the distorted videos, a six-layer CNN with the acknowledged learning ability is pretrained and finetuned by the common features of the distorted image blocks (IBs) and video blocks (VBs), respectively. Notably, the labels of the distorted IBs and VBs are predicted by the classic FR metrics. Finally, based on saliency maps and the entropy function, we conduct a pooling stage to obtain the quality scores of the distorted videos by weighting the block-level scores predicted by the trained CNN. In particular, we introduce a preprocessing and a postprocessing to reduce the impact of inaccurate labels predicted by the FR-VQA metric. Due to feature learning in the proposed framework, two kinds of experimental schemes including train-test iterative procedures on one database and tests on one database with training other databases are carried out. The experimental results demonstrate that the proposed method has high expansibility and is on a par with some state-of-the-art VQA metrics on two widely used VQA databases with various compression distortions.

Dynamic Interactions between Emotion Perception and Action Preparation for Reacting to Social Threat: A Combined cTBS-fMRI Study.

Expressions of emotion are powerful triggers for situation-appropriate responses by the observer. Of particular interest regarding the preparation of such adaptive actions are parietal and premotor cortices, given their potential for interaction with the amygdala (AMG), which is known to play a crucial role in the processing of affective information and in motor response. We set out to disentangle the respective roles of the inferior parietal lobule (IPL) and ventral premotor cortex (PMv) in humans in the processing of emotional body expressions by assessing remote effects of continuous theta burst stimulation (cTBS) in the action network and in AMG. Participants were presented with blocks of short videos showing either angry or neutral whole-body actions. The experiment consisted of three fMRI sessions: two sessions were preceded by stimulation of either right IPL (rIPL) or right PMv (rPMv); and a third session assessed baseline activity. Interestingly, whereas at baseline the left AMG did not differentiate between neutral and angry body postures, a significant difference between these conditions emerged after stimulation of either rIPL or rPMv, with much larger responses to angry than to neutral stimuli. In addition, the effects of cTBS stimulation and emotion were also observed in two other action-relevant areas, the supplementary motor area and the superior parietal cortex. Together, these results show how areas involved in action and emotion perception and in action preparation interact dynamically.

A photon recycling approach to the denoising of ultra-low dose X-ray sequences.

Clinical procedures that make use of fluoroscopy may expose patients as well as the clinical staff (throughout their career) to non-negligible doses of radiation. The potential consequences of such exposures fall under two categories, namely stochastic (mostly cancer) and deterministic risks (skin injury). According to the "as low as reasonably achievable" principle, the radiation dose can be lowered only if the necessary image quality can be maintained. Our work improves upon the existing patch-based denoising algorithms by utilizing a more sophisticated noise model to exploit non-local self-similarity better and this in turn improves the performance of low-rank approximation. The novelty of the proposed approach lies in its properly designed and parameterized noise model and the elimination of initial estimates. This reduces the computational cost significantly. The algorithm has been evaluated on 500 clinical images (7 patients, 20 sequences, 3 clinical sites), taken at ultra-low dose levels, i.e. 50% of the standard low dose level, during electrophysiology procedures. An average improvement in the contrast-to-noise ratio (CNR) by a factor of around 3.5 has been found. This is associated with an image quality achieved at around 12 (square of 3.5) times the ultra-low dose level. Qualitative evaluation by X-ray image quality experts suggests that the method produces denoised images that comply with the required image quality criteria. The results are consistent with the number of patches used, and they demonstrate that it is possible to use motion estimation techniques and "recycle" photons from previous frames to improve the image quality of the current frame. Our results are comparable in terms of CNR to Video Block Matching 3D-a state-of-the-art denoising method. But qualitative analysis by experts confirms that the denoised ultra-low dose X-ray images obtained using our method are more realistic with respect to appearance.

Power-Efficient Sum of Absolute Differences Hardware Architecture Using Adder Compressors for Integer Motion Estimation Design

Sum of absolute differences (SAD) calculation is one of the most time-consuming operations of video encoders compatible with the high efficiency video coding standard. SAD hardware architectures employ an adder tree to accumulate the coefficients from absolute difference between two video blocks. This paper exploits different adder compressors structures into the SAD hardware architecture. The architectures were synthesized to 45-nm CMOS standard cells. Synthesis results show that SAD architecture using 8–2 compressor composed with 4–2 compressors and Kogge–Stone adder in the recombination line reduces power dissipation by 25.5% on average when compared with the SAD architecture using conventional adders from a state-of-the-art synthesis tool. Our throughput analysis shows that the designed SAD units are capable of encoding full HD ( $1920\times 1080$ ) videos in real time at 30 frames/s.

Cannabis and cue-induced craving in cocaine-dependent individuals: A pilot study

BackgroundCannabis consumption is common among cocaine users; however, little is known about its effect on cocaine craving. The objective of this study was to assess whether cannabis co-use is associated with lower cue-induced cocaine craving in non-treatment-seeking cocaine-dependent individuals. MethodsData from twenty-eight cocaine-dependent men were analyzed in this pilot study. Cocaine-dependent subjects (n=12) were compared with cocaine-dependent subjects who also abused or were dependent on cannabis (n=16). After at least 72h of cocaine abstinence, verified using the Timeline Followback and a drug screening test, subjects participated in a functional magnetic resonance imaging session during which neutral and drug cue video sequences were presented. Each sequence comprised four video blocks alternating with resting blocks. We report here subjective craving measures that were collected using the Visual Analog Scale, administered before and after each video block as per standard craving measurement paradigms. ResultsCocaine craving was successfully induced, with no significant difference in cue-induced craving between the two groups. However, post-hoc analyses revealed a significant increase in pre-video cocaine craving scores over time among individuals with cannabis use disorders. ConclusionWe could not highlight significant differences in cocaine craving induction between groups, but we observed a possible deficit in craving decay in the cocaine and cannabis group. In light of this finding, methodology of craving assessment in non-treatment-seeking users, particularly when different substances are combined, should possibly include outcomes linked to craving decay. Studies examining the association between cocaine craving decay and other outcome measures, such as relapse, are also warranted.

A Video Quality Assessment Method for VoIP Applications Based on User Experience

An objective video quality assessment method is proposed to evaluate the video quality in voice over internet protocol (VoIP) applications under network distortion. The fluency and the clarity of videos are two main parts of the factors that affect user experience, thus the method evaluates these two parts to assess the distortions of videos in VoIP applications caused by codec and packet loss. The clarity of the video is measured by calculating block artifacts and frame blurring. Video blocking artifacts are measured by splitting the picture into small blocks and calculating the difference of the pixels around each border while video blurring is measured by getting edge information through Sobel operator, and counting the gradient histogram. Then the video clarity can be measured by a weighted sum of block artifacts score and blurring score using linear regression. The scores are also normalized in order to eliminate the impact of different video contents. The video fluency is calculated by counting the wrong frame in the video. Finally, a weighted sum of video clarity score and video fluency score can represent the quality of the video. The experimental results show that the objective quality scores have a strong correlation with the subjective quality scores, and the algorithm concludes two parts of user experience other than just image quality, which is more comprehensive and it can be used in video quality assessment in VoIP applications.

Video-based eye tracking for neuropsychiatric assessment.

This paper presents a video-based eye-tracking method, ideally deployed via a mobile device or laptop-based webcam, as a tool for measuring brain function. Eye movements and pupillary motility are tightly regulated by brain circuits, are subtly perturbed by many disease states, and are measurable using video-based methods. Quantitative measurement of eye movement by readily available webcams may enable early detection and diagnosis, as well as remote/serial monitoring, of neurological and neuropsychiatric disorders. We successfully extracted computational and semantic features for 14 testing sessions, comprising 42 individual video blocks and approximately 17,000 image frames generated across several days of testing. Here, we demonstrate the feasibility of collecting video-based eye-tracking data from a standard webcam in order to assess psychomotor function. Furthermore, we were able to demonstrate through systematic analysis of this data set that eye-tracking features (in particular, radial and tangential variance on a circular visual-tracking paradigm) predict performance on well-validated psychomotor tests.

Interactive Video Colorization Within a Variational Framework

This paper deals with the difficult problem of video colorization. Methods in the literature are generally based on spatio-temporal video blocks, or on frame to frame color propagation methods, each technique having its own advantages and drawbacks. In this paper , we present both a novel automatic frame-to-frame propagation approach and an interactive correction method within a variational framework. The proposed method propagates colors from an initial colorized frame to the whole grayscale video sequence. The automatic propagation results may be visually unsuitable in some cases. To overcome this limitation, a spatio-temporal functional with a user-guided correction is introduced. Two primal-dual algorithms are designed to solve the proposed variational models. Numerical results show the efficiency and the potentiality of the proposed approach in comparison with state-of-the-art methods.

Motion deblurring based on local temporal compressive sensing for remote sensing image

This paper presents a motion deblurring method which can obtain both the motion information and the recovered image based on local temporal compressive photography. In this method, video blocks are reconstructed at the corners of the image sensor during a single exposure period. The displacement vector, which will be used to build the prior point spread function (PSF) for image deblurring, is then estimated from the reconstructed videos. With the use of the prior PSF, better recovered images can be obtained with much less iteration. An experiment system is also presented to validate the effectiveness of the proposed method. The experimental results show that the proposed method could provide recovered images of high quality.

Where is the ball? Behavioral and neural responses elicited by a magic trick.

We present results from two experiments, in which subjects watched continuous videos of a professional magician repeatedly performing a maneuver in which a ball could "magically" appear under a cup. In all cases, subjects were asked to predict whether the ball would appear under the cup or not, while scalp EEG recordings were performed. Both experiments elicited strong and consistent behavioral and neural responses. In the first experiment, we used two blocks of videos with different probabilities of the ball appearing in the cup and found that, first, based on the behavioral responses, the subjects could track this probability change; and second, the different probabilities modulated the neural responses. In the second experiment, we introduced a control condition in which the magician performed the maneuver under the table, out of subjects' view. Comparing the two conditions (i.e., performing the maneuver within or out of the subjects' view), we found that, first, the magic trick dramatically biased the subjects' behavioral responses; and second, the two conditions led to differential neural responses, in spite of the fact that the stimulus triggering the evoked responses (seeing the ball in the cup) was exactly the same. Altogether, our results show how new insights into sensory and cognitive processing can be obtained using adapted magic tricks. Moreover, the approach of analyzing responses to continuous video presentations offers a more ecological setting compared to classic evoked potential paradigms, which are typically based on presenting static images flashed at the center of the screen.

Fully automatic person segmentation in unconstrained video using spatio-temporal conditional random fields

The segmentation of objects and people in particular is an important problem in computer vision. In this paper, we focus on automatically segmenting a person from challenging video sequences in which we place no constraint on camera viewpoint, camera motion or the movements of a person in the scene. Our approach uses the most confident predictions from a pose detector as a form of anchor or keyframe stick figure prediction which helps guide the segmentation of other more challenging frames in the video. Since even state of the art pose detectors are unreliable on many frames –especially given that we are interested in segmentations with no camera or motion constraints –only the poses or stick figure predictions for frames with the highest confidence in a localized temporal region anchor further processing. The stick figure predictions within confident keyframes are used to extract color, position and optical flow features. Multiple conditional random fields (CRFs) are used to process blocks of video in batches, using a two dimensional CRF for detailed keyframe segmentation as well as 3D CRFs for propagating segmentations to the entire sequence of frames belonging to batches. Location information derived from the pose is also used to refine the results. Importantly, no hand labeled training data is required by our method. We discuss the use of a continuity method that reuses learnt parameters between batches of frames and show how pose predictions can also be improved by our model. We provide an extensive evaluation of our approach, comparing it with a variety of alternative grab cut based methods and a prior state of the art method. We also release our evaluation data to the community to facilitate further experiments. We find that our approach yields state of the art qualitative and quantitative performance compared to prior work and more heuristic alternative approaches.

An efficient wavelet based video compress system based on the novel motion estimation algorithm

Image and video coding performs a significant task in several multimedia applications. In recent years, digital video communication can be found in numerous applications like broadcast services over satellite and terrestrial channels, digital video storage, wired and wireless conversational services and many more. Without processing, it is not possible to store the complete digital video. Video compression denotes the process of decreasing the amount of data required for representing the digital video images and it is the combination of spatial image compression and temporal motion compensation. So far, a lot of compression algorithms had been developed for reducing the data quantity and to render more satisfactory video quality. The main objective for carrying out this research is to construct an efficient video compression system. There are three stages involved in the proposed video compression system. In the first step of video compression, wavelet decomposition is applied to the I-frame and the resulting coefficients are quantized by employing the Listless SPECK (LSK) algorithm. The various kinds of wavelets used in the proposed LSK compression algorithm are the Daubechies, the Coiflet, the Biorthogonal and the Reverse Biorthogonal. The compression ratio gets effectively modified with different types of wavelets used. A novel block motion estimation algorithm is utilized for estimating the motion between the various video blocks. The comparison on the performance of LSK algorithm that uses different kinds of wavelets were made and the performance measures used to analyze the algorithm are the Compression Ratio (CR) and Peak Signal to Noise Ratio (PSNR). An effective estimate of the motion in the frames is obtainable with these performance measures. A highly efficient and effective video compression is achieved with the proposed scheme. Since LSK has been applied to different wavelet decompositions, the algorithm is more advantageous in terms of speed and simple hardware implementation.

Authenticated Live Integration and Verification Scheme in Hybrid Content Delivery Networks

the increase number of wireless communication environment, content distribution and streaming are became very ease. The Live video streaming over wireless network is very popular nowadays. Due to the popularity of streaming delivery of data contents over wireless networks, the development of live video streaming also has attracted much attention. This technology boosts the distribution of live video contents over CDN (Content Delivery Networks). Due to the tremendous popularity and reliability, the live video streaming suffers from huge security issues such as DOS (Denial of Services), clone attacks other malicious attacks. The proposed system protects the data from the above security issues by using a new prototype named as ALIVE. It is a shield for data, which rescues from video crime and other misbehaving activities over CDN and P2P. Alive performs the source and destination authentication while transmitting the video, so this eliminates the entry of spoofer's and unauthenticated nodes in the network. Alive utilizes an encoding technique named as cryptmask, which performs an encoding technique against copyright forgery attacks. To achieve the above goal, alive incorporated with effective encoding techniques. The experiment has been performed using NS-2 tool. KeywordsDelivery Networks, Video streaming, video source identification, wireless video streaming, video blocking.

Robust Kronecker product video denoising based on fractional-order total variation model

Most existing sparse representation-based video denoising algorithms assume video noise is additive Gaussian white noise, which is often violated in practice. In this paper, a robust Kronecker product video denoising (RKPVD) algorithm based on fractional-order total variation model is proposed to remove serious mixed Gaussian-impulse noises from the video data. Using the temporal and spatial correlations of videos, the problem of denoising mixed noises is formulated as a robust low rank video recovery minimization problem based on fractional-order total variation (FTV) model. The resulting under-determined minimization problem, which consists of nuclear norm, Kronecker product sparse ℓ1 norm and FTV, can be efficiently solved by a two-stage algorithm combined with alternating direction method (ADM). The robustness and effectiveness of the proposed RKPVD denoising algorithm on removing mixed Gaussian impulsive noise are validated in the experiments. Compared with several state-of-the-art algorithms, such as total variation (TV), sparse and redundant representation (SARR), video block matching and 3D filtering (VBM3D), robust principal component analysis (RPCA) and robust temporal spatial decomposition (RTSD), intensive experiments show that the proposed RKPVD method has a higher PSNR (peak signal-to-noise ratio) and a better visual detail preservation.

Research of NCS Algorithm in Cloud Demand System

In order to reduce the bandwidth overhead and the node request rejection rate of the cloud video-on-demand server, this paper builds the cloud auxiliary framework and proposes the neighbors and chunks selection algorithm, which contains the node selection sub-algorithm and the block download sub-algorithm. The buffer-based node selection algorithm treats the download point as the benchmark to effectively find the neighbor nodes that can provide the video data, and the buffer-based block download algorithm calculates the required minimum buffer values of the two different kinds of block download strategies under the framework, which provides the theoretical basis for the download method of the video block. The experimental results show that the algorithm in this paper achieves good effects in the aspects of the bandwidth overhead of the server, the request rejection rate and the download rate.

Traffic anomaly detection based on image descriptor in videos

The huge and ever growing volume of traffic video poses a compelling demand for efficient automatic detection of traffic anomaly. In this paper, a new traffic anomaly detection algorithm is introduced. It firstly divides a traffic video into several video cubes in temporal domain, and each video cube is divided into video blocks in spatial domain. Each image block of a video block is described using the local invariant features and the visual codebook approach. Based on the descriptor of the image block, we count the category number of the block (CNB) of a video block. Then, a Gaussian distribution model for estimating the probability of normal traffic with respect to the CNB is learned. The learned Gaussian distribution model is then used to detect the traffic anomaly from the test traffic video. Eventually, the results of all video blocks are fused to achieve the final decision. Experimental results show that the proposed algorithm performs better than two existing algorithms on both the intersection traffic videos and main road traffic videos.

Live Video Forensics: Source Identification in Lossy Wireless Networks

Video source identification is very important in validating video evidence, tracking down video piracy crimes, and regulating individual video sources. With the prevalence of wireless communication, wireless video cameras continue to replace their wired counterparts in security/surveillance systems and tactical networks. However, wirelessly streamed videos usually suffer from blocking and blurring due to inevitable packet loss in wireless transmissions. The existing source identification methods experience significant performance degradation or even fail to work when identifying videos with blocking and blurring. In this paper, we propose a method that is effective and efficient in identifying such wirelessly streamed videos. In addition, we also propose to incorporate wireless channel signatures and selective frame processing into source identification, which significantly improve the identification speed. We conduct extensive real-world experiments to validate our method. The results show that the source identification accuracy of the proposed scheme largely outperforms the existing methods in the presence of video blocking and blurring. Moreover, our method is able to identify the video source in a near-real-time fashion, which can be used to detect the wireless camera spoofing attack.

SPATIAL MOTION PATTERNS: ACTION MODELS FROM SEMI-DENSE TRAJECTORIES

A new action model is proposed, by revisiting local binary patterns (LBP) for dynamic texture models, applied on trajectory beams calculated on the video. The use of semi-dense trajectory field allows to dramatically reduce the computation support to essential motion information, while maintaining a large amount of data to ensure robustness of statistical bag of features action models. A new binary pattern, called Spatial Motion Pattern (SMP) is proposed, which captures self-similarity of velocity around each tracked point (particle), along its trajectory. This operator highlights the geometric shape of rigid parts of moving objects in a video sequence. SMPs are combined with basic velocity information to form the local action primitives. Then, a global representation of a space × time video block is provided by using hierarchical blockwise histograms, which allows to efficiently represent the action as a whole, while preserving a certain level of spatiotemporal relation between the action primitives. Inheriting from the efficiency and the invariance properties of both the semi-dense tracker Video extruder and the LBP-based representations, the method is designed for the fast computation of action descriptors in unconstrained videos. For improving both robustness and computation time in the case of high definition video, we also present an enhanced version of the semi-dense tracker based on the so-called super particles, which reduces the number of trajectories while improving their length, reliability and spatial distribution.