Temporal Scalability Research Articles

Trustworthy Authentication on Scalable Surveillance Video with Background Model Support

H.264/SVC (Scalable Video Coding) codestreams, which consist of a single base layer and multiple enhancement layers, are designed for quality, spatial, and temporal scalabilities. They can be transmitted over networks of different bandwidths and seamlessly accessed by various terminal devices. With a huge amount of video surveillance and various devices becoming an integral part of the security infrastructure, the industry is currently starting to use the SVC standard to process digital video for surveillance applications such that clients with different network bandwidth connections and display capabilities can seamlessly access various SVC surveillance (sub)codestreams. In order to guarantee the trustworthiness and integrity of received SVC codestreams, engineers and researchers have proposed several authentication schemes to protect video data. However, existing algorithms cannot simultaneously satisfy both efficiency and robustness for SVC surveillance codestreams. Hence, in this article, a highly efficient and robust authentication scheme, named TrustSSV (Trust Scalable Surveillance Video), is proposed. Based on quality/spatial scalable characteristics of SVC codestreams, TrustSSV combines cryptographic and content-based authentication techniques to authenticate the base layer and enhancement layers, respectively. Based on temporal scalable characteristics of surveillance codestreams, TrustSSV extracts, updates, and authenticates foreground features for each access unit dynamically with background model support. Using SVC test sequences, our experimental results indicate that the scheme is able to distinguish between content-preserving and content-changing manipulations and to pinpoint tampered locations. Compared with existing schemes, the proposed scheme incurs very small computation and communication costs.

Temporal Scaling of Age-Dependent Mortality: Dynamics of Aging in Caenorhabditis elegans Is Easy to Speed Up or Slow Down, but Its Overall Trajectory Is Stable.

The dynamics of aging is often described by survival curves that show the proportion of individuals surviving to a given age. The shape of the survival curve reflects the dependence of mortality on age, and it varies greatly for different organisms. In a recently published paper, Stroustrup and coauthors ((2016) Nature, 530, 103-107) showed that many factors affecting the lifespan of Caenorhabditis elegans do not change the shape of the survival curve, but only stretch or compress it in time. Apparently, this means that aging is a programmed process whose trajectory is difficult to change, although it is possible to speed it up or slow it down. More research is needed to clarify whether the "rule of temporal scaling" is applicable to other organisms. A good indicator of temporal scaling is the coefficient of lifespan variation: similar values of this coefficient for two samples indicate similar shape of the survival curves. Preliminary results of experiments on adaptation of Drosophila melanogaster to unfavorable food show that temporal scalability of survival curves is sometimes present in more complex organisms, although this is not a universal rule. Both evolutionary and environmental changes sometimes affect only the average lifespan without changing the coefficient of variation (in this case, temporal scaling is present), but often both parameters (i.e. both scale and shape of the survival curve) change simultaneously. In addition to the relative stability of the coefficient of variation, another possible argument in favor of genetic determination of the aging process is relatively low variability of the time of death, which is sometimes of the same order of magnitude as the variability of timing of other ontogenetic events, such as the onset of sexual maturation.

Simulation and experimental testbed for adaptive video streaming in ad hoc networks

This paper presents a performance evaluation of the scalable video streaming over mobile ad hoc networks. In particular, we focus on the rate-adaptive method for streaming scalable video (H.264/SVC). For effective adaptation a new cross-layer routing protocol is introduced. This protocol provides an efficient algorithm for available bandwidth estimation. With this information, the video source adjusts its bit rate during the video transmission according to the network state. We also propose a free simulation framework that supports evaluation studies for scalable video streaming. The simulation experiments performed in this study involve the transmission of SVC streams with Medium Grain Scalability (MGS) as well as temporal scalability over different network scenarios. The results reveal that the rate-adaptive strategy helps avoid or reduce the congestion in MANETs obtaining a better quality in the received videos. Additionally, an actual ad hoc network was implemented using embedded devices (Raspberry Pi) in order to assess the performance of the proposed adaptive transmission mechanism in a real environment. Additional experiments were carried out prior to the implementation with the aim of characterizing the wireless medium and packet loss profile. Finally, the proposed approach shows an important reduction in energy consumption, as the study revealed.

Performance Evaluation of Scalable Video Streaming in Mobile Ad hoc Networks

The development of video streaming services on wireless ad hoc networks is a challenge task as a consequence of different limitations such as bandwidth-constrained, variable capacity links and energy-constrained operation. Moreover, the dynamic topology of nodes causes frequent link failures and high error rates. We propose in this paper a performance evaluation of the scalable video streaming over mobile ad hoc networks. In particular, we focus on the rate-adaptive strategy for streaming scalable video (H.264/SVC). In order to provide QoS mechanisms in the routing process, a new routing protocol is introduced. This protocol estimates the available bandwidth value, which is sent to video source in order to adapt the bit rate during the video transmission. We also propose a simulation framework that supports evaluation studies for scalable video streaming. In the simulation experiments, SVC streams with combined scalability (quality and temporal scalability) were used. As quality scalability method, we used Medium Grain Scalability (MGS). The results reveal that the rate-adaptive method helps avoid or reduce the congestion in MANETs obtaining a better quality in the received videos.

Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard

This paper provides an overview of Scalable High efficiency Video Coding (SHVC), the scalable extensions of the High Efficiency Video Coding (HEVC) standard, published in the second version of HEVC. In addition to the temporal scalability already provided by the first version of HEVC, SHVC further provides spatial, signal-to-noise ratio, bit depth, and color gamut scalability functionalities, as well as combinations of any of these. The SHVC architecture design enables SHVC implementations to be built using multiple repurposed single-layer HEVC codec cores, with the addition of interlayer reference picture processing modules. The general multilayer high-level syntax design common to all multilayer HEVC extensions, including SHVC, MV-HEVC, and 3D HEVC, is described. The interlayer reference picture processing modules, including texture and motion resampling and color mapping, are also described. Performance comparisons are provided for SHVC versus simulcast HEVC and versus the scalable video coding extension to H.264/advanced video coding.

Inter-layer Texture and Syntax Prediction for Scalable Video Coding

In this paper, we demonstrate inter-layer prediction tools for scalable video coders. The proposed scalable coder is designed to support not only spatial, quality and temporal scalabilities, but also view scalability. In addition, we propose quad-tree inter-layer prediction tools to improve coding efficiency at enhancement layers. The proposed inter-layer prediction tools generate texture prediction signal with exploiting texture, syntaxes, and residual information from a reference layer. Furthermore, the tools can be used with inter and intra prediction blocks within a large coding unit. The proposed framework guarantees the rate distortion performance for a base layer because it does not have any compulsion such as constraint intra prediction. According to experiments, the framework supports the spatial scalable functionality with about 18.6%, 18.5% and 25.2% overhead bits against to the single layer coding. The proposed inter-layer prediction tool in multiloop decoding design framework enables to achieve coding gains of 14.0%, 5.1%, and 12.1% in BD-Bitrate at the enhancement layer, compared to a single layer HEVC for all-intra, low-delay, and random access cases, respectively. For the single-loop decoding design, the proposed quad-tree inter-layer prediction can achieve 14.0%, 3.7%, and 9.8% bit saving.

MCDCT-TF based video watermarking resilient to temporal and quality scaling

Heterogeneity among the end using display devices and corresponding varying system requirements make scalable version of H.264/AVC standard more popular. Existing watermarking based authentication system may not be suitable well for this new extension, because the scalability property of the codec may itself is perceived as an attack popularly known as content adaptation attack. In this paper, a blind scalable video watermarking scheme is proposed, which is robust against quality and temporal scalability. In the proposed scheme, Discrete Cosine Transform (DCT) based temporal filtering and wavelet based spatial filtering is used for choosing suitable watermark embedding zone. Experimental evidences are provided to justify the improved robustness of the proposed scheme over existing related watermarking schemes. The visual quality of the watermarked video has also been evaluated to show the efficiency of the proposed scheme.

Temporal resolution vs. visual saliency in videos: Analysis of gaze patterns and evaluation of saliency models

Temporal scalability of videos refers to the possibility of changing frame rate adaptively for efficient video transmission. Changing the frame rate may alter the spatial location that the viewers pay attention in the scene, which in turn significantly influences human׳s quality perception. Therefore, in order to effectively exploit the temporal scalability in applications, it is necessary to understand the relationship between frame rate variation and visual saliency. In this study, we answer the following three research questions: (1) Does the frame rate influence the overall gaze patterns (in an average sense over subjects)? (2) Does the frame rate influence the inter-subject variability of the gaze patterns? (3) Do the state-of-the-art saliency models predict human gaze patterns reliably for different frame rates? To answer the first two questions, we conduct an eye-tracking experiment. Under a free viewing scenario, we collect and analyze gaze-paths of human subjects watching high-definition (HD) videos having a normal or low frame rate. Our results show that both the average gaze-path and subject-wise variability of the gaze-path are influenced by frame rate variation. Then, we apply representative state-of-the-art saliency models to the videos and evaluate their performance by using the gaze pattern data collected from the eye-tracking experiment in order to answer the third question. It is shown that there exists a trade-off relation between accuracy in predicting the gaze pattern and robustness to frame rate variation, which raises necessity of further research in saliency modeling to simultaneously achieve both accuracy and robustness.

Spatio-Temporal Wavelet Based Video Compression: a Simulink Implementation for Acceleration

In this paper, we presented a Wavelet based video compression method. The wavelet transform is performed by lifting scheme methodology which allows temporal and special scalability as well as lossless reconstruction. The spatial decorrelation was elaborated with the standard JPEG2000. We exploited the inter-frames correlation thanks to a temporal wavelet transformation. A Simulink based implementation is then experimented in order to accelerate the process thanks to a co-simulation step

Temporal video transcoding from H.264/AVC-to-SVC for digital TV broadcasting

Mobile digital TV environments demand flexible video compression like scalable video coding (SVC) because of varying bandwidths and devices. Since existing infrastructures highly rely on H.264/AVC video compression, network providers could adapt the current H.264/AVC encoded video to SVC. This adaptation needs to be done efficiently to reduce processing power and operational cost. This paper proposes two techniques to convert an H.264/AVC bitstream in Baseline (P-pictures based) and Main Profile (B-pictures based) without scalability to a scalable bitstream with temporal scalability as part of a framework for low-complexity video adaptation for digital TV broadcasting. Our approaches are based on accelerating the interprediction, focusing on reducing the coding complexity of mode decision and motion estimation tasks of the encoder stage by using information available after the H.264/AVC decoding stage. The results show that when our techniques are applied, the complexity is reduced by 98 % while maintaining coding efficiency.

H.264/AVC-to-SVC temporal video transcoder for video broadcasting in wireless networks

Over the last few decades multimedia usage has changed dramatically, with networks and terminals of diverse bandwidths and capabilities coexisting. To ensure a good quality of experience, this diverse environment requires adaptability of the video stream. If video streams were compressed using Scalable Video Coding (SVC) schemes, they would be able to adapt to these heterogeneous networks and a wide range of terminals. However, since the majority of multimedia contents are compressed using H.264/AVC, they cannot benefit from this scalability. This paper makes some proposals for improving a temporal H.264/AVC-to-SVC video transcoder so that it can convert an H.264/AVC bitstream without scalability to scalable bitstreams with temporal scalability. The idea behind the proposed transcoder is to accelerate interprediction, focusing on reducing the coding complexity of the mode decision and motion estimation tasks of the encoder stage by using information available after the H.264/AVC decoding stage. To accelerate the mode decision task, a decision tree for narrowing down the Macroblock (MB) types to be checked by the SVC encoder has been developed. On the other hand, to accelerate the motion estimation task, a reduced area is created dynamically for every MB and sub-MB. Moreover, time savings increase considerably when both proposals are adjusted to work together, as opposed to doing so in an isolated way. The results show that when our joint proposal is applied, the complexity is reduced by 98 % while maintaining coding efficiency.

Locomotion Learning for an Anguilliform Robotic Fish Using Central Pattern Generator Approach

In this paper, we present locomotion learning for an Anguilliform robotic fish using a central pattern generator (CPG) approach. First, we give the overall structure of the CPG. Different from a traditional CPG that contains only coupled oscillators, our CPG consists of coupled Andronov-Hopf oscillators, an artificial neural network (ANN), and an outer amplitude modulator. Coupled oscillators, which possess a limit-cycle character, are used to generate inputs to excite the ANN. The ANN serves as a learning mechanism, from which we can obtain desired waveforms. By inputting different signals to the ANN, different desired locomotion patterns can be obtained. Outer amplitude modulator resizes the amplitudes of the ANN outputs according to task specifications. The CPG possess temporal scalability, spatial scalability, and phase-shift property; thus, we can obtain desired amplitudes, oscillation frequencies, and phase differences by tuning corresponding parameters. By extracting the swimming pattern from a real fish and using the CPG approach, we successfully generate a new swimming pattern and apply it to the robotic fish. The new pattern reserves the swimming characters of the real fish, and it is more suitable to be applied to the robotic fish. By using the new pattern, the robotic fish can perform both forward locomotion and backward locomotion, which are validated by experiments.

Balancing energy use against video quality in mobile devices

It is now common to access mobile video services through various Internet-connectable devices such as game stations, smartphones and tablets. This paper proposes a new energy management scheme for mobile devices by making use of the temporal scalability available in scalable video coding (SVC) techniques. The proposed scheme determines a quality index for each video segment, based on the pixel-wise difference between the luminance planes of consecutive frames at each frame-rate, and predicts the computation needed to decode each frame using data relating frame sizes to previous decoding times. A new dynamic voltage and frequency scaling (DVFS) scheme is then invoked to achieve the specified quality index. The proposed scheme was implemented on a smartphone and a user test was conducted to assess the acceptability of different degree of frame-rate reduction. Real measurements showed that the proposed scheme uses between 5% and 41% less energy than the Linux DVFS governor used in many smartphones, while maintaining acceptable video quality.

GOP Adaptation Coding of H.264/SVC Based on Precise Positions of Video Cuts

Hierarchical B-frame coding was introduced into H.264/SVC to provide temporal scalability and improve coding performance. A content analysis-based adaptive group of picture structure (AGS) can further improve the coding efficiency, but damages the inter-frame correlation and temporal scalability of hierarchical B-frame to different degrees. In this paper, we propose a group of pictures (GOP) adaptation coding method based on the positions of video cuts. First, the cut positions are accurately detected by the combination of motion coherence (MC) and mutual information (MI); then the GOP is adaptively and proportionately set by the analysis of MC in one scene. In addition, we propose a binary tree algorithm to achieve the temporal scalability of any size of GOP. The results for test sequences and real videos show that the proposed method reduces the bit rate by up to about 15%, achieves a performance gain of about 0.28?1.67 dB over a fixed GOP, and has the advantages of better transmission resilience and video summaries.

Adaptive HTTP Streaming with Temporal Scalability in High Efficiency Video Coding (HEVC)

The newly approved High Efficiency Video Coding Standard (HEVC) includes temporal sub-layering feature, which provides temporal scalability. Two types of pictures — Temporal Sub-layer Access Pictures and Step-wise Temporal Sub-layer Access Pictures are provided for this purpose. This paper utilizes the temporal scalability in HEVC to provide bandwidth adaptive HTTP streaming. We describe our HTTP streaming algorithm, which is media timeline aware and which dynamically switches temporal sub-layers on the server side. We performed subjective tests to determine user perception regarding acceptable frame rates when using temporal scalability of HEVC. These results are used to control the algorithm's temporal switching behavior to provide a good quality of experience to the user. We applied Internet and 3GPP error-delay patterns to validate the performance of our algorithm.

Efficient Lagrange multiplier selection for scalable video coding using temporal inter-layer dependency

Designed for scalable video coding, an efficient algorithm for Lagrange multiplier selection, named as Lagrange multiplier selection with temporal scalability, is proposed based on inter-layer dependency in the temporal domain. The inter-layer dependency between the temporal base layer and higher temporal layers is explored and used to select the Lagrange multiplier, which leads to efficient allocation of bit budgets to different temporal layers according to their impact and importance in optimization of the overall video quality. Experimental results show that compared with using the original Lagrange multiplier in the reference software model of H.264/SVC (i.e., JSVM), the proposed Lagrange multiplier achieves a performance gain of up to 0.41 dB in $$\Delta PSNR$$ and 10.61 % in bit-rate reduction at various bit-rates. At lower bit-rates, the performance gain can be further improved. Consistent performance gain is also observed when compared with two other state-of-the-art Lagrange multiplier selection techniques for scalable video coding.

Progressively refined wyner-ziv video coding for visual sensors

Wyner-Ziv video coding constitutes an alluring paradigm for visual sensor networks, offering efficient video compression with low complexity encoding characteristics. This work presents a novel hash-driven Wyner-Ziv video coding architecture for visual sensors, implementing the principles of successively refined Wyner-Ziv coding. To this end, so-called side-information refinement levels are constructed for a number of grouped frequency bands of the discrete cosine transform. The proposed codec creates side-information by means of an original overlapped block motion estimation and pixel-based multihypothesis prediction technique, specifically built around the pursued refinement strategy. The quality of the side-information generated at every refinement level is successively improved, leading to gradually enhanced Wyner-Ziv coding performance. Additionally, this work explores several temporal prediction structures, including a new hierarchical unidirectional prediction structure, providing both temporal scalability and low delay coding. Experimental results include a thorough evaluation of our novel Wyner-Ziv codec, assessing the impact of the proposed successive refinement scheme and the supported temporal prediction structures for a wide range of hash configurations and group of pictures sizes. The results report significant compression gains with respect to benchmark systems in Wyner-Ziv video coding (e.g., up to 42.03% over DISCOVER) as well as versus alternative state-of-the-art schemes refining the side-information.

A Hybrid Scheme for Authenticating Scalable Video Codestreams

A scalable video coding (SVC) codestream consists of one base layer and possibly several enhancement layers. The base layer, which contains the lowest quality and resolution images, is the foundation of the SVC codestream and must be delivered to recipients, whereas enhancement layers contain richer contour/texture of images in order to supplement the base layer in resolution, quality, and temporal scalabilities. This paper presents a novel hybrid authentication (HAU) scheme. The HAU employs both cryptographic authentication and content-based authentication techniques to ensure integrity and authenticity of the SVC codestreams. Our analysis and experimental results indicate that the HAU is able to detect malicious manipulations and locate the tampered image regions while is robust to content-preserving manipulations for enhancement layers. Although our focus in this paper is on authenticating H.264/SVC codestreams, the proposed technique is also applicable to authenticate other scalable multimedia contents such as MPEG-4 fine grain scalability and JPEG2000 codestreams.

Perceptual hash function for scalable video

Perceptual hash functions are important for video authentication based on digital signature verifying the originality and integrity of videos. They derive hashes from the perceptual contents of the videos and are robust against the common content-preserving operations on the videos. The advancements in the field of scalable video coding call for efficient hash functions that are also robust against the temporal, spatial and bit rate scalability features of the these coding schemes. This paper presents a new algorithm to extract hashes of scalably coded videos using the 3D discrete wavelet transform. A hash of a video is computed at the group-of-frames level from the spatio-temporal low-pass bands of the wavelet-transformed groups-of-frames. For each group-of-frames, the spatio-temporal low-pass band is divided into perceptual blocks and a hash is derived from the cumulative averages of their averages. Experimental results demonstrate the robustness of the hash function against the scalability features and the common content-preserving operations as well as the sensitivity to the various types of content differences. Two critical properties of the hash function, diffusion and confusion, are also examined.

Scalable video transcoding for mobile communications

Mobile multimedia contents have been introduced in the market and their demand is growing every day due to the increasing number of mobile devices and the possibility to watch them at any moment in any place. These multimedia contents are delivered over different networks that are visualized in mobile terminals with heterogeneous characteristics. To ensure a continuous high quality it is desirable that this multimedia content can be adapted on-the-fly to the transmission constraints and the characteristics of the mobile devices. In general, video contents are compressed to save storage capacity and to reduce the bandwidth required for its transmission. Therefore, if these compressed video streams were compressed using scalable video coding schemes, they would be able to adapt to those heterogeneous networks and a wide range of terminals. Since the majority of the multimedia contents are compressed using H.264/AVC, they cannot benefit from that scalability. This paper proposes a technique to convert an H.264/AVC bitstream without scalability to a scalable bitstream with temporal scalability as part of a scalable video transcoder for mobile communications. The results show that when our technique is applied, the complexity is reduced by 98 % while maintaining coding efficiency.