Playback Rate Research Articles

Self-Supervised Motion Perception for Spatiotemporal Representation Learning.

In this study, we propose a novel pretext task and a self-supervised motion perception (SMP) method for spatiotemporal representation learning. The pretext task is defined as video playback rate perception, which utilizes temporal dilated sampling to augment video clips to multiple duplicates of different temporal resolutions. The SMP method is built upon discriminative and generative motion perception models, which capture representations related to motion dynamics and appearance from video clips of multiple temporal resolutions in a collaborative fashion. To enhance the collaboration, we further propose difference and convolution motion attention (MA), which drives the generative model focusing on motion-related appearance, and leverage multiple granularity perception (MG) to extract accurate motion dynamics. Extensive experiments demonstrate SMP's effectiveness for video motion perception and state-of-the-art performance of self-supervised representation models upon target tasks, including action recognition and video retrieval. Code for SMP is available at github.com/yuanyao366/SMP.

Delay and Energy Minimization for Adaptive Video Streaming: A Joint Edge Caching, Computing and Power Allocation Approach

Adaptive bitrate streaming can confront dynamic environment and satisfy multifarious user requirements by encoding video files into different bitrate versions. It can significantly improve the user quality of experience (QoE). But at the same time, it can also lead to a further increase in video data volume. The interaction of massive video data tends to aggravate network congestion and increase transmission energy, which triggers new challenges for wireless networks in terms of delivery delay and energy consumption. In this paper, taking different video sizes and playback rate requirements into account, we aim to minimize the weighted sum of delivery delay and energy consumption for adaptive video streaming. The problem is formulated by jointly optimizing caching, computing and power allocation, which turns out to be a mixed integer nonlinear programming (MINLP) problem. To tackle it, a joint caching, computing and power allocation (JCCPA) algorithm is proposed. More specifically, the original problem is decomposed into two subproblems: 1) power allocation problem and 2) joint caching placement and computing decision problem. The power allocation problem is proved to be a quasi-convex problem, and the closed-form solution is derived. With the optimal transmit power, the joint caching placement and computing decision problem is further divided into two subproblems, which are solved in an iterative manner. Simulation results demonstrate that the proposed algorithms can effectively reduce the weighted sum of delivery delay and energy consumption (outperforms the baselines around 5%–12% within a certain range).

Cadence Feedback and Video-Based Engagement Improves Motivation and Performance during Pedalling in Stroke Patients

Video and music as a dissociative attention stimulus during exercise is known to distract from the discomfort of physical exertion and improve exercise adherence; however, the influence of video-based feedback and engagement during pedalling on the performance and motivation of pedalling in stroke patients is poorly understood. The aim of this study was to employ a novel video-based engagement paradigm for pedalling in stroke patients and evaluate its capacity to influence the cadence control, physiological output, and perceived motivation and enjoyment. Thirteen stroke patients were recruited with low-to-moderate lower-limb disability (mean age: 64.0 yrs.). A reference group of 18 healthy young adult subjects (mean age: 27.7 yrs.) was also recruited to assess the broad applicability of the techniques to a contrasting non-pathological cohort. The participants pedalled at a slow (60 RPM) and fast (100 RPM) target speed with constant resistance in 15 min pedalling bouts that included (i) baseline pedalling with real-time visual feedback of cadence deviation from the target provided only in the first 20 s (ii) real-time visual feedback of cadence data over the entire pedalling session, and (iii) real-time engagement to maintain the playback rate of a prerecorded video by pedalling at the target speed. During low speed pedalling, stroke patients demonstrated significantly smaller absolute cadence deviation during pedalling with feedback (mean difference: 1.8 RPM, p = 0.014) and video-based engagement (mean difference: 2.4 RPM, p = 0.006) compared to the baseline pedalling. For the healthy adults, feedback and video-based engagement reduced cadence deviation significantly at all speeds (p < 0.05). All but one stroke patient either enjoyed or really enjoyed the video engagement during pedalling and felt motivated to undertake this form of exercise in therapy in the future. This proof-of-concept study showed that feedback and video-based engagement may improve the targeted pedalling performance in stroke patients, and by helping dissociate subjects from physical cues associated with fatigue, may ultimately improve exercise motivation and compliance.

Intrusion Detection Technology for Wireless Sensor Networks Based on Autoregressive Moving Average

In wireless sensor networks (WSNs), aiming at the problems that internal attacks such as network congestion and high energy consumption seriously threaten the network security and normal operation, an intrusion detection technology based on traffic prediction is proposed. Firstly, the technology uses the autoregressive moving average model ARMA (autoregressive moving average) to establish the ARMA traffic prediction model for the node and then uses the predicted traffic value to obtain the traffic reception rate range through the node. Finally, the detection effect is achieved by comparing whether the actual service reception rate exceeds the prediction range. The experimental results show that, compared with the single ARMA model, under the same message playback rate, this technology has higher detection rate and lower false alarm rate and reduces the energy consumption of network nodes.

The Effect of Real-Time Video-Based Engagement and Feedback during Pedaling on Cadence Control and Exercise Motivation: A Proof-of-Concept Study.

The use of video and music as an intrinsic, dissociative attentional stimulus during exercise is thought to distract from the physical discomfort of exercise, and contribute to improved exercise adherence; however, the effects of video-based feedback and engagement during pedaling on exercise performance and motivation are poorly understood. The aims of the present study were twofold. Firstly, to develop a novel video-based engagement regime for pedaling that links pedaling cadence with the play rate of a video, and secondly, to employ an instrumented pedaling device to assess the influence of the video engagement paradigm on cadence performance and exercise motivation. Eighteen healthy subjects participated in 15-min-duration pedaling sessions while targeting a specific low cadence (60 rotations per minute) and high cadence (100 rotations per minute), including pedaling with the provision of (i) target pedaling cadence information only, (ii) visual feedback on cadence control, including pedaling duration, pedaling cadence, and cadence deviation from target, and (iii) real-time engagement, which involved pedaling at the target speed to maintain the playback rate of a pre-recorded video. Cadence deviation from the target was evaluated, and self-reported exercise motivation examined with a post-exercise survey. Pedaling-cadence deviations significantly reduced with cadence feedback at both low and high cadence (p < 0.05). Participants reported enjoying feedback and video-based engagement during pedaling, with 83% of participants feeling that engagement motivated them to perform pedaling-based exercise. In conclusion, real-time cadence control feedback and video-based engagement during pedaling for healthy individuals may improve performance in targeted pedaling tasks. Through dissociation from the physical cues associated with exercise and fatigue, feedback and engagement may ultimately increase enjoyment and exercise compliance and adherence of pedaling-based exercise. The findings may be useful in prescription and maintenance of targeted pedaling exercises for stroke rehabilitation and exercise therapy.

Video Classification for Video Service Providers : A Survey

One of the very interesting data modalities is video. From a dimensionality and size perspective, videos are one of the most interesting and intuitive data types which enable fast and easy object recognition and learning. Video classification is an important task for archiving digital contents for various video service providers. Video uploading platforms such as YouTube are collecting enormous datasets, empowering Deep Learning research. Videos being an important source to recognize any activity by the humans, video classification becomes an important and critical job for video service providers. The survey paper studies various deep learning, transfer learning and hybrid model approaches. Video data normally occurs as continuous, analog signals In order for a computer to process this video data, the analog signals must be converted to a non-continuous, digital format. In a digital format, the video data can be stored as a series of bits on a hard disk or in computer memory. A video sequence is displayed as a series of frames. Each frame is a snapshot of a moment in time of the motion-video data, and is very similar to a still image. When the frames are played back in sequence on a display device, a rendering of the original video data is created. In real-time video the playback rate is 30 frames per second. This is the minimum rate necessary for the human eye to successfully blend each video frame together into a continuous, smoothly moving image. A single frame of video data can be quite large in size. A video frame with a resolution of 512 x 482 will contain 246,784 pixels. If each pixel contains 24 bits of color information, the frame will require 740,352 bytes of memory or disk space to store. Assuming there are 30 frames per second for real-time video, a 10-second video sequence would be more than 222 megabytes in size! It is clear there can be no computer video without at least one efficient method of video data compression.

Optimized Preference-Aware Multi-Path Video Streaming with Scalable Video Coding

Most client hosts are equipped with multiple network interfaces (e.g., WiFi and cellular networks). Simultaneous access of multiple interfaces can significantly improve the users’ quality of experience (QoE) in video streaming. An intuitive approach to achieve it is to use Multi-path TCP (MPTCP). However, the deployment of MPTCP, especially with link preference, requires OS kernel update at both the client and server side, and a vast amount of commercial content providers do not support MPTCP. Thus, in this paper, we realize a multi-path video streaming algorithm in the application layer instead, by considering Scalable Video Coding (SVC), where each layer of every chunk can be fetched from only one of the orthogonal paths. We formulate the quality decisions of video chunks subject to the available bandwidth of the different paths, chunk deadlines, and link preferences as an optimization problem. The objective is to to optimize a QoE metric that maintains a tradeoff between maximizing the playback rate of every chunk and ensuring fairness among chunks. The proposed metric prefers to use bandwidth of the links to optimize a concave utility function of the chunk quality. Even though the formulation is a non-convex discrete optimization, we provide a quadratic complexity algorithm which is shown to be optimal in some special cases. We further propose an online algorithm where several challenges including bandwidth prediction errors, are addressed. Extensive emulated experiments in a real testbed with real traces of public dataset reveal the robustness of our scheme and demonstrate its significant performance improvement compared to other multi-path algorithms.

Peer-to-Peer Data Dissemination for Deadline-Sensitive Streaming in VANETs

In this paper, a mechanism of Peer-to-Peer (P2P) data dissemination for distributing deadline-sensitive streaming files in Vehicular Ad-Hoc Networks (VANETs) is proposed. In a multi-media application sensitive to deadline, the data requested by the peer needs to be received before its playback deadline. For the purpose, we adopt different strategies from the previous works, as follows. First, a distinct lookup function is designed by just collecting the part information of data distribution to reduce the network overhead. It is executed simultaneously with the task of requesting data to take advantage of the timeliness of collected information. Second, in order to utilize the limited bandwidth of VANETs efficiently, the function in requesting data schedules only data sufficient to meet the playback rate requirement of the service. Third, a method is designed to provide deadline-sensitive segment dissemination and alleviate the problem of the highly mobile hosts in VANETs. It interactively uses the estimated and actual historical values of the transmission time between two neighboring hosts and the period of two neighboring hosts remaining connected. Fourth, to further alleviate the mobility problem, a route recovery function is designed to recover a disconnected route timely, and a system parameter can be set according to host speed. Fifth, the setting of the system parameter can be a trade-off between the playing quality of the service and the capacity of the network. Compared with a very recently related work, the effectiveness of this mechanism has been verified by a lot of simulations. When the hosts in the network are static, the admission peers of the mechanism increases by 18% compared with the recent work. On the other hand, when the hosts are dynamic, the mechanism improves 10-25% and 0-14% in admission peers and timely received data over the recent work, respectively.

A Client-Based Fast Channel Change Technique Using Multiple Decoder Clocks

Rapid channel change is commonly considered a critical factor that improves subscriber quality of experience. One of the major contributors to increased channel change time is the delay due to buffering a newly-acquired media stream before the stream is presented to the user. When the decoder operates at a constant playback rate, this initial buffering avoids playback stalls. An initial buffering delay on the order of seconds or more is not uncommon, especially with advanced video compression standards such as H.264 or H.265. This article presents a novel approach, which manages audio and video decoder clocks independently and reduces the initial buffering delay significantly. The method is low-cost and can be easily implemented in software to run on most modern media processors. The method has been deployed and verified in various IPTV networks. Experiments carried out in the field show that the presented technique can provide up to 62% reduction in overall channel change time compared to conventional methods.

GroupCast: Preference-Aware Cooperative Video Streaming With Scalable Video Coding

In this paper, we propose a preference-aware cooperative video streaming system for videos encoded using scalable video coding (SVC). In the proposed system, the collaborating users are interested in watching a video together on a shared screen. However, the willingness of each user to cooperate is subject to her own constraints such as user data plans. Using SVC, videos are segmented into chunks and each chunk is encoded using layers, where each layer can be fetched through any of the collaborating users. We formulate the problem of finding the optimal quality decisions and fetching policy of the SVC layers of video chunks subject to the available bandwidth, chunk deadlines, and cooperation willingness of the different users as an optimization problem. The objective is to optimize a QoE metric that maintains a trade-off between maximizing the playback rate of every chunk and ensuring fairness among all chunks to achieve the minimum skip (stall) duration without violating any of the imposed constraints. We propose an offline algorithm to solve the non-convex optimization problem when the bandwidth prediction is non-causally known. This algorithm has a run-time complexity that is polynomial in the video length and the number of collaborating users. Furthermore, we propose an online version of the algorithm for practical scenarios, where erroneous bandwidth prediction for a short window is used. Real implementation with android devices using SVC encoded video and a public dataset of bandwidth traces reveals the robustness and performance of the proposed algorithm and shows that the algorithm significantly outperforms round robin-based mechanisms in terms of avoiding skips/stalls and fetching video chunks at their highest quality possible.

PREPARE-Playback RatE and Priority Adaptive BitRate Selection

HAS (HTTP-based Adaptive Streaming) based on ABR (Adaptive Bitrate Rate selection) has become a prevalent solution to media transmission. In order to improve the performance of HAS, many studies focus on how to reduce the unfairness, the instability and the inefficiency in a multi-clients scenario. However, the unfairness problem of the multi-clients with different priority levels and different playback rates, and the instability problem caused by the changes of the playback rate are still unresolved. To solve these problems, a novel ABR algorithm called PREPARE is proposed in this paper. In addition to normally considered throughput and client buffer information in HAS system model, the client priority and the playback rate are also incorporated to make the system model more accurate. With the feedback from servers, PREPARE algorithm is able to estimate the bandwidth much precisely and to reduce the unfairness among the multi-clients. The stability in bitrate selection is improved by introducing the buffer status in conjunction with the real-time playback rate. Moreover, the buffer underflow is prevented in download scheduling by advancing download according to the real-time playback rate. NS-3 simulator is used to evaluate the performance of PREPARE. The performance results show that PREPARE reduces the unfairness among multi-clients by more than 42% comparing to other three algorithms including ESTC, PANDA, FDASH. Meanwhile, PREPARE simultaneously keeps high stability and high efficiency.

FastScan: Robust Low-Complexity Rate Adaptation Algorithm for Video Streaming over HTTP

This paper proposes and evaluates a novel algorithm for streaming video over HTTP. The problem is formulated as a non-convex optimization problem which is constrained by the predicted available bandwidth, chunk deadlines, available video rates, and buffer occupancy. The objective is to optimize a QoE metric that maintains a tradeoff between maximizing the playback rate of every chunk and ensuring fairness among different chunks for the minimum re-buffering time. We propose FastScan, a low complexity algorithm that solves the problem. The online adaptations for dynamic bandwidth environments are proposed with imperfect available bandwidth prediction. The results of experiments driven by variable bit rate (VBR) encoded video, video platform system (dash.js), and cellular bandwidth traces of a public dataset reveal the robustness of the online version of FastScan algorithm and demonstrate its significant performance improvement as compared to the considered state-of-the-art video streaming algorithms. For example, on an experiment conducted over 100 real cellular available bandwidth traces of a public dataset that spans different available bandwidth regimes, our proposed algorithm (FastScan) achieves the minimum re-buffering (stall) time and the maximum average playback rate in every single trace as compared to Bola, Festive, BBA, RB, FastMPC, and Pensieve algorithms.

Voice Converter on Android

Voice Converter is an application developed for Android mobile platform that can help user to change his voice the way he likes. Existing voice changer mobile application provides the functions for users to change their voice into the voice of limited preset templates, this cannot see to the needs of film or animation producers, because the templates are limited and the uniqueness of the production cannot be secured. With the Voice Converter mobile application, film or animation producers can easily have access to a number of sound models that can be used to convert audio files which can be used in production later. Besides that, film makers can custom the sound of the conversion by inputting desired values of playback rate to change the audio sound in the way they would like to. The application uses speaker and microphone of the phones to play and record audio. For the recording process, the ‘MediaRecorder’ APIs is used to allow the device to capture audio, save the audio and play it back. The recordings captured can be converted into various sound models and also a model with custom parameters. The SoundPool Library is adopted to alter the playback rate of the audio. A mobile application named Voice Converter which is compatible for Android platforms is successfully developed. The functions of the application include recording and saving audio, converting input audio into existing template sound and converting recorded audio with custom parameter

Reducing Playback Rate of Audiovisual Speech Leads toaSurprising Decrease in the McGurk Effect.

We report the unexpected finding that slowing video playback decreases perception of the McGurk effect. This reduction is counter-intuitive because the illusion depends on visual speech influencing the perception of auditory speech, and slowing speech should increase the amount of visual information available to observers. We recorded perceptual data from 110 subjects viewing audiovisual syllables (either McGurk or congruent control stimuli) played back at one of three rates: the rate used by the talker during recording (the natural rate), a slow rate (50% of natural), or a fast rate (200% of natural). We replicated previous studies showing dramatic variability in McGurk susceptibility at the natural rate, ranging from 0-100% across subjects and from 26-76% across the eight McGurk stimuli tested. Relative to the natural rate, slowed playback reduced the frequency of McGurk responses by 11% (79% of subjects showed a reduction) and reduced congruent accuracy by 3% (25% of subjects showed a reduction). Fast playback rate had little effect on McGurk responses or congruent accuracy. To determine whether our results are consistent with Bayesian integration, we constructed a Bayes-optimal model that incorporated two assumptions: individuals combine auditory and visual information according to their reliability, and changing playback rate affects sensory reliability. The model reproduced both our findings of large individual differences and the playback rate effect. This work illustrates that surprises remain in the McGurk effect and that Bayesian integration provides a useful framework for understanding audiovisual speech perception.

Encoded Semantic Tree for Automatic User Profiling Applied to Personalized Video Summarization

We propose an innovative method of automatic video summary generation with personal adaptations. User interests are mined from their personal image collections. To reduce the semantic gap, we propose to extract visual representations based on a novel semantic tree (SeTree). A SeTree is a hierarchy that captures the conceptual relationships between the visual scenes in a codebook. This idea builds upon the observation that such semantic connections among the elements have been overlooked in the previous work. To construct the SeTree , we adopt a normalized graph cut clustering algorithm by conjunctively exploiting visual features, textual information, and social user-image connections. Using this technique, we obtain an 8.1% improvement of normalized discounted cumulative gain in personalized video segments ranking compared with existing methods. Furthermore, to promote the interesting parts of a video, we extract a space–time saliency map and estimate the attractiveness of segments by kernel fitting and matching. A linear function is utilized to combine the two factors, based on which the playback rate of a video is adapted to generate the summary. We play the less important segments in a fast-forward mode to keep users updated with the context. Subjective experiments were conducted which showed that our proposed video summarization approach outperformed the state-of-the-art techniques by 6.2%.

Optimized approach to cine MRI of uterine peristalsis.

To determine the optimal slice thickness, playback rate, and scan time for uterine peristalsis with 3.0T magnetic resonance imaging (MRI). In all, 23 young female volunteers underwent a 3.0T MRI scan with different slice thicknesses of 3 mm (Cine3mm ), 5 mm (Cine5mm ), and 7 mm (Cine7mm ) for 6 minutes. Subjective image quality score, signal-to-noise ratios (SNRs), and contrast-to-noise ratios (CNRs) of those MR images were evaluated by two radiologists independently. The number, intensity, and direction of uterine peristalsis with different thickness were compared at various playback rates. Also, the peristalsis frequency was counted and compared in different acquisition durations (1-6 minutes). The subjective image quality score, peristalsis number, and intensity were significantly higher in Cine7mm and Cine5mm than Cine3mm (P < 0.05), while the SNRs and CNRs of Cine7mm were significantly higher than Cine3mm (P < 0.05). Peristalsis numbers did not differ significantly at different playback rates with the same slice thickness (P = 0.548-0.962). However, peristalsis intensity at 12×, and 15× was significantly greater than that at 8× the actual speed for Cine7mm and Cine5mm (P < 0.05). The peristalsis frequency at 3, 4, 5, 6 minutes was significantly higher than that at 1 minute and 2 minutes (P < 0.05). We recommend a slice thickness of 5 mm or 7 mm and a scan time of 3 minutes for uterine peristalsis with 3.0T MRI, and a playback rate of 12× or 15× the actual speed for peristalsis observation. J. Magn. Reson. Imaging 2016;44:1397-1404.

Gender and rate effects on speech intelligibility

Older adults seeking hearing help often complain of particular difficulty understanding female voices. This contrasts with studies using young listeners with normal hearing in which female talkers have been found to be generally more intelligible than male talkers (e.g., Bradlow et al., 1996). Could some factor in addition to talker gender be causing older adults to have increased difficulty understanding female voices? Speech that has been time-compressed has been shown to be less intelligible than unprocessed speech (e.g., Gordon-Salant and Friedman, 2011), but few data exist to show whether an increased presentation rate causes an equal loss of intelligibility for male and female talkers. The present study will explore whether an increased playback rate has a greater negative effect on the intelligibility of speech produced by female versus male talkers. Subjects will listen to sentences produced by two female and two male speakers from the Utah Speaking Style Corpus presented at either their original rate or at an increased rate (1.5 times faster) and type out what they heard. The resulting data will show whether, for young normal-hearing listeners, an increased rate of speech affects the intelligibility of female talkers more than it affects the intelligibility of male talkers.

Scalable playback rate control in P2P live streaming systems

Current commercial live video streaming systems are based either on a typical client–server (cloud) or on a peer-to-peer (P2P) architecture. The former architecture is preferred for stability and QoS, provided that the system is not stretched beyond its bandwidth capacity, while the latter is scalable with small bandwidth and management cost. In this paper, we propose a P2P live streaming architecture in which by adapting dynamically the playback rate we guarantee that peers receive the stream even in cases where the total upload bandwidth changes very abruptly. In order to achieve this we develop a scalable mechanism that by probing only a small subset of peers monitors dynamically the total available bandwidth resources and a playback rate control mechanism that dynamically adapts playback rate to the aforementioned resources. We model analytically the relationship between the playback rate and the available bandwidth resources by using difference equations and in this way we are able to apply a control theoretical approach. We also quantify monitoring inaccuracies and dynamic bandwidth changes and we calculate dynamically, as a function of these, the maximum playback rate for which the proposed system able to guarantee the uninterrupted and complete distribution of the stream. Finally, we evaluate the control strategy and the theoretical model in a packet level simulator of a complete P2P live streaming system that we designed in OPNET Modeler. Our evaluation results show the uninterrupted and complete stream delivery (every peer receives more than 99 % of video blocks in every scenario) even in very adverse bandwidth changes.

Smart Streaming for Online Video Services

Bandwidth cost is a significant concern for online video service providers. Today's video streaming systems mostly use HTTP streaming, with users accessing video segments as HTTP requests. A frequently used strategy is to serve all user requests as fast as possible, as if the user is downloading a file. The downloading rate can often far exceed the playback rate, when the system is below the peak load. This is known as progressive downloading. Since users may quit before viewing the complete video, however, much of the downloaded video can be “wasted.” By studying and exploiting the predictability of users' departure behavior , the authors developed a smart streaming strategy that can significantly improve overall streaming service quality under given server bandwidth. The improvement is achieved by avoiding the waste based on predicted user departure behavior. The proposed smart streaming technique is evaluated by modeling, analysis, and simulation, as well as experimentation using a prototype implementation.

Flow-Level QoE of Video Streaming in Wireless Networks

The Quality of Experience (QoE) of streaming service is often degraded by frequent playback interruptions. To mitigate the interruptions, the media player prefetches streaming contents before starting playback, at a cost of delay. We study the QoE of streaming from the perspective of flow dynamics. First, a framework is developed for QoE when streaming users join the network randomly and leave after downloading completion. We compute the distribution of prefetching delay using partial differential equations (PDEs), and the probability generating function of playout buffer starvations using ordinary differential equations (ODEs) for CBR streaming. Second, we extend our framework to characterize the throughput variation caused by opportunistic scheduling at the base station, and the playback variation of VBR streaming. Our study reveals that the flow dynamics is the fundamental reason of playback starvation. The QoE of streaming service is dominated by the first moments such as the average throughput of opportunistic scheduling and the mean playback rate. While the variances of throughput and playback rate have very limited impact on starvation behavior.