Concurrent Video Streams Research Articles

Filtering Empty Video Frames for Efficient Real-Time Object Detection.

Deep learning models have significantly improved object detection, which is essential for visual sensing. However, their increasing complexity results in higher latency and resource consumption, making real-time object detection challenging. In order to address the challenge, we propose a new lightweight filtering method called L-filter to predict empty video frames that include no object of interest (e.g., vehicles) with high accuracy via hybrid time series analysis. L-filter drops those frames deemed empty and conducts object detection for nonempty frames only, significantly enhancing the frame processing rate and scalability of real-time object detection. Our evaluation demonstrates that L-filter improves the frame processing rate by 31-47% for a single traffic video stream compared to three standalone state-of-the-art object detection models without L-filter. Additionally, L-filter significantly enhances scalability; it can process up to six concurrent video streams in one commodity GPU, supporting over 57 fps per stream, by working alongside the fastest object detection model among the three models.

A Fast Filtering Mechanism to Improve Efficiency of Large-Scale Video Analytics

Surveillance cameras are ubiquitous around us. Emerging full-feature object-detection models can analyze surveillance videos with high accuracy but consume much computation. Directly applying these models for practical scenarios with large-scale cameras is prohibitively expensive. This, however, is wasteful and unnecessary considering that user-defined anomalies occur rarely among these videos. Therefore, we propose FFS-VA, a multi-stage Fast Filtering Mechanism for Video Analytics, to make video analytics much cost-effective. FFS-VA filters out the frames without the user-defined events by two stream-specialized filters and a cheap full-function model, to reduce the number of frames reaching the full-feature model. FFS-VA presents a global feedback-queue approach to balance the processing speeds of different filters in intra-stream and inter-stream processes. FFS-VA designs a dynamic batch technique to achieve a trade-off between throughput and latency. FFS-VA can also efficiently scale to multiple GPUs. We evaluate FFS-VA against the state-of-the-art YOLOv3 under the same hardware and video workloads. The experimental results show that under a 12.88 percent target-object occurrence rate on two GPUs, FFS-VA can support up to 30 concurrent video streams (15× more than YOLOv3) in the online case, and obtain 10× speedup when offline analyzing a stream, with an accuracy loss of less than 2 percent.

Using TensorRT for deep learning and inference applications

TensorRT是一个高性能的深度学习推理平台。它包括一个深度学习推理优化器和运行时为深度学习推理应用程序提供低延迟和高吞吐量。给出了一个使用TensorRT快速构建计算管道的例子,实现通过TensorRT执行智能视频分析的典型应用。该示例演示了使用片上解码器进行解码、使用片上标量进行视频缩放和GPU计算的4个并发视频流。为了演示的简单性，只有一个通道使用NVIDIA TensorRT执行对象标识，并在标识的对象周围生成包围框。该示例还使用视频转换器函数进行各种格式转换，使用EGLImage来演示缓冲区共享和图像显示。最后采用GPU卡V100对ResNet网络进行TensorRT加速性能的实际测试,结果表明TensorRT能够使吞吐量提升大约15倍。

A general framework for real-time analysis of massive multimedia streams

Big Data platforms provide opportunities for the management and analysis of large quantities of information, but the services they provide are often too raw, since they focus on issues of fault-tolerance, increased parallelism, and so on. An additional software layer is, therefore, needed to effectively use such architectures for advanced applications in several important real-world domains, such as scientific and health care sensors, user-generated data, supply chain systems and financial companies, to name a few. In this paper, we present RAM $$^3$$ S, a framework for the real-time analysis of massive multimedia streams, where data come from multiple data sources (such as sensors and cameras) that are widely located on the territory, with the final goal to discovery new and hidden information from the output of data sources as they occur, thus with very limited latency. We apply RAM $$^3$$ S to the use case of automatic detection of “suspect” people from several concurrent video streams, and instantiate it on top of three different open source engines for the analysis of streaming Big Data (i.e., Apache Spark, Apache Storm, and Apache Flink). The effectiveness and scalability of RAM $$^3$$ S instantiation is experimentally evaluated on real data, also comparing the performance of the three considered Big Data platforms. Such comparison is performed both on a cluster of physical machines in our datalab and on the Google Cloud Platform.

Forecasting of access network bandwidth demands for aggregated subscribers using Monte Carlo methods

In the face of relentless growth in traffic, network operators must continuously forecast bandwidth demand to properly dimension their networks and make the correct investments for the future. The consequences of under-investing are poor network performance and dissatisfied subscribers, while over-investing ties up capital that could have been better spent elsewhere. In this article we describe a model that forecasts bandwidth demands of aggregated subscribers on residential fixed access networks. For sustained bandwidth demand, we use statistical techniques to quantify the number of concurrent video streams, the shifting mix of standard definition, high definition, and ultra high definition resolutions, multicast gain, and the trend from multicast to unicast delivery of these streams. Mechanisms to cope with bursty bandwidth demand are included. The results we report can guide operators in making technology and network design choices for future FTTx deployments.

Analysis and design of peer‐assisted video‐on‐demand services

AbstractWe propose a deterministic fluid model to understand the trade‐offs in the design of peer‐assisted video‐on‐demand (VoD) services. There are three entities in this model: peers (or end users), seeders (altruistic users that own one or several complete video items), and cache servers that store and forward videos with a limited capacity. Peers join the network, download one or multiple concurrent video streams (possibly different video items), and abort the system when they wish. Peers are assumed to cooperate in a BitTorrent‐based fashion, governed by tit‐for‐tat and fair availability. The issue is to minimize the expected downloading times, choosing the set of video items that should be stored in each cache server. We first prove that the deterministic model is globally stable, and find closed expressions for the expected waiting times. Then, we introduce a combinatorial optimization problem (COP), whose nature is similar to the multiknapsack problem (where the items are videos, and knapsacks are related with the storage of cache servers). The problem turns out to be NP‐complete. Therefore, we heuristically address the problem following a GRASP (greedy randomized adaptive search procedure) methodology. Finally, we simulate the new caching methodology with real‐life traces taken from YouTube logs. The results suggest that the peer‐to‐peer philosophy is both stable and cost‐effective for on‐demand streaming purposes.

In-network adaptation of scalable video content

This thesis investigates mechanisms and applications for in-network adaptation of scalable video bit streams based on the recent H.264/Scalable Video Coding (SVC) standard. In-network adaptation refers to the adaptation of a video stream by a network element during the stream's transport through the network. The advantages of performing adaptation directly in the network are the availability of local monitoring data and a higher responsiveness according to the current networking conditions. In contrast to previous work in this field, this thesis focuses on the feasibility and realization of in-network adaptation on existing home router platforms. In this context this thesis addresses the following six research objectives. Initially, the relevant transport mechanisms for H.264/SVC and their implications on in-network adaptation (1) were analysed. In the context of this work three different Linux-based router platforms which cover a representative range of residential router devices were used as a basis for further studies and evaluations. In general these platforms can be characterized by rather modest processing capabilities and networking performance. The hardware limitations were identified and quantified in evaluations (2) using both different benchmarks and real network traffic. The offered processing power and memory throughput are roughly 10 to 100 times lower than those of a modern desktop PC. Although their application-layer networking performance is not that low, all platforms fail in fully utilizing their nominal link capacities of 100 and 1000 Mbps, respectively. Based on the known limitations the thesis proposes a stateful, packet-based adaptation mechanism for adapting scalable video bit streams (3). The approach utilizes the RTP payload format for H.264/SVC and represents a light-weight approach for in-network adaptation on the application layer. It further meets the important requirements towards a media-aware network element (MANE) to be signaling aware and to operate statefully. The mechanism was integrated in a proxy service which was deployed on all of the three platforms to prove its feasibility. Experimental evaluations with different video bit streams in standarddefinition quality demonstrate the scalability of the approach (4). The results indicate that the proxy service is able to adapt up to 16 concurrent video streams depending on the platform and video bit stream. On two of the three evaluated platforms the proposed approach even allows to handle and to adapt video streams in high-definition quality at bit rates around 15 Mbps. In addition to the proposed H.264/SVC-specific adaptation mechanism, also the applicability of generic metadata-driven adaptation on home router platforms was investigated. In particular, a proof-of-concept study of an XML-metadata-driven approach based on the MPEG-21 generic Bitstream Syntax Description (gBSD) was conducted on the platforms (5). In contrast to former evaluations that have been done on PC-based platforms, the obtained results indicate that the use of this generic adaptation cannot be recommended on such resource limited network devices. The benefits of using in-network adaptation on home router platforms are finally demonstrated in the context of high-definition streaming over IEEE 802.11 wireless networks (6). Monitoring information regarding the queueing delay, which is obviously available exclusively on the router, is used to control the adaptation of the video according to the varying throughput of the wireless link. This allows to react timely to changing conditions particularly in the case of mobile clients.

A FC/AL-Based P2P Network for Personal Archive and Sharing

Peer-to-Peer (P2P) networking technology can share a variety of contents, including the broadcast TV/Film programs. With the popularity of personal video recorders, people stores their favorite TV/film titles in the hard drive. It is clear that these recorders can easily archive and share TV/film contents over P2P networks once they are equipped with broadband connections. This is important because this emerging P2P network can provide an alternative content archive and distribution channel to the conventional unidirectional broadcasting. In other words, people can have another way to obtain TV contents that they were not able to record at the time of broadcast or purchase via the conventional DVD sale. As part of on-going research, we demonstrate that Fibre Channel -Arbitration Loop (FC-AL) technology can serve well as the storage network for personal archive and real-time content distribution. In our proposed architecture for TV content archive and sharing, the FC-AL directly connects many digital video recorders, a pool of network disks, and a content managing server.Via our integrated storage/network architecture, users can share their archives either from their own digital video recorders or the pool of networked disks. The results are the significant time-shifting for the users to watch the programs. The proposed network architecture also relieves the storage limitation of the digital video recorders, and users can help each other by archiving the “hard-to-find’ contents. In this paper, we present the technical schemes to fully utilize the performance of this emerging network. These schemes include (1) Buffering schemes to guarantee the concurrent video streams; (2) Matching schemes for communication pairs; (3) Load-balancing schemes; and (4) multiple-loop staging schemes to extend the time-shift. With the effective buffering schemes, we have demonstrated over 300 high-quality video streams can be supported by the system with less than 1display jitters.To maximally utilize the FC-AL bandwidth and full-duplex property in both directions, we have developed effective schemes to match the communication pairs between users and the pool of network disks. It is also a common situation that most playback requests are concentrated on a few popular programs. From the perspective of overall system performance, such a skewed access pattern may cause serious load-unbalancing problems because some overloaded PVRs can become performance bottlenecks even though there are still many disks having sufficient available band-width. To solve this problem, we switch playback requests as well as replicate popular program (as minimally as possible). Finally, efficient staging methods have been proposed between the networked disks in different FC-AL loops to successfully extend the time-shift from 3-day to 70 days.

Comparison of in-network versus Staggered Multicast video distribution models

This paper proposes a new video distribution service: Video-In-Network (VIN). In VIN, videos are continuously circulating in an optical network where they can be easily retrieved by VIN Nodes. A mathematical model of the VIN system was derived and used to explore the performance of VIN in terms of the maximum number of concurrent video streams which can be supported by the system. We then compared VIN with Staggered Multicast. We found that VIN has a number of advantages over Staggered Multicast. First, VIN is more scalable in terms of the number of streams/channels. Second, the startup latency of VIN is shorter than that of Staggered Multicast. Third, Staggered Multicast is a near-VOD system and the video clients usually have no specific channels to the server to request for video. In the VIN system, the video clients can request the VIN Node for particular videos for multicast or unicast on demand.

Video-on-demand server: Strategies for improving performance

Video-on-demand (VoD) is a service that has received much attention recently and the concept has become popular with computer and communication companies. Basically, VoD consists of a video server, interconnection network and user display equipment (UDE). It provides a similar service to the neighbourhood rental store. Videos or movies are distributed from the video server to users upon request through the network and users can view interactively using their UDE along with VCR-like functions (e.g. forward, rewind, pause and resume). VoD offers a convenient service since users do not have to leave their homes, which also leads to shorter response times than with today’s video stores. One important aspect of any VoD is the storage scheme used, which affects the amount and type of storage required, the maximum number of concurrent video streams and the average waiting time experienced by users of the service from the moment they choose a video to the moment video playback starts. The paper reviews some well known storage schemes, such as the phase based method and the phase based with replication method, and then proposes two new schemes that further increase the number of concurrent streams and reduce the waiting time, while reducing the total storage space required. These new schemes rely on two concepts called immediate playback RAM and sustainable playback storage. Using a case study, the new schemes are briefly compared with an existing scheme and their strengths and shortcomings are highlighted. Using some typical values, analyses are carried out that quantify the performance gain resulting from the new storage schemes.

Video-server designs for supporting very large numbers of concurrent users

We present a design for a video server in which video content is stored in special video-delivery subsystems attached directly to network components such as switches or high-speed-network ports rather than the magnetic disk storage attached to conventional computer systems. Video is preformatted and stored in the form of network packets. This design approach overcomes the CPU- and I/O-bandwidth limitations of conventional computers in executing the file-system and network-protocol code for many concurrent video streams, resulting in higher performance at a lower cost. Two designs for the approach are discussed. The first extends the packet buffer of a shared-buffer switch with additional memory for storing the video packets. The second design uses a stream controller as the interface between an array of disks and a traditional switch or network port. We have built a prototype based on the second design. To avoid interference on the disks, data is interleaved across all disks connected to a stream controller in units of fixed playback time. This also reduces the jitter in the response time of the disks and, therefore, the size of the buffers needed to maintain interruption-free delivery. The cost benefits of both approaches are discussed.

Building video-on-demand servers

We have designed and implemented a controllable software architecture for a Video?on?Demand (VOD) server. With the proposed software architecture, many system design issues can be investigated. For example, we studied several disk striping schemes in the storage subsystem and examined the impact of the disk striping schemes on the utilization of system resources. In the processing component, we observed that additional concurrent video streams can be supported by using efficient memory interleaving. Buffering with a large memory space in the processing subsystem is a common technique to alleviate the latency variance of accessing different system components. By employing user?level control and scheduling, the variance can be decreased thereby reducing the resulting buffer space needed for each video stream. In the network subsystem, we adopted a server?driven approach for investigating MPEG?2 video delivery over Asynchronous Transfer Mode (ATM) networks. The VOD server controls the pace of video transmission and reduces the complexity of the client. Since the client has limited buffer space (due to cost considerations), we have reduced the buffer requirement by regulating the transmission based on timing information embedded in the MPEG?2 streams. Our research and experimental results are based on a VOD server which is currently under construction. The prototype server is based on an SGI shared?memory multiprocessor with a mass storage system consisting of RAID?3 disk arrays. Using 30 RAID?3 disk arrays, preliminary experimental results show that the prototype server can potentially support more than 360 high?quality video streams with careful design and coordination of the different system components.

Design and Implementation of the Stony Brook Video Server

The system architecture of the Stony Brook Video Server (SBVS), which guarantees end-to-end real-time video playback in a client-server setting, is presented. SBVS employs a real-time network access protocol, RETHER, to use existing Ethernet hardware as the underlying communications media. The video server tightly integrates the bandwidth guarantee mechanisms for network transport and disk I/O. SBVS's stream-by-stream disk scheduling scheme optimizes the effective disk bandwidth without incurring significant scheduling overhead. To demonstrate the feasibility of the proposed architecture, we have implemented a prototype called SBVS-1, which can support five concurrent MPEG-1 video streams on an Intel 486DX2/EISA PC. To our knowledge, this system is the first video server that provides an end-to-end performance guarantee from the server's disks to the each user's display over standard Ethernet. This paper describes the implementation details of integrating network and I/O bandwidth guarantee mechanisms, and the performance measurements that drive and/or validate our design decisions. © 1997 by John Wiley & Sons, Ltd.