Real-time Video Streaming Research Articles

Edge computing technologies for streaming video analytics

Real-time streaming video analytics is important in applications such as intelligent surveillance, smart city, and autonomous driving. However, large-scale streaming video analytics is impractical on the cloud, due to its high computation demand, large bandwidth consumption and stringent latency requirement.The emerging edge computing paradigm can effectively solve these problems by pushing computation from the cloud to devices and servers at the network edge.To this end, this article conducts a comprehensive survey on edge computing technologies for real-time streaming video analytics.Firstly, it introduces the background of video analytics and edge computing, as well as the typical application of edge-based steaming video analytics.Then, it proposes the performance indicators and challenges faced by the existing systems.Afterwards, the key technologies in this field are introduced in detail from the device level, collaboration level, and edge/cloud server level, including model compression and selection, local caching, frame filtering, task offloading, streaming protocol, privacy protection, query optimization, inference acceleration, and edge caching.Based on the integration of above core technologies, this article proposes an edge-based large-scale video analytics platform, Argus, which provides systematic support for the real-time video stream analytics on video collection, model inference, data mining, and log management.Argus has been successfully deployed in the smart oilfield scenario.Last but not least, this article discusses the open issues on edge-based streaming video analytics, in the hope of inspiring future research ideas.

Sliding-Window Forward Error Correction Based on Reference Order for Real-Time Video Streaming

In real-time video streaming, data packets are transported over the network from a transmitter to a receiver. The quality of the received video fluctuates as the network conditions change, and it can degrade substantially when there is considerable packet loss. Forward error correction (FEC) techniques can be used to recover lost packets by incorporating redundant data. Conventional FEC schemes do not work well when scalable video coding (SVC) is adopted. In this paper, we propose a novel FEC scheme that overcomes the drawbacks of these schemes by considering the reference picture structure of SVC and weighting the reference pictures more when FEC redundancy is applied. The experimental results show that the proposed FEC scheme outperforms conventional FEC schemes.

Deep Learning-Based Auricular Point Localization for Auriculotherapy

Auriculotherapy is one of the main forms of treatment in Traditional Chinese Medicine, whose potential as an alternative medicine for both health evaluation and disease treatment has been reported in many cases. However, its efficacy highly relies on the accurate localization of auricular points, which are not easy to be remembered due to their complexity. To explore an efficient way of locating auricular points, this study proposed a deep learning-based method of automatically locating auricular points from auricular images. A self-collected dataset named EID was created for TCM auriculotherapy research, with 91 auriculotherapy-related landmark points manually annotated according to the Chinese national standardization. A deep neural network structure was trained for landmark detection, and a direction normalization module was proposed to compensate for the detection error caused by the difference between the left and right ears. The trained model was validated on dataset EID. An average NME of 0.0514±0.0023 was achieved, which outperformed similar works. In addition, a certain auricular area corresponding to the digestive system was segmented based on the localized landmarks, and the results were tested in real-time video streaming. The proposed work for both auricular landmark and area identification can be widely used in auriculotherapy education and applications.

AN EFFICIENT ADAPTATION OF UHD VIDEO STREAMING OVER INTERNET

The main challenge of multimedia applications is how to transmission the Ultra-High Definition (UHD) video streaming in real time over the internet. The real time video streaming suffer from difficulties to be flexible and efficiency cause the wide variation of the available internet bandwidth. To avoid the problems that introduces with internet, in this work the HEVC with required video network adaptive streaming are proposed and tested using different six levels of three UHD video (4K, FHD, 720p, 4CIF, CIF and QCIF). From different experiments that applied find the optimal configuration of H.265 encoding features for six levels to obtain the required PSNR with range (32-38 dB). The important part in this project is a controller that worked incorporate with the encoder (H.265) to obtain the video streaming adaptation on the available bandwidth of the channel. The controller continuously reads the status of channel buffer, then choosing the proper level of video to be transmitted over the channel. The work architecture is content two parts: First, the H.265 codec that apply on the three raw videos with optimal parameters configuration to compress them and get videos with lower bit rate and acceptable quality. Second, the compressed videos, based to controller.....

Real‐time multi‐GPU‐based 8KVR stitching and streaming on 5G MEC/Cloud environments

In this study, we propose a multi-GPU-based 8KVR stitching system that operates in real time on both local and cloud machine environments. The proposed system first obtains multiple 4 K video inputs, decodes them, and generates a stitched 8KVR video stream in real time. The generated 8KVR video stream can be downloaded and rendered omnidirectionally in player apps on smartphones, tablets, and head-mounted displays. To speed up processing, we adopt group-of-pictures-based distributed decoding/encoding and buffering with the NV12 format, along with multi-GPU-based parallel processing. Furthermore, we develop several algorithms such as equirectangular projection-based color correction, real-time CG overlay, and object motion-based seam estimation and correction, to improve the stitching quality. From experiments in both local and cloud machine environments, we confirm the feasibility of the proposed 8KVR stitching system with stitching speed of up to 83.7 fps for six-channel and 62.7 fps for eight-channel inputs. In addition, in an 8KVR live streaming test on the 5G MEC/cloud, the proposed system achieves stable performances with 8 K@30 fps in both indoor and outdoor environments, even during motion.

Application of active queue management for real-time adaptive video streaming.

Video streaming currently dominates global Internet traffic. Live streaming broadcasts events in real-time, with very different characteristics compared to video-on-demand (VoD), being more sensitive to variations in delay, jitter, and packet loss. The use of adaptive streaming techniques over HTTP is massively deployed on the Internet, adapting the video quality to instantaneous condition of the network. Dynamic Adaptive Streaming over HTTP (DASH) is the most popular adaptive streaming technology. In DASH, the client probes the network quality and adjusts the quality of requested video segment according to the bandwidth fluctuations. Therefore, DASH is an over-the-top application using unmanaged networks to distribute content in the best possible quality. In order to maintain a seamless playback, VoD applications commonly use a large reception buffer. However, in live streaming, the use of large buffers is not allowed because of the induced delay. Active Queue Management (AQM) arises as an alternative to control the congestion in router’s queue, pressing the traffic sources to reduce their transmission rate when it detects incipient congestion. In this article, we evaluate the performance of recent AQM strategies for real-time adaptive video streaming. Furthermore, we propose a new AQM algorithm to improve the user-perceived video quality. The results show that the proposed method achieves better performance than competing AQM algorithms and improves the video quality in terms of average peak signal-to-noise ratio while keeping the fairness among concurrent flows.

Многофункциональная система безопасности угольных шахт – визуализация событий (горнотехнических процессов) с рабочего места шахтера

The paper reviews the use of mobile video monitoring equipment in coal mines. The most common option is the use of stationary video cameras with real-time video streaming to the mine dispatcher's control monitor via cables. Despite all the benefits of the information obtained, this method has certain limitations due to the specific features of the mine atmosphere, i.e. high humidity and dust levels, as well as the impossibility to organize video monitoring over the entire length of the mine workings. Therefore, mobile video monitoring equipment, both portable and vehicle-based, is efficient supplement to the stationary video cameras. The portable devices include smart phones and the battery-powered head lights with an integrated video camera, which have recently become very popular. In both cases, an important consideration, in addition to the actual video capturing, is the issue of transmitting video data to the top level, i.e. to the mine dispatcher's control panel. The following options are possible: connection to the mine wireless network hotspots via radio channel, reading the information in the lamp rooms when leaving the mine and real-time broadcasting from the mine to the top level. The assumption is made that in order to implement the fastest (and the most efficient) way that works without delays between capturing and transmitting of video data to the daylight surface, such as the latter of the options above, a communications infrastructure based on wireless and cable networks needs to be deployed in the mine workings. The required infrastructure is present in a number of systems designed to locate miners inside the mine workings as part of a multifunctional security system, which enables continuous radio communication of individual devices with infrastructure nodes and, therefore, real-time video data transmission.

Video Packet Distribution Scheme for Multimedia Streaming Services in VANETs.

By leveraging the development of mobile communication technologies and due to the increased capabilities of mobile devices, mobile multimedia services have gained prominence for supporting high-quality video streaming services. In vehicular ad-hoc networks (VANETs), high-quality video streaming services are focused on providing safety and infotainment applications to vehicles on the roads. Video streaming data require elastic and continuous video packet distributions to vehicles to present interactive real-time views of meaningful scenarios on the road. However, the high mobility of vehicles is one of the fundamental and important challenging issues for video streaming services in VANETs. Nevertheless, previous studies neither dealt with suitable data caching for supporting the mobility of vehicles nor provided appropriate seamless packet forwarding for ensuring the quality of service (QoS) and quality of experience (QoE) of real-time video streaming services. To address this problem, this paper proposes a video packet distribution scheme named Clone, which integrates vehicle-to-vehicle and vehicle-to-infrastructure communications to disseminate video packets for video streaming services in VANETs. First, an indicator called current network quality information (CNQI) is defined to measure the feature of data forwarding of each node to its neighbor nodes in terms of data delivery ratio and delay. Based on the CNQI value of each node and the trajectory of the destination vehicle, access points called clones are selected to cache video data packets from data sources. Subsequently, packet distribution optimization is conducted to determine the number of video packets to cache in each clone. Finally, data delivery synchronization is established to support seamless streaming data delivery from a clone to the destination vehicle. The experimental results show that the proposed scheme achieves high-quality video streaming services in terms of QoS and QoE compared with existing schemes.

Application Development for Mask Detection using Raspberry Pi

Abstract: In our daily life, humans are discovering new technology, and scientists also discovering new viruses. In recently scientists are discovered COVID-19 (coronavirus). COVID-19 pandemic has apace affected our everyday life-disrupting world trade and movements. Carrying a protecting mask has become a brand new tradition. Our planned technique detects the face from the video properly then it identifies if it's a mask wear on that or not. This project aims to develop a face mask detection system to detect any kind of face mask. The current study used OpenCV, Python, and Tensor Flow to detect whether a person wearing a face mask or not. The model was tested with real-time video streams. Keywords: Face mask detection, Face detection, mask detection, coronavirus, OpenCV, Tensorflow, Deeplearning

Drowsiness Detection for Motorized Vehicles Using Machine Learning

Abstract: Driver drowsiness is one of the major causes for most of the accidents in the world. Detecting the driver's eye tiredness is the easiest way for measuring the drowsiness of the driver. The advent of high-speed motorized vehicles drowsy driving accidents has claimed the lives of millions of people across the globe. To avoid such accidents, proposes a Machine Learning based system drowsiness system for motorized vehicles with alarm and Web Push Notifications to notify the driver before any accident occurs. The driver's face is captured by a real-time camera system, and the eye borders are detected by a pre-trained machine learning model from the real-time video stream. Then each eye is represented by 6 – coordinates (x, y) starting from the left corner of the eye and then working clockwise around the eye. The EAR (Ear Aspect Ratio) is calculated across 20 consecutive frames, and if it falls below a certain threshold, it sounds an alarm and sends the details of the nearest coffee shop to your mobile device via a Web Push Notification. When the alarm is activated, it also displays a list of nearby coffee shops to help the driver stay awake. Keywords: Machine Learning, SVM, MOR, EAR

Social Distancing for Covid-19 Monitoring System

The battle of corona-virus and mankind is possible to be tackled as long as we maintain the basic norm of social distancing and wearing masks amongst ourselves as it is through our droplets from the respiratory tract that the virus spreads. With the increasing demand for man-force and people requiring to go to their workplaces post lockdown, it is very necessary that we save each other from the virus. In this project, we will go through a detailed explanation of how we can use Python, AI and Deep Learning to monitor social distancing at public places and workplaces are keeping a safe distance from each other by analyzing real-time video streams from the camera and also detect facial mask monitoring using OpenCV and Python. To ensure if people are following social distancing protocols in public places and workplaces, we wanted to develop a tool that can monitor if people are keeping a safe distance from one another, wearing masks or not by processing real-time video footage from the camera. People at workplaces, factories, shops can integrate this tool into their security camera systems and can monitor whether people are keeping a safe distance from each other or not along with that we detect facial mask monitoring using Python with help of haar-cascade algorithm to see whether a person is wearing a mask or not. We are also planning to include thermal screening detection to measure the temperature of the subjects, a dashboard which will display a live report of corona cases around the world. We will also include an alert system that will send a notification to the authorities if the social distancing is not followed or if the temperature exceeds the threshold. The authorities can take suitable measures to isolate the subject and thus prevent the spread of Covid-19.

Real-time Camera and Video Streaming Through Optimized Settings of Ethernet AVB in Vehicle Network System

Real-time Camera and Video Streaming Through Optimized Settings of Ethernet AVB in Vehicle Network System

Application Layer ARQ Algorithm for Real-Time Multi-Source Data Streaming in UAV Networks.

Because of the specific characteristics of Unmanned Aerial Vehicle (UAV) networks and real-time applications, the trade-off between delay and reliability imposes problems for streaming video. Buffer management and drop packets policies play a critical role in the final quality of the video received by the end station. In this paper, we present a reactive buffer management algorithm, called Multi-Source Application Layer Automatic Repeat Request (MS-AL-ARQ), for a real-time non-interactive video streaming system installed on a standalone UAV network. This algorithm implements a selective-repeat ARQ model for a multi-source download scenario using a shared buffer for packet reordering, packet recovery, and measurement of Quality of Service (QoS) metrics (packet loss rate, delay and, delay jitter). The proposed algorithm MS-AL-ARQ will be injected on the application layer to alleviate packet loss due to wireless interference and collision while the destination node (base station) receives video data in real-time from different transmitters at the same time. Moreover, it will identify and detect packet loss events for each data flow and send Negative-Acknowledgments (NACKs) if packets were lost. Additionally, the one-way packet delay, jitter, and packet loss ratio will be calculated for each data flow to investigate the performances of the algorithm for different numbers of nodes under different network conditions. We show that the presented algorithm improves the QoS of the video data received under the worst network connection conditions. Furthermore, some congestion issues during deep analyses of the algorithm’s performances have been identified and explained.

Real‐time CVSA decals recognition system using deep convolutional neural network architectures

The Commercial Vehicle Safety Alliance (CVSA) aims to achieve uniformity, compatibility and reciprocity of commercial motor vehicle inspections and enforcement by certified inspectors dedicated to driver and vehicle safety. Commercial vehicles that pass a CVSA inspection are eligible for a decal representing a commitment to safety. In this work, we propose a two-step automatic CVSA decal recognition system using deep convolutional neural network architectures. The first step localizes a vehicle's windshield and the CVSA decal within, and classifies the decal colour. The CVSA decal is cropped and used as input to the second stage, which localizes and classifies a digit and the corner-cut of a CVSA decal. With the corner cut, colour, and digit, the system can determine the decal's date of issue. We use as our baseline the MobileDet architecture, customizing the backbone to our tasks. Our first custom architecture is larger than the baseline because it needs more representational power to detect small decals within an image. The second architecture is much smaller because digit and corner-cut recognition is a simpler task. Our custom architectures reduce processing time and exceed accuracies relative to pre-trained architectures. We implemented our models on different edge hardware accelerators (e.g. the Google Coral, Nvidia Jetsons, and Intel NCS) and compared the performance when processing a real-time video stream. Our system can predict frames at 173.31 FPS on an NVIDIA Jetson AGX Xavier with 98.5% mean average precision @ 0.5 IoU.

Improving Perceived Quality of Live Adaptative Video Streaming

Quality of service (QoS) requirements for live streaming are most required for video-on-demand (VoD), where they are more sensitive to variations in delay, jitter, and packet loss. Dynamic Adaptive Streaming over HTTP (DASH) is the most popular technology for live streaming and VoD, where it has been massively deployed on the Internet. DASH is an over-the-top application using unmanaged networks to distribute content with the best possible quality. Widely, it uses large reception buffers in order to keep a seamless playback for VoD applications. However, the use of large buffers in live streaming services is not allowed because of the induced delay. Hence, network congestion caused by insufficient queues could decrease the user-perceived video quality. Active Queue Management (AQM) arises as an alternative to control the congestion in a router’s queue, pressing the TCP traffic sources to reduce their transmission rate when it detects incipient congestion. As a consequence, the DASH client tends to decrease the quality of the streamed video. In this article, we evaluate the performance of recent AQM strategies for real-time adaptive video streaming and propose a new AQM algorithm using Long Short-Term Memory (LSTM) neural networks to improve the user-perceived video quality. The LSTM forecast the trend of queue delay to allow earlier packet discard in order to avoid the network congestion. The results show that the proposed method outperforms the competing AQM algorithms, mainly in scenarios where there are congested networks.

UAV-Based and WebRTC-Based Open Universal Framework to Monitor Urban and Industrial Areas.

Nowadays, we are observing a rapid development of UAV-based monitoring systems, which are faced with more and more new tasks, such as high temporal resolution and high spatial resolution of measurements, or Artificial Intelligence on board. This paper presents the open universal framework intended for fast prototyping or building a short series of specialized flying monitoring systems able to work in urban and industrial areas. The proposed framework combines mobility of UAV with IoT measurements and full-stack WebRTC communications. WebRTC offers simultaneous transmission of both a real-time video stream and the flow of data coming from sensors, and ensures a kind of protection of data flow, which leads to preserving its near-real-time character and enables contextual communication. Addition of the AI accelerator hardware makes this system AI-ready, i.e., the IoT communication hub, which is the air component of our system, is able to perform tasks of AI-supported computing. The exemplary prototype of this system was evaluated in terms of the ability to work with fast-response sensors, the ability to work with high temporal and high spatial resolutions, video information in poor visibility conditions and AI-readiness. Results show that prototypes based on the proposed framework are able to meet the challenges of monitoring systems in smart cities and industrial areas.

Augmented Reality Maintenance Assistant Using YOLOv5

Maintenance professionals and other technical staff regularly need to learn to identify new parts in car engines and other equipment. The present work proposes a model of a task assistant based on a deep learning neural network. A YOLOv5 network is used for recognizing some of the constituent parts of an automobile. A dataset of car engine images was created and eight car parts were marked in the images. Then, the neural network was trained to detect each part. The results show that YOLOv5s is able to successfully detect the parts in real time video streams, with high accuracy, thus being useful as an aid to train professionals learning to deal with new equipment using augmented reality. The architecture of an object recognition system using augmented reality glasses is also designed.

A Case Study on Real-time Live Video Streaming Content

With the development of new media, great changes are taking place in the way people get information. The change is the use of video content that can deliver content in a more three-dimensional way than words or photos. After 2016, the number of live video streaming content providers and users has increased. In this paper the write takes the 1 personal live video streaming content as the research object. And the write takes live video streaming content on YouTube live or Douyu TV as a research example. In this paper, the writer analyzes the digital information content in the live video streaming case. And the writer expounds the necessity of these visual information and the characteristics of real-time live video streaming content. Especially since 2020, because of the influence of the COVID-19, the live video streaming industry has begun to combine with the traditional industry. It is expected that the integration of digital cutting-edge technology and live video streaming will not only provide diversity in the content, but also create more social value for the video content consumption culture. Therefore, The writer thinks it is necessary to conduct in-depth research on the social responsibility of real-time live content in the future.

A Safety Measuring Tool to Maintain Social Distancing On COVID-19 Using Deep Learning Approach

Due to this recent wave of the Deadly corona virus disease, which has spread widely all over the world, so physical or Social distancing has become a mandatory precaution to avoid close contact. As of January 2021, the count of affected cases is 10.8 million and recovered cases are 10.5 million and the death rate is 155 thousand. In conformity, we have proposed a system that elucidates the usage of Python, computer vision, and deep learning to monitor whether people are maintaining Social distance at public areas in roads, pathways, work places and, even in hospitals. For the purpose of monitoring social distances among people, we have developed a social distancing detection tool that can keep track of safe distance and determine the distance between people in order to reduce the impact of the corona virus pandemic by analyzing real-time video Streaming, such as CCTV or Camera which can be added with the security camera system at hospitals, workplaces, markets, Jewelers shop etc. As an output, the detection tool will throw A Voice alert or a Notification, so that people would be remained to maintain a social distance.

Real-Time Video Streaming and Control of Cellular-Connected UAV System: Prototype and Performance Evaluation

Unmanned aerial vehicles (UAVs) play an increasingly important role in military, public, and civilian applications, where providing connectivity to UAVs is crucial for its real-time control, video streaming, and data collection. Considering that cellular networks offer wide area, high speed, and secure wireless connectivity, cellular-connected UAVs have been considered as an appealing solution to provide UAV connectivity with enhanced reliability, coverage, throughput, and security. Due to the nature of UAVs mobility, the throughput, reliability and End-to-End (E2E) delay of UAVs communication under various flight heights, video resolutions, and transmission frequencies remain unknown. To evaluate these parameters, we develop a cellular-connected UAV testbed based on the Long Term Evolution (LTE) network with its uplink video transmission and downlink control&command (CC) transmission. We also design algorithms for sending control signal and controlling UAV. The indoor experimental results provide fundamental insights for the cellular-connected UAV system design from the perspective of transmission frequency, adaptability, and link outage, respectively.