Video Streaming Research Articles

Vehicle Detection in Traffic Monitoring

Traffic Monitoring is a challenging task on crowded roads. Traffic Monitoring procedures are manual, expensive, time consuming and involve human operators. Large-scale storage and analysis of video streams were not possible due to limited availability. However, it is now possible to implement object detection and tracking, behavioral analysis of traffic patterns, number plate recognition and surveillance on video streams produced by traffic monitoring. In Big data, video datasets are so large that typical database systems are not able to store and analyses the datasets. Storing and Processing big volumes of data requires Scalability, Fault Tolerance and Availability. Thus, Big Data and Cloud computing are two compatible concepts as cloud enables big data for traffic monitoring. In this paper, we proposed vehicle detection for traffic monitoring. We have applied Support Vector Machine (SVM) machine learning algorithm for detecting vehicles.

Enabling lightweight immersive user interaction in smart buildings through learning-based mobile panorama streaming

Smart buildings integrate users’ digitized wearables with their physical surroundings, creating a seamless and interactive user experience. This is achieved through the utilization of multiple sensors, video streaming, artificial intelligence, and edge computing. These technologies gather extensive data and provide users with a wide range of applications, such as 3D audio/video in AR/VR, localization, virtual tours, and vigilant monitoring. Nevertheless, the current AR/VR devices face limitations due to the bulkiness and discomfort of the hardware used for on-body sensing, such as headsets and specialized glasses. These components often become uncomfortable during prolonged usage, posing a challenge for creating an immersive system that combines lightweight interaction with high-quality presentation.This paper presents a comprehensive system designed to enable immersive interaction in smart buildings with a focus on lightweight solutions. The system consists of the following components: (1). A lightweight panoramic imaging framework to address the challenges related to hardware size and functionality. (2). A learning-based video transcoding cost prediction framework for efficient load balancing. (3). A layered networking architecture designed to facilitate high-quality mobile panorama live streaming. Collectively, these components offer lightweight interaction paired with enhanced presentation quality. Our experimental results demonstrate the effectiveness of the system design, showcasing its seamless operation across different times, geographical locations, and heterogeneous wireless networks.

Intelligent Monitoring System for the Elderly based on Posture Recognition

In recent years, due to the gradual increase in the elderly population, elderly care has become an increasingly urgent issue, and the most common risk faced by the elderly is accidental falls. In this paper, an intelligent monitoring system for the elderly is studied, which includes a communication module, a sensor module and a state detection module. It can monitor the status of the elderly in real time and feed back to the server to activate the buzzer alarm function. In the communication module, WI-FI communication between esp32 microcontroller and PC is established to realize real-time transmission of pressure sensor data. The C/S communication method between Raspberry PI and PC was established to realize the transmission of real-time video data. In the sensor module, the pressure sensor is used for data acquisition, and the RF algorithm is used to realize the binary classification prediction of whether the human body is in bed or out of bed. In the state detection module, the yolov8 algorithm combined with the GSRF algorithm is used to classify the image data into five action categories, namely, falling, waving, sitting, standing and walking, to predict the state of the human body in the real-time video stream with an accuracy of 90%. After the sensor module detects the state of the human body out of bed, the state detection module is started, and the results are transmitted back to the back-end server and uploaded to the Web page to realize the function of real-time monitoring of the state of the elderly.

Web Real-Time Communication Using Gesture

Gesture recognition has emerged as a promising technology for enhancing human-computer interaction in various domains. In this paper, we propose a novel approach to gesture recognition using WebRTC (Web Real-Time Communication), a powerful web technology that enables real-time communication between web browsers. Our approach leverages the capabilities of WebRTC to capture live video streams from webcams and process them in real-time for gesture recognition. We present a detailed methodology for integrating WebRTC with a custom deep learning techniques. We evaluate the performance of our system using a diverse set of hand gestures and demonstrate its effectiveness in real-world scenarios. Our experimental results show an average recognition accuracy of 90%, outperforming existing methods in terms of both accuracy and computational efficiency. Potential applications of our approach in interactive web applications, gaming, and augmented reality systems. This work contributes the research on gesture recognition and demonstrates the feasibility of using WebRTC for real-time gesture recognition applications. Key Words: gesture recognition, WebRTC , Real-time communication

An empirical assessment of the use of an algorithm factory for video delivery operations.

Video service providers are moving from focusing on Quality of Service (QoS) to Quality of Experience (QoE) in their video networks since the users' demand for high-quality video content is continually growing. By focusing on QoE, video service providers can provide their subscribers with a more personalized and engaging experience, which can help increase viewer satisfaction and retention. This focus shift requires not only a more sophisticated approach to network management and new tools and technologies to measure and optimize QoE in their networks but also a novel approach to video delivery operations. This paper describes the components, interactions, and relationships of an algorithm factory for video delivery operation that assures high QoE for video streaming services. The paper also showcases the results of gradually implementing an algorithm factory in the video industry. Using a dataset from 2016 to 2022, we present the case of a European PayTV service provider that achieved improved performance measured by both objective and subjective metrics. The use of an algorithm factory significantly improved the PayTV service provider's performance. The study found a fivefold increase in the speed of critical incident resolution and a 59% reduction in the number of critical incidents, all while expanding the customer base and maintaining the same level of labor resources. The case also demonstrates a strong positive relation between the productivity measures of the PayTV operator and their survey-based quality ratings. These results underscore the importance of flawless QoS and operational excellence in delivering QoE to meet the evolving demands of viewers. The paper adds to the existing literature on relationships between operational efficiency, innovation, and subjective quality. The paper further offers empirical evidence from the PayTV industry. The insights provided are expected to benefit both traditional and over-the-top (OTT) video service providers in their quest to stay ahead in the rapidly evolving video industry. It may also translate to other service providers in similar industries committed to supporting high-quality service delivery.

Robust and efficient FISTA-based method for moving object detection under background movements

Moving object detection is a fundamental task in many video processing applications, such as video surveillance. The robustness and efficiency of background subtraction make it one of the most common methods for detecting moving objects from a video stream. However, adapting background models with moving cameras remains challenging. Major issues include maintaining the model given viewpoint alterations and compensating motions considering depth variations. Moreover, gradual illumination changes, dynamic backgrounds, and complex motions intensify over time in moving camera scenarios, further complicating background model maintenance. In this context, this paper proposes a novel Robust and Online Tensor-based model named ROTAB that incorporates a more implicit consideration of the relationship between sequential frames than the previous methods, allowing for better adaptation to background changes. Moreover, we propose an Improved version of FISTA named IFISTA that employs two strategies to reduce oscillatory behavior and minimize iterations, improving stability and efficiency. Practically, the combination of IFISTA and ROTAB (IFISTA-ROTAB) demonstrates suitable performance for real-time applications. Quantitative and qualitative experiments are conducted on three large-scale datasets, namely CDnet 2014, BMC 2012 and LASIESTA showing the superiority of IFISTA-ROTAB with a gain from two up to seven percent in average.

Beyond 100G three-dimensional flexible coherent PON with time, frequency, and power resource-allocation capabilities

Emerging services such as 8K/16K high-quality video streaming, virtual reality/augmented reality, beyond-5G mobile Internet, and edge computing are driving the need for even higher speed, capacity, and flexibility in optical access networks. Coherent passive optical networks (CPONs) have garnered significant attention in recent years due to their superior receiver sensitivity and high flexibility for 100G speeds and beyond. Research interest has surged in flexible, multi-dimensional multiplexing schemes that go beyond traditional time-division multiplexing (TDM). To further leverage the flexibility offered by CPONs, this paper proposes and experimentally demonstrates what we believe to be a novel three-dimensional flexible coherent PON (3D FLCS-CPON) for downstream applications. This system offers resource-allocation capabilities in the time, frequency, and power domains. By utilizing digital subcarrier multiplexing and power allocation in both the time and frequency domains, we achieve more flexible and adaptable access rates compared to traditional TDM and frequency-division multiplexing (FDM). We present in detail the operating principles of 3D FLCS-CPON in the downstream, along with steps for power calibration between subcarriers. As a proof of concept, we demonstrate the feasibility of 3D FLCS-CPON in the downstream using four subcarriers, achieving a peak data rate of 250Gb/s/λ over a 20-km fiber.

Digital Image and Video Processing: Algorithms and Applications

Many of the techniques that are used in digital image and video processing were developed in the 1960s at Bell Laboratories. These techniques have applications in a variety of fields, including medical imaging, videophone, character recognition, satellite imagery, and wire-photo standards conversion. Additional applications include enhancement of photographs or vidoes. The early stages of image and video processing were developed with the intention of enhancing the overall quality of the image or video. For the purpose of enhancing the visual effect of humans, it is intended for human beings. When it comes to image and video processing, the input is an image of poor quality, and the output is an image and video of higher quality. Research on algorithms and applications of digital image and video processing is the primary purpose of this study, which aims to investigate these topics extensively. The methodology employed in this study is qualitative research technique. In accordance with the findings of this research, "Image Processing" refers to the process of analyzing images with the objective of determining the significance of objects and identifying them. Image analysts analyze data that has been remotely sensed and attempt to detect, identify, classify, measure, and evaluate the significance of physical and cultural objects, as well as their patterns and spatial relationships. One subcategory of signal processing is known as video processing, and it is distinguished by the fact that the signals that are input and output are video files or video streams. Technology such as television sets, videocassette recorders (VCRs), DVD players, and other devices all make use of video processing algorithms. The processing of images and videos is extremely useful in a variety of contexts.

DLLBVS: Design of a High-Efficiency Deep Learning based Low-BER Video Streaming Model for High-Noise Wireless Networks

Design of video streaming models for high-noise networks is a multimodal process that requires efficient channel modelling, noise analysis, error-specific stream control, intelligent data augmentation, etc. This article proposes design of a high-efficiency deep learning based low-BER (Bit Error Rate) Video streaming model for high-noise wireless networks. The proposed model initially uses a Long-Short-Term Memory (LSTM) block for estimation of frame level features, and then uses Orthogonal Frequency Division Multiple Access (OFDMA) based modulation platform for transmission and reception of processed video frames. The OFDMA model is cascaded with a chaotic communication module, which assists in improving data fidelity under different noise conditions. The chaotic communication module is optimized using Grey Wolf Optimizer (GWO), which assists in setting up its hyperparameters for better efficiency under real-time communication scenarios. Video frames were pre-processed using Dual Neural Networks that assisted in estimation of differential frame information sets. Due to estimation of this differential frame information, streaming speed is improved, which assists in increasing number of frames transmitted per second, thereby improving streaming performance for different video types. This frame information is further processed by an iterative Gated Recurrent Unit (GRU) based VARMAx Model, which assists in predicting frame removal instances, thus assisting in faster & low error communications. The GRU VARMAx Model optimizes data flow, thus reducing congestion and improving throughput. The model was tested under AWGN (Additive While Gaussian Noise), Rayleigh, & Rician channel types, and its efficiency was compared with standard streaming techniques in terms of communication delay, Bit Error Rate, Peak to Average Power Ratio (PAPR), throughput, communication jitter and computational complexity levels. Based on this comparison it was observed that the proposed model showcased 3.5% lower communication delay, 8.3% lower BER, 5.9% lower PAPR, 10.5% higher throughput, 3.9% lower jitter and 6.4% lower computational complexity, which makes it highly useful for a wide variety of real-time streaming scenarios.

Artificial intelligence methods in information‑measuring and control systems

In modern agriculture, there is a need for efficient methods of monitoring and controlling the condition of agricultural objects. This requires the development of technologies capable of automatically recognizing and counting these objects based on data obtained from small unmanned aerial vehicles. Aim is to develop and implement a method for automatic recognition and counting of agricultural objects based on processing a stream of frames captured by a small unmanned aerial vehicle, in order to improve the efficiency of monitoring and managing agricultural resources. Results. A system has been developed that uses computer vision methods to analyze the video stream and demonstrates high accuracy in recognizing and counting haystacks in the captured frames. Experimental results confirm the effectiveness of the proposed approach. Practical Significance. The developed system has significant practical value for agriculture by automating the process of monitoring and controlling the condition of agricultural objects. This can lead to improved land resource utilization efficiency and reduced manual labor costs.

THE STUDY OF THE VIDEO TRANSCODING EFFICIENCY DEPENDING ON THE NUMBER OF DECODERS AND ENCODERS WORKER THREADS

In present study the efficiency of transcoding H.264 compressed video stream depending on the number of decoders and encoders worker threads was investigated. Tests were done using FFmpeg open source libraries based on Intel and ARM platforms. Both CPUs used were quad core and could run up to 4 threads. The effect of the number of decoders and encoder worker threads was studied upon the same input test video. The obtained results indicate that the greater the number of encoder threads produces the greater the fps value. At the same time, using the default encoder threads number provides better performance than the explicitly set value of 4 encoder threads.

A RESEARCH OF THE LATEST APPROACHES TO VISUAL IMAGE RECOGNITION AND CLASSIFICATION

Context. The paper provides an overview of current methods for recognizing and classifying visual images in static images or video stream. The paper will discuss various approaches, including machine learning, current problems of these methods and possible improvements. The biggest challenges of the visual image retrieval and classification task are discussed. The main emphasis is placed on the review of such promising algorithms as SSD, YOLO, R-CNN, an overview of the principles of these methods, network architectures.
 Objective. The aim of the work is to analyze existing studies and find the best algorithm for recognizing and classifying visual images for further activities.
 Method. Primary method is to compare different factors of algorithms in order to select the most perspective one. There are different marks to compare, like image processing speed, accuracy. There are a number of studies and publications that propose methods and algorithms for solving the problem of finding and classifying images in an image [3–6]. It should be noted that most promising approaches are based on machine learning methods. It is worth noting that the proposed methods have drawbacks due to the imperfect implementation of the Faster R-CNN, YOLO, SSD algorithms for working with streaming video. The impact of these drawbacks can be significantly reduced by applying the following solutions: development of combined identification methods, processing of edge cases – tracking the position of identified objects, using the difference between video frames, additional preliminary preparation of input data. Another major area for improvement is the optimization of methods to work with real-time video data, as most current methods focus on images.
 Results. As an outcome of the current research we have found an optimal algorithm for further researches and optimizations.
 Conclusions. Analysis of existent papers and researches has demonstrated the most promising algorithm for further optimizations and experiments. Also current approaches still have some space for further. The next step is to take the chosen algorithm and investigate possibilities to enhance it.

U.S. tweens’ reactions to unboxing videos: Effects of sponsorship disclosure and advertising training

ABSTRACT YouTube unboxing videos have become a popular genre among youth, and hosts often receive financial or other incentives for showcasing products. Although sponsored relationships must be explicitly disclosed per legal regulations, not all videos comply. This study assesses US tweens’ reactions to unboxing videos with varying cues of a sponsored relationship between the host and product manufacturer, using a 3 (sponsored; non-sponsored; sponsorship unaddressed cue) x 2 (advertising training; no advertising training) randomized experimental design. The sample consisted of 215 tweens between ages 8–13 years. Half of the participants were randomly assigned to watch an advertising literacy training video before the unboxing video. Tweens detect a higher selling intent within the sponsored video (relative to the non-sponsored video), only after viewing the advertising literacy training video. The training video enhances the positive relationship between perceived informative intent and purchase intention, while weakening the positive relationship between perceived selling intent and purchase intention. Findings suggest tweens may have a unique mental schema for unboxing-style online videos which is a hybrid of informational and advertisement. Brief training videos could be used on video streaming platforms to develop and cue tweens’ advertising literacy.

Adaptive Video Streaming in Multi-Tier Computing Networks: Joint Edge Transcoding and Client Enhancement

With the advancement of multimedia technology and wireless networks, there is a growing demand for high-quality video streaming. Delivering stable video streaming in extremely dynamic wireless networks, nevertheless, is still an open problem. Recent developments in client computing and mobile edge computing (MEC) technologies have both shown promise in enhancing the adaptive bitrate (ABR) streaming services. In this paper, we consider a video streaming system in multi-tier computing networks, enabled by joint edge-side video transcoding and client-side video enhancement. By “enhancement,” we mean that the client improves the video chunk quality via client-side image processing modules. In particular, we aim to design a joint bitrate adaptation, edge transcoding, and client image-processing algorithm, maximizing the quality of experience (QoE) of streaming services. The majority of the prior art has concentrated on super-resolution-enabled video streaming. Contrarily, we show that the video enhancement method outperforms the super-resolution approach in terms of signal-to-noise ratio and frames per second, implying a superior alternative for client-side processing in ABR streaming. We formulate the problem as an event-triggered Markov decision process (E-MDP), and propose a deep reinforcement learning (DRL)-based framework, named EDTEA. To deal with the delayed feedback induced by multi-tier computing, the entropy and the expected re-buffering terms are introduced to the objective and the reward, respectively. Extensive simulations based on real-world videos and bandwidth traces manifest that compared with state-of-the-art approaches, EDTEA provides <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$10.4\%\sim 78.4\%$</tex-math></inline-formula> extra QoE while reducing re-buffering time by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$85.5\%\sim 91.7\%$</tex-math></inline-formula> .

NUMBER PLATE DETECTION USING MACHINE LEARNING

This project seeks to develop a real-time, Python-based, license plate recognition system leveraging deep learning's efficiency and robustness. Employing custom- trained convolutional neural networks alongside targeted object detection algorithms, the system aims to accurately locate and extract license plate information from images and video streams. Extensive data preprocessing and comprehensive model training optimize performance under varying lighting, angles, and minor occlusions. Integration into a user-friendly application further enhances accessibility and practical utility. This project's contribution lies in its applicability to real-world scenarios like traffic management, law enforcement, and vehicle tracking, demonstrating the power of advanced computer vision and machine learning for practical problem-solving in the automotive domain.

Learning Audio and Video Bitrate Selection Strategies via Explicit Requirements

Mobile video streaming dominates today's network traffic, and adaptive bitrate (ABR) algorithms have been routinely adopted for transmitting media content across dynamic mobile networks. State-of-the-art ABR algorithms mainly alter video bitrate without considering audio bitrate as they consider the impact on the video negligible due to their small size. However, to bring users an immersive experience, recent content providers have applied high-quality audio with large sizes, like stereophonic sound. Therefore, improper audio bitrate selection will adversely affect video bitrate selection, leading to undesirable audio/video combinations (the highest video quality with the lowest audio quality, and vice versa) and frequent playback interruptions. To address these inefficiencies, we propose a Self-Play reinforcement learning-based Audio-aware ABR algorithm named SPA to learn strategies for audio and video bitrate selections. By learning from explicit goals, SPA can match the actual requirements and attain good performance. By conducting trace-driven and testbed-based experiments, we observe SPA's considerable superiority compared to existing approaches, including reducing the undesirable combinations by up to 34.17× and achieving zero stall time across 88.57% of traces. We also invite 35 volunteers to join a subjective test, and the result shows that 33/35 people consider SPA provides them with a satisfactory viewing experience.

Design and Control of Ultrasound Image-guided Prostatic Brachytherapy Robot

Background:: With the continuous development of medical robots, they have become a powerful tool for surgery and treatment. A prostate brachytherapy robot is a kind of robot widely used in prostate cancer treatment; its main function is to accurately implant radioactive seeds into prostate tissue to kill cancer cells. Using prostate brachytherapy robots for surgery can reduce damage to surrounding tissue during surgery, thereby reducing the risk of external electrolysis and metastasis of tumor cells. Objective:: This study aimed to design a prostate brachytherapy robot that is more in line with practical usage needs, based on the method of manually performing prostate brachytherapy surgery in clinical practice. Finally, its feasibility has been demonstrated through brachytherapy experiments. Methods:: Firstly, the actual requirements for prostate particle implantation surgery have been analyzed, which has been followed by reviewing previous literature and patents to determine the design scheme of the robot. Finally, SOLIDWORKS software has been used for mechanical design of the robot, and the control system of the robot has been designed based on Arduino. Results:: A robotic arm type prostate particle implantation robot has been successfully designed. We have installed a visual module on the prostate particle implantation robot, which has enabled the robot system to have image transmission monitoring function to assist in the calibration of the puncture position before surgery. Finally, we have employed other sensors to complete the particle implantation experiment, and successfully confirmed its superiority through comparing errors. Conclusion:: This work has analyzed and studied the clinical process of prostate particle implantation surgery, and designed a prostate particle implantation robot, including the mechanical design and control system design of the robot. We have also simulated the scene of prostate particle implantation surgery in the human body through particle implantation experiments, and successfully and accurately implanted radioactive particles to the target. The experimental results have proven the feasibility of the robot. We have made improvements in execution efficiency by using a parallelogram-based robotic arm as the representation of the prostate particle implantation robot and equipped it with a visual module. The visual module collects a video stream of the puncture needle's perspective and provides feedback to the control end to assist in the calibration of the puncture position before surgery, greatly reducing the probability of secondary injury to patients.

P‐9.3: A Typical Interconnect Model for High Bandwidth Video Streaming Transmission in Mixed‐Reality Devices

P‐9.3: A Typical Interconnect Model for High Bandwidth Video Streaming Transmission in Mixed‐Reality Devices

Interactive technology of high voltage tower acceptance UAV Based on AR and YOLOv7

To address many shortcomings of the current unmanned aerial vehicle (UAV) inspection mode of the transmission line high-voltage towers and improve the inspection efficiency, the UAV tower foreign matter detection technology based on Augmented Reality (AR) and You Only Look Once: Unified, Real-Time Object Detection (YOLOv7) has been developed. Firstly, the system architecture, including UAV, ground station, and AR glasses is established. Then, the AR scene is built by using the Unity engine, and the interactive control mode between AR glasses and UAV is designed. Subsequently, the mapping mode of head action and UAV command is devised to synchronize the UAV action and head action. Finally, the tower is detected for foreign objects by using YOLOv7 vision detection technology. The experimental results validate the efficacy of the built AR scene, demonstrating seamless functionality in completing the video streaming and attitude data transmission functions. The use of AR glasses can achieve interactive control of the UAV attitude. In addition, the YOLOv7 visual detection algorithm can accurately detect the attachment of foreign objects to the high-voltage tower.

Control of Video Buffering for Video Streams Decoded from Cyclic Structures

Control of Video Buffering for Video Streams Decoded from Cyclic Structures