Video Streaming Research Articles

Toward an interface affordance model for online streaming video platforms

Toward an interface affordance model for online streaming video platforms

INTELLIGENT VIDEO ANALYSIS TECHNOLOGY FOR AUTOMATIC FIRE CONTROL TARGET RECOGNITION BASED ON MACHINE LEARNING

Context. Target recognition is a priority in military affairs. This task is complicated by the fact that it is necessary to recognize moving objects, different terrain and landscape create obstacles for recognition. Combat actions can take place at different times of the day, accordingly, it is necessary to take into account the perspective of lighting and general lighting. It is necessary to detect the object in the video by segmenting the video frames, recognize and classify. Objective of the study is to develop a technology for the analysis of the development of a technology for recognizing targets in real time as a component of the fire control system, due to the use of artificial intelligence, YOLO and machine learning. Method. The article develops a video stream analysis technology for automatic target recognition of the fire control system based on machine learning. The paper proposes the development of a target recognition module as a component of the fire control system within the framework of the proposed information technology using artificial intelligence. The YOLOv8 pattern recognition model family was used to develop the target recognition module. The methods used during the study of the formed dataset. – Bounding Box: Noise – Up to 15% of pixels (limiting frame: adding salt and pepper noise to the image – up to 15% of pixels). – Bounding Box: Blur – Up to 2.5px (bounding box: adding Gaussian blur to the image – up to 2.5 pixels). – Cutout – 3 boxes with 10% size each (cut out a part of the image – 3 boxes of 10% size each). – Brightness Between –25% and +25% (changing the brightness of the image to increase the resistance of the model to changes in lighting and camera settings – from –25% to +25%). – Rotation – Between –15 and +15 (rotation of the image object – clockwise or counterclockwise by degrees from –15 to +15). – Flip – Horizontal (flip the image object horizontally). Results. The data is collected from open sources, in particular, from videos posted in open sources on the YouTube platform. The main task of data preprocessing is the classification of three classes of objects on video or in real time – APC, BMP and TANK. The dataset is formed using the Roboflow platform based on the labeling tools and subsequently the augmentation tools. The dataset consists of 1193 unique images – approximately equally for each class. The training was conducted using Google Colab resources. It took 100 epochs to train the model. Conclusions. Analysis is performed according to mAP50 (average precision as 0.85), mAP50-95 (0.6), precision (0.89) and recall (0.75). Big losses are due to the fact that the background was not taken into account during the research – training the module on the basis of confirmed data (images) of the background without technology

Spectrogram based Wi-Fi usage activity classification using deep learning

Abstract The article presents and characterizes the ability of a neural network to distinguish between different Wi-Fi activities based on their distinct spectral features. To achieve this objective, we have built a database containing spectrogram images retrieved by over-the-air measurements of signals emitted in 802.11ax Wi-Fi networks deployed in both the 2.4 GHz and 5 GHz frequency bands. The dataset consists of spectrograms labeled by six distinct user activities, namely file download, speed test, video streaming, file upload, video call, and voice over IP call (VoIP). The network training parameters such as learning rate and validation frequency were optimized to enhance model performance. The influence of different activation functions, ReLU, leaky ReLU, eluLayer, and swishLayer from Matlab was also evaluated to achieve neural network fine-tunning. Results demonstrate the effectiveness of the proposed neural network architecture in accurately classifying Wi-Fi usage patterns, achieving test accuracies above 98%.

Neural network-based navigation filter for monocular pose and motion tracking of noncooperative spacecraft

This paper presents a neural network-based multiplicative extended kalman filtering (MEKF) to track the pose (attitude and position) and 6D motion (angular velocity and linear velocity) of a known tumbling spacecraft using monocular camera for close-proximity operations. The proposed MEKF estimates pose and 6D motion of the target spacecraft relative to servicing spacecraft by combining dynamic and kinematic information with pseudo-measurement information provided by convolutional neural network (CNN). Specifically, a modified EfficientNet-B3 CNN architecture is constructed for recovering pose directly, while the 2D and 6D motion mapping equation is derived to recover 6D motion from monocular video stream. The former is recovered by using the CNN to learn the implicit mapping relationship between image and pose. The latter is recovered using motion mapping equation which combines pose pseudo-measurements with information of two consecutive image frames. The advantage of the proposed method is that it can provide full relative state pseudo-measurements including pose and 6D motion to the filter merely using a single monocular camera as sensor. Subsequently, the Failed Spacecraft Close-range Observation Dataset (FSCOD) is constructed to analyze the performance of the proposed method. The FSCOD consists of training and validation images independent of dynamic trajectories and test images induced by dynamic trajectories of tumbling spacecraft, both of which are generated by Unreal Engine 4 (UE4). Numerical simulations are conducted utilizing sequential images to evaluate the performance of the proposed MEKF and the modified EfficientNet-B3. The results show that the estimation accuracy of the modified EfficientNet-B3 is approximate to that of the UrsoNet, whereas its network parameters are only half of the UrsoNet. Moreover, the proposed MEKF has higher steady-state pose estimation accuracy than the UrsoNet baseline method and is robust to initial estimation errors and measurement noise.

Adaptive Video Streaming in Integrated Satellite-Terrestrial Networks: A Low-Complexity Model Predictive Control Approach

Adaptive Video Streaming in Integrated Satellite-Terrestrial Networks: A Low-Complexity Model Predictive Control Approach

Personalization at Scale: Big Data Technologies in Digital Content Delivery and Their Effect on Viewer Experience

This article examines the transformative impact of big data technologies on the digital content delivery industry, focusing on their application in enhancing customer experience through video streaming services. Through a comprehensive case study of a leading provider, we investigate the implementation of Hadoop and AWS Big Data product suites to optimize content delivery, personalize user recommendations, and enable dynamic ad insertion. Our findings demonstrate significant improvements in service reliability, viewer engagement, and advertising effectiveness. The article reveals how Big Data analytics facilitate real-time content optimization and personalized user experiences, increasing viewer retention and satisfaction. Furthermore, we explore the challenges and limitations encountered during implementation, providing insights into the practical applications of Big Data in the evolving landscape of digital media consumption. This article contributes to the growing body of literature on Big Data applications in media and entertainment, offering valuable insights for practitioners and researchers in digital content delivery.

Übersichtsarbeiten / Review Articles. Neue Medien und Sucht im Kindes- und Jugendalter / New Media and Addiction in Youth

New Media and Addiction in Youth New media, such as digital games, social media, and video streaming services, have emerged as fundamental channels for socialization and leisure activities among young people. Platforms such as TikTok, Fortnite, and YouTube do not only provide entertainment, enjoyment, and opportunities for social connection, but also carry the risk of excessive use. Particularly vulnerable youth, who turn to media to escape real-world problems and conflicts, can quickly become trapped in a vicious cycle, resulting in problematic usage habits with significant psychosocial consequences. This review presents the current state of research on Digital Media Use Disorders (DMUD) in youth. It outlines diagnostic criteria, presents recent data on epidemiology, etiology, and treatment approaches, and discusses implications for prevention. DMUD in youth remains a serious health issue in the aftermath of the pandemic. Parents and schools play a crucial role in guiding children and adolescents towards media literacy and self-regulated use of digital media. But also game and platform providers, who often employ manipulative mechanisms that prevent a self-regulated use, need to be held accountable. The introduction of the European Digital Services Act represents a crucial step in this direction.

Разработка и применение алгоритмов фильтрации и обработки видеопотока на ПЛИС

The paper is devoted to the details of implementation of machine vision algorithms on FPGA. The algorithms were implemented using the Vitis HLS high-level synthesis tool. The main parameter in the realization of the algorithms is speed; since it is necessary to proceed processing of the incoming video stream with minimum delay.

Leveraging openCV and image processing for crime identification through facial recognition

To maintain the safety and security of communities, law enforcement agencies provide vital functions such as crime prevention and identification. Identifying people with a criminal record efficiently and accurately is one of the biggest hurdles in this field. This study delves into a fresh strategy for taking on this problem by making use of state-of-the-art image processing methods and machine learning algorithms. The project's main goal is to identify convicted felons by utilizing OpenCV, a robust open-source computer vision toolkit, and different facial recognition algorithms. By giving them a tool that guarantees a faster reaction rate and improved accuracy in suspect identification, the system is aimed to boost the capacities of law enforcement organizations. This project is based on the tenet of criminal psychology, which states that criminals are prone to committing new crimes. The system's goal is to detect and track repeat criminals both before and after they commit crimes by making use of this information. The method works by taking pictures of people who are known to be criminals and putting them in a database. From there, they can be compared to either live video streams or static photographs taken by surveillance cameras. In order to analyze and identify facial characteristics effectively, algorithms like LBPH and the Haar cascade classifier are also used. The system's robustness and efficiency are guaranteed by integrating these approaches into the OpenCV framework. This research makes a valuable contribution to the field of crime prevention by offering law enforcement authorities a dependable and scalable alternative. This system provides a streamlined interface for storing, retrieving, and matching facial photos. It was built using Python 3.5 and uses SQLite to manage the database of offenders' details. Proactively monitoring persons with a criminal record is made easier and faster with the suggested method. Suspect identification is also made easier. This research showcases the possibility of integrating machine learning and image processing to enhance public safety and crime prevention initiatives. It is an early step towards more complete video surveillance systems.

COBIRAS: Offering a Continuous Bit Rate Slide to Maximize DASH Streaming Bandwidth Utilization

Reaching close-to-optimal bandwidth utilization in dynamic adaptive streaming over HTTP (DASH) systems can, in theory, be achieved with a small discrete set of bit rate representations. This includes typical bit rate ladders used in state-of-the-art DASH systems. In practice, however, we demonstrate that bandwidth utilization, and consequently the quality of experience (QoE), can be improved by offering a continuous set of bit rate representations, i.e., a continuous bit rate slide (COBIRAS). Moreover, we find that the buffer fill behavior of different standard adaptive bit rate (ABR) algorithms is sub-optimal in terms of bandwidth utilization. To overcome this issue, we leverage COBIRAS’ flexibility to request segments with any arbitrary bit rate and propose a novel ABR algorithm MinOff , which helps maximizing bandwidth utilization by minimizing download off-phases during streaming. To avoid extensive storage requirements with COBIRAS and to demonstrate the feasibility of our approach, we design and implement a proof-of-concept DASH system for video streaming that relies on just-in-time encoding ( JITE ), which reduces storage consumption on the DASH server. Finally, we conduct a performance evaluation on our testbed and compare a state-of-the-art DASH system with few bit rate representations and our JITE DASH system, which can offer a COBIRAS, in terms of bandwidth utilization and video QoE for different ABR algorithms.

Integration of terrestrial radio and visual radio: A case study of m radio 98.8 Surabaya

In the process of developing transmission technology and internet applications, radio broadcasting institutions are constantly evolving. In an effort to improve management performance, any changes in radio broadcasting technology are always adaptable. This study utilises a qualitative approach that is still rarely used to review the integration of terrestrial and visual radio. Data collection techniques were used through field observations, interviews, and documentation studies. After data collection was completed, data analysis was conducted systematically in three aspects: data reduction, data display, and conclusion drawing. In contrast to the transmission of radio broadcasts over the air or voice broadcasts, this study found that radio broadcasts can also be distributed through audio and video streaming on the internet (also known as the network) in the form of voice, text, images, and video.

User-Perceived Capacity: Theory, Computation, and Achievable Policies.

User-perceived throughput is a novel performance metric attracting a considerable amount of recent attention because it characterizes the quality of the experience in mobile multimedia services. For instance, it gives a data rate of video streaming with which a user will not experience any lag or outage in watching video clips. However, its performance limit remains open. In this paper, we are interested in the achievable upper bound of user-perceived throughput, also referred to as the user-perceived capacity, and how to achieve it in typical wireless channels. We find that the user-perceived capacity is quite limited or even zero with channel state information at the receiver (CSIR) only. When both CSIR and channel state information at the transmitter (CSIT) are available, the user-perceived throughput can be substantially improved by power or even rate adaptation. A constrained Markov decision process (CMDP)-based approach is conceived to compute the user-perceived capacity with joint power-rate adaptation. It is rigorously shown that the optimal policy obeys a threshold-based rule with time, backlog, and channel gain thresholds. With power adaptation only, the user-perceived capacity is equal to the hard-delay-constrained capacity in our previous work and achieved by joint diversity and channel inversion.

The Evolution of Volumetric Video: A Survey of Smart Transcoding and Compression Approaches

Volumetric video, the capture and display of three-dimensional (3D) imagery, has emerged as a revolutionary technology poised to transform the media landscape, enabling immersive experiences that transcend the limitations of traditional 2D video. One of the key challenges in this domain is the efficient delivery of these high-bandwidth, data-intensive volumetric video streams, which requires innovative transcoding and compression techniques. This research paper explores the state-of-the-art in volumetric video compression and delivery, with a focus on the potential of AI-driven solutions to address the unique challenges posed by this emerging medium.

Log Volume Measurement and Counting Based on Improved Cascade Mask R-CNN and Deep SORT

Logs require multiple verifications to ensure accurate volume and quantity measurements. Log end detection is a crucial step in measuring log volume and counting logs. Currently, this task primarily relies on the Mask R-CNN instance segmentation model. However, the Feature Pyramid Network (FPN) in Mask R-CNN may compromise accuracy due to feature redundancy during multi-scale fusion, particularly with small objects. Moreover, counting logs in a single image is challenging due to their large size and stacking. To address the above issues, we propose an improved log segmentation model based on Cascade Mask R-CNN. This method uses ResNet for multi-scale feature extraction and integrates a hierarchical Convolutional Block Attention Module (CBAM) to refine feature weights and enhance object emphasis. Then, a Region Proposal Network (RPN) is employed to generate log segmentation proposals. Finally, combined with Deep SORT, the model tracks log ends in video streams and counts the number of logs in the stack. Experiments demonstrate the effectiveness of our method, achieving an average precision (AP) of 82.3, APs of 75.3 for small, APm of 70.9 for medium, and APl of 86.2 for large objects. These results represent improvements of 1.8%, 3.7%, 2.6%, and 1.4% over Mask R-CNN, respectively. The detection rate reached 98.6%, with a counting accuracy of 95%. Compared to manually measured volumes, our method shows a low error rate of 4.07%.

MNCATM: A Multi-Layer Non-Uniform Coding-Based Adaptive Transmission Method for 360° Video

With the rapid development of multimedia services and smart devices, 360-degree video has enhanced the user viewing experience, ushering in a new era of immersive human–computer interaction. These technologies are increasingly integrating everyday life, including gaming, education, and healthcare. However, the uneven spatiotemporal distribution of wireless resources presents significant challenges for the transmission of ultra-high-definition 360-degree video streaming. To address this issue, this paper proposes a multi-layer non-uniform coding-based adaptive transmission method for 360° video (MNCATM). This method optimizes video caching and transmission by dividing non-uniform tiles and leveraging users’ dynamic field of view (FoV) information and the multi-bitrate characteristics of video content. First, the video transmission process is formalized and modeled, and an adaptive transmission optimization framework for a non-uniform video is proposed. Based on this, the optimization problem required by the paper is summarized, and an algorithm is proposed to solve the problem. Simulation experiments demonstrate that the proposed method, MNCATM, outperforms existing transmission schemes in terms of bandwidth utilization and user quality of experience (QoE). MNCATM can effectively utilize network bandwidth, reduce latency, improve transmission efficiency, and maximize user experience quality.

A geometric model with stochastic error for abnormal motion detection of portal crane bucket grab

Abnormal swing angle detection of bucket grabs is crucial for efficient harbor operations. In this study, we develop a practically convenient swing angle detection method for crane operation, requiring only a single standard surveillance camera at the fly-jib head, without the need for sophisticated sensors or markers on the payload. Specifically, our algorithm takes the video images from the camera as input. Next, a fine-tuned ‘the fifth version of the You Only Look Once algorithm’ (YOLOv5) model is used to automatically detect the position of the bucket grab on the image plane. Subsequently, a novel geometric model is constructed, which takes the pixel position of the bucket grab, the steel rope length provided by the Programmable Logic Controller (PLC) system, and the optical lens information of the camera into consideration. The key parameters of this geometric model are statistically estimated by a novel iterative algorithm. Once the key parameters are estimated, the algorithm can automatically detect swing angles from video streams. Being analytically simple, the computation of our algorithm is fast, as it takes about 0.01 s to process one single image generated by the surveillance camera. Therefore, we are able to obtain an accurate and fast estimation of the swing angle of an operating crane in real-time applications. Simulation studies are conducted to validate the model and algorithm. Real video examples from Qingdao Seaport under various weather conditions are analyzed to demonstrate its practical performance.

GestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents

Existing gesture interfaces only work with a fixed set of gestures defined either by interface designers or by users themselves, which introduces learning or demonstration efforts that diminish their naturalness. Humans, on the other hand, understand free-form gestures by synthesizing the gesture, context, experience, and common sense. In this way, the user does not need to learn, demonstrate, or associate gestures. We introduce GestureGPT, a free-form hand gesture understanding framework that mimics human gesture understanding procedures to enable a natural free-form gestural interface. Our framework leverages multiple Large Language Model agents to manage and synthesize gesture and context information, then infers the interaction intent by associating the gesture with an interface function. More specifically, our triple-agent framework includes a Gesture Description Agent that automatically segments and formulates natural language descriptions of hand poses and movements based on hand landmark coordinates. The description is deciphered by a Gesture Inference Agent through self-reasoning and querying about the interaction context (e.g., interaction history, gaze data), which is managed by a Context Management Agent. Following iterative exchanges, the Gesture Inference Agent discerns the user’s intent by grounding it to an interactive function. We validated our framework offline under two real-world scenarios: smart home control and online video streaming. The average zero-shot Top-1/Top-5 grounding accuracies are 44.79%/83.59% for smart home tasks and 37.50%/73.44% for video streaming tasks. We also provide an extensive discussion that includes rationale for model selection, generalizability, and future research directions for a practical system etc.

From incipience to status quo: Development from theatre to streaming

This paper highlights the profound impact of streaming platforms on the profitability and distribution of mainstream films. Through digital innovations that have since transformed the entertainment industry, consumers have turned to video streaming services such as YouTube and Netflix, advancing accessible and affordable content instead of traditional theatrical releases. This transition, fueled by changes in consumer behaviour and accelerated by the COVID-19 pandemic, has reshaped the film industry’s financial and cultural basis, with streaming platforms offering a viable and often preferable alternative to cinemas. Using a mixed-methods approach, the study analyses historical box office trends and industry professionals’ opinions to investigate the rise of streaming and the oscillating state of theatres. While box office revenues remain volatile, especially for mid-budget and independent films, streaming continues to grow: hitting a projected global market value over $184.3 billion by 2027. Insights from this research are crucial for future filmmakers, industry professionals, business leaders, and entrepreneurs to navigate the rapidly changing landscape of film distribution.

Spatiotemporal Information, Near-Field Perception, and Service for Tourists by Distributed Camera and BeiDou Positioning System in Mountainous Scenic Areas

The collaborative use of camera near-field sensors for monitoring the number and status of tourists is a crucial aspect of smart scenic spot management. This paper proposes a near-field perception technical system that achieves dynamic and accurate detection of tourist targets in mountainous scenic areas, addressing the challenges of real-time passive perception and safety management of tourists. The technical framework involves the following steps: Firstly, real-time video stream signals are collected from multiple cameras to create a distributed perception network. Then, the YOLOX network model is enhanced with the CBAM module and ASFF method to improve the dynamic recognition of preliminary tourist targets in complex scenes. Additionally, the BYTE target dynamic tracking algorithm is employed to address the issue of target occlusion in mountainous scenic areas, thereby enhancing the accuracy of model detection. Finally, the video target monocular spatial positioning algorithm is utilized to determine the actual geographic location of tourists based on the image coordinates. The algorithm was deployed in the Tianmeng Scenic Area of Yimeng Mountain in Shandong Province, and the results demonstrate that this technical system effectively assists in accurately perceiving and spatially positioning tourists in mountainous scenic spots. The system demonstrates an overall accuracy in tourist perception of over 90%, with spatial positioning errors less than 1.0 m and a root mean square error (RMSE) of less than 1.14. This provides auxiliary technical support and effective data support for passive real-time dynamic precise perception and safety management of regional tourist targets in mountainous scenic areas with no/weak satellite navigation signals.

Introduction: mapping the music and video streaming landscape and its disruptions in Southeast Asia

ABSTRACT This article introduces the special issue of Mapping the Music and Video Streaming Landscape and its Disruptions in Southeast Asia by identifying three ways in which the special issue engages with the broader scholarship on streaming. In it, we discuss how our special issue converses with scholarly debates about the disruptive nature of streaming, how the unique qualities of streaming infrastructure in Southeast Asia resonates with scholarship on other global South contexts, and how both the commonalities and variations within Southeast Asia that our special issue makes apparent can be likened to what other scholars have observed of the disjunctive and overlapping nature of the European market.