Traditional Video Research Articles

A Dual-Channel and Frequency-Aware Approach for Lightweight Video Instance Segmentation

Video instance segmentation, a key technology for intelligent sensing in visual perception, plays a key role in automated surveillance, robotics, and smart cities. These scenarios rely on real-time and efficient target-tracking capabilities for accurate perception and intelligent analysis of dynamic environments. However, traditional video instance segmentation methods face complex models, high computational overheads, and slow segmentation speeds in time-series feature extraction, especially in resource-constrained environments. To address these challenges, a Dual-Channel and Frequency-Aware Approach for Lightweight Video Instance Segmentation (DCFA-LVIS) is proposed in this paper. In feature extraction, a DCEResNet backbone network structure based on a dual-channel feature enhancement mechanism is designed to improve the model’s accuracy by enhancing the feature extraction and representation capabilities. In instance tracking, a dual-frequency perceptual enhancement network structure is constructed, which uses an independent instance query mechanism to capture temporal information and combines with a frequency-aware attention mechanism to capture instance features on different attention layers of high and low frequencies, respectively, to effectively reduce the complexity of the model, decrease the number of parameters, and improve the segmentation efficiency. Experiments show that the model proposed in this paper achieves state-of-the-art segmentation performance with few parameters on the YouTube-VIS dataset, demonstrating its efficiency and practicality. This method significantly enhances the application efficiency and adaptability of visual perception intelligent sensing technology in video data acquisition and processing, providing strong support for its widespread deployment.

Cost-Effective and Practical: Cable Cameras as Video Laryngoscopes in Emergency Medicine

Abstract I am Dr. Emre Bülbül, an emergency medicine specialist, proposing the use of cable cameras as video laryngoscopes. Cable cameras can provide high-resolution images at a low cost, making them a practical alternative to traditional laryngoscopes, especially in emergency situations and in resource-limited healthcare facilities. The benefits of using cable cameras as video laryngoscopes include: Cost-Effectiveness; Traditional video laryngoscopes are expensive, but cable cameras are affordable, allowing for more efficient use of healthcare budgets. Easy Accessibility: Cable cameras can be easily found and used in all types of healthcare facilities, preventing time loss in emergencies. This innovative solution can improve the quality of healthcare services, particularly in regions with limited resources. By addressing this topic in your journal, you can raise awareness among healthcare professionals and recipients.

A Real-Time Posture Detection Algorithm Based on Deep Learning

With the development of machine vision and multimedia technology, posture detection and related algorithms have become widely used in the field of human posture recognition. Traditional video surveillance methods have the disadvantages of slow detection speed, low accuracy, interference from occlusions, and poor real-time performance. This paper proposes a real-time pose detection algorithm based on deep learning, which can effectively perform real-time tracking and detection of single and multiple individuals in different indoor and outdoor environments and at different distances. First, a corresponding pose recognition dataset for complex scenes was created based on the YOLO network. Then, the OpenPose method was used to detect key points of the human body. Finally, the Kalman filter multi-object tracking method was used to predict the state of human targets within the occluded area. Real-time detection of human postures (sitting, stand up, standing, sit down, walking, fall down, and lying down) is achieved with corresponding alarms to ensure the timely detection and processing of emergencies.

Two‐stage video anomaly detection based on dual‐stream networks and multi‐instance learning

AbstractTo promptly detect abnormal events in surveillance videos, this article designs a video anomaly detection method based on multiple instance learning. Generally, abnormal events occur less frequently compared to normal events. Traditional video surveillance relies on manual operation to monitor scenes and detect abnormal events by watching surveillance videos. However, watching surveillance footage is a labor‐intensive task, and prolonged observation can lead to visual fatigue and lack of concentration, which in turn results in missed detections and false positives [1]. Therefore, it is crucial to develop intelligent algorithms for video anomaly detection. The method can detect whether segments of a video contain abnormal events. First, the I3D network is used as a feature extractor to capture spatiotemporal features from the input video. Then, the spatiotemporal information is processed and input into a segment‐level anomaly detector based on multiple instance learning for detection. The authors treat abnormal videos as positive bags and normal videos as negative bags, and automatically learn a deep anomaly ranking model that can predict abnormal segments. Finally, the results of the training were tested and analyzed, demonstrating that the model is capable of detecting abnormal traffic segments.

Comparative educational effectiveness of AI generated images and traditional lectures for diagnosing chalazion and sebaceous carcinoma

Sebaceous carcinoma is difficult to distinguish from chalazion due to their rarity and clinicians’ limited experience. This study investigated the potential of AI-generated image training to improve diagnostic skills for these eyelid tumors compared to traditional video lecture-based education. Students from Orthoptics, Optometry, and Vision Research (n = 55) were randomly assigned to either an AI-generated image training group or a traditional video lecture group. Diagnostic performance was assessed using a 50-image quiz before and after the intervention. Both groups showed significant improvement in overall diagnostic accuracy (p < 0.001), with no significant difference between groups (p = 0.124). In the AI group, all 25 chalazion images showed improvement, while only 6 out of 25 sebaceous carcinoma images improved. The video lecture group showed improvement in 19 out of 25 chalazion images and 24 out of 25 sebaceous carcinoma images. The proportion of images with improved accuracy was significantly higher in the AI group for chalazion (P = 0.022) and in the video group for sebaceous carcinoma (P < 0.001). These findings suggest that AI-generated image training can enhance diagnostic skills for rare conditions, but its effectiveness depends on the quality and quantity of patient images used for optimization. Combining AI-generated image training with traditional video lectures may lead to more effective educational programs. Further research is needed to explore AI’s potential in medical education and improve diagnostic skills for rare diseases.

VIGOR: Reviving Cloud Gaming Sessions

Cloud gaming is a multi-billion dollar industry. A client in cloud gaming sends its movement to the game server on the Internet, which renders and transmits the resulting video back. In order to provide a good gaming experience, a latency below 80 ms is required. This means that video rendering, encoding, transmission, decoding, and displaying have to finish within that time frame, which is especially challenging to achieve due to server overload, network congestion, and losses. In this paper, we propose VIGOR, a new method to revive cloud gaming sessions by recovering lost or corrupted video frames. Unlike traditional video frame recovery, VIGOR uses game states to enhance recovery accuracy significantly and utilizes partially decoded frames to recover lost portions. We develop a holistic system that consists of (i) efficiently extracting game states, (ii) modifying H.264 video decoder to generate a mask to indicate which portions of video frames need recovery, and (iii) designing a novel neural network to recover either complete or partial video frames. VIGOR is extensively evaluated using iPhone 12 and laptop implementations, and we demonstrate the utility of game states in the game video recovery and the effectiveness of our overall design.

Strategies to alleviate flickering: Bayesian and smoothing methods for deep learning classification in video

PurposeIncreasing reliance on autonomous systems requires confidence in the accuracies produced from computer vision classification algorithms. Computer vision (CV) for video classification provides phenomenal abilities, but it often suffers from “flickering” of results. Flickering occurs when the CV algorithm switches between declared classes over successive frames. Such behavior causes a loss of trust and confidence in their operations.Design/methodology/approachThis “flickering” behavior often results from CV algorithms treating successive observations as independent, which ignores the dependence inherent in most videos. Bayesian neural networks are a potential remedy to this issue using Bayesian priors. This research compares a traditional video classification neural network to its Bayesian equivalent based on performance and capabilities. Additionally, this work introduces the concept of smoothing to reduce the opportunities for “flickering.”FindingsThe augmentation of Bayesian layers to CNNs matched with an exponentially decaying weighted average for classifications demonstrates promising benefits in reducing flickering. In the best case the proposed Bayesian CNN model reduces flickering by 67% while maintaining both overall accuracy and class level accuracy.Research limitations/implicationsThe training of the Bayesian CNN is more computationally demanding and the requirement to classify frames multiple times reduces resulting framerate. However, for some high surety mission applications this is a tradeoff the decision analyst may be willing to make.Originality/valueOur research expands on previous efforts by first using a variable number of frames to produce the moving average as well as by using an exponentially decaying moving average in conjunction with Bayesian augmentation.

Analyzing the applicability of psychometric QoE modeling for projection-based point cloud video quality assessment

Point cloud video delivery will be an important part of future immersive multimedia. In it, objects represented as sets of points are embedded within a video which is streamed and displayed to remote users. This opens possibilities towards remote presence scenarios such as tele-conferencing, remote education and virtual training. Due to its infeasibly high bandwidth requirements, encoding is unavoidable. The introduced artifacts and network degradations can have an important but unpredictable impact on the end-user’s Quality of Experience (QoE). Thus, real-time quality monitoring and prediction mechanisms are key to allow for fast countermeasures in case of QoE decrease. Since current state-of-the-art research is focusing on either continuous QoE monitoring of traditional video streaming services or objective delivery optimizations of point cloud content without any QoE validation, we believe this work brings a valuable contribution to current literature. Therefore, we present a no-reference (NR) QoE model, consisting of KMeans clustering and sigmoidal mapping, that works on video-level, group-of-pictures (GOP)-level and frame-level granularity. Results show the value of the sigmoidal mapping across all granularity levels. The clustering algorithm shows its value at the video-level and in the role of an outlier detector on the more fine-grained levels. Satisfying results are yet obtained with correlation values often going above 0.700 on GOP- and frame-level while maintaining root mean squared error (RMSE) below 10 on a 0–100 scale. In addition, a Command Line Interface (CLI) Video Metric Tool is presented that allows for easy and modular calculation of NR metrics on a given video.

Adaptive Prediction Structure for Learned Video Compression

Learned video compression has developed rapidly and shown competitive rate-distortion performance compared with the latest traditional video coding standard H.266 (VVC). However, existing works were restricted to fixed prediction direction and GoP size. The inflexibility on prediction structure hinders learned video compression towards optimal compression efficiency in diverse motion scenarios. In this paper, we propose to advance learned video compression with adaptive prediction structure decision. Specifically, we propose a unified compression framework that supports both forward prediction and bi-directional prediction. The framework can flexibly switch to different prediction direction to achieve better prediction performance. Meanwhile, we propose a low-complexity prediction structure decision algorithm, where prediction direction and GoP size are adaptively determined based on motion complexity to achieve optimal compression efficiency. Experimental results demonstrate that the proposed unified framework with adaptive decision algorithm improves compression efficiency of pure forward prediction-based or bi-directional prediction-based framework with neglectable ( \(0.9\%\) ) encoding time increment. Meanwhile, it achieves comparable compression performance with VVC and recent learned video coding methods.

Time Series Classification of Badminton Pose using LSTM with Landmark Tracking

Traditional methods of analyzing badminton matches, such as video movement analysis, are time-consuming, prone to errors, and rely heavily on manual annotation. This creates challenges in accurately and efficiently classifying badminton actions and player poses. This paper aims to develop an accurate time series classification method for badminton poses using landmark tracking. The proposed method integrates Long Short-Term Memory (LSTM) networks with landmark tracking to classify badminton poses in a time series, addressing the limitations of traditional video analysis techniques. The dataset consists of 30 respondents performing three distinct activities—lob, smash, and serve—under two conditions: good and bad execution. The approach combines LSTM networks with landmark tracking data, utilizing intra-class variation from a multi-view dataset to enhance pose classification accuracy. The LSTM model achieved high accuracy in classifying badminton poses, successfully detecting serves, lobs, and smashes in real-time with over 90% accuracy. Additionally, the system improved match analysis, achieving 85% accuracy in detection and classification, demonstrating the effectiveness of combining landmark tracking with machine learning for sports analysis. This study underscores the importance of pose estimation in badminton analysis, particularly through landmark tracking, which significantly improves the accuracy of classifying player poses and contributes to the advancement of automated sports analysis.

HVASR: Enhancing 360-degree video delivery with viewport-aware super resolution

In recent years, 360-degree videos have gained significant traction due to their capacity to provide immersive experiences. However, the adoption of 360-degree videos substantially escalates bandwidth demands, necessitating approximately four to ten times more bandwidth than traditional video formats do. This presents a considerable challenge in maintaining high-quality videos in environments characterized by limited bandwidth or unstable networks. A trend has emerged where client-side computational power and deep neural networks are employed to enhance video quality while mitigating bandwidth requirements within contemporary video delivery systems. These approaches segment a video into discrete chunks and apply super resolution (SR) models to each segment, streaming low-resolution (LR) chunks alongside their corresponding SR models to the client. Although these methods enhance both video quality and transmission efficiency for conventional videos, they impose greater computational resource demands when applied to 360-degree content, thereby constraining widespread implementation. This paper introduces an innovative method called HVASR for 360-degree videos that leverages viewport information for more precise segmentation and minimizes model training costs as well as bandwidth requirements. Additionally, HVASR incorporates a viewport-aware training strategy that is aimed at further enhancing performance while reducing computational expenses. The experimental results demonstrate that HVASR achieves an average utility increase ranging from 12.46% to 40.89% across various scenes.

PERCEPTION OF ACTIVE VIDEO GAMING IN ENHANCEMENT OF PHYSICAL ACTIVITY AMONG ADOLESCENTS

Background: Active Video Games (AVGs) require players to perform movements that are mirrored by characters on-screen, integrating physical activity into gaming. This synergy of technology and exercise provides a dynamic way to remain active by engaging in digital entertainment, redefining traditional video gameplay. Objective: This study aims to assess adolescents' perceptions of active video gaming and explore its association with physical activity, considering AVGs potential to alter exercise behaviors. Methods: A cross-sectional study was conducted with 385 adolescents aged 12-25 years. The study was divided into three phases: initial demographic data collection, assessing awareness of AVGs through a questionnaire, and evaluating perceptions of AVGs via diverse inquiry methods to understand their impact on physical activity levels. Results: Findings indicate a robust correlation between active video gaming and increased physical activity among adolescents, with a notably positive perception of AVGs. The majority of participants (68%) reported a favorable view towards AVGs as a form of exercise, and 62% acknowledged an increase in their physical activity levels due to AVGs. Conclusion: AVGs demonstrate significant potential as a motivational tool for increasing physical activity among youth, making exercise more accessible and appealing through gamified experiences. There remains a keen interest among adolescents to learn more about the benefits of AVGs.

HADNet: A Novel Lightweight Approach for Abnormal Sound Detection on Highway Based on 1D Convolutional Neural Network and Multi-Head Self-Attention Mechanism

Video surveillance is an effective tool for traffic management and safety, but it may face challenges in extreme weather, low visibility, areas outside the monitoring field of view, or during nighttime conditions. Therefore, abnormal sound detection is used in traffic management and safety as an auxiliary tool to complement video surveillance. In this paper, a novel lightweight method for abnormal sound detection based on 1D CNN and Multi-Head Self-Attention Mechanism on the embedded system is proposed, which is named HADNet. First, 1D CNN is employed for local feature extraction, which minimizes information loss from the audio signal during time-frequency conversion and reduces computational complexity. Second, the proposed block based on Multi-Head Self-Attention Mechanism not only effectively mitigates the issue of disappearing gradients, but also enhances detection accuracy. Finally, the joint loss function is employed to detect abnormal audio. This choice helps address issues related to unbalanced training data and class overlap, thereby improving model performance on imbalanced datasets. The proposed HADNet method was evaluated on the MIVIA Road Events and UrbanSound8K datasets. The results demonstrate that the proposed method for abnormal audio detection on embedded systems achieves high accuracy of 99.6% and an efficient detection time of 0.06 s. This approach proves to be robust and suitable for practical applications in traffic management and safety. By addressing the challenges posed by traditional video surveillance methods, HADNet offers a valuable and complementary solution for enhancing safety measures in diverse traffic conditions.

The potential effects of deepfakes on news media and entertainment

AbstractDeepfakes are synthetic media, such as pictures, music and videos, created with generative artificial intelligence (GenAI) tools, a technique that builds upon machine learning and multi-layered neural networks trained on large datasets. Today, anyone can create deepfakes online, without any knowledge about the underpinning technology. This fact opens up creative opportunities at the same time as it creates individual and societal challenges. Some identified challenges are fake news, bullying, defamation, media manipulation and democracy damage. The aim of this study was to analyse and discuss how deepfakes could affect entertainment, education and digital news media in positive and negative ways. The chosen research strategy was a qualitative cross-sectional study to create a snapshot of current attitudes in blogs, online discussion fora and from a selected expert panel. Firstly, blogs and internet discussion fora on deepfakes and AI were analysed inductively. Secondly, results from the previous step were used to create a question scheme for semi-structured interviews with some selected experts that have relevant knowledge in the field of AI and deepfakes. Finally, interviews answered were analysed thematically using the Technology Affordances and Constraints Theory as an analytical lens. Findings show that deepfakes at the same time have a wide variety of affordances, but also some critical constraints to consider. In the affordance part, there are the potential use of deepfakes for speculative and provoking concepts, not only as entertainment, but also to illustrate educational scenarios that are hard or impossible to capture in traditional videos, images or sound recordings. Furthermore, there are many affordances for entertainment and the creation of environments where realism is not an important condition. Some examples are video game graphics, animated film background music in games and music videos. However, the conclusion is that the findings comprise more of constraints than of affordances, where the serious negative aspect of deepfakes can lead to psychological, financial and social harm. Many images and videos in news media have already been manipulated during the last decades, but deepfakes can take this to a new and more problematic level where the overall trust might disappear. The author’s recommendation is to further increase the focus on source criticism and that discussions on which sources to trust should be an early introduced part of educational programmes.

The Effectiveness of Interactive Videos in Increasing Student Engagement in Online Learning

The shift towards online learning has highlighted the need for innovative approaches to maintain student engagement. Traditional video lectures often fail to captivate students, decreasing attention and learning outcomes. Interactive videos incorporating quizzes, clickable hotspots, and branching scenarios have emerged as a potential solution to this challenge. This study aimed to evaluate the effectiveness of interactive videos in increasing student engagement in online learning environments compared to traditional video lectures. Additionally, it sought to identify the most impactful interactive elements and their correlation with learning outcomes. A mixed-methods approach was employed, involving 200 undergraduate students from diverse disciplines. Participants were randomly assigned to either an experimental group using interactive videos or a control group using traditional video lectures for a 6-week online course. Data collection methods included pre-and post-tests, engagement metrics (time spent, interaction frequency), and qualitative feedback through surveys and focus group discussions. Results demonstrated a significant increase in student engagement for the interactive video group, with 45% higher interaction rates and 30% longer viewing times than the control group. Quiz-based interactions and branching scenarios were found to be the most effective elements. Moreover, the experimental group showed a 25% improvement in learning outcomes as measured by post-test scores. In conclusion, this study provides strong evidence for the efficacy of interactive videos in enhancing student engagement and learning outcomes in online education. These findings suggest that incorporating interactive elements into video content can significantly improve the online learning experience, offering educators a valuable tool to address engagement challenges in digital learning environments.

STMixer: A One-Stage Sparse Action Detector.

Traditional video action detectors typically adopt the two-stage pipeline, where a person detector is first employed to generate actor boxes and then 3D RoIAlign is used to extract actor-specific features for action recognition. This detection paradigm requires multi-stage training and inference, and the feature sampling is only constrained inside the box, failing to effectively leverage richer context information outside. Recently, several query-based action detectors have been proposed to predict action instances in an end-to-end manner. However, they still lack adaptability in feature sampling and decoding, thus suffering from the issues of inferior performance or slower convergence. In this paper, we propose two core designs for a more flexible one-stage sparse action detector. First, we present a query-based adaptive feature sampling module, which endows the detector with the flexibility of mining a group of discriminative features from the entire spatio-temporal domain. Second, we devise a decoupled feature mixing module, which dynamically attends to and mixes video features along the spatial and temporal dimensions respectively for better feature decoding. Based on these designs, we instantiate two detection pipelines, that is, STMixer-K for keyframe action detection and STMixer-T for action tubelet detection. Without bells and whistles, our STMixer detectors obtain the state-of-the-art results on five challenging spatio-temporal action detection benchmarks for keyframe action detection or action tube detection.

Anomaly detection in crowded scenes: technologies, challenges and opportunities

The paper discusses advancements in intelligent video surveillance systems, particularly focused on anomaly detection in crowded environments. These systems aim to enhance public safety by automatically detecting unusual behavior and potential threats in real-time. Traditional video surveillance, relying heavily on human monitoring, faces limitations like reduced concentration and slow response times. In contrast, intelligent surveillance uses machine learning and AI algorithms to process vast amounts of video data, identifying patterns that deviate from normal behavior. Crowd anomaly detection is essential in densely populated areas like transportation hubs, stadiums, and public squares. The diversity of anomalies, ranging from minor disruptions to serious threats such as theft or terrorist attacks, presents a challenge for these systems. Anomalies can be difficult to detect due to their unpredictable nature, and what constitutes an anomaly varies depending on the context. The paper highlights the need for robust systems that can adapt to various environmental conditions and distinguish between normal variations and genuine threats. While supervised machine learning models show promise, they often require large amounts of labeled data, which is difficult to obtain in real-world settings. Unsupervised models and deep learning techniques, such as Convolution Neural Networks, have been effective in analyzing crowd behavior and detecting anomalies. However, these methods still face challenges, including scalability, high false positive rates, and the need for real-time processing in large-scale environments. The paper concludes by addressing the limitations of current crowd anomaly detection methods, such as their computational costs, ethical concerns, and inability to detect novel anomalies. It suggests directions for future research, including the integration of advanced learning techniques to improve system performance and scalability

Spatiotemporal analysis using deep learning and fuzzy inference for evaluating broiler activities

Observing poultry activity is crucial for assessing their health status; however, the inspection process is often time-consuming and labor-intensive, particularly in cases involving large numbers of chickens. Inexperienced breeders may also misjudge their activity levels, potentially missing opportunities for prevention and treatment. This study integrates traditional video surveillance with an advanced monitoring system to identify various broiler behaviors in a breeding environment. A two-stage deep learning approach is employed: in the first stage, the broilers are detected, and in the second stage, five key body points (head, abdomen, two legs, and tail) are identified. A skeleton-based model is then developed centered around the abdomen, with six angles calculated using trigonometric methods. These angles are analyzed by a long short-term memory network to estimate behaviors such as “Standing”, “Walking”, “Resting”, “Eating”, “Preening”, and “Flapping”, selecting the behavior with the highest probability. Dual-layer fuzzy logic inference systems were used to evaluate the proportion of time broilers spent in static versus dynamic states, providing a robust determination of their activity levels. Validated in a mixed-sex breeding environment, the proposed system achieved accuracies of at least 85.2% for identifying broiler type, 79.2% for identifying body parts, and 50.8% for identifying behaviors. The activity level evaluation results were consistent with those conducted by experienced poultry experts.

Информатизация в административном судопроизводстве: текущее состояние и перспективы

The article is dedicated to the issues of using information technology and its variation, digital technologies, in contemporary administrative proceedings. The author reviews problems of using traditional video conferences and electronic document flow as well as problems of using artificial intelligence and machine learning systems.

Immersive storytelling for pro-environmental behaviour change: The Green Planet augmented reality experience

Immersive media is becoming more mainstream but its role in challenging social norms and individuals’ behaviours is less understood. We implemented a survey at The Green Planet Augmented Reality Experience, one of the largest immersive experiences in the United Kingdom, investigating the potential of nature-themed augmented reality (AR) experiences to encourage sustainability beliefs and pro-environmental behaviours, with a one-month follow-up. A video condition group was also recruited to watch a 2D screen-capture video of the AR experience. Results suggest that the AR group had increased in sustainability attitudes and pro-environmental behaviours after (vs. before) the experience, and this persisted at follow-up one month later. When compared to the video group, the AR group scored higher in sustainability beliefs immediately after engaging with the media (but did not differ in pro-environmental behavioural intentions). At a one-month follow-up, both groups appeared to have declined in sustainability beliefs and behaviours in comparison to immediately after media exposure (but were still greater when compared to before exposure). While the two groups did not differ in sustainability beliefs, AR visitors reported to have performed more pro-environmental behaviours in the month following the experience and even increased significantly more since before attending the experience when compared to those in the video condition. Finally, we built a model of cognitive mechanisms of longitudinal pro-environmental behaviour change by media, identifying the sense of presence and connection with nature as mechanisms through which immersive media can promote greater behaviour change when compared to traditional video media. Limitations and industry implications are discussed.