Real-time Video Research Articles

A place to call our home: innovative rural physical therapy training in Canada

BackgroundTo promote rural practice and increase enrollment in the entry-level Master of Science Physical Therapy program at the University of Alberta, the Department of Physical Therapy developed a rural physical therapy satellite campus located in a farming region in central Alberta. A distributed learning format was used to connect the rural cohort to the main urban campus. Real time video conferencing was used to connect the two campuses for all lectures, seminars and clinical skills classes. This evaluation aimed to describe a unique rural training program for physical therapy students and its effectiveness in promoting work in rural communities after graduation.MethodsPhysical Therapy students in the first three years (2012–2015) of commencing the rural satellite program (n = 280) were surveyed, and six focus groups were held to capture student experiences, satisfaction and engagement. Data were collected on employment locations of the 2012–2019 graduates’ first physical therapy position and current employment.ResultsSurvey results suggested comparable levels of satisfaction and engagement for all physical therapy students regardless of campus. Focus group data revealed that students quickly accepted the distributed learning technological interface, enjoyed their local campuses, and felt connected to instructors and student colleagues. Compared to the overall physical therapy workforce, a higher percentage of physical therapists graduating from the rural campus reported working in rural centers for both their first and current jobs.ConclusionRegardless of campus, students were satisfied and equally engaged in the physical therapy program. Students who completed the physical therapy program in a rural setting tended to work rurally after graduation. A distributed learning model may be useful for other healthcare training programs to promote engagement in rural health.

Investigation of Deep Learning Model for Vehicle Classification

The usage of automobiles in cities and metropolitan areas has increased drastically throughout the years and there is a need to monitor the flow of road traffic to improve the traffic congestion and safety. One of the best ways to monitor the traffic is using an artificial intelligence and machine learning. An automatic vehicle tracking system based on artificial intelligence and machine learning can offers capability to analyse the real-time traffic video data for the purpose of traffic surveillance. The computer vision is one of the subsets in machine learning that can train the computer to understand the visual data and perform specific tasks such as object detection and classification. A Vision-based system can be proposed to detect road accidents, predict traffic congestion and further road traffic analytics. This can improve the safety in transportation where it can recognize types of vehicles on the road, detecting road accidents, predicting the traffic congestion and further road traffic analytics. In the context of road traffic monitoring, the parameters of the traffic such as the type and number of vehicles that passes through must be recorded in order to gain valuable insights and make prediction such as the occurrence of traffic congestion. However, this requires reliable informative and accurate data as input for analytics. Therefore, in this research the deep learning model for vehicle classification is investigated to detect, classify types of vehicles and further predictive analytics. The vehicle classification is proposed based on Single Shot Detector (SSD) architecture model. The proposed model is tested on five different classes of vehicles with a total of 1263 images. Experimental results show that SSD model able to achieve 0.721 of precision, 0.741 of recall and 0.731 of F1 Score. Finally, the result show that the SSD model is more accurate among all the models for all the performance measure with the difference of more than 0.052 of precision, 0.706 of recall and 0.05 of F1 Score.

Object Detection on Real-Time Video with FPN and Modified Mask RCNN Based on Inception-ResNetV2

Object Detection on Real-Time Video with FPN and Modified Mask RCNN Based on Inception-ResNetV2

Evaluating the Impact of Direct, Direct Video, and Indirect Video Laryngoscopy Training on the Proficiency of Medical Students in Performing Direct Laryngoscopy: A High-Fidelity Manikin-Based Assessment.

Endotracheal intubation (ETI), a potentially lifesaving intervention employed frequently in the emergent setting, is a manual skill that improves with repetitive practice and high-quality feedback. Classically, ETI centered around Direct Laryngoscopy (DL); however, with the advent and recent availability of Indirect Video Laryngoscopy (IVL) and Direct Video Laryngoscopy (DVL), studies have demonstrated varying results on the benefit of Video Laryngoscopy (VL) in training. We hypothesize that a training program centered on DVL, allowing students to visualize the anatomy and simultaneously receive instructor feedback via a real-time video feed, will practically improve student performance in DL. Our study of first-year medical students from the Edward Via College of Osteopathic Medicine (n = 21) randomized participants to one of three cohorts: DL, IVL, and DVL in a manikin-based simulation laboratory evaluated on successful intubation, time to successful intubation, dental injury, Numeric Rating Scale (NRS) to assess the trainee's perception of their performance and confidence level of performing intubation in a real-life scenario. Our results did not demonstrate a statistically significant difference between the three training modalities based on the outcomes assessed. Although IVL and DVL cohorts achieved 100% success following training, compared to 71% in the DL cohort, the results were not statistically significant. This is potentially due to our limited sample size, as our sample did not meet the calculated 162 participants for 80% power. These findings suggest that a larger sample size may be required to determine if there is a significant difference in outcomes for these training modalities.

Drone-based mobile full-duplex optical communication

Wireless optical communication plays a vital role in mobile communication networks, and both drones and vehicles are essential mobile communication nodes. Here, image identification technology, gimbal, and full-duplex optical communication systems are integrated to manage the challenging issues of light alignment and target tracking for mobile full-duplex optical communication under the transmission control protocol/internet protocol (TCP/IP) scheme. Both drones and vehicles are equipped with the proposed setup to realize full-duplex wireless optical data transmission at a communication rate of 2 Mbps in both air-to-ground and air-to-air scenarios. By further integrating wireless-fidelity (Wi-Fi) modules, drone-based mobile optical communication systems (MOCSs) can be interconnected with the network via the TCP/IP scheme, and real-time video transmission can be demonstrated. This work provides a feasible route toward a mobile optical communication network (MOCN) for diverse applications.

Prototype for Multi-UAV Monitoring–Control System Using WebRTC

Most unmanned aerial vehicle (UAV) ground control station (GCS) solutions today are either web-based or native applications, primarily designed to support a single UAV. In this paper, our research aims to provide an open, universal framework intended for rapid prototyping, addressing these objectives by developing a Web Real-Time Communication (WebRTC)-based multi-UAV monitoring and control system for applications such as automated power line inspection (APOLI). The APOLI project focuses on identifying damage and faults in power line insulators through real-time image processing, video streaming, and flight data monitoring. The implementation is divided into three main parts. First, we configure UAVs for hardware-accelerated streaming using the GStreamer framework on the NVIDIA Jetson Nano companion board. Second, we develop the server-side application to receive hardware-encoded video feeds from the UAVs by utilizing a WebRTC media server. Lastly, we develop a web application that facilitates communication between clients and the server, allowing users with different authorization levels to access video feeds and control the UAVs. The system supports three user types: pilot/admin, inspector, and customer. Our research aims to leverage the WebRTC media server framework to develop a web-based GCS solution capable of managing multiple UAVs with low latency. The proposed solution enables real-time video streaming and flight data collection from multiple UAVs to a server, which is displayed in a web application interface hosted on the GCS. This approach ensures efficient inspection for applications like APOLI while prioritizing UAV safety during critical scenarios. Another advantage of the solution is its integration compatibility with platforms such as cloud services and native applications, as well as the modularity of the plugin-based architecture offered by the Janus WebRTC server for future development.

SimCapture app video performance assessment versus real-time instructor-based performance evaluation of undergraduates in neonatal resuscitation-an agreement study.

Undergraduates are trained in the basic neonatal resuscitation programme (NRP) and evaluated for skill acquisition by NRP performance evaluation test (PET). Video use improves the validity of assessment when video-information adequacy, intrusiveness to students, and educational purpose are adequately balanced. We evaluated whether there was a difference between instructor's real-time assessment and video-based assessment done by another independent assessor using videos recorded via the SimCapture application. Undergraduates were trained in basic neonatal resuscitation and were evaluated at the end for skill acquisition by PET as a part of a randomized control trial (RCT). Video recording of the PET assessment was done with the SimCapture app. Independent assessors evaluated recorded videos to evaluate and validate real-time instructor performance evaluation time assessments. We conducted an agreement study to evaluate the difference between a real-time instructor and video assessments. We trained 143 students; 139 videos were evaluated. The Bland-Altman plot showed good agreement. For the post-test result, 66 (47.5%) passed, and 73 (52.5%) failed according to the real-time assessment, while 61 (43.8%) passed and 78 (56.1%) failed according to the video assessment. The agreement between the two assessments was 94.9%, with a kappa value of 0.898. Indicating the need for positive pressure ventilation (PPV), 138 (99.3%) did correctly in real-time assessment, while 133 (95.6%) did correctly in video assessment with agreement of 96.4% but low kappa of 0.27. We conclude that the instructors' assessment was excellent and didn't differ from assessments obtained from recorded videos and vice-versa.

Quality of Experience Optimization for Real-Time XR Video Transmission With Energy Constraints

Quality of Experience Optimization for Real-Time XR Video Transmission With Energy Constraints

Holo-Stroke: Assessing for Immersive Stroke Care Through Stroke Hologram Teleportation.

Background: Augmented reality enables the wearer to see both their physical environment and virtual objects. Holograms could allow 3D video of providers to be transmitted to distant sites, allowing patients to interact with virtual providers as if they are in the same physical space. Our aim was to determine if Tele-Stroke augmented with Holo-Stroke, compared with Tele-Stroke alone, could improve satisfaction and perception of immersion for the patient. Methods: Kinect cameras positioned at 90-degree intervals around the hub practitioner were used. Cameras streamed real-time optical video to a unity point-cloud program where the data were stitched together in a 360-degree view. The resultant hologram was positioned in 3D space and was visible through the head-mounted display by the patient. Radiology images were shared in Tele-Stroke and via hologram. Likert satisfaction questions were administered. Wilcoxon signed-rank testing was used. Results: Each of the 30 neurology clinic participants scored both Tele-Stroke and Holo-Stroke. Out of these, 29 patients completed the assessments (1 failure owing to computer reboot). Average age was 52 years, with 53.3% of the patients being female, 70.0% being White, and 13.3% being Hispanic. Likert scale score median "Overall" was 32 Tele-Stroke versus 48 Holo-Stroke (p < 0.00001), "Immersion" was 5 versus 10 (p < 0.00001), "Beneficial Technique" was 6 versus 10 (p < 0.00001), and "Ability to See Images" was 5 versus 10 (p < 0.00001). Discussion: Holo-Stroke 3D holographic Tele-Stroke exams resulted in feasibility, satisfaction, and high perception of immersion for the patient. Patients were enthusiastic for the more immersive, personal discussion with their provider and a robust way to experience radiology images. Though further assessments are needed, Holo-Stroke can help the provider "be there, not just see there!"

Real-Time Lightweight Video Super-Resolution With RRED-Based Perceptual Constraint

Real-Time Lightweight Video Super-Resolution With RRED-Based Perceptual Constraint

Towards faster yet accurate video prediction for resource-constrained platforms

Video prediction has emerged as a critical task in the field of computer vision. While conventional deep learning-based video prediction models excel in accuracy, they often impose significant computational overhead. Moreover, they tend to disregard crucial factors like inference latency and memory consumption, which may render them unsuitable for real-time video prediction applications. This issue is further exacerbated by the necessity to deploy many of these applications on resource-limited embedded devices. In this study, we introduce an innovative faster yet accurate video prediction (FAVP) model designed to mitigate inference latency and memory footprint, all the while maintaining predictive accuracy. Our model employs an encoder-convertor-decoder architecture with a novel multi-layer convertor. Each layer within the convertor uses consistent kernel sizes to precisely capture dimensional features, while varying kernel sizes between layers effectively capture both local and global features. This combination of uniformity within layers and diversity between layers enhances the model’s ability to grasp dynamic changes. Furthermore, we demonstrate that replacing conventional large-kernel convolutions with involutions significantly trims the model’s parameter count and inference latency, without compromising prediction accuracy. Through comprehensive experiments conducted on resource-adequate x86 platforms, utilizing KTH, TrafficBJ, Human3.6M, Moving MNIST, and KTH-Enhanced datasets, we showcase our model’s exceptional prediction performance, particularly in terms of inference latency. Additionally, we evaluate the model on resource-constrained NVIDIA Jetson Nano platform using KITTI & CalTech Pedestrian dataset. The results underscore our model’s superior inference speed and its competence in meeting real-time video prediction requirements, even under stringent resource limitations.

Real-time traffic conflict prediction at signalized intersections using vehicle trajectory data and deep learning

Real-time conflict prediction at signalized intersections is crucial for urban road safety management. This study developed a real-time conflict prediction framework for signalized intersections using video data real-time recognition technology and deep learning techniques, incorporating lane-level information and feature interactions. The modeling framework consists of three stages: real-time video data extraction and processing, the development of a Deep and Cross Network (DCN)-based real-time traffic conflict prediction model, and conflict-driven factor interpretability analysis through SHapley Additive exPlanations (SHAP). In the first stage, an efficient automated trajectory extraction system is designed to obtain vehicle trajectories in real time for dynamic traffic parameters and conflict frequency identification. In the second stage, a DCN model is developed to construct the relationships between dynamic traffic parameters, including their interactions, and traffic conflicts. In the third stage, SHAP explores the impact mechanisms of different dynamic traffic parameters on traffic conflicts. The model’s predictive performance and interpretability are evaluated using intersection video data from Changsha City, China. The results show that: (1) In real-time traffic conflict prediction at signalized intersections across different modified time-to-conflict thresholds (1.5 s and 3.0 s), the DCN model consistently outperformed statistical and machine learning models. (2) High traffic flow on main and secondary roads at signalized intersections significantly increases the complexity and frequency of conflicts, with varying sensitivity depending on the interaction of traffic flow, speed, and platoon length. (3) The proposed framework provides a safety measurement standard for data-driven road safety management methods.

Spatial and Temporal Detection With Attention for Real-Time Video Analytics at Edges

Spatial and Temporal Detection With Attention for Real-Time Video Analytics at Edges

Fermented and Unfermented Cocoa Beans for Quality Identification Using Image Features

Fermented cocoa bean products are one of the high-quality requirements of the cocoa processing industry. On an automated industrial scale, early identification of cocoa bean quality is essential in the processing industry. This study aims to identify the condition of quality cocoa beans based on fermentation and non-fermentation characteristics. This study applies analysis based on static images taken using a camera with a distance variation of 5 cm, 10 cm, and 15 cm in both classes, with 500 image data each. The Feature extraction Approach uses the Oriented Gradient (HOG) method with a Support Vector Machine (SVM) classification technique. Image analysis of both object classes was also performed with a color change to show the dominance of the color pattern on the skin of the cocoa beans to be analyzed. The results showed that fermented cocoa beans show a color pattern and texture that tends to be darker and coarser than non-fermented cocoa beans. Computational results with performance analysis using Receiver Operating Characterisic (ROC) on both classes showed the results that the distance of 5 cm and 15 cm has 100% accuracy, but based on the best performance, comprehensively seen in terms of Precision, Recall, and F1-Score shows the best value is at a distance of 15 cm. The results of this research based on the literature review conducted have better achievements, thus enabling further research on the development of conveyor models with real-time video data for automation systems.

지리교육에서 FPV 드론을 활용한 창의적 학습 전략 탐구 : 예비 교사를 대상으로 한 적용 사례 연구

FPV drones are unmanned aerial vehicles equipped with cameras that allow the operator to view real-time video feed from the drone's perspective while controlling it. These drones open new possibilities in educational settings, particularly in promoting creative learning in geography education. This study explores creative learning strategies using FPV drones, where pre-service teachers collected and analyzed spatial data in real time to gain a deeper understanding of geographical concepts and spatial awareness. To achieve this, 42 pre-service teachers participated in practical activities conducted in Marine City and Cheongsapo in Busan, where they explored and analyzed actual terrains through drones, gaining immersive and practical problem-solving experiences that are difficult to achieve through traditional learning methods. The results of the study indicate that learning with FPV drones significantly enhanced geographic reasoning and spatial awareness, confirming that such tools are effective not only in geography education but also in promoting creative learning across various educational fields. In conclusion, this study suggests that FPV drones can play a crucial role in educational innovation and serve as valuable resources for the development and implementation of technology-based learning strategies in the future.

Eye Based Interactive Communication System

Eye-based communication systems, like the Blink-To-Speak language, are essential tools for individuals with motor neuron disorders, enabling them to articulate their needs and emotions. These systems leverage eye movements as a means of interaction, providing a vital communication channel for those with severe speech impairments. By interpreting specific eye gestures, users can effectively convey their thoughts, fostering a sense of independence and improving their quality of life. Many of the existing eye-tracking technologies tend to be complex and costly, limiting their accessibility in low-income regions. Using a mobile phone camera, the system captures real-time video frames to detect and track the user’s eyes, leveraging advanced facial landmark detection. Four primary eye movements—Left, Right, Up, and Blink—form the basis of the Blink-To-Live communication model, allowing users to communicate a range of more than 60 daily commands. By translating sequences of three eye gestures, the system generates readable sentences that are displayed on a screen and can be vocalized through synthesized speech. Unlike other sensor-dependent systems, Blink-To-Live is designed for affordability, flexibility, and ease of use, making it accessible without the need for specialized software or hardware. A prototype has been tested on a range of participants, showing positive results in terms of ease of use, flexibility, and cost-effectiveness

Recognition of Real-Time Video Activities Using Stacked Bi-GRU with Fusion-based Deep Architecture

Recognizing and understanding human activities in real-time videos is a challenging task due to the complex nature of video data and the need for efficient and accurate analysis. This research pioneers a breakthrough in video activity recognition by introducing a robust framework leveraging the power of a stacked Bidirectional Long Short-Term Memory (Bi-LSTM) and Gated Recurrent Unit (GRU) architecture, harmonized within a fusion-based deep model. The stacked Bi-LSTM-GRU model capitalizes on its dual recurrent architecture, capturing nuanced temporal dependencies within video sequences. The fusion-based deep architecture synergizes spatial and temporal features, enabling the model to discern intricate patterns in human activities. To further enhance the discriminative power of the model, we introduce a fusion module in the proposed deep architecture. The fusion module integrates multi-modal features extracted from different levels of the network hierarchy, allowing for a more comprehensive representation of video activities. We demonstrate the efficacy of our approach through rigorous experimentation on UCF50, UCF101, and HMDB51 datasets. In experiments on the UCF50 dataset, our model achieves an accuracy of 97.01% and 95.86% on training and validation sets respectively, showcasing its proficiency in discerning activities across a diverse range of scenarios. The evaluation extends to the UCF101 dataset, where the proposed approach achieves a competitive accuracy of 97.62% and 96.93% on training and validation sets, surpassing previous benchmarks by a margin of approx 1%. Further-more, on the challenging HMDB51 dataset, the model demonstrates a robust accuracy of 89.71%and 88.88% on training and validation sets, solidifying its efficacy in intricate action recognition tasks.

Model of integrated mental health video consultations for people with depression or anxiety in primary care (PROVIDE-C): assessor masked, multicentre, randomised controlled trial

ObjectiveTo evaluate whether an integrated mental health video consultation approach (PROVIDE model) can improve symptoms compared with usual care in adults with depression and anxiety disorders attending primary care.DesignAssessor masked,...

Cutting-Edge Methods in Visage Expression Recognition and Ocular Tracking for Analyzing Remote Video Interviews

This paper focuses to augment an in the flesh image and video processor, capacitated with an artificial intelligence (AI) arbitrator that can speculate a job interviewee’s behavioral adeptness according to the facial dictions or expressions. We put forward a sentiment analysis model using machine learning and CNN for the reinforcement of decision-making in the job interview process. This is bought up by histogram of oriented gradients and support vector machine (HOG-SVM) with the addition of convolutional neural network (CNN) recognition in real-time video recorded interviews. The goal of applying this technology, is to develop a method that could automatically decode a candidate’s behavior by his or her facial language (or micro expressions) based on the behavioral ecology view of facial displays (BECV) which is different from the classical perspective of recognizing emotional states. This paper dispenses a trailblazing technique of determining the performance of a candidate in a video interview. To do this, a delineation of the analysis of sentiments and eye tracking technique study is intricated in which the results can be processed on a single screen to select the right person for the hire. Key Words: Artificial Intelligence(AI), Histogram of Oriented Gradients(HOG), Support Vector Machine(SVM), Convolutional Neural Network(CNN), Behavioral Ecology View of Facial Displays, Eye Tracking Technique, AVI(Automatic Visual Inspection).

Mustang: Improving QoE for Real-Time Video in Cellular Networks by Masking Jitter

The advent of 5G and interactive live broadcasting has led to a growing trend of people preferring real-time interactive video services on mobile devices, particularly mobile phones. In this work, we measure the performance of Google congestion control (GCC) in cellular networks, which is the default congestion control algorithm for Web Real-Time Communication (WebRTC). Our measurements show that GCC sometimes makes bitrate decisions which are harmful to quality of experience (QoE) in cellular networks with high jitter. We further find that the frame delivery time (FDT) in the player can mitigate network jitter and maintain QoE. Moreover, the receiving rate is better to reflect the network congestion than RTT in cellular networks. Based on these measurements and findings, we propose Mustang, an algorithm designed to overcome the jitter in cellular networks. Mustang makes use of the FDT and receiving rate as feedback information to the sender. Then the sender adjusts its sending rate based on the information to guarantee QoE. We have implemented Mustang in WebRTC and evaluated it in both emulated and real cellular networks. The experimental results show that Mustang can improve WebRTC’s QoS and QoE performance. For QoS, Mustang increases the sending rate by 72.1% and has similar RTT and packet loss when compared with GCC, while it is about 30% better for QoE.