Real-time Video Research Articles

Self-Identifying Alcohol Scene Detection from Video and Showing the Message Alcohol Consumption is Injurious to Health

Abstract: The proposed project aims to detect alcohol consumption from real time video footage using machine learning and show the message "Alcohol consumption is injurious to health".

Security Drone for Surveillance in Military

Abstract: This paper presents the design and development of a sophisticated military security drone equipped with advanced components including Pixhawk flight controller, GPS module, camera, receiver, telemetry system, BLDC (Brushless Direct Current) motors, and electronic speed controller (ESC). The Pixhawk flight controller serves as the brain of the drone, managing flight dynamics, navigation, and mission planning with remarkable precision and reliability. Integrated with a highaccuracy GPS module, the drone achieves autonomous navigation capabilities, enabling it to execute pre-defined flight paths and perform missions with minimal human intervention. A high-resolution camera system is incorporated into the drone, providing real-time video surveillance and reconnaissance capabilities. This camera system is complemented by a receiver module, facilitating remote control and data transmission between the drone and ground control station.Furthermore, the telemetry system enables remote monitoring of the drone's vital parameters, including altitude, speed, battery status, and sensor readings. This real-time telemetry data enhances situational awareness and facilitates informed decision-making during missions.The propulsion system of the drone consists of BLDC motors and electronic speed controllers, offering efficient and reliable propulsion for sustained flight operations. These components are carefully integrated to optimize power consumption, flight endurance, and maneuverability, ensuring the drone's effectiveness in various operational scenarios. In summary, the integration of Pixhawk flight controller, GPS module, camera, receiver, telemetry, BLDC motors, and electronic speed controller in the design of the military security drone represents a significant advancement in unmanned aerial technology. The resulting drone platform offers enhanced capabilities for surveillance, reconnaissance, and security applications, contributing to the effectiveness and efficiency of military operations in diverse environments

Artificial intelligence-assisted system for the assessment of Forrest classification of peptic ulcer bleeding: a multicenter diagnostic study.

Inaccurate Forrest classification may significantly affect clinical outcomes, especially in high risk patients. Therefore, this study aimed to develop a real-time deep convolutional neural network (DCNN) system to assess the Forrest classification of peptic ulcer bleeding (PUB). A training dataset (3868 endoscopic images) and an internal validation dataset (834 images) were retrospectively collected from the 900th Hospital, Fuzhou, China. In addition, 521 images collected from four other hospitals were used for external validation. Finally, 46 endoscopic videos were prospectively collected to assess the real-time diagnostic performance of the DCNN system, whose diagnostic performance was also prospectively compared with that of three senior and three junior endoscopists. The DCNN system had a satisfactory diagnostic performance in the assessment of Forrest classification, with an accuracy of 91.2% (95%CI 89.5%-92.6%) and a macro-average area under the receiver operating characteristic curve of 0.80 in the validation dataset. Moreover, the DCNN system could judge suspicious regions automatically using Forrest classification in real-time videos, with an accuracy of 92.0% (95%CI 80.8%-97.8%). The DCNN system showed more accurate and stable diagnostic performance than endoscopists in the prospective clinical comparison test. This system helped to slightly improve the diagnostic performance of senior endoscopists and considerably enhance that of junior endoscopists. The DCNN system for the assessment of the Forrest classification of PUB showed satisfactory diagnostic performance, which was slightly superior to that of senior endoscopists. It could therefore effectively assist junior endoscopists in making such diagnoses during gastroscopy.

Multi-modal Emotion Recognition Using Neural Networks: Deploy on Cloud

This paper explores recent developments in emotion recognition, focusing on methodologies employing Multilayer Perceptron (MLP) classifiers. Emotion recognition is crucial for human-machine interaction across various domains, including healthcare, surveillance systems, and intelligent learning environments. There are many deep learning models that have been employed for emotion detection, this emphasizes the effectiveness of MLP classifiers. The project at hand centers on predicting emotions such as happiness, calmness, fear, and disgust solely utilizing MLP classifiers. The system is trained on audio data, with recordings matched in frequency, time, and speed to the raw data used for training. Additionally, Convolutional Neural Network (CNN) models are used here to analyze facial expressions in real-time video streams by providing neat and complete approach to emotion recognition. Upon deployment on the AWS cloud platform, the system provides real-time emotion predictions in textual format via a web interface created using Flask. Furthermore, the system offers the capability to analyze emotions from live video streams, presenting continuous feedback on emotions through a face-outlined box updated every three seconds. This functionality serves to confirm individuals' emotions and provide self-awareness regarding their emotional states. This project focuses on giving efficiency by using MLP classifiers in conjunction with CNN models for emotion recognition tasks, thereby contributing to the advancement of human-machine interaction. Key Words: MLP Classifier, CNN, AWS Cloud, Flask, emotion, audio, video

TLS-RWKV: Real-Time Online Action Detection with Temporal Label Smoothing

Online action detection (OAD)is a challenging task that involves predicting the ongoing action class in real-time streaming videos, which is essential in the field of autonomous driving and video surveillance. In this article, we propose an approach for OAD based on the Receptance Weighted Key Value (RWKV) model with temporal label smooth. The RWKV model captures temporal dependencies and computes efficiently at the same time, which makes it well-suited for real-time applications. Our TLS-RWKV model demonstrates advancements in two aspects. First, we conducted experiments on two widely used datasets, THUMOS’14 and TVSeries. Our proposed approach demonstrates state-of-the-art performance with 71.8% mAP on THUMOS’14 and 89.7% cAP on TVSeries. Second, our proposed approach demonstrates impressive efficiency, running at over 600 FPS and maintaining a competitive mAP of 59.9% on THUMOS’14 with RGB features alone. Notably, this efficiency surpasses the prior state-of-the-art model, TesTra, by more than two times. Even when executed on a CPU, our model maintains a commendable speed, exceeding 200 FPS. This high efficiency makes our model suitable for real-time deployment, even on resource-constrained devices. These results showcase the effectiveness and competitiveness of our proposed approach in OAD.

ARSIP: Automated Robotic System for Industrial Painting

This manuscript addresses the critical need for precise paint application to ensure product durability and aesthetics. While manual work carries risks, robotic systems promise accuracy, yet programming diverse product trajectories remains a challenge. This study aims to develop an autonomous system capable of generating paint trajectories based on object geometries for user-defined spraying processes. By emphasizing energy efficiency, process time, and coating thickness on complex surfaces, a hybrid optimization technique enhances overall efficiency. Extensive hardware and software development results in a robust robotic system leveraging the Robot Operating System (ROS). Integrating a low-cost 3D scanner, calibrator, and trajectory optimizer creates an autonomous painting system. Hardware components, including sensors, motors, and actuators, are seamlessly integrated with a Python and ROS-based software framework, enabling the desired automation. A web-based GUI, powered by JavaScript, allows user control over two robots, facilitating trajectory dispatch, 3D scanning, and optimization. Specific nodes manage calibration, validation, process settings, and real-time video feeds. The use of open-source software and an ROS ecosystem makes it a good choice for industrial-scale implementation. The results indicate that the proposed system can achieve the desired automation, contingent upon surface geometries, spraying processes, and robot dynamics.

A real-time wearable AR system for egocentric vision on the edge

Real-time performance is critical for Augmented Reality (AR) systems as it directly affects responsiveness and enables the timely rendering of virtual content superimposed on real scenes. In this context, we present the DARLENE wearable AR system, analysing its specifications, overall architecture and core algorithmic components. DARLENE comprises AR glasses and a wearable computing node responsible for several time-critical computation tasks. These include computer vision modules developed for the real-time analysis of dynamic scenes supporting functionalities for instance segmentation, tracking and pose estimation. To meet real-time requirements in limited resources, concrete algorithmic adaptations and design choices are introduced. The proposed system further supports real-time video streaming and interconnection with external IoT nodes. To improve user experience, a novel approach is proposed for the adaptive rendering of AR content by considering the user’s stress level, the context of use and the environmental conditions for adjusting the level of presented information towards enhancing their situational awareness. Through extensive experiments, we evaluate the performance of individual components and end-to-end pipelines. As the proposed system targets time-critical security applications where it can be used to enhance police officers’ situational awareness, further experimental results involving end users are reported with respect to overall user experience, workload and evaluation of situational awareness.

Sunda Script Detection Using You Only Look Once Algorithm

The Sundanese script is a writing system used in the Sundanese language, one of the regional languages of West Java, Indonesia. This study investigates the use of the YOLO v8 algorithm for the real-time video detection of Sundanese script. Various versions of YOLO v8, including YOLO v8n, v8s, v8m, v8l, and v8x, were tested to determine the most effective model. After a comprehensive evaluation involving the analysis of mean Average Precision (mAP), F1-Confidence, and precision, the study selected the YOLO v8s model as the primary detection method. YOLO v8s demonstrated superior performance with the highest mAP of 98.835%, an F1-Confidence of 98%, and a precision of 76,2%. This choice was based on a balance between high accuracy and computational efficiency. The results indicate significant potential for object recognition technology in the learning and preservation of Sundanese script.

Real-time video surveillance on highways using combination of extended Kalman Filter and deep reinforcement learning

Highways, as one of the main arteries of transit and transportation in today's world, play a fundamental role in accelerating transportation, and for this reason, continuous monitoring of them is of great importance. Among these, monitoring compliance with transportation laws by vehicles is of utmost importance; for automation, efficient and vehicle-specific models can be used. In this article, a new method for video surveillance of highways is presented using an extended Kalman filter (EKF) and reinforcement learning models. There are three primary stages to the suggested approach. During the first stage, the extended Kalman filter (EKF) is used to identify and track multiple targets. Next, in the second stage, a convolutional neural network (CNN) processes each detected moving item to determine the kind of vehicle. During this stage, the CNN model's ideal configuration is ascertained using a new optimization approach that combines Particle Swarm Optimization (PSO) and reinforcement learning. After identifying the type of vehicle, in the third phase, the proposed method uses a separate CNN model for each target vehicle to assess its compliance with transportation safety principles. It should be mentioned that each vehicle's associated CNN model is configured during this phase using the suggested optimization methodology. Investigations have been conducted into the effectiveness of the suggested method in identifying violations of road safety laws as well as how well it performed in the two phases of vehicle type identification. According to the findings, the suggested approach can identify the kind of vehicle with 98.72% accuracy, which is at least 3.41% better than the approaches that were compared. On the other hand, this model can detect the violation of road safety laws for each vehicle with an average accuracy of 91.5%, which shows at least a 3.49% improvement compared to the other methods.

Adaptive wavelet transform defogging scheme for real-time video restoration with field programmable gate array implementation

Adaptive wavelet transform defogging scheme for real-time video restoration with field programmable gate array implementation

Real-time video surveillance based human fall detection system using hybrid haar cascade classifier

Real-time video surveillance based human fall detection system using hybrid haar cascade classifier

Use of a Novel Beyeonics One Three-dimensional Head-mounted Digital Visualization Platform in Vitreoretinal surgeries.

Ophthalmic microscopes have been crucial in visualizing surgical fields, but their limitations in enhancing the surgical view through digital image processing have prompted the development of digital surgical microscopes. The Beyeonics One microscope, a novel digital microscope, offers ophthalmic surgeons a 3D visualization platform and an augmented reality (AR) surgical headset, potentially improving surgical decision-making and outcomes. While its initial use has been described in cataract and corneal surgeries, its application in vitreoretinal surgery remains relatively unexplored. In this interventional case series, we collected data from the medical records of patients who underwent vitreoretinal surgery using the Beyeonics One 3D visualization platform at the Tel Aviv Medical Center. A total of 36 eyes from 36 subjects were included. Surgical techniques included retinal detachment surgeries and macular surgeries, performed by experienced surgeons. The surgical visualization was facilitated by the Beyeonics One 3D head-mounted display (HMD) platform. The procedures were uneventful, and none intra- or postoperative complications were reported, and surgeons did not experience any signal delay in the real-time video. The Beyeonics One microscope offers several potential advantages in vitreoretinal surgery, including digital image processing, enhanced depth perception through the 3D HMD platform, and hands-free image control using head gestures. While this study demonstrates the feasibility and safety of the Beyeonics One microscope, addressing limitations related to hazy views and optimizing image quality are crucial for consistent visualization.

Real-time mask-wearing detection in video streams using deep convolutional neural networks for face recognition

This research aims to develop a real-time mask-wearing detection system using deep convolutional neural networks (CNNs). This is crucial in the coronavirus disease 2019 (COVID-19) pandemic to alert individuals who are not wearing masks early on, thereby reducing the spread of the virus. Since COVID-19 primarily spreads through respiratory droplets and mask-wearing is recommended, our proposed study utilizes computer vision techniques, specifically image processing, to detect masked and unmasked faces. We employ a customized CNN architecture consisting of five convolutional layers, followed by max-pooling layers and fully connected (FC) layers. The final output layer utilizes softmax activation for classification. The model is updated with optimized layer configurations and parameter values. We are developing an application that uses a digital camera as an input device. The application utilizes a dataset comprising 11,792 image samples, which are used for training and testing purposes with the 80:20 ratio. Real-time testing is conducted using human subjects captured by the camera. The experimental results demonstrate that the CNN method achieves a classification accuracy of 99% on the training data and 98.83% during real-time video testing. These findings suggest that the real-time mask detection system using CNN performs effectively.

A hybrid facial features extraction-based classification framework for typhlotic people

Facial features play a vital role in the real-time cloud-based applications. Since, most of the conventional models are difficult to detect heterogeneous facial features due to high computational memory and time for the internet of things (IoT) based video surveillance mechanisms. Video based facial features identification and extraction include a large number of candidates features which are difficult to detect the contextual similarity of the facial key points due to noise and computational memory. In order to resolve these issues, a hybrid multiple features extraction measures are implemented on the real-time video dataset to extract key points using the cloud-based classifier. In this work, a hybrid classifier is used to classify the key facial points in the cloud computing environment. Experimental results show that the proposed hybrid multiple feature extraction-based frameworks have better computational efficiency in terms of error rate, recall, precision, and accuracy than the conventional models.

Intelligent Technique for Moving Object Detection from Problematic Video Captured through Camera Sensor

Aim: The significant aim of the proposed work is to develop an adaptive method to compute the threshold during run-time and update it adaptively for each pixel in the testing phase. It classifies motion-oriented pixels from the scene for moving objects using background subtraction and enhances using post-processing. Background: According to the huge demand for surveillance system, society is looking towards an intelligent video surveillance system that detect and track moving objects from video captured through a surveillance camera. So, it is very crucial and highly recommended throughout the globe in numerous domains such as video-based surveillance, healthcare, transportation, and many more. Practically, this research area faces lots of challenging issues such as illumination variation, cluttered background, camouflage, etc. So, this paper has developed an adaptive background subtraction method to handle such challenging problems. Objective: To focus and study the problematic video data captured through the camera sensor. To handle challenging issues available in real-time video scenes. To develop a background subtraction method and update the background model adaptively for moving object detection. Method: The proposed method has been accomplished using the following sections: Background model construction Automatic generation of threshold Background subtraction Maintenance of background model Results: The qualitative analysis of the proposed work is experimented with publicly available datasets and compared with considered state-of-the-art methods. In this work, library sequence (thermal data) of CDNET and other color video frame sequences Foreground aperture, Waving Tree and Camouflage are considered from Microsoft’s Wallflower. The quantitative values depicted in Table- 1. This work demonstrate the better performance of the proposed method as compared to state-ofthe- art methods. It also generates better outcomes and handles the problem of a dynamic environment and illumination variation. Conclusion: Currently, the world is demanding computer vision-based security and surveillancebased applications for society. This work has provided a method for the detection of moving information using an adaptive method of background subtraction approach for moving object detection in video scenes. The performance evaluation depicts better average results as compared to considered peer methods.

Captionomaly: A Deep Learning Toolbox for Anomaly Captioning in Social Surveillance Systems

Real-time video stream monitoring is gaining huge attention lately with an effort to fully automate this process. On the other hand, reporting can be a tedious task, requiring manual inspection of several hours of daily clippings. Errors are likely to occur because of the repetitive nature of the task causing mental strain on operators. There is a need for an automated system that is capable of real-time video stream monitoring in social systems and reporting them. In this article, we provide a tool aiming to automate the process of anomaly detection and reporting. We combine anomaly detection and video captioning models to create a pipeline for anomaly reporting in descriptive form. A new set of labels by creating descriptive captions for the videos collected from the UCF-Crime (University of Central Florida-Crime) dataset has been formulated. The anomaly detection model is trained on the UCF-Crime, and the captioning model is trained with the newly created labeled set UCF-Crime video description (UCFC-VD). The tool will be used for performing the combined task of anomaly detection and captioning. Automated anomaly captioning would be useful in the efficient reporting of video surveillance data in different social scenarios. Several testing and evaluation techniques were performed. Source code and dataset: https://github.com/Adit31/Captionomaly-Deep-Learning-Toolbox-for-Anomaly-Captioning.

코로나19 펜데믹 이후 비대면 미술치료 연구 동향 분석

This study aims to suggest the usefulness and development direction of non-face-to-face art therapy by analyzing research trends on non-face-to-face art therapy after the COVID-19 pandemic. The papers subject to analysis are studies on non-face-to-face art therapy after COVID-19, with a total of 73 papers published from 2020 to 2023. The research results are as follows. First, the most research on non-face-to-face art therapy was conducted in 2022, and there were many master's theses. Second, the most common research method was qualitative research, and many studies were non-face-to-face and real-time video, and did not record whether or not IRB review was approved. Third, most non-face-to-face art therapy sessions were 6 to 10 sessions, lasting less than 90 minutes, and many studies did not record media preparation. Fourth, the number of study subjects was 2-5, and most of them were adults and had emotional problems. Fifth, the research topic was emotion-related, and most of the research involved verifying effects. These research results were able to provide direction and basic data for non-face-to-face art therapy required in the future post-corona era and explore the possibility of applying non-face-to-face art therapy to clinical sites.

Twenty-five years of real-time surveillance video analytics: a bibliometric review

Twenty-five years of real-time surveillance video analytics: a bibliometric review

Driver Drowsiness Detection using CNN

Abstract: This research introduces a Driver Drowsiness Detection system employing Convolutional Neural Networks (CNN). The system analyses real-time facial features from in-vehicle cameras to determine a driver's alertness level. Trained on diverse datasets, the CNN model demonstrates high accuracy in identifying drowsiness signs, making it suitable for real-world deployment. This system contributes to road safety by providing timely alerts to prevent accidents caused by driver fatigue. As road safety remains a critical concern, the development of intelligent systems to mitigate driver-related risks has become imperative. Driver drowsiness is a major factor contributing to road accidents, emphasizing the need for robust and real-time detection mechanisms. This research presents a novel approach for Driver Drowsiness Detection using Convolutional Neural Networks (CNN). The proposed system utilizes CNN architecture to analyse facial features extracted from real-time video streams captured by an in-vehicle camera. Facial landmarks and expressions are processed to determine the driver's level of alertness. The CNN model is trained on a diverse dataset comprising both drowsy and alert facial expressions, ensuring its adaptability to different driving conditions and individual characteristics. The system's effectiveness is evaluated through extensive experiments using various datasets and scenarios. The results demonstrate the CNN's capability to accurately identify signs of driver drowsiness, achieving high precision and recall rates. Furthermore, the model's real-time processing capabilities make it suitable for deployment in practical, on-road scenarios. The proposed Driver Drowsiness Detection system presents a promising solution to enhance road safety by providing timely alerts to drivers when signs of drowsiness are detected. This research contributes to the ongoing efforts to integrate artificial intelligence into vehicle safety systems, ultimately reducing the risk of accidents caused by driver fatigue.

Violence Detection using Deep Learning

In an era where technological advancements are reshaping various aspects of society, crime prevention and public safety have emerged as critical areas of concern.This project proposes a novel approach to crime analytics by harnessing the power of live footage and deep learning technologies.The invention of Crime Analytics Using Live Footage represents a groundbreaking advancement in the field of crime prevention and law enforcement. This technology harnesses the power of real-time video analysis, predictive analytics, and data-driven decision-making to proactively detect and prevent criminal activities, ensuring public safety and enhancing the efficiency of law enforcement efforts.The system's core functionality lies in its ability to analyze live video footage from a network of surveillance cameras, leveraging advanced computer vision algorithms and machine learning techniques. This real-time analysis enables the immediate identification of potential threats, suspicious behavior, and emerging criminal activities within monitored areas.