Video Research Articles

Solar-blind photonic integrated chips for real-time on-chip communication

The monolithically integrated self-driven photoelectric detector (PD) with the light-emitting diode (LED) epitaxial structure completely relies on the built-in electric field in the multi-quantum wells region to separate the photogenerated carriers. Here, we propose a novel superlattices–electron barrier layer structure to expand the potential field region and enhance the detection capability of the integrated PD. The PD exhibits a record-breaking photo-to-dark current ratio of 5.14 × 107, responsivity of 110.3 A/W, and specific detectivity of 2.2 × 1013 Jones at 0 V bias, respectively. A clear open-eyed diagram of the monolithically integrated chip, including the PD, LED, and waveguide, is realized under a high-speed communication rate of 150 Mbps. The obtained transient response (rise/decay) time of 2.16/2.28 ns also illustrates the outstanding transient response capability of the integrated chip. The on-chip optical communication system is built to achieve the practical video signals transmission application, which is a formidable contender for the core module of future large-scale photonic integrated circuits.

Enhanced Area Defense: Magnetic Launcher and Sensor Integration

In this article, the most appropriate design method for a field defense system that operates with magnetic field laws by taking audio and video signals as reference is described. The study is important because it eliminates the operator (soldier) factor in the battlefield. Taking into account the effect of capacitor voltage, capacitance value and accelerator winding inductance value, the main criteria for determining the power supply and armature structure of the electromagnetic launcher are proposed. This proposal is made with MATLAB/Simulink software based mathematical model and the differences are explained by comparing with ANSYS Maxwell simulation. The simulated results show that the speed difference between the models is 7%. Additionally, the design methods of the audio and video-based positioning systems that control the system are explained in the study. In this system where signal receiving microphones are positioned in triangular form, a linear algorithm using the time difference method is utilized. By comparing the theoretical mathematical model with the experimental simulation model, the accuracy of the method and the method function is proved. In this study, a deep learning-based target detection system that operates with the YOLO v2 algorithm is used to increase the system's mission execution capacity. The operator is eliminated by switching the system with the signal received from the positioning systems.

A machine learning-driven web application for sign language learning.

Addressing the increasing demand for accessible sign language learning tools, this paper introduces an innovative Machine Learning-Driven Web Application dedicated to Sign Language Learning. This web application represents a significant advancement in sign language education. Unlike traditional approaches, the application's unique methodology involves assigning users different words to spell. Users are tasked with signing each letter of the word, earning a point upon correctly signing the entire word. The paper delves into the development, features, and the machine learning framework underlying the application. Developed using HTML, CSS, JavaScript, and Flask, the web application seamlessly accesses the user's webcam for a live video feed, displaying the model's predictions on-screen to facilitate interactive practice sessions. The primary aim is to provide a learning platform for those who are not familiar with sign language, offering them the opportunity to acquire this essential skill and fostering inclusivity in the digital age.

Convergence region size research of television signal superposition iteration method for measurement machine vision system

Background. Image processing is used in machine vision measurement systems, which are widely used in industrial applications. One of the important tasks in processing is image superposition. Registration methods must fulfill two basic requirements: satisfactory registration quality and practical processing time. One of the methods that meets these requirements is iterative superposition. Aim. The aim is to study the size of the area (region of convergence) for the values of the parameters of the initial approximation, which ensures satisfactory superposition. The size of the region determines the performance of the iterative algorithm. Methods. The size of the region of convergence was determined experimentally: correlation analysis and statistical simulation. Results. Experiments have shown if a sampling step is no more than 1/10 of the size of the image fragment, then the probability of correct registration is almost equal to one. If this threshold is exceeded, then the probability decreases. Conclusion. The size of the region of convergence of the iterative image registration method has been established. It is possible to design processing algorithms and predict processing time by obtaining results. The superposition method was introduced into a machine vision system to recognize contact wire clamps on the railway. Image superposition in this system is carried out in real time.

HUMAN FOLLOWING CART

The development of a human-following robot utilizing Raspberry Pi and camera technology represents a significant advancement in autonomous robotics, offering practical solutions for various industries. This project integrates a Raspberry Pi microcomputer, a camera, and ultrasonic sensors to create a robot capable of tracking and following a human operator. The system employs computer vision algorithms, facilitated by a Haar Cascade Classifier, to detect faces in real-time video feeds. The video feed is processed to determine the position of the human within the frame, enabling the robot to navigate accordingly. Ultrasonic sensors are used to measure the distance between the robot and the human, ensuring that a safe and optimal following distance is maintained. The motors and motor drivers are controlled based on the visual and distance data, allowing the robot to move forward, backward, or turn as needed. This combination of technologies enhances worker safety by reducing physical strain and improves operational efficiency by automating the transportation of goods, tools, and supplies. The implementation of this system demonstrates the practical application of affordable and accessible technologies like the Raspberry Pi in creating intelligent, autonomous robots. The project highlights the potential for such robots to revolutionize workflows in warehouses, manufacturing plants, and hospitals, making advanced automation more attainable for a wide range of businesses. The integration of computer vision and sensor-based navigation in human-following robots marks a significant step forward in the evolution of intelligent workspaces and autonomous systems.

AI Assistant for Visually Impaired

This paper presents an advanced assistive technol- ogy system aimed at improving accessibility and independence for visually impaired individuals. Utilizing artificial intelligence (AI) and computer vision techniques, the system provides real- time auditory feedback to users about their surroundings. The core components of the system include image captioning, face recognition, and depth estimation, all integrated to offer a comprehensive understanding of the environment. The system captures live video feed through a webcam, processes the images using pre-trained models like CLIPSeg for image segmentation and DPT for depth estimation, and generates textual descriptions of detected objects and their spatial distances. These descriptions are translated from English to Kannada using Google’s transla- tion services and converted into speech with the gTTS library, ensuring accessibility for Kannada-speaking users. Additionally, the system employs face recognition to identify known individuals in the vicinity, providing personalized auditory notifications. The combination of these technologies enables the system to offer context-aware assistance, helping visually impaired users to navigate and interact with their surroundings more effectively. Experimental results demonstrate the system’s capability to deliver accurate, real-time feedback, highlighting its potential to significantly enhance the quality of life for visually impaired individuals.

Bimodal deep learning model for subjectively enhanced emotion classification in films

This research delves into the concept of color grading in film, focusing on how color influences the emotional response of the audience. The study commenced by recalling state-of-the-art works that process audio–video signals and associated emotions by machine learning. Then, assumptions of subjective tests for refining and validating an emotion model for assigning specific emotional labels to selected film excerpts were presented. The insights gained from these subjective evaluations facilitated the creation of a comprehensive database of movie excerpts. This database was subsequently employed to both train and evaluate the efficacy of deep learning models. The latter half of the study shifts focus to the intelligent analysis of audio and video signals that form film excerpts. This involved exploring diverse methodologies for parameterizing these signals. Models that demonstrated the highest accuracy on the test dataset on audio/video only were amalgamated to forge a bimodal model, which integrates both audio and video signals for emotion classification. The bimodal model exhibited superior accuracy in tests compared to a model that solely relied on video signal classification. This enhancement in performance was achieved with only a marginal increase in the complexity and the number of parameters within the model.

Уточнение границ областей уверенного приема телевизионного сигнала в одночастотной сети DVB-T2

One of the ways to clarify the zones of a television signal’s confident reception in single-frequency networks (SFN) DVB-T2 deployed on previously existing infrastructure of analog signal’s television is considered. A simulation model of the SFN broadcasting consist of module of the propagation of radio waves, module of a transmitting station, module of a receiving station and a geoinformation system. It has been revealed that the DVB-T2 broadcasting modes, the effective antenna suspension heights and the effective radiated power of transmitters, which are used in DVB-T2 broadcasting, can lead not only to the appearance of areas of intra-character interference with reduced television signal quality inside the service area, but also to the effect of the so-called "intra-network amplification". The discrepancy between the used protective interval (PI) and the topology of the studied OSS has been established.

Advanced Tactical Helmet for Military Use

Abstract: Advanced features on a tactical helmet designed for military use. Soldiers in today's military operations must have improved situational awareness, strong communication skills, and superior protection when fighting. A new generation of tactical helmets with enhanced features is required because the standard helmets now in use do not adequately fulfil these criteria. To improve army performance and survivability The helmet has several extra features in addition to its extensive ESP32 CAM for real-time data visualization, communication systems for seamless inter-squad communication, and improved ballistic protection materials. Among these characteristics include the incorporation of a helmet camera's First-Person-View (FPV) live video feed into a base station, along with the base station's face detection capabilities for enhanced identification. SolidWorks was used in the design process to produce a unique 3D-printed helmet, while ESP32 and ESP8266 were used to power a custom mainboard that effectively controlled the helmet's functions. this creative solution aims to significantly improve military operations, increase survivability, and ensure the safety of soldiers on the battlefield.

Manual Acupuncture Manipulation Recognition Method via Interactive Fusion of Spatial Multiscale Motion Features

Manual acupuncture manipulation (MAM) is essential in traditional Chinese medicine treatment. MAM action recognition is important for junior acupuncturists’ training and education; however, there are obvious personalized differences in hand gestures among expert acupuncturists for the same type of MAM. In addition, during the MAM operations, the magnitude and frequency of the expert acupuncturists’ hand shape and relative needle-holding finger position changes are tiny and fast, resulting in difficulties in observing MAM action details. Thus, we propose a Spatial Multiscale Interactive Fusion MAM Recognition Network to solve the difficulties in MAM recognition. First, this paper presents an optical flow-based hand motion contour global feature extraction method for acupuncture hand shape. Second, to explore the motion rule between the needle-holding fingers during the MAM operations, we design a quantitative description method of the relative motion of the needle-holding fingers: an “interactive attention module,” which achieves feature fusion and mines the correlation between different scales of MAM action features. Finally, the proposed MAM recognition method was validated by 20 acupuncturists from the Beijing University of Traditional Chinese Medicine and 10 from the Beijing Zhongguancun Hospital who participated in the MAM video signal collection. The proposed recognition method achieves the highest average validation accuracy of 95.3% and the highest test accuracy of 96.0% for four typical MAMs, proving its feasibility and effectiveness.

Head-locked, world-locked, or conformal diminished-reality? An examination of different AR solutions for pedestrian safety in occluded scenarios

Many collisions between pedestrians and cars are caused by poor visibility, such as occlusion by a parked vehicle. Augmented reality (AR) could help to prevent this problem, but it is unknown to what extent the augmented information needs to be embedded into the world. In this virtual reality experiment with a head-mounted display (HMD), 28 participants were exposed to AR designs, in a scenario where a vehicle approached from behind a parked vehicle. The experimental conditions included a head-locked live video feed of the occluded region, meaning it was fixed in a specific location within the view of the HMD (VideoHead), a world-locked video feed displayed across the street (VideoStreet), and two conformal diminished reality designs: a see-through display on the occluding vehicle (VideoSeeThrough) and a solution where the occluding vehicle has been made semi-transparent (TransparentVehicle). A Baseline condition without augmented information served as a reference. Additionally, the VideoHead and VideoStreet conditions were each tested with and without the addition of a guiding arrow indicating the location of the approaching vehicle. Participants performed 42 trials, 6 per condition, during which they had to hold a key when they felt safe to cross. The keypress percentages and responses from additional questionnaires showed that the diminished-reality TransparentVehicle and VideoSeeThrough designs came out most favourably, while the VideoHead solution caused some discomfort and dissatisfaction. An analysis of head yaw angle showed that VideoHead and VideoStreet caused divided attention between the screen and the approaching vehicle. The use of guiding arrows did not contribute demonstrable added value. AR designs with a high level of local embeddedness are beneficial for addressing occlusion problems when crossing. However, the head-locked solutions should not be immediately dismissed because, according to the literature, such solutions can serve tasks where a salient warning or instruction is beneficial.

Multimodality Cardiovascular Imaging for Totally Video-Guided Thorascopic Cardiac Surgery.

Totally video-guided thorascopic cardiac surgery (TVTCS) represents one of the most minimally invasive access routes to the heart. Its feasibility and safety can be guaranteed by an experienced surgeon with skilled operative techniques under the guidance of a video signal via thoracoscopy and the imaging from transesophageal echocardiography. At present, this surgical approach has been applied for atrioventricular valve disease, atrial septum defects plus and partial anomalous pulmonary venous drainage, cardiac tumors, hypertrophic obstructive cardiomyopathy, aortic valve disease, and atrial fibrillation. Multimodality cardiovascular imaging, including echocardiography, X-ray, computed tomography (CT), magnetic resonance imaging (MRI) and cardiac catheterization, provides morphologic characteristics and function status of the cardiovascular system and a comprehensive view of the target anatomy. In this review, the benefits of multimodality cardiovascular imaging are summarized for the clinical practice of TVTCS, including the preoperative preparation, intraoperative guidance and postoperative supervision. The disease categories are also individually reviewed on the basis of multimodality cardiovascular imaging, to ensure the feasibility and safety for TVTCS. Cardiovascular imaging technologies not only confirm who is a candidate for this surgical technique, but also provide technical support during the procedure and for postop follow to assess the clinical outcomes. Multimodality cardiovascular imaging is instrumental to provide the requirements to solve the problems for conduction of TVTCS; and to provide individualized protocols with high-resolution and real-time dynamic imaging fusion.

TeleAware Robot: Designing Awareness-augmented Telepresence Robot for Remote Collaborative Locomotion

Telepresence robots can be used to support users to navigate an environment remotely and share the visiting experience with their social partners. Although such systems allow users to see and hear the remote environment and communicate with their partners via live video feed, this does not provide enough awareness of the environment and their remote partner's activities. In this paper, we introduce an awareness framework for collaborative locomotion in scenarios of onsite and remote users visiting a place together. From an observational study of small groups of people visiting exhibitions, we derived four design goals for enhancing the environmental and social awareness between social partners, and developed a set of awareness-enhancing techniques to add to a standard telepresence robot - named TeleAware robot. Through a controlled experiment simulating a guided exhibition visiting task, TeleAware robot showed the ability to lower the workload, facilitate closer social proximity, and improve mutual awareness and social presence compared with the standard one. We discuss the impact of mobility and roles of local and remote users, and provide insights for the future design of awareness-enhancing telepresence robot systems that facilitate collaborative locomotion.

Virtual Trailroom using Machine Learning

The virtual trail room system receives a real-time video feed from a camera and processes data with the OpenCV computer vision library and Haarcascade classifier. In order to correctly identify faces in the video feed, the Haarcascade classifier is trained on a sizable dataset of human faces.The virtual room system may function independently, which eliminates the need for additional employees and associated expenditures, in contrast to typical security systems that demand the presence of physical security officers.

DRIVER DROWSINESS DETECTION SYSTEM

The proposed system utilizes facial landmark detection and eye aspect ratio calculation to detect drowsiness in real-time from a webcam feed. It employs the dlib library for face detection and facial landmark prediction. The eye aspect ratio (EAR) is computed from the distances between specific points on the eyes. When the EAR falls below a predefined threshold for a certain number of consecutive frames, it triggers an alert indicating potential drowsiness. The alert includes visual cues on the video feed, an auditory alert using the playsound library, and even sends an instant WhatsApp message using pywhatkit to notify a designated contact about the drowsiness. This system serves as a useful tool for monitoring driver fatigue or alertness in various contexts, promoting safety and awareness. Its capacity to swiftly identify and alert users to the onset of drowsiness represents a significant leap forward in safety technology. By mitigating the risk of accidents stemming from fatigue-induced impairments, this project underscores the pivotal role of computer vision in real-time human behavior monitoring, offering a versatile and invaluable tool for fostering safety and awareness across various domains. Keywords: Drowsiness Detection; Python; Whatsapp messaging; Webcam feed; Computer Vision; Alert System; Real-time monitoring; Eye-Aspect Ratio; Frame Processing; Facial Recognition.

Automated Stand-Alone Bot for Intruder Detection

The Automated Standalone Bot for Intruder Detection System represents a cutting-edge approach to bolstering security measures through the deployment of an intelligent surveillance system. By harnessing sophisticated computer vision techniques, this system is capable of vigilantly monitoring its surroundings and swiftly identifying potential security threats with remarkable accuracy. Through continuous real-time analysis of video feeds and the application of advanced image processing algorithms, the bot can promptly detect any unauthorized individuals or suspicious activities within its designated area of coverage. Upon detecting a potential intruder, the system initiates an immediate response protocol, which may include activating alarms, notifying security personnel, or implementing deterrent measures to deter any further intrusion attempts. What sets this system apart is its ability to continually refine its intruder detection capabilities over time, leveraging machine learning algorithms to adapt to evolving threats and minimize false positives. Moreover, the system demonstrates remarkable discernment by distinguishing between human subjects, animals, and other objects, thereby reducing the occurrence of false alarms and enhancing its overall reliability. This comprehensive approach not only fortifies security measures but also instills a greater sense of confidence in the system's ability to safeguard the premises effectively. Key Words: Automated Standalone Bot, Intruder Detection System, Security Measures, Intelligent System, Computer Vision Techniques, Surveillance, Real-time Analysis, Video Feeds, Image Processing Algorithms, Swift Detection, Unauthorized Individuals, Suspicious Activities, Response Protocol, Alarms, Notification, Deterrent Measures, Machine Learning Algorithms, Adaptation, False Positives, Reliability, Human Subjects, Animals, False Alarms, Premises Security

Bridging the Communication Gap - Sign Language Communication System

Now, imagine a world where words are not spoken, and emotions are not expressed. For the Deaf community, this is not a mere thought experiment but a daily reality. Indeed, they are forced to communicate primarily through sign language e, therefore, regularly feel alienated in a society where words are the primary tool for translating thoughts and feelings. Under such circumstances, most Deaf and hard-of-hearing people are struggling to communicate the most effectively. They usually opt for interpreters or their own signaling, but these alternatives are not the most efficient or effective choices. While signaling is the most intuitive and meaningful alternative, its grammar and semantic variations make it impossible to comprehend for those people who are not within the culture. Therefore, we have also developed a software prototype that could interpret sign language automatically. This paper presents a new method to recognize Indian sign language alphabets A-Z and digits (0-9) in a real- time video feed by employing the Bag of Visual Words model (BOVW). Furthermore, it not only predicts the labels of the signs but also gives the output as text and speech. The segmentation process here comprehends skin color detection and background subtraction. Speeded Up Robust FeaturesSurf features have been extracted again from the images, and histograms are built to map the signs to text labels. Furthermore, we have utilized algorithms such as Support Vector Machine (SVM) and Convolutional Neural Networks (CNN) for binary classification. Finally, we have established an interactive Graphical User Interface (GUI to make this process more user-friendly

Towards automatic inspection of nuclear fuel elements in spent fuel pools: Audio analysis

In this article, we propose and explore a novel step in the digitization of the mapping of the spent fuel pool of nuclear power plants, in which the audio signal from the operator's microphone is used to obtain the identification codes of those components that are in each of the cells of the pool. In this way, we have not only an acquisition system but also a verification system that can be used in combination with the outcome of the analysis of the video signal. We developed an algorithm that uses at its core one of the latest models of multilingual Automatic Speech Recognition to transcribe audio signal, and with a post-processing of the timed transcriptions we build the identification code of fuel heads and other components. Results show a very high accuracy in audios from real recording of Spanish nuclear facilities, and the methodology proposed is easily extensible to other nuclear facilities in the world.

Pipe Inspection Robot

Abstract: A pipe inspection robot is device that is inserted into pipes to check for obstruction or damage. These robots are traditionally manufactured offshore, are extremely expensive, and are often not adequately supported in the event or malfunction. This had resulted in associated environmental services limited. A New Zealand utilize of this equipment, facing significant periods of down time as they wait for their robots to be the repaired. Recently, they were informing that several robots were no longer supported. This project was conceived to redesign the electronics control systems one of these PIR, utilizing the existing mechanical platform. Requirements for the robot were that it must operate reliably in confined, dark and wet environments and provides a human wear with a digital video feed of the internal status of the pipes. There robot should as much as possible incorporate off the shaft components, cheap, and potentially onsite repair. This project details the redesign and constructions of such robots. Its employees their electronic boards integrated with mechanical components and provides video feedback via custom graphical interface although at the prototypes state the electronics has been successful with cost of less than a length of the original robot purchase prize

Connected Robotics for Intelligent Surveillance: An IOT Approach

Abstract: The robot employs the HTTP Communication protocol, enabling users to access live video streaming from the OV2640 camera through a standard web browser. The web interface includes intuitive directional buttons for seamless control, allowing users to command the robot to move left, right, forward, and backward. The locomotion of the robot is achieved through two DC motors integrated with a Robot chassis and an L293D motor driver module. This paper presents the development and implementation of an innovative IoT-based Surveillance Robot designed for remote control, utilizing a Wi-Fi module and a smartphone interface. The portable espionage robot integrates cutting-edge technology, featuring an AI Thinker ESP32-CAM module with an ESP32-S processor, an OV2640 camera, motor drivers, and a robust chassis with two moving wheels. The ESP32-CAM allows the capture and storage of images on a microSD card, facilitating efficient data management. The real-time video feed is transmitted wirelessly to the user's smartphone over a Wi-Fi infrastructure, providing a livestreaming experience. This wireless surveillance robot finds applications in various scenarios such as war fields, disaster management, and beyond, showcasing its versatility in remote monitoring and reconnaissance tasks.