Low-cost Webcam Research Articles

DETECTION AND RECOGNITION OF TRAFFIC SIGNS FROM DATA COLLECTED BY THE MOBILE MAPPING SYSTEM

Abstract. Autonomous vehicles and high-resolution maps are key elements of future transport systems. Detection and recognition of traffic signs is an important element for the safe driving of autonomous vehicles and the development of high-resolution maps. In this study, it is aimed to accurately detect and identify traffic signs based on the data collected by the mobile mapping system in order to ensure the safe movement of autonomous vehicles in traffic. A low-cost method is proposed with the ResNet-50 model for an autonomous vehicle to automatically detect and recognise traffic signs while moving on the road. As a result of the model training, 0.99 accuracy and 0.016 loss were obtained. The success of the method was first observed on images randomly selected from the dataset. Then, a real-time test was performed on a low-cost webcam. The tests showed that the handled method detects and identifies the traffic sign quickly and accurately.

Low-Cost Webcam Camera and GNSS Integration for Updating Home Data using AI Principles

PDAM Surya Sembada Surabaya City responds to Surabaya's water service demand by pioneering a cost-effective mobile mapping tech. This innovation integrates 3 MP webcams, GNSS, and IMU sensors to update customer building data efficiently. Precise photo data with location and movement info within 1-meter proximity captures accurate building coordinates. Applying georeferencing and Cosine Similarity reduces redundant data by 50%. GNSS, IMU sensors, and VGG-16 model aid data collection and analysis. This comprehensive approach optimizes PDAM Surabaya's operations, ensuring timely and accurate customer updates amid Surabaya's population growth, promising enhanced service delivery while managing costs.

Object Recognition and Localization for Pick-and-Place Task using Difference-based Dynamic Movement Primitives

In this paper, a framework is developed for three dimensional (3D) object recognition, localization, and anthropomorphic manipulation using YOLOv5, two low-cost webcams, and a Franka Emika (FE) Panda manipulator. Autonomous pick-and-place tasks are executed in a human-like manner using impedance control and novel difference-based Dynamic Movement Primitives (DMP) trajectory planning. The high accuracy vision system is developed with two webcams by using triangular relation to determine the object's position. The difference-based DMP algorithm and task execution plan are used to learn unique trajectories for different objects based on human demonstration and the object's coordinates. By integrating these components together, experimental studies on pick-and-place tasks are successfully performed for two different objects.

Markerless Motion Capture System Based on Webcams Using OpenPose

Recently, webcam-based motion capture systems have attracted tremendous attention as alternative low-cost motion analysis tools, owing to the rapid advancement of markerless technologies. Although the upper and lower limbs are moved simultaneously in most function activities in daily life, most existing studies focused on kinematic data observed in the hip and knee joints during gait. The purpose of this study was to measure the kinematic data of the upper and lower limb simultaneously during modified squat, using low-cost webcams and to compare them to those of the 3D marker-based motion capture system. The modified squat was performed nine times in total without rest in between. The modified squat consisted of hip and knee flexion with elbow flexion. During modified squat, the kinematic data were collected using 4 RGB cameras and 13 Vicon cameras. The positions of the joints were obtained using OpenPose and the 3D positions were reconstructed using OpenCV. Angular trajectories from the Vicon motion capture system tended to be very similar to the ones from the webcam-based motion capture system. There was no significant difference in the kinematic data at the elbow between two different capture systems. Additionally, the minimum joint angles at the hip and knee were not significantly different. It was confirmed that the markerless motion capture system based on webcam can measure angular joint motions at the same level as the Vicon 3D motion capture system with markers. The webcam-based motion capture system will enhance the usability in clinical settings.

Medical image interpretation training with a low‐cost eye tracking and feedback system: A preliminary study

All medical students must learn to detect tumours in chest X-ray films. Eye tracking technology may be able to improve the training of this skill, with the potential benefit of saving thousands of lives each year. Eye trackers can record where students actually look in relation to known abnormalities in images, allowing the automated provision of corrective feedback during medical training. Research using dedicated eye tracking units suggests their potential as pedagogical tools but this hardware can be expensive, which prevents their use by medical students. To overcome this financial barrier, this project used eye tracking based on low-cost webcams to develop a system with the potential to allow all medical students to improve their interpretation skills. In self-study mode, students can practice and improve based on automated performance feedback in a browser-based system. Improvements in performance over time can be determined and mistakes can be played back and analysed for correction. Eight medical trainees used the system on a custom dataset of 60 chest X-ray images and were able to improve their decision times by self-study alone. The system was also rated highly for its ability to provide valuable objective information during subsequent one-on-one instruction sessions.

A 3D-printed robotic system for fully automated multiparameter analysis of drinkable water samples

This work describes a 3D-printed robotic system named RSAWA (robotic system for automatic water analysis) for fully automated water analysis. RSAWA consists of a robotic arm coupled to a syringe pump, temperature and conductivity sensors, a low-cost webcam as colorimetric detector, and a 96-well microplate placed on a 3D-printed platform. The robotic system is controlled by software and it performs all analytical procedures. RSAWA was applied to measure conductivity (CDT), pH, total alkalinity (TA), total hardness (TH), chloride (Cl−), nitrite (NO2−), total dissolved phosphorus (TP), and total iron (TI) in drinkable water samples. A simple circuit was designed for conductivity determinations, while colorimetric pH determinations were carried out using Hue values extracted from digital images and a pH universal indicator. HSV histograms were used to calculate Pearson's correlation coefficients, allowing the construction of accurate titration curves. In addition to achieving sample throughputs of 112 h−1 for TA and TH determinations and 92 h−1 for Cl− determinations, RSAWA produced 99.5% less waste than the corresponding reference methods during titrations. Colorimetric measurements were performed through RGB vector norms calculated from digital images were used as analytical signals. Limits of quantification (μg L−1) were 6.83, 13.0 and 1.5 mg L−1 for NO2−, TP, and TI determinations, respectively. Sample throughputs (samples h−1) were 83 for NO2− and TP and 72 for TI with a 98.5% reduction in waste generation. Thus, RSAWA is a low-cost, feasible, and environmentally friendly alternative to quickly and accurately determine several chemical and physicochemical parameters in aqueous samples.

A robust photoplethysmographic imaging for contactless heart and respiratory rates measurement using a simple webcam

Video photoplethysmography has been an area of interest for researchers from different domains driven by the demand of low cost, comfortable, simple and portable technology for physiological parameters measurements. In this paper we propose a robust and simple method for remotely measure heart and respiratory activities through video photoplethysmographic (vPPG) signal recordings using a low-cost webcam. The vPPG signal is extracted from colour video recording of a human face in an ambient light environment. The image and signal processing operations are minimised and optimised. Three critical aspects which are performances, reduced computational time and low computational complexity are focused. Experimental heart rate, breathing rate, and heart rate variability obtained results on 20 healthy subjects show a high correlation with those obtained using an approved contact sensor. The heart rate error obtained with the proposed method is ±2 beats per minute (bpm), this value is acceptable for standard heart diagnosis diseases.

Associating Facial Expressions and Upper-Body Gestures with Learning Tasks for Enhancing Intelligent Tutoring Systems

Learning involves a substantial amount of cognitive, social and emotional states. Therefore, recognizing and understanding these states in the context of learning is key in designing informed interventions and addressing the needs of the individual student to provide personalized education. In this paper, we explore the automatic detection of learner’s nonverbal behaviors involving hand-over-face gestures, head and eye movements and emotions via facial expressions during learning. The proposed computer vision-based behavior monitoring method uses a low-cost webcam and can easily be integrated with modern tutoring technologies. We investigate these behaviors in-depth over time in a classroom session of 40 minutes involving reading and problem-solving exercises. The exercises in the sessions are divided into three categories: an easy, medium and difficult topic within the context of undergraduate computer science. We found that there is a significant increase in head and eye movements as time progresses, as well as with the increase of difficulty level. We demonstrated that there is a considerable occurrence of hand-over-face gestures (on average 21.35%) during the 40 minutes session and is unexplored in the education domain. We propose a novel deep learning approach for automatic detection of hand-over-face gestures in images with a classification accuracy of 86.87%. There is a prominent increase in hand-over-face gestures when the difficulty level of the given exercise increases. The hand-over-face gestures occur more frequently during problem-solving (easy 23.79%, medium 19.84% and difficult 30.46%) exercises in comparison to reading (easy 16.20%, medium 20.06% and difficult 20.18%).

3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision.

The design of markerless systems to reconstruct human motion in a timely, unobtrusive and externally valid manner is still an open challenge. Artificial intelligence algorithms based on automatic landmarks identification on video images opened to a new approach, potentially e-viable with low-cost hardware. OpenPose is a library that t using a two-branch convolutional neural network allows for the recognition of skeletons in the scene. Although OpenPose-based solutions are spreading, their metrological performances relative to video setup are still largely unexplored. This paper aimed at validating a two-cameras OpenPose-based markerless system for gait analysis, considering its accuracy relative to three factors: cameras' relative distance, gait direction and video resolution. Two volunteers performed a walking test within a gait analysis laboratory. A marker-based optical motion capture system was taken as a reference. Procedures involved: calibration of the stereoscopic system; acquisition of video recordings, simultaneously with the reference marker-based system; video processing within OpenPose to extract the subject's skeleton; videos synchronization; triangulation of the skeletons in the two videos to obtain the 3D coordinates of the joints. Two set of parameters were considered for the accuracy assessment: errors in trajectory reconstruction and error in selected gait space-temporal parameters (step length, swing and stance time). The lowest error in trajectories (~20 mm) was obtained with cameras 1.8 m apart, highest resolution and straight gait, and the highest (~60 mm) with the 1.0 m, low resolution and diagonal gait configuration. The OpenPose-based system tended to underestimate step length of about 1.5 cm, while no systematic biases were found for swing/stance time. Step length significantly changed according to gait direction (p = 0.008), camera distance (p = 0.020), and resolution (p < 0.001). Among stance and swing times, the lowest errors (0.02 and 0.05 s for stance and swing, respectively) were obtained with the 1 m, highest resolution and straight gait configuration. These findings confirm the feasibility of tracking kinematics and gait parameters of a single subject in a 3D space using two low-cost webcams and the OpenPose engine. In particular, the maximization of cameras distance and video resolution enabled to achieve the highest metrological performances.

A Low-Cost Vision-Based Monitoring of Computer Numerical Control (CNC) Machine Tools for Small and Medium-Sized Enterprises (SMEs).

In the new era of manufacturing with the Fourth Industrial Revolution, the smart factory is getting much attention as a solution for the factory of the future. Despite challenges in small and medium-sized enterprises (SMEs), such as short-term strategies and labor-intensive with limited resources, they have to improve productivity and stay competitive by adopting smart factory technologies. This study presents a novel monitoring approach for SMEs, KEM (keep an eye on your machine), and using a low-cost vision, such as a webcam and open-source technologies. Mainly, this idea focuses on collecting and processing operational data using cheaper and easy-to-use components. A prototype was tested with the typical 3-axis computer numerical control (CNC) milling machine. From the evaluation, availability of using a low-cost webcam and open-source technologies for monitoring of machine tools was confirmed. The results revealed that the proposed system is easy to integrate and can be conveniently applied to legacy machine tools on the shop floor without a significant change of equipment and cost barrier, which is less than $500 USD. These benefits could lead to a change of monitoring operations to reduce time in operation, energy consumption, and environmental impact for the sustainable production of SMEs.

Object acquisition and selection using automatic scanning and eye blinks in an HCI system

This paper presents an object acquisition and selection approach in human computer interaction systems. In this approach, objects placed over computer screen are automatically scanned and the user performs voluntary eye blinks for object selection when the focus comes over the object of interest. Here, scanning means moving the focus over objects placed on the computer screen one by one and the scanning time is the time taken to move focus from one object to the next object. The user is not required to perform any physical movement, the moving part is only the eye lids. A low cost webcam and MATLAB software with computer vision toolbox are required to implement the proposed approach. The performance of the proposed approach has been compared with the Camera Mouse for selection of text and graphic objects. The Camera Mouse utilizes facial feature tracking for mouse cursor control and dwell time for object selection. Three experiments were performed for evaluation of the proposed method in which ten healthy users voluntarily participated. The proposed method has given significantly better performance than the Camera Mouse when selection of text objects was performed in an html file. For selection of graphic objects placed on computer screen, where page scrolling is not required, no significant difference has been found in the performance of both the systems. The proposed method has also been evaluated for performing mouse analogous operations using eye blinks and a performance comparison has been made with state-of-the-art methods.

Reliability of a low-cost webcam recording system for three-dimensional lower limb gait analysis

ABSTRACT The purposes of this study were to develop and evaluate the test-retest reliability of a specific low-cost three-dimensional webcam recording system (3D-WCRS) and compare its reliability to a standard motion analysis system. Twenty healthy volunteers comprised of 5 males and 15 females with a mean age of 22.90 years and mean BMI of 22.72 kg/m2 were investigated for angles of hip, knee and ankle joints in three planes while walking at a self-selected speed. Intraclass correlation coefficients (ICCs) were used to evaluate as well as compare the test-retest reliability of the 3D-WCRS and standard motion analysis system. Standard error of measurement (SEM) was also analyzed for the purposes of the study. The results exhibited excellent test-retest reliability for the 3D-WCRS (ICCs ranged between 0.93 and 0.99, p = 0.001) in the three joints and planes. The standard motion analysis system demonstrated excellent reliability for all joints and planes (ICCs ranged between 0.99 and 1.00, p = 0.001). Minimal SEM values were observed in both the 3D-WCRS and standard motion analysis systems. Therefore, the developed low-cost 3D-WCRS exhibits good to excellent test-retest reliability. The test-retest reliability of the 3D-WCRS is likely to be comparable to a standard motion analysis system.

Modelling and control of weld height reinforcement in the GMAW process

In a recent research related to technological application in monitoring and control of welding, the use of various approaches to improve the productivity and quality through the development of techniques and automatic control systems is observed. This paper presents a methodology for modelling and control of the weld height reinforcement, allowing adjusting the process parameters, which can be implemented in the field of research or, in certain cases, in the industrial sector as an approach to the control of weld bead geometries. This was developed for the geometric characteristic studied in an integrated system of images acquisition, modelling and control system. The weld bead formation is measured through an acquisition system and processing of images using a low-cost web cam. The wire feed speed, the welding speed and voltage are predicted using an artificial neural network based on the desired weld height reinforcement. The control system is based on fuzzy logic.

Reliable sagittal plane kinematic gait assessments are feasible using low-cost webcam technology

Three-dimensional (3-D) motion capture systems are commonly used for gait analysis because they provide reliable and accurate measurements. However, the downside of this approach is that it is expensive and requires technical expertise; thus making it less feasible in the clinic. To address this limitation, we recently developed and validated (using a high-precision walking robot) a low-cost, two-dimensional (2-D) real-time motion tracking approach using a simple webcam and LabVIEW Vision Assistant. The purpose of this study was to establish the repeatability and minimal detectable change values of hip and knee sagittal plane gait kinematics recorded using this system. Twenty-one healthy subjects underwent two kinematic assessments while walking on a treadmill at a range of gait velocities. Intraclass correlation coefficients (ICC) and minimal detectable change (MDC) values were calculated for commonly used hip and knee kinematic parameters to demonstrate the reliability of the system. Additionally, Bland-Altman plots were generated to examine the agreement between the measurements recorded on two different days. The system demonstrated good to excellent reliability (ICC>0.75) for all the gait parameters tested on this study. The MDC values were typically low (<5°) for most of the parameters. The Bland-Altman plots indicated that there was no systematic error or bias in kinematic measurements and showed good agreement between measurements obtained on two different days. These results indicate that kinematic gait assessments using webcam technology can be reliably used for clinical and research purposes.

Automatic Selection of Webcam Photoplethysmographic Pixels Based on Lightness Criteria

We propose, in this study, an original method that was developed to remotely measure the instantaneous pulse rate using photoplethysmographic signals that were recorded from a low-cost webcam. The method is based on a prior selection of pixels of interest using a custom segmentation that used the face lightness distribution to define different sub-regions. The most relevant sub-regions are automatically selected and combined by evaluating their respective signal to noise ratio. Performances of the proposed technique were evaluated using an approved contact sensor on a set of seven healthy subjects. Different experiments while reading, with motion or while performing common tasks on a computer were conducted in the laboratory. The proposed segmentation technique was compared with other benchmark methods that were already introduced in the scientific literature. The results exhibit high degrees of correlation and low pulse rate absolute errors, demonstrating that the segmentation we propose in this study outperform available region-of-interest selection methods.

Peripheral vasomotor activity assessment using a continuous wavelet analysis on webcam photoplethysmographic signals.

Vasoconstriction and vasodilation phenomena reflect the relative changes in the vascular bed. They induce particular modifications in the pulse wave magnitude. Webcams correspond to remote sensors that can be employed to measure the pulse wave in order to compute the pulse frequency. Record and analyze pulse wave signal with a low-cost webcam to extract the amplitude information and assess the vasomotor activity of the participant. Photoplethysmographic signals obtained from a webcam are analyzed through a continuous wavelet transform. The performance of the proposed filtering technique was evaluated using approved contact probes on a set of 12 healthy subjects after they perform a short but intense physical exercise. During the rest period, a cutaneous vasodilation is observable. High degrees of correlation between the webcam and a reference sensor were obtained. Webcams are low-cost and non-contact devices that can be used to reliably estimate both heart rate and peripheral vasomotor activity, notably during physical exertion.

Real-time motorized electrical hospital bed control with eye-gaze tracking

Patients with motor neuron disease and most terminal patients cannot use their hands or arms, and so they need another person for their all needs. However, the mental functions and memories of such patients are generally sound, and they can control their eyes. Using an eye-gaze tracking technique, we have realized a real-time system for such patients. The system controls a motorized electrical hospital bed (EHB) by eye gaze with 4 degrees of freedom, using a low-cost webcam. Contactless systems that require calibration cannot be used for EHB control. The system developed in this work does not require any calibration process and it is contactless. These properties are the most innovative part of the proposed approach. To begin, the system detects the eye region and computes the iris centers. It then tracks the centers and moves a mouse pointer on a screen with the eye gaze. The specific movements of the mouse pointer are evaluated as position changing requests and the completed movements of the mouse pointer change the EHB position electrically. The communication between the computer and the EHB is provided by a relay control card driven by Arduino Mega. The system works under day/artificial lighting conditions successfully with or without eyeglasses. The system was tested with 30 volunteers on the EHB safely and was completed with 90% success (the exceptions being people with slanted eyes).

Mouse Cursor Control System Based on Hand Gesture

The apparition on market of the low-cost webcams with, at least, satisfactory qualities open up new directions regarding the implementation of human computer interaction (HCI) interfaces. The paper presents a HCI interface for mouse cursor control. The purpose of the implemented solution is to control the mouse cursor by user hand gestures captured through a webcam. For improving the gesture recognition based on the fluctuation of illuminance levels the finger strips color detection was used. The results reveal the good behavior of the system in low light condition.

A gender classification scheme based on multi-region feature extraction and information fusion for unconstrained images

Since gender classification has been interesting in many applications, we proposed a gender classification scheme based on multi-region feature extraction and information fusion in the paper. The proposed gender classification scheme is composed of three parts: pre-processing, multi-region feature extraction, and gender classifier. Before extracting useful information from multiple regions in a facial image, face detection and face orientation correction are performed in the pre-processing. Multi-region feature extraction measures three kinds of features from eyes, internal face, and hair. Since the three kinds of features have their particular properties, a classifier based on decision-level information fusion is utilized to combine these features for gender classification. To evaluate the proposed scheme, a large number of unconstrained images containing different-size faces are captured by using a low-cost webcam and digital cameras. Experimental results show that our proposed scheme can detect facial regions and the location of eyes well. Furthermore, the accuracy of the proposed gender classification scheme is higher than 96 %. These experimental results demonstrate that the proposed scheme can deal with unconstrained images for gender classification.

Development of a hybrid mental spelling system combining SSVEP-based brain–computer interface and webcam-based eye tracking

Abstract The goal of this study was to develop a hybrid mental speller that can effectively prevent unexpected typing errors in the steady-state visual evoked potential (SSVEP)-based mental speller by simultaneously using the information of eye-gaze direction detected by a low-cost webcam without calibration. In the implemented hybrid mental speller, a character corresponding to the strongest SSVEP response was typed only when the position of the selected character coincided with the horizontal eye-gaze direction (‘left’, ‘no direction’, or ‘right’) detected by the webcam-based eye tracker. When the character detected by the SSVEP-based mental speller was located in the direction opposite the eye-gaze direction, the character was not typed at all (a beep sound was generated instead), and thus the users of the speller did not need to correct the mistyped character using a ‘BACKSPACE’ key. To verify the feasibility and usefulness of the developed hybrid mental spelling system, we conducted online experiments with ten healthy participants, each of whom was asked to type 15 English words consisting of a total of 68 characters. As a result, 16.6 typing errors could be prevented on average, demonstrating that the proposed hybrid strategy could effectively enhance the performance of the SSVEP-based mental spelling system.