Webcam Research Articles

Application of a Self-developed, Low-budget Indocyanine Green Camera in Surgical Imaging – a Single Institution’s Experiences

Introduction: Indocyanine green is a fluorescent dye, the use of which is becoming more and more widespread in different areas of surgery. Several international studies deal with the dye’s usefulness in intraoperative angiography, the localization of tumors, the more precise identification of anatomical structures, the detection of lymph nodes and lymph ducts, etc. The application of the dye is safe, but a suitable equipment park is required for its use, which entails relatively high costs.Objectives: The aim of our research is to create a detector system on a low budget, to be used safely in everyday practice and to illustrate its operation with practical examples at our own institute.Methods: By modifying a web camera, using filter lenses and special LEDs, we created a device suitable for exciting and detecting indocyanine green fluorescence. We prove its excellent versatility during the following procedures at our institute: breast tumor surgery, kidney transplantation, bowel resection, parathyroid surgery and liver tumor resection.Results: The finished camera has an LED light source with a peak wavelength of 780 nm, and the incoming light is filtered by a bandpass filter with a center wavelength of 832 nm. A low budget ($112), easy-to-use tool was created, which is suitable for taking advantage of the opportunities provided by indocyanine green.

Development of a proficiency-based training curriculum for beginners in endoscopic endonasal surgery

ObjectivePrecise endoscopic skills are critical for endoscopic endonasal surgery (EES) to ensure patient safety, as well as to succeed as a surgeon. Among these skills, basic skills such as hand-eye coordination and bimanual dexterity are the most important skills for novice surgeons. However, there is no proficiency-based training for the basic skills in EES. In this study, we aimed to develop a proficiency-based training curriculum for beginners in EES using tractable tasks and to assess the curriculum's validity and educational benefits. MethodsTraining tasks for basic surgical skills of EES were created by experts, using a low-cost and simple simulator. Subsequently, a proficiency-based training curriculum was developed through a preliminary study. The face and content validity of the curriculum was evaluated by expert surgeons. The construct validity was confirmed by comparing the scores of the medical students with those of the experts. In addition, the educational benefits of the curriculum were assessed by comparing the scores of the medical students before and after the implementation of the curriculum. ResultsA proficiency-based training curriculum using six tractable tasks was developed in the preliminary study. Replication of real surgical situation, camera navigation, and instrument operability under endoscopy were all highly appreciated (4.6, 4.6, and 4.8 out of 5 points, respectively), which indicates face validity. The content validity of the curriculum was demonstrated by the results of the questionnaire (4.4 out of 5 points). The curriculum was implemented by 12 medical students. The initial student scores were significantly lower than the expert scores (mean score: 61.9 vs. 100.0, p < 0.001), and each coefficient of variation of the student scores was higher than that of the expert scores (p < 0.01), supporting the construct validity of the curriculum. The mean student composite scores were statistically improved after the training (61.9 vs. 93.3, p < 0.0001). In addition, improvement of five other medical students’ EES skills through this training curriculum was confirmed in cadaver dissection (1.7 vs. 3.7, p < 0.0001). ConclusionThe validity of the proficiency-based training curriculum developed through the preliminary study was established based on the questionnaire of experts and the difference between student and expert scores. In addition, the educational benefits were demonstrated by the students’ learning curves. The low-cost and simple simulator seems appropriate as an entry model for beginners in EES, and this curriculum can provide rapid intervention and objective assessment of basic skills in EES.

Vote Now Application

The E-voting system offers internet voting. To stop voter fraud, we use two different levels of security. As the first level of security, a web camera records the voter's face and uploads it to the database. After that, the person's face is compared to the face in the database to verify its authenticity. The two faces are compared with the help of the Local Binary Pattern algorithm. The plan's foundation is combining two images and assigning a value to a central pixel. Those central pixels will have values of either 0 or 1. If the value is less than a pixel, a histogram of the labels is calculated and used as a descriptor.

Differentiating depression using facial expressions in a virtual avatar communication system.

Depression has a major effect on the quality of life. Thus, identifying an effective way to detect depression is important in the field of human-machine interaction. To examine whether a combination of a virtual avatar communication system and facial expression monitoring potentially classifies people as being with or without depression, this study consists of three research aims; 1) to understand the effect of different types of interviewers such as human and virtual avatars, on people with depression symptoms, 2) to clarify the effect of neutral conversation topics on facial expressions and emotions in people with depression symptoms, and 3) to compare verbal and non-verbal information between people with or without depression. In this study, twenty-seven participants-fifteen in the control group and twelve in the depression symptoms group-were recruited. They were asked to talk to a virtual avatar and human interviewers on both neutral and negative conversation topics and to score PANAS; meanwhile, facial expressions were recorded by a web camera. Facial expressions were analyzed by both manual and automatic analyses. In the manual analysis, three annotators counted gaze directions and reacting behaviors. On the other hand, automatic facial expression detection was conducted using OpenFace. The results of PANAS suggested that there was no significance between different interviewers' types. Furthermore, in the control group, the frequency of look-downward was larger in negative conversation topics than in neutral conversation topics. The intensity of Dimpler was larger in the control group than in the depression symptoms group. Moreover, the intensity of Chin Raiser was larger in neutral conversation topics than in negative conversation topics in the depression symptoms group. However, in the control groups, there was no significance in the types of conversation topics. In conclusion, 1) there was no significance between human and virtual avatar interviewers in emotions, facial expressions, and eye gaze patterns, 2) neutral conversation topics induced less negative emotion in both the control and depression symptoms group, and 3) different facial expressions' patterns between people with, or without depression, were observed in the virtual avatar communication system.

A novel hybrid biometric software application for facial recognition considering uncontrollable environmental conditions

Human face recognition research in biometric applications is a popular research subject because of its various applications and challenges, including camera type, pose, light or illumination, resolutions, wearing or not wearing glasses, and expressions, among others. This study presents a novel hybrid biometric software application for facial recognition considering uncontrollable environmental conditions. The proposed system uses two features namely, Laplace of Gaussian filter-based Discrete Wavelet Transform (LGDWT) and Discrete Cosine Transform Compressed based Log Gabor Filter (DCTLGF). The combined LGDWT and DCTLGF features were used by a Multiclass Support Vector Machine (MSVM) classifier to create the desired class label of individual faces. Our work was tested on a face dataset comprising 25 people of 200 face images which are taken using a five-megapixel low-goal web camera and yielded good results in different bounds in contrast to existing techniques.

Rotation Invariant Graph Neural Network for 3D Point Clouds

In this paper we propose a novel rotation normalization technique for point cloud processing using an oriented bounding box. We use this method to create a point cloud annotation tool for part segmentation on real camera data. Custom data sets are used to train our network for classification and part segmentation tasks. Successful deployment is completed on an embedded device with limited processing power. A comparison is made with other rotation-invariant features in noisy synthetic datasets. Our method offers more auxiliary information related to the dimension, position, and orientation of the object than previous methods while performing at a similar level.

Exploring an Affective and Responsive Virtual Environment to Improve Remote Learning

Online classes are typically conducted by using video conferencing software such as Zoom, Microsoft Teams, and Google Meet. Research has identified drawbacks of online learning, such as “Zoom fatigue”, characterized by distractions and lack of engagement. This study presents the CUNY Affective and Responsive Virtual Environment (CARVE) Hub, a novel virtual reality hub that uses a facial emotion classification model to generate emojis for affective and informal responsive interaction in a 3D virtual classroom setting. A web-based machine learning model is employed for facial emotion classification, enabling students to communicate four basic emotions live through automated web camera capture in a virtual classroom without activating their cameras. The experiment is conducted in undergraduate classes on both Zoom and CARVE, and the results of a survey indicate that students have a positive perception of interactions in the proposed virtual classroom compared with Zoom. Correlations between automated emojis and interactions are also observed. This study discusses potential explanations for the improved interactions, including a decrease in pressure on students when they are not showing faces. In addition, video panels in traditional remote classrooms may be useful for communication but not for interaction. Students favor features in virtual reality, such as spatial audio and the ability to move around, with collaboration being identified as the most helpful feature.

Design and implementation monitoring robotic system based on you only look once model using deep learning technique

&lt;span lang="EN-US"&gt;The need for robotics systems has become an urgent necessity in various fields, especially in video surveillance and live broadcasting systems. The main goal of this work is to design and implement a rover robotic monitoring system based on raspberry pi 4 model B to control this overall system and display a live video by using a webcam (USB camera) as well as using you only look once algorithm-version five (YOLOv5) to detect, recognize and display objects in real-time. This deep learning algorithm is highly accurate and fast and is implemented by Python, OpenCV, PyTorch codes and the Context Object Detection Task (COCO) 2020 dataset. This robot can move in all directions and in different places especially in undesirable places to transmit live video with a moving camera and process it by the YOLOv5 model. Also, the robot system can receive images, videos, or YouTube links and process them with YOLOv5. Raspberry Pi is controlled remotely by connecting to the network through Wi-Fi locally or publicly using the internet with a remote desktop connection application. The results were very satisfactory and proved the high-performance efficiency of the robot.&lt;/span&gt;

Detection Method of River Floating Objects based on Edge Computing

For the detection of objects floating in the river, most of the traditional intelligent video monitoring methods are used to monitor through manual viewing. During the traditional video data transmission and processing, the massive amount of data will put pressure on the background such as network and computing power. The new effective method is to crop and optimize YOLOV5s, add a specific image pre-processing module, and deploy it by edge computing, embed a SOC (System on Chip) chip in the web camera for real-time processing of video data. Simulation experiments prove that the above method can improve the accuracy and speed of the target detection network.

Attendance Monitoring System Design Based on Face Segmentation and Recognition

Aim: The proposed work aim was to monitor real-time attendance using face recognition in every institutional sector. It is one of the key concerns in every organization. Background: Nowadays, most organizations spend a lot of time marking attendance for a large number of individuals manually. Many technologies like Radio Frequency Identification (RFID) and biometric systems are introduced to overcome the manual attendance system. when using these RFID and biometric people need to stand in queue to make their presence. Objective: The main objective of the system is to provide an automated attendance system with the help of face recognition owing to the difficulty in the manual as well as other traditional attendance systems. Methods: The proposed work was done through face recognition using Machine Learning. Face recognition is a part of biometric characteristic of a human. It was easy to process than other biometric measurements like fingerprint, iris scan, hand scan, retina scan. The Haarcascade classifier will detect a face, and the LBPH algorithm will recognize the face. The experiment performs on the creation of real-time face data. Results: Using the web camera connected to the computer, face detection and recognition are performed, and recognized faces mark as attended. Here, the admin module and teacher modules are implemented with different functionalities to monitor attendance. Conclusion: Experiment results get 94.5% accuracy in face detection and 98.5% accuracy in face recognition by using the Haarcascade classifier and LBPH algorithm. This application system will be simple to implement, accurate, and efficient in monitoring attendance in real-time.

Improved Real-Time Monocular SLAM Using Semantic Segmentation on Selective Frames

Monocular simultaneous localization and mapping (SLAM) is emerging in advanced driver assistance systems and autonomous driving, because a single camera is cheap and easy to install. Conventional monocular SLAM has two major challenges leading inaccurate localization and mapping. First, it is challenging to estimate scales in localization and mapping. Second, conventional monocular SLAM uses inappropriate mapping factors such as dynamic objects and low-parallax areas in mapping. This paper proposes an improved real-time monocular SLAM that resolves the aforementioned challenges by efficiently using deep learning-based semantic segmentation. To achieve the real-time execution of the proposed method, we apply semantic segmentation only to downsampled keyframes in parallel with mapping processes. In addition, the proposed method corrects scales of camera poses and three-dimensional (3D) points, using estimated ground plane from road-labeled 3D points and the real camera height. The proposed method also removes inappropriate corner features labeled as moving objects and low parallax areas. Experiments with eight video sequences demonstrate that the proposed monocular SLAM system achieves significantly improved and comparable trajectory tracking accuracy, compared to existing state-of-the-art monocular and stereo SLAM systems, respectively. The proposed system can achieve real-time tracking on a standard CPU potentially with a standard GPU support, whereas existing segmentation-aided monocular SLAM does not.

Development of an Interactive Touchless Technology Based on Static-Electricity-Induced Luminescence

Touchless technology has garnered significant interest in recent years because of its effectiveness in combating infectious diseases such as the novel coronavirus (COVID-19). The goal of this study was to develop an inexpensive and high-precision touchless technology. A base substrate was coated with a luminescent material that emitted static-electricity-induced luminescence (SEL), and it was applied at high voltage. An inexpensive web camera was used to verify the relationship between the non-contact distance to a needle and the applied-voltage-triggered luminescence. The SEL was emitted at 20-200 mm from the luminescent device upon voltage application, and the web camera detected the SEL position with an accuracy of less than 1 mm. We used this developed touchless technology to demonstrate a highly accurate real-time detection of the position of a human finger based on SEL.

Gesture Recognition Based Virtual Mouse and Keyboard

Today, computer vision has advanced to the point where a machine can recognise its owner by running a straight forward picture processing application. People are employing this in this period of growth because vision in a variety of everyday activities, including Face Recognition, Color Detection, Autopilot Cars, etc. In this project, computer vision, or a camera, is utilised to recognise hand gestures to create a virtual mouse and keyboard. The computer's camera will scan the image of various hand gestures made by a user, and the computer's mouse cursor will move and even carry out various functions using various movements in accordance with the gestures. Similar to how some other motions can be used to access keyboard features.Without a wire or other external devices, it will function as a virtual mouse and keyboard. The project's webcam is its sole piece of hardware, and Python is used to code on the Anaconda platform. This project uses cutting-edge Machine Learning and Computer Vision algorithms to recognise hand movements, and it operates without the need for any additional hardware. Index Terms - Face Recognition, Machine Learning, Virtual Mouse And Keyboard, Anaconda Platform

Domain randomisation and CNN-based keypoint-regressing pose initialisation for relative navigation with uncooperative finite-symmetric spacecraft targets using monocular camera images

Vision-based relative navigation technology is a key enabler of several areas of the space industry such as on-orbit servicing, space debris removal, and formation flying. A particularly demanding scenario is navigating relative to a non-cooperative target that does not offer any navigational aid and is unable to stabilise its attitude. This research integrates a convolutional neural network (CNN) and an EPnP-solver in a pose initialisation system. The system’s performance is benchmarked on images gathered from the European Proximity Operations Simulator EPOS 2.0 laboratory. A synthetic dataset is generated using Blender as a rendering engine. A segmentation-based pose estimation CNN is trained using the synthetic dataset and the resulting pose estimation performance is evaluated on a set of real images gathered from the cameras of the EPOS 2.0 robotic close-range relative navigation laboratory. It is demonstrated that a synthetic-image-trained CNN-based pose estimation pipeline is able to successfully perform in a close-range visual relative navigation setting on real camera images of a 6-facet symmetrical spacecraft.

Effective Detection of the Machinability of Stainless Steel from the Aspect of the Roughness of the Machined Surface

Reliable measurement of surface roughness (Ra) is extremely important for quality control of production processes. The cost of the equipment and the duration of the measurement process are very high. The aim of this work is to develop a device for non-destructive measurement of specific roughness levels on stainless steel using computer vision. The device should be structurally simple, affordable, accurate, and safe for practical use. The purpose of the device is to effectively detect the level of roughness of the treated surface obtained by the water jet cutting process. On the basis of the obtained results, it is possible to adjust the parameters during the cutting process. The principle of operation of the device is based on measuring the intensity of the visible spectrum of the light reflected from the surface of the sample to be measured and correlating these values with the values of the measured roughness. After testing several variants of the device, the so-called vertical measurement variant was developed using the following equipment: violet light LED, optical filter and light splitter, USB 2.0 web camera, Arduino microcontroller, personal computer, and LabView programming interface.

2D Camera-Based Air-Writing Recognition Using Hand Pose Estimation and Hybrid Deep Learning Model

Air-writing is a modern human–computer interaction technology that allows participants to write words or letters with finger or hand movements in free space in a simple and intuitive manner. Air-writing recognition is a particular case of gesture recognition in which gestures can be matched to write characters and digits in the air. Air-written characters show extensive variations depending on the various writing styles of participants and their speed of articulation, which presents quite a difficult task for effective character recognition. In order to solve these difficulties, this current work proposes an air-writing system using a web camera. The proposed system consists of two parts: alphabetic recognition and digit recognition. In order to assess our proposed system, two character datasets were used: an alphabetic dataset and a numeric dataset. We collected samples from 17 participants and asked each participant to write alphabetic characters (A to Z) and numeric digits (0 to 9) about 5–10 times. At the same time, we recorded the position of the fingertips using MediaPipe. As a result, we collected 3166 samples for the alphabetic dataset and 1212 samples for the digit dataset. First, we preprocessed the dataset and then created two datasets: image data and padding sequential data. The image data were fed into the convolution neural networks (CNN) model, whereas the sequential data were fed into bidirectional long short-term memory (BiLSTM). After that, we combined these two models and trained again with 5-fold cross-validation in order to increase the character recognition accuracy. In this work, this combined model is referred to as a hybrid deep learning model. Finally, the experimental results showed that our proposed system achieved an alphabet recognition accuracy of 99.3% and a digit recognition accuracy of 99.5%. We also validated our proposed system using another publicly available 6DMG dataset. Our proposed system provided better recognition accuracy compared to the existing system.

Educational Innovation Faced with COVID-19: Deep Learning for Online Exam Cheating Detection

Because the COVID-19 epidemic has limited human activities, it has touched almost every sector. Education is one of the most affected areas. To prevent physical touch between students, schools and campuses must adapt their complete learning system to an online environment. The difficulty with this technique arises when the teachers or lecturers administer exams. It is difficult to oversee pupils one by one online. This research proposes the development of a computer program to aid in this effort. By applying deep learning models, this program can detect a person’s activities during an online exam based on a web camera. The reliability of this system is 84.52% based on the parameter F1-score. This study built an Indonesian-language web-based application. Teachers and lecturers in Indonesia can use this tool to evaluate whether students are cheating on online exams. Unquestionably, this application is a tool that may be utilized to develop distance learning educational technology in Indonesia.

Digital Performance and Its Discontents (or, Problems of Presence in Pandemic Performance)

For at least thirty years, scholars have debated the centrality of physical and embodied presence as essential to the experience of theatre and performance. A debate that was perhaps largely academic suddenly became a shared reality when COVID-19 shuttered live venues, closed universities and moved artists of all kinds online. Suddenly, much of the theatre world began acting for the (web)camera. This article considers the changing attitudes toward notions of theatrical presence before and throughout the pandemic and what these changes might mean for the future of live arts. In particular, it posits the intersection of theatre, film and media history as key to understanding digitally enhanced contemporary experiences and expectations of post-pandemic performances.

A Solution of Pattern Recognition Based on Deep Learning for Robotic Manipulator to Pick Up and Drop Objects

This article presents the pattern recognition solution based on two deep learning modules supporting the robotic manipulator to pick up and drop objects. The first deep learning module performs image processing to recognize identified objects. The second module is used to train object pick and drop tasks based on the recognition results of the first module. To check the feasibility of the proposed pattern recognition solution, several tests are performed on a real robot arm model with 6 degrees of freedom with the constraints of joint variables limited from -170 degrees to 170 degrees. After performing 84 tests in more than 8 hours on GeForce RTX 3080 GPU with changes in object features (shape and colour), pick up or drop location, the statistical results show that the robot can be done exactly as required with up to 94% accuracy with a low-cost USB camera.

Determination and use of external color filter for improved camera colorimetry

Most digital cameras have spectral characteristics that make it impossible for the camera to provide accurate colorimetric information of a recorded scene. We extend previous work on using multiple filtered exposures by the use of a newly available capability of creating manufacturable filters using online software. Multiple criteria are considered in the determination of an optimal color filter on a training dataset. These criteria include the Vora-Value, figure of merit (FOM), training average ΔE, and training maximum ΔE. Our method includes the use of a realistic imaging noise model. We use two optimization methods: minimum mean square error (MMSE), which requires knowledge of the color sensitivities of the camera’s internal RGB filters and knowledge of the covariance matrices of the data ensemble and the noise, and a pseudoinverse method that uses only the recorded data and knowledge of the values of the training data. Filters were selected using the training data and the performance is measured on independent testing data. The results from both optimization methods show that filters chosen using the training ΔE criteria consistently outperformed the theoretical FOMs. It is also shown that the MMSE method is very sensitive to small perturbations of the internal filter sensitivities. Real camera images were recorded using the optimally determined filters from both methods. Measured errors provide confirmation of the simulated results.