Computer Vision Techniques Research Articles

The potential of short-wave infrared hyperspectral imaging and deep learning for dietary assessment: a prototype on predicting closed sandwiches fillings

IntroductionAccurate measurement of dietary intake without interfering in natural eating habits is a long-standing problem in nutritional epidemiology. We explored the applicability of hyperspectral imaging and machine learning for dietary assessment of home-prepared meals, by building a proof-of-concept, which automatically detects food ingredients inside closed sandwiches.MethodsIndividual spectra were selected from 24 hyperspectral images of assembled closed sandwiches, measured in a spectral range of 1116.14 nm to 1670.62 nm over 108 bands, pre-processed with Standard Normal Variate filtering, derivatives, and subsampling, and fed into multiple algorithms, among which PLS-DA, multiple classifiers, and a simple neural network.ResultsThe resulting best performing models had an accuracy score of ~80% for predicting type of bread, ~60% for butter, and ~ 28% for filling type. We see that the main struggle in predicting the fillings lies with the spreadable fillings, meaning the model may be focusing on structural aspects and not nutritional composition.DiscussionFurther analysis on non-homogeneous mixed food items, using computer vision techniques, will contribute toward a generalizable system. While there are still significant technical challenges to overcome before such a system can be routinely implemented in studies of free-living subjects, we believe it holds promise as a future tool for nutrition research and population intake monitoring.

Fault Detection in Induction Machines Using Learning Models and Fourier Spectrum Image Analysis

Induction motors are essential components in industry due to their efficiency and cost-effectiveness. This study presents an innovative methodology for automatic fault detection by analyzing images generated from the Fourier spectra of current signals using deep learning techniques. A new preprocessing technique incorporating a distinctive background to enhance spectral feature learning is proposed, enabling the detection of four types of faults: healthy motor coupled to a generator with a broken bar (HGB), broken rotor bar (BRB), race bearing fault (RBF), and bearing ball fault (BBF). The dataset was generated from three-phase signals of an induction motor controlled by a Direct Torque Controller under various operating conditions (20–1500 rpm with 0–100% load), resulting in 4251 images. The model, based on a Visual Geometry Group (VGG) architecture with 19 layers, achieved an overall accuracy of 98%, with specific accuracies of 99% for RAF, 100% for BRB, 100% for RBF, and 95% for BBF. A new model interpretability was assessed using explainability techniques, which allowed for the identification of specific learning patterns. This analysis introduces a new approach by demonstrating how different convolutional blocks capture particular features: the first convolutional block captures signal shape, while the second identifies background features. Additionally, distinct convolutional layers were associated with each fault type: layer 9 for RAF, layer 13 for BRB, layer 16 for RBF, and layer 14 for BBF. This methodology offers a scalable solution for predictive maintenance in induction motors, effectively combining signal processing, computer vision, and explainability techniques.

Patch-Wise Deep Learning Method for Intracranial Stenosis and Aneurysm Detection-the Tromsø Study

Intracranial atherosclerotic stenosis (ICAS) and intracranial aneurysms are prevalent conditions in the cerebrovascular system. ICAS causes a narrowing of the arterial lumen, thereby restricting blood flow, while aneurysms involve the ballooning of blood vessels. Both conditions can lead to severe outcomes, such as stroke or vessel rupture, which can be fatal. Early detection is crucial for effective intervention. In this study, we introduced a method that combines classical computer vision techniques with deep learning to detect intracranial aneurysms and ICAS in time-of-flight magnetic resonance angiography images. The process began with skull-stripping, followed by an affine transformation to align the images to a common atlas space. We then focused on the region of interest, including the circle of Willis, by cropping the relevant area. A segmentation algorithm was used to isolate the arteries, after which a patch-wise residual neural network was applied across the image. A voting mechanism was then employed to identify the presence of atrophies. Our method achieved accuracies of 76.5% for aneurysms and 82.4% for ICAS. Notably, when occlusions were not considered, the accuracy for ICAS detection improved to 85.7%. While the algorithm performed well for localized pathological findings, it was less effective at detecting occlusions, which involved long-range dependencies in the MRIs. This limitation was due to the architectural design of the patch-wise deep learning approach. Regardless, this can, in the future, be mitigated in a multi-scale patch-wise algorithm.

AtOMICS: A deep learning-based Automated Optomechanical Intelligent Coupling System for testing and characterization of silicon photonics chiplets

Abstract Recent advances in silicon photonics promise to revolutionize modern technology by improving performance of everyday devices in multiple fields [1]. However, as the industry moves into a mass fabrication phase, the problem of effective testing of integrated silicon photonics devices remains to be solved. A cost-efficient manner that reduces schedule risk needs to involve automated testing of multiple devices that share common characteristics such as input-output coupling mechanisms, but at the same time needs to be generalizable to multiple types of devices and scenarios. In this paper we present a neural network-based automated system designed for in-plane fiber-chip-fiber testing, characterization, and active alignment of silicon photonic devices that use process-design-kit library edge couplers. The presented approach combines state-of-the-art computer vision techniques with time-series analysis, in order to control a testing setup that can process multiple devices and can be easily tuned to incorporate additional hardware. The system can operate at vacuum or atmospheric pressures and maintains stability for fairly long time periods in excess of a month.

In-situ quality monitoring during embedded bioprinting using integrated microscopy and classical computer vision.

Despite significant advances in bioprinting technology, current hardware platforms lack the capability for process monitoring and quality control. This limitation hampers the translation of the technology into industrial GMP-compliant manufacturing settings. As a key step towards a solution, we developed a novel bioprinting platform integrating a high-resolution camera for in-situ monitoring of extrusion outcomes during embedded bioprinting. Leveraging classical computer vision and image analysis techniques, we then created a custom software module for assessing print quality. This module enables quantitative comparison of printer outputs to input points of the CAD model's 2D projections, measuring area and positional accuracy. To showcase the platform's capabilities, we then investigated compatibility with various bioinks, dyes, and support bath materials for both 2D and 3D print path trajectories. In addition, we performed a detailed study on how the rheological properties of granular support hydrogels impact print quality during embedded bioprinting, illustrating a practical application of the platform. Our results demonstrated that lower viscosity, faster thixotropy recovery, and smaller particle sizes significantly enhance print fidelity. This novel bioprinting platform, equipped with integrated process monitoring, holds great potential for establishing auditable and more reproducible biofabrication processes for industrial applications.

A Comparative Analysis of YOLOv9, YOLOv10, YOLOv11 for Smoke and Fire Detection

Forest fires cause extensive environmental damage, making early detection crucial for protecting both nature and communities. Advanced computer vision techniques can be used to detect smoke and fire. However, accurate detection of smoke and fire in forests is challenging due to different factors such as different smoke shapes, changing light, and similarity of smoke with other smoke-like elements such as clouds. This study explores recent YOLO (You Only Look Once) deep-learning object detection models YOLOv9, YOLOv10, and YOLOv11 for detecting smoke and fire in forest environments. The evaluation focuses on key performance metrics, including precision, recall, F1-score, and mean average precision (mAP), and utilizes two benchmark datasets featuring diverse instances of fire and smoke across different environments. The findings highlight the effectiveness of the small version models of YOLO (YOLOv9t, YOLOv10n, and YOLOv11n) in fire and smoke detection tasks. Among these, YOLOv11n demonstrated the highest performance, achieving a precision of 0.845, a recall of 0.801, a mAP@50 of 0.859, and a mAP@50-95 of 0.558. YOLOv11 versions (YOLOv11n and YOLOv11x) were evaluated and compared against several studies that employed the same datasets. The results show that YOLOv11x delivers promising performance compared to other YOLO variants and models.

Real-Time Driver Drowsiness Detection System Using Machine Learning

Driver fatigue is a leading cause of road accidents globally, with significant fatalities and injuries reported annually. This paper presents a real-time Driver Drowsiness Detection System leveraging computer vision and machine learning techniques. Using Convolutional Neural Networks (CNNs), the system monitors key behavioural indicators, such as eye closure and yawning frequency, from live video feeds. Preprocessing techniques like grayscale conversion and normalization ensure robustness under varied conditions. Alerts are triggered immediately upon detecting drowsiness, enhancing road safety. The system’s non-intrusive design ensures scalability and affordability, making it suitable for integration into Advanced Driver Assistance Systems (ADAS). This paper also discusses challenges and potential enhancements, such as IoT integration and edge computing. Key Words: Driver Drowsiness Detection, Machine Learning, CNNs, Real-Time Monitoring, Road Safety.

Learn math through motion: a technology-enhanced embodied approach with augmented reality for geometry learning in K-12 classrooms

ABSTRACT Despite the growing attention to the technology-enhanced embodied learning approach in mathematics education and its pedagogical potential, much remains unexplored regarding its implementation in real-world classroom contexts and student engagement in embodied learning activity. Addressing this gap, we introduce an embodied learning game leveraging computer vision and Augmented Reality (AR) techniques. We implemented the embodied AR game into fourth-grade classrooms to explore its impact on students’ geometry understanding, attitudes, and engagement patterns. Our analysis revealed significant enhancements in students’ geometry understanding from pre- to post-test, whereas their attitudes remained unchanged. A multiple regression analysis on students’ in-game log and hand gesture data showed that only time spent on making-angle activities predicts students’ cognitive learning gain, suggesting that not all hand gestures were meaningful to students’ learning. In addition to demonstrating the pedagogical benefits of cost-effective AR-based embodied learning in classroom settings, our findings highlight the importance of thoughtfully designing embodied actions that are both educational and engaging.

Neural Radiance Fields (NeRF) for 3D Reconstruction of Monocular Endoscopic Video in Sinus Surgery.

To validate the use of neural radiance fields (NeRF), a state-of-the-art computer vision technique, for rapid, high-fidelity 3-dimensional (3D) reconstruction in endoscopic sinus surgery (ESS). An experimental cadaveric pilot study. Academic medical center. Complete bilateral endoscopic sinus surgery was performed on 3 cadaveric specimens, followed by postsurgical nasal endoscopy using a 0° rigid endoscope. NeRF was utilized to generate 3D reconstructions from the monocular endoscopic video feed. Reconstructions were calibrated, scaled, and then co-registered to postoperative computed tomography (CT) image sets to assess accuracy. Reconstruction error was determined by comparing ethmoid sinus measurements on NeRF reconstructions and CT image sets. NeRF-based 3D scene reconstructions were successfully generated and co-registered to corresponding CT images for 5 out of 6 cadaveric nasal cavity sides. The mean reconstruction errors and standard error of the mean (SEM) for ethmoid length and height were 0.17 (SEM 0.59) and 0.70 (SEM 0.44) mm, respectively. NeRF demonstrates significant potential for dynamic, high-fidelity 3D surgical field reconstruction in ESS, offering submillimeter accuracy comparable to postoperative CT data in cadaveric specimens. This innovative approach may ultimately augment dynamic real-time intraoperative navigation through co-registration of the 3D reconstruction with preoperative imaging to potentially reduce the risk of injury to critical structures, optimize surgical completeness and, thereby, improve surgical outcomes. Further refinement and validation in live surgical settings are necessary to fully realize its clinical utility.

From video to vital signs: a new method for contactless multichannel seismocardiography

Seismocardiography (SCG) is a technique that non-invasively measures the chest wall’s local vibrations caused by the heart’s mechanical activity. Traditionally, SCG signals have been recorded using accelerometers placed at a single location on the chest wall. This study presents an innovative, cost-effective SCG method that utilizes standard smartphone videos to capture data from multiple chest locations. The analysis of vibrations from multiple points can offer a more thorough understanding of the heart’s mechanical activity compared to signals obtained solely from a single chest location. Our approach employs computer vision and deep learning techniques to extract and improve the resolution of multichannel SCG maps obtained by video capture of chest movement. We attached a grid of patterned stickers to the chest surface and recorded videos of chest movements during different respiratory phases. Using a deep learning-based object detector and a template tracking method, we tracked the stickers across video frames and extracted the corresponding SCG signals from sticker displacements. We also developed a robust algorithm to estimate heart rate (HR) from these chest videos and identify the optimal chest location for HR estimation. The method was tested on 28 chest videos captured from 14 healthy participants. The results demonstrated that our method effectively extracted multichannel SCG maps and enhanced their resolution with a mean squared error of 0.1078 and 0.0418 for right-to-left and head-to-foot SCG signals, respectively. We observed intersubject chest vibration patterns corresponding to cardiac events including opening and closure of the heart valves. Moreover, our algorithm accurately estimated HR from 1968 SCG signals extracted from the videos compared to the gold-standard HR measured from each subject’s electrocardiogram (bias ± 1.96 SD = 0.04 ± 2.14 bpm; r = 0.99, p < 0.001). The findings from this study underscore the potential of our approach in developing a cardiac monitoring tool using a smartphone that would be widely accessible to the general public and might provide more timely detection of diseases.

DIKOApp: An AI-Based Diagnostic System for Knee Osteoarthritis.

The diagnosis of knee osteoarthritis is challenging due to its complex nature and various contributing factors. With the advancement of artificial intelligence (AI) technology, some computer vision-based methods have been developed to address this task. However, when applied in practice, these methods encounter numerous challenges. Training a powerful AI model to effectively analyze a wide range of medical images is crucial. On the other hand, collecting and accurately labeling a significant number of medical images in the real world is necessary. Specifically, when dealing with knee images from specific regions like Vietnam, certain unique biological characteristics make it difficult to utilize and trust previously published studies. To effectively address these challenges, we introduce DIKOApp, an automatic diagnostic application for knee osteoarthritis based on the DIKO framework, trained on a dataset specifically built for the Vietnamese population. This framework is designed with two stages that leverage medical knowledge and computer vision techniques. The DIKO framework leverages efficient data sampling and augmentation framework to handle medical images in the real world more effectively. When evaluated using a real-world knee image dataset from Vietnamese individuals, the DIKO model demonstrates impressive performance with an accuracy of 89.34% and an F1-score of 0.88. By utilizing the capabilities of the DIKO framework, DIKOApp shows practical and promising real-world potential, enabling doctors and healthcare service providers to diagnose pathological conditions more accurately while requiring less diagnostic time, thereby improving the lives of patients.

Motion adaptive vision-based vibration measurement and modal identification for the roof masts of a tall building

On May 18, 2021, occupants in Saige Plaza Building felt significant building motions together with its roof masts caught in obvious vibrations (May 18 vibration event), which were recorded by a surveillance camera installed on the roof of the building. This study aims to investigate the vibration characteristics of masts by processing the video data using computer vision technique. A motion adaptive vision-based vibration measurement method (M-DAVIM) is firstly proposed, focusing on dealing with the adverse motion effects of the camera itself on measuring dynamic displacements. The M-DAVIM incorporates time-domain and frequency domain correction procedures to reduce noise caused by camera self-vibration, and utilizes a CNN-based object tracking method to search objects that blurred by camera shaking. Indoor periodic vibration tests and field tests of photovoltaic panels demonstrated that M-DAVIM outperforms previous vision-based methods in accurately measuring displacements under unfavorable conditions, such as target rotation, motion blurring, and background interference. The proposed M-DAVIM was then applied to measure the dynamic displacements of the roof masts of Saige Building and identify their modal parameters (frequencies and damping ratios) based on the limited video data. A vibration component of 7.60 Hz was identified as the camera self-vibration and was effectively corrected by the M-DAVIM method. Based on finite element analysis and given the wind condition, the twin-mast might mainly experience the vortex-induced vibration at 2.12 Hz with two masts vibrating synchronously in-plane along opposite directions during May 18 to 22, 2021. This study demonstrates the robustness and effectiveness of the M-DAVIM and shows its potential application for long-term field monitoring of large-scale structures under severe outdoor environments.

Experimental study of bubble cavitation on a NACA 0015 hydrofoil by a computer vision approach

The present study aims to investigate the fundamental dynamics of cavitation through an experimental approach. The study involves developing and testing a computer vision technique to measure the critical characteristics of a cavitation state. The experimental campaign was designed to investigate the dynamics of bubble cavitation developing on a NACA 0015 hydrofoil. The hydrofoil was tested under working conditions with an angle of attack of zero and a flow velocity of V=8[m/s]. The experiments were performed for several cavitation numbers, acquiring a dataset of high-speed videos for each. These videos were analyzed using the methodology presented to extract quantitative data that described the observed cavitation state. The article describes each phase of the developed technique in detail, illustrating step by step the solutions adopted to solve the problems faced. The analyses’ results highlight the potential of this approach. Indeed, the characteristic features of the cavitation conditions were measured with high accuracy. These quantities are difficult to access using the techniques commonly used to observe cavitation. The results provide valuable insight for an in-depth understanding of the cavitation phenomenon and the detrimental effects it generates.

Intelligent Assessment Through Human Motion Recognition

Human motion recognition (HMR) has emerged as a pivotal technology across various domains, enabling intelligent assessment of performance and behavior. This survey examines advancements in HMR methodologies, including computer vision techniques, deep learning, and machine learning algorithms, highlighting how they facilitate real-time analysis and enhance decision-making processes. Additionally, address the challenges inherent in data quality, privacy concerns, and algorithmic limitations that impact the effectiveness of HMR systems. This research introduces an innovative approach employing computer vision techniques to evaluate performance and deliver instant feedback on body positioning during exercise routines. This method enables immediate self-adjustment and encouragement, even in the absence of professional supervision. Our system utilizes a versatile learning framework to analyze live expert demonstrations or recorded video content, promoting correct form and enhancing exercise outcomes

Novel Framework for Artificial Bubble Image Generation and Boundary Detection Using Superformula Regression and Computer Vision Techniques

Bubble multiphase systems are crucial in industries such as biotechnology, medicine, oil and gas, and water treatment. Optical data analysis provides critical insights into bubble characteristics, such as the shape and size, complementing physical sensor data. Existing detection techniques rely on classical computer vision algorithms and neural network models. While neural networks achieve a higher accuracy, they require extensive annotated datasets, and classical methods often struggle with complex systems due to their lower accuracy. This study proposes a novel framework to address these limitations. Using Superformula parameter regression, we introduce an advanced border detection method for accurately identifying gas inclusions and complex-shaped objects in multiphase environments. The framework also includes a new approach for generating realistic artificial bubble images based on physical flow conditions, leveraging the Superformula to create extensive, labeled datasets without manual annotation. Tested on real bubble flows in mass transfer equipment, the algorithms enable bubble classification by shape and size, enhance detection accuracy, and reduce development time for neural network solutions. This work provides a robust method for object detection and dataset generation in multiphase systems, paving the way for more precise modeling and analysis.

Enhancing Robot Autonomy: Path Planning via Hand-Drawn Sketches and Computer Vision Integration

This investigation delves into an advanced method for enhancing robot autonomy by integrating hand-drawn sketches with computer vision systems to optimize route planning effectively. The procedure starts with the capture of image data, which is then processed through multiple stages until the sketches are turned into exact coordinates that guide a robot. Canny edge detection, a specific computer vision technique, is employed to detect the edges of the drawn lines, facilitating precise and dependable navigation for autonomous robot operations. Initially, the image is subjected to noise reduction and edge enhancement before converting the sketch lines into Cartesian coordinates. This step is succeeded by point filtering and ordering to ascertain accurate path adherence by the robot, with no coordinates overlooked. Scale adjustments are also made to match digital image coordinates with the real-world setting, thereby improving the robot’s interaction based on the received visual inputs. The program has been tested in Robotic Operating System 2 (ROS) using MoveIt2 and also in RoboDK software, where it was used to verify the results. It is important to note that path planning, which focuses on determining an optimal route from start to finish, is a subset of motion planning, which also considers the dynamics and kinematics of the robot's movement along that route.

Оценка осанки школьников с применением видеоанализа движений, выделением параметров анатомических точек и критической длительности поз

This article discusses the development and implementation of an algorithm for the automatic analysis of schoolchildren's posture using video recordings. The study aims to create an effective tool for detecting posture deviations through biomechanical features, such as shoulder and head tilt angles, distances between anatomical points, and other parameters. The algorithm involves the use of the BlazePose neural network for extracting body key points, identifying irrelevant frames, and analyzing time-series data. The research methodology is based on the application of computer vision techniques and biomechanical feature analysis, followed by data visualization and automated report generation. The results demonstrate that the proposed algorithm effectively identifies posture deviations, providing visual feedback for the prevention and correction of potential disorders. The automation of the process enables large-scale monitoring of schoolchildren's posture and contributes to the prevention of chronic musculoskeletal disorders.

Optimizing Quality Control: A Comprehensive Analysis of Computer Vision Methods for Assessing Vegetables and Fruits

Efficient quality control in the agriculture sector, particularly regarding the inspection of vegetables and fruits, stands as a critical necessity in today's health-focused industry. Conventional fruit grading methods, ill-suited for large-scale production, demand an automated, non-invasive, and economically feasible substitute. Computer vision emerges as a promising avenue, leveraging image analysis and machine learning algorithms to evaluate the quality of produce. The convergence of computer vision and image processing technologies in contemporary agriculture has brought about a substantial transformation in quality assessment methodologies. This paper conducts an in-depth exploration of the amalgamation of computer vision and image processing techniques for the evaluation of agricultural produce quality. Through a comprehensive review, this scientific analysis investigates the integration of computer vision and image processing techniques in agricultural quality assessment. It scrutinizes key studies, their practical implementations, outcomes, and the research voids they reveal. Technological progressions within the agricultural domain have the potential to amplify productivity and curtail the circulation of flawed or substandard products. Moreover, this study deliberates on the forthcoming trends in computer vision technology applications, accentuating their prospective influence on the vegetables and fruits industry.

An enhanced classification system of various rice plant diseases based on multi-level handcrafted feature extraction technique.

The rice plant is one of the most significant crops in the world, and it suffers from various diseases. The traditional methods for rice disease detection are complex and time-consuming, mainly depending on the expert's experience. The explosive growth in image processing, computer vision, and deep learning techniques provides effective and innovative agriculture solutions for automatically detecting and classifying these diseases. Moreover, more information can be extracted from the input images due to different feature extraction techniques. This paper proposes a new system for detecting and classifying rice plant leaf diseases by fusing different features, including color texture with Local Binary Pattern (LBP) and color features with Color Correlogram (CC). The proposed system consists of five stages. First, input images acquire RGB images of rice plants. Second, image preprocessing applies data augmentation to solve imbalanced problems, and logarithmic transformation enhancement to handle illumination problems has been applied. Third, the features extraction stage is responsible for extracting color features using CC and color texture features using multi-level multi-channel local binary pattern (MCLBP). Fourth, the feature fusion stage provides complementary and discriminative information by concatenating the two types of features. Finally, the rice image classification stage has been applied using a one-against-all support vector machine (SVM). The proposed system has been evaluated on three benchmark datasets with six classes: Blast (BL), Bacterial Leaf Blight (BLB), Brown Spot (BS), Tungro (TU), Sheath Blight (SB), and Leaf Smut (LS) have been used. Rice Leaf Diseases First Dataset, Second Dataset, and Third Dataset achieved maximum accuracy of 99.53%, 99.4%, and 99.14%, respectively, with processing time from [Formula: see text]. Hence, the proposed system has achieved promising results compared to other state-of-the-art approaches.

ENHANCING SURVEILLANCE SECURITY: USING GENERATIVE ADVERSARIAL NETWORKS AND COMPUTER VISION TO PREVENT IDENTITY SPOOFING IN FACIAL RECOGNITION SYSTEMS

Facial recognition systems have become integral to modern security infrastructures, offering a reliable method for identity verification. However, these systems are vulnerable to spoofing attacks, where adversaries use images, videos, or 3D masks to impersonate individuals and bypass authentication. Traditional anti-spoofing methods often fail to detect sophisticated attacks, necessitating the development of more robust solutions. The primary objective of this research was to explore the potential of GANs in generating synthetic data to train advanced facial recognition models and improve their resistance to spoofing. The study aimed to evaluate the effectiveness of integrating GANs with computer vision techniques for detecting and mitigating spoofing attempts in real-time surveillance scenarios. The results indicate that GAN-based models significantly improve the ability of facial recognition systems to identify and reject spoofed identities. By generating adversarial examples, the GANs enhanced the training process, enabling the facial recognition system to learn subtle patterns indicative of spoofing, such as inconsistencies in facial texture or lighting. The integration of computer vision techniques further strengthened the model's performance, allowing it to detect spoofing attacks in various conditions, including different angles, lighting variations, and partial occlusions. The system demonstrated increased accuracy in distinguishing genuine identities from spoofed ones, with improved adaptability to real-world challenges. This study demonstrates that GANs, when combined with computer vision, offer a promising solution to enhance the security of facial recognition systems against identity spoofing. The approach improves the resilience of these systems in real-world applications by detecting more sophisticated spoofing techniques. However, limitations include the need for large, high-quality datasets to train the models effectively and the computational resources required for real-time processing.