Vision Algorithms Research Articles

Constructing a Classification Scheme - and its Consequences: A Field Study of Learning to Label Data for Computer Vision in a Hospital Intensive Care Unit

Research on data annotation for artificial intelligence (AI) has demonstrated that biases, power, and culture impact the ways that annotators apply labels to data and subsequently affect downstream AI systems. However, annotators can only apply labels that are available to them in the annotation classification scheme. Drawing on a 3-year ethnographic study of an R&D collaboration between medical and AI researchers, we argue that the construction of the classification schema itself -- decisions about what kinds of data can and cannot be collected, what activities can and cannot be detected in the data, what the possible annotation classes ought to be, and the rules by which an item ought to be classified into each class -- dramatically shape the annotation process, and through it, the AI. We draw on Bowker and Star's [9] classification theory to detail how the creation of a training data codebook for a computer vision algorithm in hospital intensive care units (ICUs) evolved from its original, clinically-driven goal of classifying complex clinical activities into a narrower goal of identifying physical objects and simpler activities in the ICU. This work reinforces how trade-offs and decisions made long before annotators begin labeling data are highly consequential to the resulting AI system.

High-fidelity pose estimation for real-time extended reality (XR) visualization for cardiac catheterization

Extended reality (XR) technologies are emerging as promising platforms for medical training and procedural guidance, particularly in complex cardiac interventions. This paper presents a high-fidelity methodology to perform real-time 3D catheter tracking and visualization during simulated cardiac interventions. A custom 3D-printed setup with mounted cameras enables biplane video capture of a catheter. A computer vision algorithm processes the biplane images in real-time to reconstruct the 3D catheter trajectory represented by any designated number of points along its length. This method accurately localizes the catheter tip within 1 mm and can reconstruct any arbitrary catheter configuration. The tracked catheter data is integrated into an interactive Unity-based scene rendered on the Meta Quest 3 headset. The visualization seamlessly combines a reconstructed 3D patient-specific heart model with the dynamically tracked catheter, creating an immersive extended reality training environment. Our experimental study, involving six participants, demonstrated that the 3D visualization provided by the proposed XR system significantly outperformed 2D visualization in terms of speed and user experience. This suggests that the XR system has the potential to enhance catheterization training by improving spatial comprehension and procedural skills. The proposed system demonstrates the potential of XR technologies to transform percutaneous cardiac interventions through improved visualization and interactivity.

RUDIE: Robust approach for underwater digital image enhancement

Processing underwater digital images is critical in ocean engineering, biology, and environmental studies, focusing on challenges such as poor lighting, image de-scattering, and color restoration. Due to environmental conditions on the sea floor, improving image contrast and clarity is essential for underwater navigation and obstacle avoidance. Particularly in turbid, low-visibility waters, we require robust computer vision techniques and algorithms. Over the past decade, various models for underwater image enrichment have been proposed to address quality and visibility issues under dynamic and natural lighting conditions. This research article aims to evaluate various image improvement methods and propose a robust model that improves image quality, addresses turbidity, and enhances color, ultimately improving obstacle avoidance in autonomous systems. The proposed model demonstrates high accuracy compared to traditional models. The result analysis indicates the proposed model produces images with greatly improved visibility and exceptional color accuracy. Furthermore, research can unlock new possibilities for underwater exploration, monitoring, and intervention by advancing the state-of-the-art models in this domain.

Advances and challenges in UAV navigation based on visual SLAM

Abstract. In recent years, unmanned aerial vehicles (UAVs) have seen extensive use in fields such as agriculture, search and rescue, commercial, and military operations, driving the demand for autonomous navigation capabilities. Though GPS is the traditional method used for navigation, it becomes problematic in harsh environments like deserts. The Visual Simultaneous Localization and Mapping technology offers a solution to enhance UAV navigation in complex environments by constructing maps and localizing the UAV simultaneously in real time. This paper presents state-of-the-art visual SLAM technology developed for UAV navigation regarding algorithms like Oriented FAST and Rotated BRIEF SLAM (ORB-SLAM) and Large-Scale Direct Monocular SLAM (LSD-SLAM), whereby their performance is also discussed, with its positives and negatives. In this regard, the latest progress and challenges in applications are reviewed and analyzed through relevant literature from the databases of PubMed, IEEE, and Google Scholar in the past five years. The novelty of this paper lies in the comprehensive evaluation of the application performance of different visual SLAM algorithms in UAV navigation and the proposal of future research directions.

Metasurface-enabled 3D imaging via local bright spot gray scale matching using the structured light dot array.

Three-dimensional (3D) imaging is widely utilized in various applications, such as light detection, autonomous vehicles, and machine vision. However, conventional 3D imaging systems often rely on bulky optical components. Metasurfaces, as next-generation optical devices, possess flexible wavefront modulation capabilities and excellent combination with computer vision algorithms. Here, we propose a large field-of-view (FOV) structured light dot array projection device based on a metasurface, covering a 2π-FOV, for projecting coded point clouds in Fourier space. We explore a local bright spot gray scale matching algorithm for depth extraction, enabling 3D imaging. This algorithm simplifies the data processing flow and optimizes depth extraction and feature matching processes through a customized region gray scale comparison. As a result, it effectively reduces computational complexity and enhances tolerance to image quality fluctuations. The proposed approach provides new possibilities for developing compact and high-performance planar 3D optical imaging devices, which will drive the advancement of fields such as computer vision and artificial intelligence.

Real-time modeling of transient crustal deformation through the quantification of uncertainty deduced from GNSS data

We propose a new method for real-time uncertainty monitoring of earthquake and volcano source models using data from the global navigation satellite system (GNSS) and explore its application concerning observation station placement. The Geospatial Information Authority of Japan operates two main types of GNSS earth observation network system (GEONET) coordinates for crustal deformation monitoring on different time scales: post-processing analysis values and real-time GEONET analysis system for rapid deformation monitoring (REGARD). REGARD uses the Markov Chain Monte Carlo (MCMC) method termed real-time automatic uncertainty estimation of a coseismic single rectangular model using GNSS data (RUNE) for single rectangular fault model estimation to handle uncertainty. Thus far, no GNSS monitoring system can automatically detect transient crustal deformation events, such as volcanic activity and earthquake swarms, on timescales of a day or less. We extended RUNE and developed a core program for a new monitoring system for earthquake and volcanic source models and their uncertainties. Our program achieved automatic and stable MCMC utilization for rectangular fault, dike, Mogi, and spheroid models by increasing the computational speed, improving search efficiency, and adjusting hyperparameters. The program automatically determines the standard deviation of the likelihood function assuming a normal distribution with weights for each observation station. The calculation time was within 15 s for 1 × 106 samples on a standard 1U server. We assessed the reliability of the developed method using synthetic and observed GNSS data from the 2015 Sakurajima volcanic event. The results were consistent with the assumed model and previous studies and indicated an advantage in automatically quantifying uncertainty in a short computation time. Based on MCMC samples, we developed a new visualization algorithm to indicate areas on a map in which the number of observation stations should be expanded. We assessed the reliability using data from the 2023 Noto Peninsula earthquake [Mj 6.5]. The results indicate that the algorithm is helpful in studying the placement of stations. The above model extensions and their application are essential to achieve a rapid quantitative understanding of disaster events near urban areas and for utilizing this information in emergency response activities.Graphical

Waste Segregator: Waste Management Prototype

Smart Waste Dustbins utilize IoT, AI, and computer vision for efficient waste management. They feature multiple compartments, sensors, and automatic lid mechanisms for real-time waste sorting. Computer vision and AI algorithms accurately identify and segregate waste. Smart Waste Dustbins monitor fill levels and offer a user-friendly interface for public engagement. Sensors include ultrasonic, moisture, and metal sensors for proximity, moisture analysis, and metal item segregation. Smart Waste Dustbins revolutionize waste management, enhancing efficiency in segregation, recycling, and collection processes, leading to reduced environmental impact and improved resource utilization for sustainable communities.

Exploring Computer Vision Algorithms to Evaluate The Quality of Espresso Crema

Exploring Computer Vision Algorithms to Evaluate The Quality of Espresso Crema

Autonomous Drone Navigation Using Computer Vision

Abstract: Autonomous drone navigation with computer vision is an innovative technology that allows drones to move through intricate surroundings without needing human control. This system uses live visual information to identify objects, steer clear of obstacles, and determine routes, improving operational safety and precision. The project's main objective is to create a visionbased navigation system that combines object detection and obstacle avoidance algorithms through deep learning methods like YOLO (You Only Look Once), alongside real-time sensor fusion. The drone uses computer vision algorithms to process aerial images and automatically changes its flight path to prevent crashes. A personalized dataset of aerial images is generated and utilized for improving object detection skills in various environments. In order to guarantee practicality in real- world situations, the system undergoes validation through simulations and field tests on different terrains, focusing on its resilience in changing environments. Improvements in navigation accuracy and obstacle detection are accomplished by implementing adaptive pathplanning and integrating multiple sensors, guaranteeing the drone's efficient operation in real-life situations. The goal of this method is to enhance the drone's ability to make decisions, minimize human mistakes, and expand its range of potential uses in areas like surveillance, agriculture, and disaster response

Biocompatible Acellular Dermal Matrix-Based Neuromorphic Device with Ultralow Voltage, Ion Channel Emulation, and Synaptic Forgetting Visualization Computation.

Neuromorphic bioelectronics aim to integrate electronics with biological systems yet encounter challenges in biocompatibility, operating voltages, power consumption, and stability. This study presents biocompatible neuromorphic devices fabricated from acellular dermal matrix (ADM) derived from porcine dermis using low-temperature supercritical CO2 extraction. The ADM preserves the natural scaffold structure of collagen and minimizes immunogenicity by eliminating cells, fats, and noncollagenous impurities, ensuring excellent biocompatibility. The ADM-based devices emulate biological ion channels with biphasic membrane current modulation, exhibiting temperature dependency and pH sensitivity. It operates at an ultralow voltage of 1 mV and demonstrates reliable synaptic modulation exceeding 4 × 104 endurance cycles. The activation voltage can be theoretically as low as 59 μV, comparable to brainwave signals with a power of merely 7 aJ/event. Furthermore, a brain-like forgetting visualization algorithm is developed, leveraging the synaptic forgetting plasticity of ADM-based devices to achieve complex computing tasks in a highly energy-efficient manner. Neuromorphic devices based on ADM not only hold potential in implantable biointerfaces due to their exceptional biocompatibility, ultralow voltage, and power but also provide a feasible way for energy-efficient computing paradigms through a synergistic hardware-software approach.

Machine Learning Based Mechanical Fault Diagnosis and Detection Methods: A Systematic Review

Abstract Mechanical fault diagnosis and detection are crucial for enhancing equipment reliability, economic efficiency, production safety, and energy conservation. In the era of Industry 4.0, artificial intelligence (AI) has emerged as a significant tool for mechanical fault diagnosis and detection, attracting considerable attention from both academia and industry. This review focuses on the application of AI techniques in mechanical fault diagnosis and detection using artificial intelligence techniques based on the existing research. It examines various AI algorithms including k-nearest neighbors, support vector machine, artificial neural network, deep learning, reinforcement learning, computer vision, and transformer algorithm integrating theoretical foundations with practical applications in industrial production. Furthermore, a comprehensive overview of these algorithms applications in mechanical fault diagnosis and detection is provided. Finally, a critical assessment highlights the advantages and limitations of these techniques, while forecasting the developmental trajectories of future intelligent diagnostic technologies based on machine learning. This review serves to bridge the gap between researchers in AI and fault diagnosis, contributing significantly to the field.

Distect: automatic sample-position tracking for X-ray experiments using computer vision algorithms.

Soft X-ray spectroscopy is an important technique for measuring the fundamental properties of materials. However, for measurements of samples in the sub-millimetre range, many experimental setups show limitations. Position drifts on the order of hundreds of micrometres during thermal stabilization of the system can last for hours of expensive beam time. To compensate for drifts, sample tracking and feedback systems must be used. However, in complex sample environments where sample access is very limited, many existing solutions cannot be applied. In this work, we apply a robust computer vision algorithm to automatically track and readjust the sample position in the dozens of micrometres range. Our approach is applied in a complex sample environment, where the sample is in an ultra-high vacuum chamber, surrounded by cooled thermal shields to reach sample temperatures down to 2.5 K and in the center ofa superconducting split coil. Our implementation allows sample-position tracking and adjustment in the vertical direction since this is the dimension where drifts occur during sample temperature change in our setup. The approach can be easily extended to 2D. The algorithm enables a factor of ten improvement in the overlap of a series of X-ray absorption spectra in a sample with a vertical size down to 70 µm. This solution can be used in a variety of experimental stations, where optical access is available and sample access by other means is reduced.

Accuracy of warm ischemia time measurement using a surgical intelligence software in partial nephrectomies: A validation study

AbstractObjectivesThe objectives of this study are to compare the accuracy of warm ischemia times (WITs) derived by a surgical artificial intelligence (AI) software to those documented in surgeon operative reports during partial nephrectomy procedures and to assess the potential of this technology in evaluating postoperative renal function.Patients and methodsA surgical AI software (Theator Inc., Palo Alto, CA) was used to capture and analyse videos of partial nephrectomies performed between October 2023 and April 2024. The platform utilized computer vision algorithms to detect clamp placement and removal, enabling precise WIT measurement. Expert‐reviewed surgical videos served as the ground truth. Platform‐derived WITs were compared to those in surgeon operative reports using paired‐sample t‐tests. Additionally, we analysed the correlation between platform‐derived WITs and postoperative creatinine levels extracted from electronic health records (EHRs) integrated via health level seven (HL7) messaging protocols.ResultsOf 64 eligible cases, 61 were included in the final analysis. Platform‐derived WITs were within 1 min of the ground truth in all procedures, within 30 s in 97%, and within 10 s in over 80%. The mean difference between platform‐derived WITs and ground truth was 8.3 s, significantly lower than the 2.45 min difference for operative reports (p < 0.001). No significant correlation was found between platform‐derived WIT and postoperative creatinine changes, aligning with the view that WIT may not independently determine postoperative renal function. Although not the primary goal of this study, significant correlations were observed between WIT, tumour size and RENAL score.ConclusionThis study demonstrates the high accuracy of a surgical intelligence platform in measuring WIT during partial nephrectomies. The findings support the use of AI‐based surgical time measurement for precise intraoperative documentation and highlight the potential of integrating these data with EHRs to advance research on surgical outcomes.

Skin feature point tracking using deep feature encodings

AbstractFacial feature tracking is a key component of imaging ballistocardiography (BCG) where accurate quantification of the displacement of facial keypoints is needed for good heart rate estimation. Skin feature tracking enables video-based quantification of motor degradation in Parkinson’s disease. While traditional computer vision algorithms like Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Lucas-Kanade method (LK) have been benchmarks due to their efficiency and accuracy, they often struggle with challenges like affine transformations and changes in illumination. In response, we propose a pipeline for feature tracking, that applies a convolutional stacked autoencoder to identify the most similar crop in an image to a reference crop containing the feature of interest. The autoencoder learns to represent image crops into deep feature encodings specific to the object category it is trained upon. We train the autoencoder on facial images and validate its ability to track skin features in general using manually labelled face and hand videos of small and large motion recorded in our lab. Our evaluation protocol is comprehensive, including quantification of errors in human annotation. The tracking errors of distinctive skin features (moles) are so small that we cannot exclude the fact that they stem from the manual labelling based on a $$\chi ^2$$ χ 2 -test. With a mean error of 0.6–3.3 pixels, our method outperformed the other methods in all but one scenario. More importantly, our method was the only one that did not diverge. We also compare our method with the latest state-of-the-art transformer for feature matching by Google—Omnimotion. Our results indicate that our method is superior at tracking different skin features under large motion conditions and that it creates better feature descriptors for tracking, matching, and image registration compared to both traditional algorithms and the latest Omnimotion.

Deep Neural Network-Based Cigarette Filter Defect Detection System with FPGA Acceleration for Online Recognition

In the cigarette manufacturing industry, machine vision and artificial intelligence algorithms have been employed to improve production efficiency by detecting product defects. However, achieving both high accuracy and real-time defect detection for cigarettes with complex patterns remains a challenge. To address these issues, this study proposes a model based on RESNET18, combined with a feature enhancement algorithm, to improve detection accuracy. Additionally, a method is designed to deploy the model on a field-programmable gate array (FPGA) with high parallel processing capabilities to achieve high-speed detection. Experimental results demonstrate that the proposed detection model achieves a detection accuracy of 95.88% on a cigarette filter defect dataset with an end-to-end detection speed of only 9.38 ms.

Autonomous Positioning for Mobile Vehicles Based on Visual‐Inertial Fusion in Challenging Dark Roadway Scenes

ABSTRACTAccurate positioning for autonomous driven underground mining vehicles (UMVs) and coal mine robots (CMRs) is indeed one of the cores in the intelligentization of coal mining. Completely different from positioning on the ground and in parking scenes, there will be great difficulties in realizing the accurate active positioning for UMVs shuttled in dark and narrow roadways. We propose an effective visual and inertial fusion positioning method for autonomous CMRs and roadheaders in challenging roadway scenarios based on the odometer‐aided inertial navigation system and visual pose estimation system. Velocity information of the odometer is adapted to restrain the error accumulation of inertial positioning based on a Kalman filter. The hybrid visual feature detection algorithm is put forward to improve the accuracy and robustness of visual observation information in a dark environment. Autonomous experiments for CMRs and roadheaders are separately performed in the narrow roadway and dark passageway to demonstrate the applicability of our localization method. The proposed approach outperforms the subsystems and existing methods in accuracy and has outstanding stability.

Real-time detection of coke particles in blast furnace operations using machine learning: Case of a steel plant in India

The efficiency of blast furnace operations in a steel plant relies on the quality of raw materials, including coke. The particle size distribution (PSD) of coke plays a vital role in ensuring the efficiency of the blast furnace. Conventional methods of measuring average particle size are time-consuming and labour-intensive leading to delayed operations. To address the issue, various studies have used different techniques for real-time measurement of coke particle size, including laser diffraction, acoustic sounding, X-ray computed tomography, and computer vision. While these methods have shown successful applications, there have been limitations in their lower prediction accuracy and detection speed. Therefore, our study proposes a practical approach that leverages the integration of computer vision algorithms to enhance accuracy and detection speed. The proposed methodology is compared with the conventional method of measuring particle size in the lab at a blast furnace. Our results reveal that the YOLOv8 algorithm outperforms the conventional method, providing efficient detection of 5.419 frames per second and with a mean absolute error of [Formula: see text]1.77 mm within the lab results. YOLOv8 can perform object segmentation providing much more precise polygon masks of the coke particles instead of simply providing rectangular dimensions using traditional object detection, as explored in previous studies. This approach enables real-time monitoring of PSD and timely alerts to the onsite team, significantly improving the efficiency of blast furnace operations.

Deep learning-powered efficient characterization and quantification of microplastics

Characterizing and quantifying microplastics (MPs) are time-consuming and labor-intensive tasks traditionally. This paper presents an artificial intelligence (AI) framework aiming to automate these tasks by integrating computer vision and deep learning techniques. The approach leverages Fourier Transform Infrared (FTIR) spectra and visual images. Primary novelties of this research involve the development of: (1) an AI framework integrating efforts of data processing, analytics, visualization, and human-computer interaction; (2) a method for transforming FTIR data into contour images; (3) data augmentation strategies for resolving data scarcity and imbalance issues; (4) deep learning models for identifying MPs; (5) computer vision algorithms for quantifying MPs; and (6) an engineer-friendly graphic user interface (GUI) for enhancing data accessibility. The AI framework has been applied to polyethylene, polypropylene, polystyrene, polyamide, ethylene-vinyl acetate, and cellulose acetate. Results confirmed the efficacy of the framework, exhibiting high accuracy scores in classification (98 %), segmentation (99 %), and quantification (96 %) tasks. This research advances the capability of automatic assessment of MPs.

Asynchronous Visual-Inertial Odometry for Event Cameras

Abstract. Event cameras are bio-inspired visual sensors that output changes in pixel-level brightness asynchronously instead of standard intensity frames at a fixed rate. These cameras offer reliable visual information in high-speed motion and high dynamic range (HDR) scenes, addressing the limitations of traditional cameras in such scenarios. Therefore, research of integrating event cameras into established visual algorithms holds significant value. In this study, based on traditional Visual-Inertial Odometry (VIO) frameworks, we proposed an innovative asynchronous monocular event-based inertial odometry method to fully exploit the benefits of event cameras. First, the corner features are extracted separately from the raw event stream and the time surface map, followed by uniform feature selection to accurately describe three-dimensional spatial geometry. Then, feature tracking is achieved by integrating these two event representation methods. In addition, our method obtains stable and high frequency state estimation by fusing event and IMU measurements through graph optimization. We validate the effectiveness of our proposed approach, comparing with several state-of-the-art EVIO systems and VIO systems.

Visual SLAM system based on dynamic semantic features

Aiming at the problem that dynamic objects (such as pedestrians, vehicles, and animals) in real scenes affect the positioning and mapping accuracy of visual SLAM (Simultaneous Localization and Mapping), a YOLOv3-ORB-SLAM3 algorithm is proposed based on ORB-SLAM3 (Oriented FAST and Rotated BRIEF-Simultaneous Localization and Mapping 3). The algorithm adds a semantic thread on the basis of ORB-SLAM3, and adopts a dual-thread mechanism for dynamic and static scene feature extraction: the semantic thread uses YOLOv3 to perform semantic recognition target detection on dynamic objects in the scene, and removes outliers from the extracted dynamic area feature points; the tracking thread extracts scene area features through ORB features, and obtains static scene features combined with semantic information and sends them to the backend, thereby eliminating the interference of dynamic scenes on the system and improving the positioning accuracy of the visual SLAM algorithm. Verified using the TUM (Technical University of Munich) dataset, the results show that the ATE (Average Treatment Effect) index of the YOLOv3-ORB-SLAM3 algorithm in the monocular mode is about 30% lower than that of the ORB-SLAM3 algorithm, and the ATE index of the dynamic sequence in the RGB-D (Red, Green and Blue-Depth) mode is reduced 10%, while the static sequence has no significant decrease.