Computer Vision Models Research Articles

Core outcomes measures in dental computer vision studies (DentalCOMS)

ObjectivesTo improve reporting and comparability as well as to reduce bias in dental computer vision studies, we aimed to develop a Core Outcome Measures Set (COMS) for this field. The COMS was derived consensus based as part of the WHO/ITU/WIPO Global Initiative AI for Health (WHO/ITU/WIPO AI4H). MethodsWe first assessed existing guidance documents of diagnostic accuracy studies and conducted interviews with experts in the field. The resulting list of outcome measures was mapped against computer vision modeling tasks, clinical fields and reporting levels. The resulting systematization focused on providing relevant outcome measures whilst retaining details for meta-research and technical replication, displaying recommendations towards (1) levels of reporting for different clinical fields and tasks, and (2) outcome measures. The COMS was consented using a 2-staged e-Delphi, with 26 participants from various IADR groups, the WHO/ITU/WIPO AI4H, ADEA and AAOMFR. ResultsWe assigned agreed levels of reporting to different computer vision tasks. We agreed that human expert assessment and diagnostic accuracy considerations are the only feasible method to achieve clinically meaningful evaluation levels. Studies should at least report on eight core outcome measures: confusion matrix, accuracy, sensitivity, specificity, precision, F-1 score, area-under-the-receiver-operating-characteristic-curve, and area-under-the-precision-recall-curve. ConclusionDental researchers should aim to report computer vision studies along the outlined COMS. Reviewers and editors may consider the defined COMS when assessing studies, and authors are recommended to justify when not employing the COMS. Clinical significanceComparing and synthesizing dental computer vision studies is hampered by the variety of reported outcome measures. Adherence to the defined COMS is expected to increase comparability across studies, enable synthesis, and reduce selective reporting.

Diabetic Retinopathy Features Segmentation without Coding Experience with Computer Vision Models YOLOv8 and YOLOv9.

Computer vision is a powerful tool in medical image analysis, supporting the early detection and classification of eye diseases. Diabetic retinopathy (DR), a severe eye disease secondary to diabetes, accompanies several early signs of eye-threatening conditions, such as microaneurysms (MAs), hemorrhages (HEMOs), and exudates (EXs), which have been widely studied and targeted as objects to be detected by computer vision models. In this work, we tested the performances of the state-of-the-art YOLOv8 and YOLOv9 architectures on DR fundus features segmentation without coding experience or a programming background. We took one hundred DR images from the public MESSIDOR database, manually labelled and prepared them for pixel segmentation, and tested the detection abilities of different model variants. We increased the diversity of the training sample by data augmentation, including tiling, flipping, and rotating the fundus images. The proposed approaches reached an acceptable mean average precision (mAP) in detecting DR lesions such as MA, HEMO, and EX, as well as a hallmark of the posterior pole of the eye, such as the optic disc. We compared our results with related works in the literature involving different neural networks. Our results are promising, but far from being ready for implementation into clinical practice. Accurate lesion detection is mandatory to ensure early and correct diagnoses. Future works will investigate lesion detection further, especially MA segmentation, with improved extraction techniques, image pre-processing, and standardized datasets.

A defect detection method for Akidzuki pears based on computer vision and deep learning

To quickly and accurately detect defects in Akidzuki (Pyrus pyrifolia Nakai) pears after harvest, this study aims to develop a method for Akidzuki pear defect detection based on computer vision and deep learning models. It mainly consists of obtaining high-quality images using an image acquisition system and proposing a new YOLO-AP detection model to identify defects in Akidzuki pears. The model uses YOLOv5 as the main architecture. The GhostDynamicConv (GDC) module is obtained by replacing the standard convolution in the Ghost module with a dynamic convolution. The C3-GhostDynamicConv (C3-GDC) module is obtained by replacing the Bottleneck module of C3 in Neck with the GDC module, simplifying the network while improving the model's accuracy. Meanwhile, Bottleneck Attention Module (BAM) is introduced after C3-GDC to refine the intermediate features. In addition, the original bounding box loss function is replaced with Wise-IoUv3 (WIoUv3) to accelerate the model convergence. The results demonstrate that YOLO-AP performs better in Akidzuki pear defect detection, with a mAP@0.5 of 0.939, a recall of 0.921, and a detection speed of 454.5 fps (2.2 ms per image). These values represent a 4.2 %, 3.5 %, and 1.4 % improvement over the baseline model. Comparing YOLO-AP with the updated YOLOv9 and other detection models, YOLO-AP is more accurate and faster. These findings demonstrated that the proposed method can detect Akidzuki pear defects in real time, efficiently and accurately, providing technical support for post-harvest defect detection.

Visual Detection of Traffic Incident through Automatic Monitoring of Vehicle Activities

Intelligent transportation systems (ITSs) derive significant advantages from advanced models like YOLOv8, which excel in predicting traffic incidents in dynamic urban environments. Roboflow plays a crucial role in organizing and preparing image data essential for computer vision models. Initially, a dataset of 1000 images is utilized for training, with an additional 500 images reserved for validation purposes. Subsequently, the Deep Simple Online and Real-time Tracking (Deep-SORT) algorithm enhances scene analyses over time, offering continuous monitoring of vehicle behavior. Following this, the YOLOv8 model is deployed to detect specific traffic incidents effectively. By combining YOLOv8 with Deep SORT, urban traffic patterns are accurately detected and analyzed with high precision. The findings demonstrate that YOLOv8 achieves an accuracy of 98.4%, significantly surpassing alternative methodologies. Moreover, the proposed approach exhibits outstanding performance in the recall (97.2%), precision (98.5%), and F1 score (95.7%), underscoring its superior capability in accurate prediction and analyses of traffic incidents with high precision and efficiency.

AI-based detection and identification of low-level nuclear waste: a comparative analysis

Ensuring environmental safety and regulatory compliance at Department of Energy (DOE) sites demands an efficient and reliable detection system for low-level nuclear waste (LLW). Unlike existing methods that rely on human effort, this paper explores the integration of computer vision algorithms to automate the identification of such waste across DOE facilities. We evaluate the effectiveness of multiple algorithms in classifying nuclear waste materials and their adaptability to newly emerging LLW. Our research introduces and implements five state-of-the-art computer vision models, each representing a different approach to the problem. Through rigorous experimentation and validation, we evaluate these algorithms based on performance, speed, and adaptability. The results reveal a noteworthy trade-off between detection performance and adaptability. YOLOv7 shows the best performance and requires the highest effort to detect new LLW. Conversely, OWL-ViT has lower performance than YOLOv7 and requires minimal effort to detect new LLW. The inference speed does not strongly correlate with performance or adaptability. These findings offer valuable insights into the strengths and limitations of current computer vision algorithms for LLW detection. Each developed model provides a specialized solution with distinct advantages and disadvantages, empowering DOE stakeholders to select the algorithm that aligns best with their specific needs.

Semi-supervised learning for efficient water leakage segmentation in tunnel infrastructure

Water leakage segmentation based on computer vision allows for efficient monitoring and maintenance, ensuring the safety and integrity of tunnel infrastructure. In this study, a semisupervised learning method with self-training via pseudo-labels is proposed to overcome the dependence of conventional computer vision models on high-quality labeled data. Strong data augmentation is first injected into the unlabeled data to establish a robust self-training baseline and differentiate similar predictions between teacher and student networks. To mitigate the accumulation of erroneous pseudo-labels and their potential impact on performance, we chose to retrain using only reliable, unlabeled images. The unlabeled images are prioritized based on their overall stability. By considering image-level contextual information, the selection process offers a more suitable approach for segmentation compared to conventional pixelwise methods. The experimental results highlight the method’s capability to enhance model performance with a limited amount of labeled data, surpassing the effectiveness of other semisupervised methods.

Human Eyes-Inspired Recurrent Neural Networks Are More Robust Against Adversarial Noises.

Humans actively observe the visual surroundings by focusing on salient objects and ignoring trivial details. However, computer vision models based on convolutional neural networks (CNN) often analyze visual input all at once through a single feedforward pass. In this study, we designed a dual-stream vision model inspired by the human brain. This model features retina-like input layers and includes two streams: one determining the next point of focus (the fixation), while the other interprets the visuals surrounding the fixation. Trained on image recognition, this model examines an image through a sequence of fixations, each time focusing on different parts, thereby progressively building a representation of the image. We evaluated this model against various benchmarks in terms of object recognition, gaze behavior, and adversarial robustness. Our findings suggest that the model can attend and gaze in ways similar to humans without being explicitly trained to mimic human attention and that the model can enhance robustness against adversarial attacks due to its retinal sampling and recurrent processing. In particular, the model can correct its perceptual errors by taking more glances, setting itself apart from all feedforward-only models. In conclusion, the interactions of retinal sampling, eye movement, and recurrent dynamics are important to human-like visual exploration and inference.

A novel approach for large-scale characterization of residential cooking-generated PM with computer vision and low-cost sensors

Cooking is one of the major sources of indoor particulate matter (PM), which poses significant health risks and is a severe health hazard. Current studies lack an economical and effective analytical framework for quantifying inhalable particles (PM10) and fine particulate matter (PM2.5) from residential cooking activities on a large scale under real-world scenarios. This study bridges this gap by employing computer vision (CV) technology and readily available sensors. We collected data over a month in real-world settings, including cooking videos and air quality data (indoor PM10, PM2.5, CO2, temperature, relative humidity, and outdoor PM10 and PM2.5 concentrations). To classify high-emission (pan-frying, stir-frying, deep-frying) and low-emission (stewing, steaming, boiling, non-cooking) activities, we developed and validated a robust CV model named “Cooking-I3D.” This model leverages a pre-trained Two-Stream Inflated 3D ConvNet (I3D) architecture. We then assessed the efficacy of the CV-predicted cooking method in PM characterization using a first-order multivariate autoregressive model, controlling for environmental factors. The Cooking-I3D model achieved exceptional performance, boasting an accuracy of 95 % and an Area Under the Curve (AUC) of 0.98. Our results indicate that a single 6-minute high-emission cooking event triggers a 21–25 % increase in indoor PM concentrations and a 23–24 % increase in the indoor/outdoor ratio, with relative errors in these estimates ranging from 10 to 21 %. This innovative method offers a powerful tool for long-term assessment of cooking-related indoor air pollution and facilitates precision exposure assessment in human health studies.

Proposal of Innovative Methods for Computer Vision Techniques in Maritime Sector

Computer vision (CV) techniques have been widely studied and applied in the shipping industry and maritime research. The existing literature has primarily focused on enhancing image recognition accuracy and precision for water surface targets by refining CV models themselves. This paper introduces innovative methods to further improve the accuracy of detection and recognition using CV models, including using ensemble learning and integrating shipping domain knowledge. Additionally, we present a novel application of CV techniques in the maritime domain, expanding the research perspective beyond the traditional focus on the accurate detection and recognition of water surface targets. Specifically, a novel solution integrating a CV model and the transfer learning method is proposed in this paper to address the challenge of relatively low-speed and high-charge internet services on ocean-going vessels, aiming to improve the online video viewing experience while conserving network resources. This paper is of importance for advancing further research and application of CV techniques in the shipping industry.

Improving Marvel Hero Classification through Dataset Curation

lores the development of a computer vision model for classifying Marvel superheroes such as Black Widow, Hulk, Iron Man, and Spider-Man. Utilizing a curated dataset sourced from Kaggle, the research emphasizes the critical role of dataset quality in refining model accuracy. Insights gained include adjustments to neural network configurations and leveraging Edge Impulse for enhanced performance. The findings highlight effective strategies for optimizing classification accuracy in complex image recognition tasks.PART 2:This part explores the application of Bayesian logistic regression to model the relationship between temperature and the probability of O-ring failure. By leveraging Bayesian inference techniques, analyzing historical data to quantify the risk associated with temperature variations and emphasize the importance of probabilistic approaches in safety-critical decision-making.The Space Shuttle Challenger disaster on January 28, 1986, remains a poignant case study in aerospace engineering failure. The investigation concluded that the failure of O-ring seals in cold temperatures led to the tragic loss of the shuttle and its crew.

RL-CWtrans Net: multimodal swimming coaching driven via robot vision.

In swimming, the posture and technique of athletes are crucial for improving performance. However, traditional swimming coaches often struggle to capture and analyze athletes' movements in real-time, which limits the effectiveness of coaching. Therefore, this paper proposes RL-CWtrans Net: a robot vision-driven multimodal swimming training system that provides precise and real-time guidance and feedback to swimmers. The system utilizes the Swin-Transformer as a computer vision model to effectively extract the motion and posture features of swimmers. Additionally, with the help of the CLIP model, the system can understand natural language instructions and descriptions related to swimming. By integrating visual and textual features, the system achieves a more comprehensive and accurate information representation. Finally, by employing reinforcement learning to train an intelligent agent, the system can provide personalized guidance and feedback based on multimodal inputs. Experimental results demonstrate significant advancements in accuracy and practicality for this multimodal robot swimming coaching system. The system is capable of capturing real-time movements and providing immediate feedback, thereby enhancing the effectiveness of swimming instruction. This technology holds promise.

Towards real-time monitoring of insect species populations

Insect biodiversity and abundance are in global decline, potentially leading to a crisis with profound ecological and economic consequences. Methods and technologies to monitor insect species to aid in preservation efforts are rapidly being developed yet their adoption has been slow and focused on specific use cases. We propose a computer vision model that works towards multi-objective insect species identification in real-time and on a large scale. We leverage an image data source with 16 million instances and a recent improvement in the YOLO computer vision architecture to present a quick and open-access method to develop visual AI models to monitor insect species across climatic regions.

A circadian behavioral analysis suite for real-time classification of daily rhythms in complex behaviors.

Measuring animal behavior over long timescales has been traditionally limited to behaviors that are easily measurable with real-time sensors. More complex behaviors have been measured over time, but these approaches are considerably more challenging due to the intensive manual effort required for scoring behaviors. Recent advances in machine learning have introduced automated behavior analysis methods, but these often overlook long-term behavioral patterns and struggle with classification in varying environmental conditions. To address this, we developed a pipeline that enables continuous, parallel recording and acquisition of animal behavior for an indefinite duration. As part of this pipeline, we applied a recent breakthrough self-supervised computer vision model to reduce training bias and overfitting and to ensure classification robustness. Our system automatically classifies animal behaviors with a performance approaching that of expert-level human labelers. Critically, classification occurs continuously, across multiple animals, and in real time. As a proof-of-concept, we used our system to record behavior from 97 mice over two weeks to test the hypothesis that sex and estrogen influence circadian rhythms in nine distinct home cage behaviors. We discovered novel sex- and estrogen-dependent differences in circadian properties of several behaviors including digging and nesting rhythms. We present a generalized version of our pipeline and novel classification model, the "circadian behavioral analysis suite," (CBAS) as a user-friendly, open-source software package that allows researchers to automatically acquire and analyze behavioral rhythms with a throughput that rivals sensor-based methods, allowing for the temporal and circadian analysis of behaviors that were previously difficult or impossible to observe.

High-resolution mapping of urban Aedes aegypti immature abundance through breeding site detection based on satellite and street view imagery

Identification of Aedes aegypti breeding hotspots is essential for the implementation of targeted vector control strategies and thus the prevention of several mosquito-borne diseases worldwide. Training computer vision models on satellite and street view imagery in the municipality of Rio de Janeiro, we analyzed the correlation between the density of common breeding grounds and Aedes aegypti infestation measured by ovitraps on a monthly basis between 2019 and 2022. Our findings emphasized the significance (p ≤ 0.05) of micro-habitat proxies generated through object detection, allowing to explain high spatial variance in urban abundance of Aedes aegypti immatures. Water tanks, non-mounted car tires, plastic bags, potted plants, and storm drains positively correlated with Aedes aegypti egg and larva counts considering a 1000 m mosquito flight range buffer around 2700 ovitrap locations, while dumpsters, small trash bins, and large trash bins exhibited a negative association. This complementary application of satellite and street view imagery opens the pathway for high-resolution interpolation of entomological surveillance data and has the potential to optimize vector control strategies. Consequently it supports the mitigation of emerging infectious diseases transmitted by Aedes aegypti, such as dengue, chikungunya, and Zika, which cause thousands of deaths each year.

Multi-Task Intelligent Monitoring of Construction Safety Based on Computer Vision

Effective safety management is vital for ensuring construction safety. Traditional safety inspections in construction heavily rely on manual labor, which is both time-consuming and labor-intensive. Extensive research has been conducted integrating computer-vision technologies to facilitate intelligent surveillance and improve safety measures. However, existing research predominantly focuses on singular tasks, while construction environments necessitate comprehensive analysis. This study introduces a multi-task computer vision technology approach for the enhanced monitoring of construction safety. The process begins with the collection and processing of multi-source video surveillance data. Subsequently, YOLOv8, a deep learning-based computer vision model, is adapted to meet specific task requirements by modifying the head component of the framework. This adaptation enables efficient detection and segmentation of construction elements, as well as the estimation of person and machine poses. Moreover, a tracking algorithm integrates these capabilities to continuously monitor detected elements, thereby facilitating the proactive identification of unsafe practices on construction sites. This paper also presents a novel Integrated Excavator Pose (IEP) dataset designed to address the common challenges associated with different single datasets, thereby ensuring accurate detection and robust application in practical scenarios.

Benchmarking analysis of computer vision algorithms on edge devices for the real-time detection of digital dermatitis in dairy cows

Digital dermatitis (DD) is a bovine claw disease responsible for ulcerative lesions on the coronary band of the foot. It causes significant animal welfare and economic losses to the cattle industry. Early detection of DD can lead to prompt treatment and decrease lameness. Current detection and staging methods require a trained individual to evaluate the interdigital space on each foot for clinical signs of DD. Computer vision (CV), a type of artificial intelligence for image analysis, has demonstrated promising results on object detection tasks. However, farms require robust solutions that can be deployed in harsh conditions including dust, debris, humidity, precipitation, other equipment issues. The study aims to train, deploy, and benchmark DD detection models on edge devices. Images were collected from commercial dairy farms with the camera facing the interdigital space on the plantar surface of the foot. Images were scored for M-stages of DD by a trained investigator using the M-stage DD classification system. Models were trained to detect and score DD lesions and embedded on an edge device. The Tiny YOLOv4 model deployed on a CV specific integrated camera module connected to a single board computer achieved a mean average precision (mAP) of 0.895, an overall prediction accuracy of 0.873, and a Cohen’s kappa of 0.830 for agreement between the computer vision model and the trained investigator. The model reached a final inference speed of 40 frames per second (FPS) and ran stably without any interruptions. The CV model was able to detect DD lesions on an edge device with high performance and speed. The CV tool can be used for early detection and prompt treatment of DD in dairy cows. Real-time detection of DD on edge device will improve health outcomes, while simultaneously decreasing labor costs. We demonstrate that the deployed model can be a low-power and portable solution for real-time detection of DD on dairy farms. This result is a step towards applying CV algorithms to veterinary medicine and implementing real-time detection of health outcomes in precision farming.

Promote computer vision applications in pig farming scenarios: high-quality dataset, fundamental models, and comparable performance1

Computer vision is widely recognized as an influential technology in the field of precision management of animals. Emerging studies have demonstrated the potential to improve pig health and welfare through animal surveillance systems and computer vision (CV) algorithms. However, the lack of benchmark datasets and robust fundamental algorithms restrict CV applications for the commercial use. This study aims to bridge the gap between technology development and commercial applications in pig farming scenarios by introducing a general-purpose dataset (PigLife), comparing benchmark performances of foundational CV algorithms and model development workflows. The PigLife dataset contains video clips and images (38 short video clips, 2K image frames, 22K pig instances) across most pig production phases in a typical commercial pig farm: Breeding and Gestation, Farrow to Wean, Weaning & Nursery, and Growth to Finish. Three detection algorithms (Faster R-CNN, RetinaNet, TridentNet) and three segmentation algorithms (Mask R-CNN, MViTv2, Point-Rend) were trained on the PigLife dataset from scratch. Fine-tuning of pre-trained models (YOLO8-m, Faster-RCNN-r50) and no-training from zero-shot models (CLIP-SAM, Grouddino-HQSAM) were also evaluated to suggest faster CV development workflows for commercial applications in pig farming. This study emphasizes the necessity of a benchmark dataset for evaluating the robustness of algorithms and identifying the remaining difficulties and challenges across various algorithms. Furthermore, developing CV models from pre-trained algorithms or zero-shot models showed better performance and a faster process, which could reduce barriers when developing high-performance CV products in pig production industry.

Computer Vision Techniques for Morphological Analysis and Identification of Two Pseudo-nitzschia Species

The diversity of phytoplankton influences the structure of and processes that occur in marine ecosystems, with size and other morphological traits being crucial for nutrient uptake and retention in the euphotic zone. Our research introduces a machine learning method that can facilitate the analysis of phytoplankton functional traits from image data. We use computer vision to identify and quantify phytoplankton species and estimate size-related traits based on cell morphology. The study uses transfer learning, where generic, pre-trained YOLOv8 computer vision models are fine-tuned with microscope image data from the Adriatic Sea. The study shows that, for this task, it is possible to effectively fine-tune models trained on out-of-domain images and that this is possible with a small training dataset. The results show high accuracy in detecting and segmenting phytoplankton cells from the microscopic images of the two selected phytoplankton taxa. For detection, the model achieves AP scores of 88.1% for Pseudo-nitzschia cf. delicatissima and 90.9% for Pseudo-nitzschia cf. calliantha, while for segmentation, the scores are 88.4% for Pseudo-nitzschia cf. delicatissima and 91.2% for Pseudo-nitzschia cf. calliantha. Compared to manual image analysis, the developed automatic method significantly increases the number of samples that can be processed.

Hybrid deep learning models for the screening of Diabetic Macular Edema in optical coherence tomography volumes

Several studies published so far used highly selective image datasets from unclear sources to train computer vision models and that may lead to overestimated results, while those studies conducted in real-life remain scarce. To avoid image selection bias, we stacked convolutional and recurrent neural networks (CNN-RNN) to analyze complete optical coherence tomography (OCT) cubes in a row and predict diabetic macular edema (DME), in a real-world diabetic retinopathy screening program. A retrospective cohort study was carried out. Throughout 4-years, 5314 OCT cubes from 4408 subjects who attended to the diabetic retinopathy (DR) screening program were included. We arranged twenty-two (22) pre-trained CNNs in parallel with a bidirectional RNN layer stacked at the bottom, allowing the model to make a prediction for the whole OCT cube. The staff of retina experts built a ground truth of DME later used to train a set of these CNN-RNN models with different configurations. For each trained CNN-RNN model, we performed threshold tuning to find the optimal cut-off point for binary classification of DME. Finally, the best models were selected according to sensitivity, specificity, and area under the receiver operating characteristics curve (AUROC) with their 95% confidence intervals (95%CI). An ensemble of the best models was also explored. 5188 cubes were non-DME and 126 were DME. Three models achieved an AUROC of 0.94. Among these, sensitivity, and specificity (95%CI) ranged from 84.1–90.5 and 89.7–93.3, respectively, at threshold 1, from 89.7–92.1 and 80–83.1 at threshold 2, and from 80.2–81 and 93.8–97, at threshold 3. The ensemble model improved these results, and lower specificity was observed among subjects with sight-threatening DR. Analysis by age, gender, or grade of DME did not vary the performance of the models. CNN-RNN models showed high diagnostic accuracy for detecting DME in a real-world setting. This engine allowed us to detect extra-foveal DMEs commonly overlooked in other studies, and showed potential for application as the first filter of non-referable patients in an outpatient center within a population-based DR screening program, otherwise ended up in specialized care.

Transitioning sleeping position detection in late pregnancy using computer vision from controlled to real-world settings: an observational study

Sleeping on the back after 28 weeks of pregnancy has recently been associated with giving birth to a small-for-gestational-age infant and late stillbirth, but whether a causal relationship exists is currently unknown and difficult to study prospectively. This study was conducted to build a computer vision model that can automatically detect sleeping position in pregnancy under real-world conditions. Real-world overnight video recordings were collected from an ongoing, Canada-wide, prospective, four-night, home sleep apnea study and controlled-setting video recordings were used from a previous study. Images were extracted from the videos and body positions were annotated. Five-fold cross validation was used to train, validate, and test a model using state-of-the-art deep convolutional neural networks. The dataset contained 39 pregnant participants, 13 bed partners, 12,930 images, and 47,001 annotations. The model was trained to detect pillows, twelve sleeping positions, and a sitting position in both the pregnant person and their bed partner simultaneously. The model significantly outperformed a previous similar model for the three most commonly occurring natural sleeping positions in pregnant and non-pregnant adults, with an 82-to-89% average probability of correctly detecting them and a 15-to-19% chance of failing to detect them when any one of them is present.