Computer Vision Approach Research Articles

Development Of A Vision- based Anti-drone Identification Friend Or Foe Model To Recognize Birds And Drones Using Deep Learning

ABSTRACT Recently, the growing use of drones has paved the way for limitless applications in all the domains. However, their malicious exploitations have affected the airspace safety, making them double-edged weapons. Therefore, intelligent anti-drone systems capable of recognizing and neutralizing airborne targets become highly required. In the existing literature, most of the attention has been centered on recognizing drones as unique airborne target, whereas the real challenge is to distinguish between drones and non-drone targets. To address this issue, this study develops an Identification Friend or Foe (IFF) model able to classify the aerial targets in foe or friend categories by determining whether the aerial target is a drone or bird, respectively. To achieve this objective, artificial intelligence and computer vision approaches have been combined through transfer learning, data augmentation and other techniques in our model. Another contribution of this work is the study of the impact of depth on the classification performance, which is demonstrated through our experiments. A comparison is performed based on eight models, where EfficientNetB6 shows the best results with 98.12% accuracy, 98.184% precision, 98.115% F1 score and 99.85% Area Under Curve (AUC). The computational results demonstrate the practicality of the developed model.

Fluorescent Neuronal Cells v2: multi-task, multi-format annotations for deep learning in microscopy

Fluorescent Neuronal Cells v2 is a collection of fluorescence microscopy images and the corresponding ground-truth annotations, designed to foster innovative research in the domains of Life Sciences and Deep Learning. This dataset encompasses three image collections wherein rodent neuronal cell nuclei and cytoplasm are stained with diverse markers to highlight their anatomical or functional characteristics. Specifically, we release 1874 high-resolution images alongside 750 corresponding ground-truth annotations for several learning tasks, including semantic segmentation, object detection and counting. The contribution is two-fold. First, thanks to the variety of annotations and their accessible formats, we anticipate our work will facilitate methodological advancements in computer vision approaches for segmentation, detection, feature extraction, unsupervised and self-supervised learning, transfer learning, and related areas. Second, by enabling extensive exploration and benchmarking, we hope Fluorescent Neuronal Cells v2 will catalyze breakthroughs in fluorescence microscopy analysis and promote cutting-edge discoveries in life sciences.

Computer Vision Approach for Phenotypic Characterization of Horticultural Crops

Computer Vision Approach for Phenotypic Characterization of Horticultural Crops

P153 Artificial Intelligence to predict interleukin-23 signalling activity from H&E stained whole slide images of Inflammatory Bowel Disease

Abstract Background Interleukin23 (IL-23) is a cytokine that plays a crucial role in the pathogenesis of inflammatory bowel disease (IBD), making it a highly validated therapeutic target. Understanding the role of IL-23 in IBD at the histopathological level is crucial for determining effective treatment strategies, providing insights into IBD patients who fail to respond to targeted therapies, or predicting those who are likely to lose response. In this context, there is a surge in utilizing artificial intelligence (AI) for histopathological data in IBD and other disease indications. Here, we present an automated computer vision approach to predict IL-23 signalling activity directly from routinely stained Hematoxylin and Eosin (H&E) images Methods A total of 1502 samples with matched clinical data and H&E biopsy images were included from 991 Crohn’s disease (CD) and 511 ulcerative colitis (UC) samples. IL-23 signalling activity was calculated using gene set variation analysis on RNA-seq data collected from the same tissue biopsies. The data were obtained from the IBD Plexus program of the Crohn’s & Colitis Foundation. The proposed approach is based on vision transformers (ViTs) which is a type of deep learning model. ViTs divide the input image into fixed-size patches, transform it into linear embedding, and analyze it with the self-attention mechanism. This enables ViTs to incorporate relationships between different patches of the input image to identify regions predictive of IL-23. Our approach was trained in a weakly supervised manner to automatically identify tissue regions that correlate with IL-23 signaling activity. The model produces interpretable heatmaps to interrogate model predictions and allow clinicians to visualize and interpret the significance of different tissue regions predictive of IL-23 Results We performed 5-fold cross-validation on the splits obtained at the patient level, retaining the data distribution of IL-23 signaling activity, biopsy location, and diagnosis. We separately validated the performance of the proposed model on both disease categories, including CD and UC. The proposed approach achieved an area under the curve (AUC) of 0.82 ± 0.04 on unseen data from CD and an AUC of 0.80 ± 0.02 for UC. The 5-fold results for both disease categories are shown below Conclusion The presented results highlight the significance of computational pathology algorithms to identify IL-23 signalling activity from H&E images. Pathological interpretation from the heatmaps may help understand disease pathomechanism and optimize the treatment options for IBD patients by timely identification of IL-23 status. We are further validating the clinical utility of such heatmaps and expanding the use of H&E to predict other patient-centric endpoints

A Proof-of-Concept Computer Vision Approach for Measurement of Tympanic Membrane Perforations.

Size, an important characteristic of a tympanic membrane perforation (TMP), is commonly assessed with gross estimation via visual inspection, a practice which is prone to inaccuracy. Herein, we demonstrate feasibility of a proof-of-concept computer vision model for estimating TMP size in a small set of perforations. An open-source deep learning architecture was used to train a model to segment and calculate the area of a perforation and the visualized tympanic membrane (TM) in a set of endoscopic images of mostly anterior and relatively small TMPs. The model then computed relative TMP size by calculating the ratio of perforation area to TM area. Model performance on the test dataset was compared to ground-truth manual annotations. In a validation survey, otolaryngologists were tasked with estimating the size of TMPs from the test dataset. The primary outcome was the average absolute error of model size predictions and clinician estimates compared to sizes determined by ground-truth manual annotations. The model's average absolute error for size predictions was a 0.8% overestimation for all test perforations. Conversely, among the 38 survey respondents, the average clinician error was a 11.0% overestimation (95% CI, 5.2-16.7%, p = 0.003). In a small sample of TMPs, we demonstrated a computer vision approach for estimating TMP size is feasible. Further validation studies must be done with significantly larger and more heterogenous datasets. N/A Laryngoscope, 134:2906-2911, 2024.

A computer vision approach to monitor activity in commercial broiler chickens using trajectory-based clustering analysis

To monitor changes in broiler behaviour related to animal health and welfare, farmers typically observe their flocks using manual observation. However, due to the labour intensive and continuous aspect of this task, the analysis of broiler behaviour could be automated using camera technology. This paper proposes a proof-of-concept camera surveillance system based on the automated detection and tracking of broilers to monitor activity bouts using unsupervised 2D trajectory clustering. Firstly, a convolutional neural network-based detector was trained and tested on our labelled dataset which resulted in a precision, recall and f score of 0.98, 0.90 and 0.94, respectively. Using a tracking-by-detection approach, the proposed system was able to track chickens across video frames with a multi-object tracking accuracy of 74.7%. A component-based feature saliency Gaussian mixture model (CFSGMM) was subsequently generated and applied to objectively cluster the trajectories based on their spatiotemporal information. Nineteen features were extracted from the trajectories, representing both static and dynamic characteristics of broiler movement, and three activity classes were identified: ‘least active/resting,’ ‘active’ and ‘highly-active.’ The proposed method was validated on one-minute monocular video sequences. CFSGMM was applied to cluster 2D trajectories relating to broiler activity bouts within the commercial rearing environment with an agreement ranging from 6.0 to 99.7% when compared to human observation. We demonstrate the potential of the computer vision system to monitor overt, short-term changes in broiler activity associated with on-farm events and discuss the opportunities of leveraging the technology to monitor longer term changes in welfare state. It is anticipated that further development of the detection and tracking systems will improve the performance of the trajectory clustering method.

High-statistics pedestrian dynamics on stairways and their probabilistic fundamental diagrams

Staircases play an essential role in crowd dynamics, allowing pedestrians to flow across large multi-level public facilities such as transportation hubs, shopping malls, and office buildings. Achieving a robust quantitative understanding of pedestrian behavior in these facilities is a key societal necessity. What makes this an outstanding scientific challenge is the extreme randomness intrinsic to pedestrian behavior. Any quantitative understanding necessarily needs to be probabilistic, including average dynamics and fluctuations. To this purpose, large-scale, real-life trajectory datasets are paramount.In this work, we analyze the data from an unprecedentedly high statistics year-long pedestrian tracking campaign, in which we anonymously collected millions of trajectories of pedestrians ascending and descending stairs within Eindhoven Central train station (The Netherlands). This has been possible thanks to a state-of-the-art, faster than real-time, computer vision approach hinged on 3D depth imaging, sensor fusion, and YOLOv7-based depth localization. We consider both free-stream conditions, i.e. pedestrians walking in undisturbed, and trafficked conditions, unidirectional/bidirectional flows. We report on Eulerian fields (density, velocity and acceleration), showing how the walking dynamics changes when transitioning from stairs to landing. We then investigate the (mutual) positions of pedestrian as density changes, considering the crowd as a “compressible” physical medium. In particular, we show how pedestrians willingly opt to occupy fewer space than available, accepting a certain degree of compressibility. This is a non-trivial physical feature of pedestrian dynamics and we introduce a novel way to quantify this effect. As density increases, pedestrians strive to keep a minimum distance d≈0.6m (two treads of the staircase) from the person in front of them. Finally, we establish first-of-kind fully resolved probabilistic fundamental diagrams, where we model the pedestrian walking velocity as a mixture of a slow and fast-paced component (both in non-negligible percentages and with density-dependent characteristic fluctuations). Notably, averages and modes of velocity distribution turn out to be substantially different. Our results, of which we include probabilistic parametrizations based on few variables, are key towards improved facility design and realistic simulation of pedestrians on staircases.

Inferring storefront vacancy using mobile sensing images and computer vision approaches

Storefront vacancy has been a widespread and worldwide phenomenon, raising concerns about the changing characteristic of the retail landscape, loss of community vitality, and hollowing out of cities. Although the causes leading to this phenomenon have been extensively debated, little granular data are available to evaluate the issue in a timely manner. Therefore, this study aims to develop a data-driven approach to capture the commercial structure of vacant storefronts on a store-by-store basis as well as to analyze their evolution patterns. First, street-level images were collected using mobile sensing in a low-cost, large-scale and efficient manner; then, a storefront vacancy estimation model was developed using computer vision techniques to infer the storefront location, operation status, business category, and vacancy rates. Three volunteers spent five days collecting street-level images from an urban area of 964 km2 in the case city of Xining, China. As a result, 93,069 stores were identified in the city in March 2022, of which 25,488 were vacant. Moreover, the storefront vacancy rate increased significantly after the epidemic, from 21.8% in 2018 to 30.0% in 2022. Stores in shopping, catering, and life services had the maximum vacancies. The factors that had the greatest impact on storefront vacancy were, in order of importance, far from commercial zonings, low population density, and far from the urban center. However, these factors influenced the vacancy in diverse and complex ways, and in the future, urban planning strategies to address vacancy issues should be well considered and differentiated.

Designing a computer-vision-based artifact for automated quality control: a case study in the food industry

AbstractReducing waste through automated quality control (AQC) has both positive economical and ecological effects. In order to incorporate AQC in packaging, multiple quality factor types (visual, informational, etc.) of a packaged artifact need to be evaluated. Thus, this work proposes an end-to-end quality control framework evaluating multiple quality control factors of packaged artifacts (visual, informational, etc.) to enable future industrial and scientific use cases. The framework includes an AQC architecture blueprint as well as a computer vision-based model training pipeline. The framework is designed generically, and then implemented based on a real use case from the packaging industry. As an innovate approach to quality control solution development, the data-centric artificial-intelligence (DCAI) paradigm is incorporated in the framework. The implemented use case solution is finally tested on actual data. As a result, it is shown that the framework’s implementation through a real industry use case works seamlessly and achieves superior results. The majority of packaged artifacts are correctly classified with rapid prediction speed. Deep-learning-based and traditional computer vision approaches are both integrated and benchmarked against each other. Through the measurement of a variety of performance metrics, valuable insights and key learnings for future adoptions of the framework are derived.

Deep Learning Driven Automated Red Palm Weevil Detection Using Sparrow Search Optimization

In recent decades, Red Palm Weevils (RPW) have been demonstrated as a harmful pest of palm trees worldwide, predominantly in the Middle East. The RPW is produced massive damage to several palm varieties. Primary detection of the RPW is a complex problem to optimum date production while the recognition is avoided by palm trees as to be influenced by RPW. Several studies are driven to determine a precise approach for the detection, localization, and classification of RPW pests. Employing computer vision (CV) technology with pattern detection is verified that further productive once utilized for identifying and classifying insects. Thus, the automated method decreases either the problem or labor effort required for enhancing the farmer's income. The farmers can be stimulated to enhance the productivity of date fruit once this has been done. With this motivation, this article focuses on the design of automated RPW pest detection using sparrow search optimization with deep learning (RPWPD-SSODL) technique. The presented RPWPD-SSODL algorithm mostly focused on the detection and classification of RPW using computer vision approaches. To accomplish this, the RPWPD-SSODL technique employs bilateral filtering (BF) for noise removal. Next, the RPWPD-SSODL technique uses Dense-RefineDet object detector with ShuffleNet model as a backbone network. For improving the recognition solution, the hyperparameter tuning of the ShuffleNet model can be optimally adjusted using the SSO algorithm. To validate the simulation results of the RPWPD-SSODL technique, a wide-ranging simulation outcome is implemented. The simulation values potrayed the improvement of the RPWPD-SSODL algorithm over other approaches under several measures.

The unmet promise of trustworthy AI in healthcare: why we fail at clinical translation.

Artificial intelligence (AI) has the potential to revolutionize healthcare, for example via decision support systems, computer vision approaches, or AI-based prevention tools. Initial results from AI applications in healthcare show promise but are rarely translated into clinical practice successfully and ethically. This occurs despite an abundance of "Trustworthy AI" guidelines. How can we explain the translational gaps of AI in healthcare? This paper offers a fresh perspective on this problem, showing that failing translation of healthcare AI markedly arises from a lack of an operational definition of "trust" and "trustworthiness". This leads to (a) unintentional misuse concerning what trust (worthiness) is and (b) the risk of intentional abuse by industry stakeholders engaging in ethics washing. By pointing out these issues, we aim to highlight the obstacles that hinder translation of Trustworthy medical AI to practice and prevent it from fulfilling its unmet promises.

Vehicle Perception From Satellite.

Satellites are capable of capturing high-resolution videos. It makes vehicle perception from satellite become possible. Compared to street surveillance, drive recorder or other equipments, satellite videos provide a much broader city-scale view, so that the global dynamic scene of the traffic are captured and displayed. Traffic monitoring from satellite is a new task with great potential applications, including traffic jams prediction, path planning, vehicle dispatching, etc. Practically, limited by the resolution and view, the captured vehicles are very tiny (a few pixels) and move slowly. Worse still, these satellites are in Low Earth Orbit (LEO) to capture such high-resolution videos, so the background is also moving. Under this circumstance, traffic monitoring from the satellite view is an extremely challenging task. To attract more researchers into this field, we build a large-scale benchmark for traffic monitoring from satellite. It supports several tasks, including tiny object detection, counting and density estimation. The dataset is constructed based on 12 satellite videos and 14 synthetic videos recorded from GTA-V. They are separated into 408 video clips, which contain 7,336 real satellite images and 1,960 synthetic images. 128,801 vehicles are annotated totally, and the number of vehicles in each image varies from 0 to 101. Several classic and state-of-the-art approaches in traditional computer vision are evaluated on the datasets, so as to compare the performance of different approaches, analyze the challenges in this task, and discuss the future prospects.

Robust sign language detection for hearing disabled persons by Improved Coyote Optimization Algorithm with deep learning

<abstract> <p>Sign language (SL) recognition for individuals with hearing disabilities involves leveraging machine learning (ML) and computer vision (CV) approaches for interpreting and understanding SL gestures. By employing cameras and deep learning (DL) approaches, namely convolutional neural networks (CNN) and recurrent neural networks (RNN), these models analyze facial expressions, hand movements, and body gestures connected with SL. The major challenges in SL recognition comprise the diversity of signs, differences in signing styles, and the need to recognize the context in which signs are utilized. Therefore, this manuscript develops an SL detection by Improved Coyote Optimization Algorithm with DL (SLR-ICOADL) technique for hearing disabled persons. The goal of the SLR-ICOADL technique is to accomplish an accurate detection model that enables communication for persons using SL as a primary case of expression. At the initial stage, the SLR-ICOADL technique applies a bilateral filtering (BF) approach for noise elimination. Following this, the SLR-ICOADL technique uses the Inception-ResNetv2 for feature extraction. Meanwhile, the ICOA is utilized to select the optimal hyperparameter values of the DL model. At last, the extreme learning machine (ELM) classification model can be utilized for the recognition of various kinds of signs. To exhibit the better performance of the SLR-ICOADL approach, a detailed set of experiments are performed. The experimental outcome emphasizes that the SLR-ICOADL technique gains promising performance in the SL detection process.</p> </abstract>

Computer Vision Approaches for Automated Bee Counting Application

Many application from the bee colony health state monitoring could be efficiently solved using a computer vision techniques. One of such challenges is an efficient way for counting the number of incoming and outcoming bees, which could be used to further analyse many trends, such as the bee colony health state, blooming periods, or for investigating the effects of agricultural spraying. In this paper, we compare three methods for the automated bee counting over two own datasets. The best performing method is based on the ResNet-50 convolutional neural network classifier, which achieved accuracy of 87% over the BUT1 dataset and the accuracy of 93% over the BUT2 dataset.

Face Detection and Counting: Recent Advances and Future Research Directions

In recent years, face detection and counting have undergone a remarkable transfor-mation due to the emergence of deep learning techniques, particularly Convolutional Neural Networks (CNNs). These methods have surpassed traditional computer vision approaches, benefiting from vast datasets and robust computational resources. CNN’s, the cornerstone of contemporary face recognition systems, excel in accurately identifying faces across diverse conditions and environments. The ascendancy of CNNs in face detection stems from their capacity to autonomously learn hierarchical features from raw pixel data, obviating the need for manual feature engineering. Consequently, they yield resilient systems capable of accommodating real-world variations like lighting, pose, expression, and occlusion. Their scalability enables the efficient processing of extensive image databases, rendering them invaluable for applications requiring face counting amidst crowded scenes. Nonetheless, challenges persist, particularly in deploying these systems on resources-constrained devices. The computational complexity and storage demand of deep CNNs necessitate ongoing exploration of lightweight network architectures that balances accuracy with reduced computational footprint. The historical Violajones algorithm, while foundational, has been eclipsed by the superior performance of CNNs. Deep learning’s prowess lies in its adaptability to various tasks and its capacity to continuously refine itself with more data. By learning features directly from data, CNNs excel in capturing intricate patterns crucial for precise face detection and counting. Deep learning, especially through CNNs, has revolutionized face detection and counting by delivering unparal-leled accuracy and robustness. However, ongoing efforts are crucial to address challenges in the efficiency and accessibility of these systems for broader deployment across various applications and devices.

Wavelet scattering‐ and object detection‐based computer vision for identifying dengue from peripheral blood microscopy

AbstractDengue fever infection is a global health concern. Early disease detection is crucial for averting complications and fatality. Characteristic morphological changes in lymphocytes can be observed on a peripheral blood smear (PBS) in cases of dengue infection. In this research, we have developed automated computer vision models for dengue detection on PBS images using two approaches: wavelet scattering transform (WST)‐based feature engineering and classification and You Only Look Once (YOLO)‐based deep transfer learning for object detection. In the former, Morlet wavelet scattering features extracted from lymphocytes were used as input for five shallow classifiers for image classification. Among these, the support vector machine achieved the best results of 98.7% accuracy using 10‐fold cross‐validation. In the latter, computer vision‐enabled object detection was implemented using five YOLOv8 scaled variants. Among these, YOLOv8s and YOLOv8l attained identical best mean accuracy of 99.3% ± 1.4% across five independent experiments. Our results confirmed the feasibility and excellent diagnostic accuracy for both WST‐ and YOLOv8‐enabled computer vision approaches for diagnosing dengue infection in PBS images. This research incorporates deep machine learning along with AI technology to enhance understanding and capabilities in automated Dengue diagnosis. The significance of this research extends to the broader domain of mosquito‐borne illnesses. However, it is important to note that the findings are limited to the dataset used by the researchers.

Telecommunication Fiber Box Detection Using YOLO in Urban Environment

The Fiber Distribution Panel (FDP) box is an essential piece of internet access hardware because it provides users with high-speed data networking and functions as a cable organizer to reduce wire clutter. After installing the FDP, an inspection must be performed to ensure that all necessary components are present. However, This examination is still done manually; the technician snaps a picture of the panel and sends it to its supervisor for verification, which is time-consuming and often prone to errors. In addition to images captured in low-light and complex environments, it makes it more difficult for humans to identify the components with just a naked eye. On this matter, a much more efficient method to assess the FDP installation work is very much needed. Therefore, using computer vision approaches, we utilize a deep learning algorithm to perform object detection and automate the assessment of FDP installation components based on visual data. One of the deep learning models established in the literature is the You Only Look Once (YOLO) model, a one-stage deep learning object detection algorithm that employs a fully conventional approach to generate highly accurate real-time predictions. This paper uses YOLOv5s to identify the fiber box and its relevant components, even in urban environments. Experimentations show that YOLO successfully identified the installation parts with a mean average precision score of 86% at a 0.5 confidence level, even with limited data.

Telecommunication Fiber Box Detection Using YOLO in Urban Environment

The Fiber Distribution Panel (FDP) box is an essential piece of internet access hardware because it provides users with high-speed data networking and functions as a cable organizer to reduce wire clutter. After installing the FDP, an inspection must be performed to ensure that all necessary components are present. However, This examination is still done manually; the technician snaps a picture of the panel and sends it to its supervisor for verification, which is time-consuming and often prone to errors. In addition to images captured in low-light and complex environments, it makes it more difficult for humans to identify the components with just a naked eye. On this matter, a much more efficient method to assess the FDP installation work is very much needed. Therefore, using computer vision approaches, we utilize a deep learning algorithm to perform object detection and automate the assessment of FDP installation components based on visual data. One of the deep learning models established in the literature is the You Only Look Once (YOLO) model, a one-stage deep learning object detection algorithm that employs a fully conventional approach to generate highly accurate real-time predictions. This paper uses YOLOv5s to identify the fiber box and its relevant components, even in urban environments. Experimentations show that YOLO successfully identified the installation parts with a mean average precision score of 86% at a 0.5 confidence level, even with limited data.

Detection of Traditional Costumes: A Computer Vision Approach

Traditional attire has assumed a pivotal role within the contemporary fashion industry. The objective of this study is to construct a computer vision model tailored to the recognition of traditional costumes originating from five distinct countries, namely India, Korea, Japan, Tanzania, and Vietnam. Leveraging a dataset comprising 1,608 images, we proceeded to train the cutting-edge computer vision model YOLOv8. The model yielded an impressive overall mean average precision (MAP) of 96%. Notably, the Indian sari exhibited a remarkable MAP of 99%, the Tanzanian kitenge 98%, the Japanese kimono 92%, the Korean hanbok 89%, and the Vietnamese ao dai 83%. Furthermore, the model demonstrated a commendable overall box precision score of 94.7% and a recall rate of 84.3%. Within the realm of the fashion industry, this model possesses considerable utility for trend projection and the facilitation of personalized recommendation systems.

Vortex and Core Detection using Computer Vision and Machine Learning Methods

The identification of vortices and cores is crucial for understanding airflow motion in aerodynamics. Currently, numerous methods in Computer Vision and Machine Learning exist for detecting vortices and cores. This research develops a comprehensive framework by combining classic Computer Vision and state-of-the-art Machine Learning techniques for vortex and core detection. It enhances a CNN-based method using Computer Vision algorithms for Feature Engineering and then adopts an Ensemble Learning approach for vortex core classification, through which false positives, false negatives, and computational costs are reduced. Specifically, four features, i.e., Contour Area, Aspect Ratio, Area Difference, and Moment Centre, are employed to identify vortex regions using YOLOv5s, followed by a hard voting classifier based on Random Forest, Adaptive Boosting, and Xtreme Gradient Boosting algorithms for vortex core detection. This novel approach differs from traditional Computer Vision approaches using mathematical variables and image features such as HAAR and SIFT for vortex core detection. The findings show that vortices are detected with a high degree of statistical confidence by a fine-tuned YOLOv5s model, and the integrated technique produces an accuracy score of 97.56% in detecting vortex cores conducted on a total of 133 images generated from a rotor blade NACA0012 simulation. Future work will focus on framework generalisation with a larger and more diverse dataset and intelligent threshold development for more efficient vortex and core detection.