Computer Vision Research Articles

Fit-Sense AI

Sustaining a healthy lifestyle has grown more crucial in the fast-paced world of today. But not everybody has access to a personal trainer for fitness. This is where technology can become extremely important in increasing the accessibility and appeal of fitness. A project called "Fit- Sense AI” is an AI-powered fitness Trainer that uses OpenCV for computer vision, MediaPipe for position estimation, and artificial intelligence (AI) to build a virtual fitness trainer that you can use on your computer. The goal of this project is to offer people a customized, interactive fitness experience. It can follow the user's motions in real time and provide feedback and instructions for different exercises by using computer vision techniques. The AI fitness trainer will assess your technique while you're doing strength training or yoga poses and provide suggestions for improvement

Handwritten Character Recognition: A Comparative Study

Handwriting recognition is a technique used to interpret intelligible handwritten input and convert it into digital text using Machine Learning tools. This research paper provides a comparison of the application of CRNN and CNN for handwriting recognition, using a dataset containing about 370,000 handwritten names. Our experiments demonstrate that the CRNN hybrid model produces the highest accuracy compared to the CNN model. This paper summarises contributions reported on the A-Z Handwritten Alphabets in .csv format dataset for handwritten character recognition. This dataset has been extensively used to validate novel techniques in computer vision. This paper makes a distinction between those works using some kind of data augmentation and works using the original dataset

Medicinal Plant Classification and it's Identification

Indian Ayurvedic Medicine reached international level The basis of Ayurveda is herbal medicine. The pharmaceutical industry is starting to pay more. Be careful with herbs because their side effects and consequences are less than modern medicine and at the same time they are very expensive. The company efficiently and reliably uses leaf images from many deep and machine learning algorithms developed in recent years. In this study, leaves of 45 different medicinal plants were used and the deep learning method was used. A number of classification and identification methods were used using computer vision technology to achieve the best results. There are many medicinal plants according to the classification of the paper plant

An Improved Fire Detection Approach Based On Yolo-v8 for Smart Cities

Systems for detecting fires are essential for preventing property damage and saving lives. defending people and property. Conventional techniques frequently depend on sensor-based strategies, which have limitations in intricate settings. In order to improve accuracy and efficiency, this study suggests an intelligent fire detection system that makes use of machine learning and computer vision techniques. The technology analyzes video streams in real time using deep learning algorithms to identify fire incidents based on visual patterns and attributes. Future research on fire detection systems will benefit from the information this study will provide for smoker and fire detection issues in both indoor and outdoor situations. The improved fire detection technique for smart cities that is based on the YOLOv8 algorithm is the smart fire detection system (SFDS), which uses deep learning to identify fire-specific properties in real-time. The SFDS strategy may be more cost-effective, reduce false alarms, and improve fire detection accuracy when compared to traditional methods. It can also be extended to find other intriguing aspects of smart cities, such as gas leakage or flooding. The proposed smart city framework consists of four primary levels: the application layer (i), cloud layer (iii), fog layer (ii), and internet of things layer (iv). The recommended technique uses fog, cloud computing, and the Internet of Things layer to collect and understand data in real time. This reduces the chance of damage to persons or property and enables faster reaction times. The SFDS demonstrated state- of-the-art performance in terms of precision and recall, with a high precision rate of 97.1% across all classes. Among the potential applications are intelligent security systems, forest fire monitoring, and public space fire safety management.

Tradescantia response to air and soil pollution, stamen hair cells dataset and ANN color classification

Tradescantia plant is a complex system that is sensible to environmental factors such as water supply, pH, temperature, light, radiation, impurities, and nutrient availability. It can be used as a biomonitor for environmental changes; however, the bioassays are time-consuming and have a strong human interference factor that might change the result depending on who is performing the analysis. We have developed computer vision models to study color variations from Tradescantia clone 4430 plant stamen hair cells, which can be stressed due to air pollution and soil contamination. The study introduces a novel dataset, Trad-204, comprising single-cell images from Tradescantia clone 4430, captured during the Tradescantia stamen-hair mutation bioassay (Trad-SHM). The dataset contain images from two experiments, one focusing on air pollution by particulate matter and another based on soil contaminated by diesel oil. Both experiments were carried out in Curitiba, Brazil, between 2020 and 2023. The images represent single cells with different shapes, sizes, and colors, reflecting the plant's responses to environmental stressors. An automatic classification task was developed to distinguishing between blue and pink cells, and the study explores both a baseline model and three artificial neural network (ANN) architectures, namely, TinyVGG, VGG-16, and ResNet34. Tradescantia revealed sensibility to both air particulate matter concentration and diesel oil in soil. The results indicate that Residual Network architecture outperforms the other models in terms of accuracy on both training and testing sets. The dataset and findings contribute to the understanding of plant cell responses to environmental stress and provide valuable resources for further research in automated image analysis of plant cells. Discussion highlights the impact of turgor pressure on cell shape and the potential implications for plant physiology. The comparison between ANN architectures aligns with previous research, emphasizing the superior performance of ResNet models in image classification tasks. Artificial intelligence identification of pink cells improves the counting accuracy, thus avoiding human errors due to different color perceptions, fatigue, or inattention, in addition to facilitating and speeding up the analysis process. Overall, the study offers insights into plant cell dynamics and provides a foundation for future investigations like cells morphology change. This research corroborates that biomonitoring should be considered as an important tool for political actions, being a relevant issue in risk assessment and the development of new public policies relating to the environment.

The Poses for Equine Research Dataset (PFERD)

Studies of quadruped animal motion help us to identify diseases, understand behavior and unravel the mechanics behind gaits in animals. The horse is likely the best-studied animal in this aspect, but data capture is challenging and time-consuming. Computer vision techniques improve animal motion extraction, but the development relies on reference datasets, which are scarce, not open-access and often provide data from only a few anatomical landmarks. Addressing this data gap, we introduce PFERD, a video and 3D marker motion dataset from horses using a full-body set-up of densely placed over 100 skin-attached markers and synchronized videos from ten camera angles. Five horses of diverse conformations provide data for various motions from basic poses (eg. walking, trotting) to advanced motions (eg. rearing, kicking). We further express the 3D motions with current techniques and a 3D parameterized model, the hSMAL model, establishing a baseline for 3D horse markerless motion capture. PFERD enables advanced biomechanical studies and provides a resource of ground truth data for the methodological development of markerless motion capture.

A Mobile-Based System for Detecting Plant Leaf Diseases Using Deep Convolutional Neural Network

Plant diseases pose a significant challenge to agriculture in India, causing substantial losses in crop production each year. Limited-resource farmers often struggle with diagnosing these diseases accurately, especially when relying on visual observation of leaf symptoms. Hence, there is an urgent need to enhance the detection, monitoring, and prediction of crop diseases to mitigate agricultural losses effectively. To address this challenge, we propose a mobile-based system empowered by Machine Learning (ML) and computer vision tailored for the Indian context. Our system utilizes Convolutional Neural Networks (CNNs) as the core deep learning engine to classify 37 disease categories commonly found in Indian crops. We have curated a comprehensive dataset comprising 96,206 images of plant leaves, encompassing both healthy and infected, for training, validation, and testing purposes. The user interface of our system is designed as an Android mobile application, enabling farmers to capture images of diseased plant leaves effortlessly. Upon image capture, the system swiftly identifies the disease category and presents it to the user along with a confidence percentage. This functionality empowers farmers to take timely action to protect their crops, thereby reducing reliance on incorrect fertilizers that may exacerbate plant stress. Furthermore, we evaluated the performance of our system using various metrics, including classification accuracy and processing time. Our results indicate an impressive overall classification accuracy of 94% across the 37 disease classes prevalent in 14 different crop species commonly cultivated in India. In essence, our ML-powered mobile-based solution offers Indian farmers a robust tool for efficient crop disease diagnosis and management, ultimately contributing to the enhancement of agricultural productivity and sustainability in the country.

Oxide semiconductor based deep‐subthreshold operated read‐out electronics for all‐printed smart sensor patches

AbstractThe ability to fabricate an entire smart sensor patch with read‐out electronics using commercial printing techniques may have a wide range of potential applications. Although solution‐processed oxide thin film transistors (TFTs) are capable of providing high mobility electron transport, resulting in large ON‐state current and power output, there is hardly any literature report that uses the printed oxide TFTs at the sensor interfaces. Here, printed amorphous indium‐gallium‐zinc oxide (a‐IGZO)‐based deep‐subthreshold operated TFTs that comprise signal amplifiers and analog‐to‐digital converters (ADCs) that can successfully digitalize the analog sensor signals up to a frequency range of 1 kHz are reported. In addition, exploiting the high current oxide TFTs, a current drive circuit placed after the ADC unit has been found useful in producing easy‐to‐detect visual recognition of the sensor signal at a predefined threshold crossover. Notably, the entire smart sensor patch is demonstrated to operate at a low supply voltage of ≤2 V, thereby ensuring that it can be an on‐chip energy source compatible and standalone detection unit.

One-Shot Face Stylization

The abstract delves into the intricate realm of one-shot face stylization, a captivating domain within computer vision and deep learning. It revolves around the art of transforming a target face using a reference image, navigating the delicate balance between retaining facial recognition and infusing desired stylistic traits. This pursuit has garnered considerable attention owing to its myriad applications across digital art, entertainment, and personalized products. The abstract scrutinizes the essential components of one-shot face stylization, highlighting the pivotal role of deep neural networks, particularly generative adversarial networks (GANs). These networks are adept at crafting bespoke facial images by assimilating information from both the target and reference faces, leveraging the reference image as a guiding beacon. The crux of success in one-shot facial stylization lies in the meticulous orchestration of the fading process, which harmonizes the preservation of identity with the enhancement of artistic technique. As advancements in this field continue to unfold, the potential ramifications span far and wide, promising to revolutionize creative expression and customization across diverse industries, spanning from digital art and animation to virtual avatars and social media filters. Key Words: One-Shot face stylization, Generative adversarial network, Facial recognition, Digital art, Personalization

Air Handwriting using AI and ML

Air Handwriting using AI and ML

Parallel Hierarchical Multi-View Feature Fusion Based on Canonical Correlation Analysis for Mammogram Retrieval

Due to the diversity of image sources, content-based multi-source image fusion and retrieval have shown promising capabilities in computer vision tasks, and especially when applied in Computer-Aided Diagnosis (CAD) to automate and improve the accuracy of medical image analysis. The combination of computer vision and CAD systems has the potential to revolutionize healthcare by augmenting the expertise of clinicians, improving overall diagnostic accuracy and helping experts in the clinical decision-making process by classifying and retrieving similar annotated clinical images to a given query. In the context of multi-view mammography interpretation, the concept of multi-view feature fusion has recently been studied to improve retrieval performance while effectively guaranteeing the complementarity of both MLO and CC views. However, conventional multi-view feature fusion makes descriptors long and lacks to take into consideration the relationship between descriptors. To deal with this issue, we propose two hierarchical multi-view feature fusion methods, for multi-view mammogram retrieval, based on the Canonical Correlation Analysis (CCA), which is the most commonly used multivariate parametric test. In fact, we have adapted CCA to determine the relationship between two descriptors by processing latent correlation factors. Moreover, after extracting descriptors for each view, a comparative study of texture and shape fusion descriptors is proposed in order to identify the more discriminative features for multi-view mammogram retrieval. Then, a query-dependent distance metric preserving both visual resemblance and semantic similarity is carried out to dynamically determine the more appropriate distance measure for each query image. Extensive experiments on the challenging Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) have demonstrated the effectiveness of the proposed hierarchical multi-view feature fusion for mammogram retrieval, which outperforms the performance achieved either by conventional fused information or by single view information. To improve the transparency of our paper, the source code of the proposed method and the related dataset (including readme files) are publicly accessible through the following GitHub link: https://github.com/ABDERRAHIMMAR/Multi-View-Feature-Fusion-for-Mammogram-Retrieval . This open-access resource empowers researchers and practitioners to delve deeper into our methodology, fostering collaboration and advancements in the field of computer-aided diagnosis and medical image analysis.

Robust detection and refinement of saliency identification

Salient object detection is an increasingly popular topic in the computer vision field, particularly for images with complex backgrounds and diverse object parts. Background information is an essential factor in detecting salient objects. This paper suggests a robust and effective methodology for salient object detection. This method involves two main stages. The first stage is to produce a saliency detection map based on the dense and sparse reconstruction of image regions using a refined background dictionary. The refined background dictionary uses a boundary conductivity measurement to exclude salient object regions near the image's boundary from a background dictionary. In the second stage, the CascadePSP network is integrated to refine and correct the local boundaries of the saliency mask to highlight saliency objects more uniformly. Using six evaluation indexes, experimental outcomes conducted on three datasets show that the proposed approach performs effectively compared to the state-of-the-art methods in salient object detection, particularly in identifying the challenging salient objects located near the image's boundary. These results demonstrate the potential of the proposed framework for various computer vision applications.

Status of the Deep Learning-Based Shattered Pellet Injection Shard Tracking at ASDEX Upgrade

Plasma disruptions pose an intolerable risk to large tokamaks, such as ITER. If a disruption can no longer be avoided, ITER’s last line of defense will be the Shattered Pellet Injection. An experimental test bench was created at ASDEX Upgrade to inform the design decisions for controlling the shattering of the pellets and develop the techniques for the generation of the fragment distributions necessary for optimal disruption mitigation. In an effort to analyze the videos resulting from the more than 1000 tests and determine the impact of different settings on the resulting shard cloud, an analysis pipeline, based on traditional computer vision (CV), was created. This pipeline enabled the analysis of 173 of the videos, but at the same time showed the limits of traditional CV when applied in applications with a highly heterogeneous dataset such as this. We created a machine learning-based (ML) alternative as a drop-in replacement to the original image processing code using a semantic segmentation model to exploit the innate adaptability and robustness of deep learning models. This model is capable of labeling the entire dataset quickly, accurately and reliably. This contribution details the implementation of the ML model and the current state and future plans of the project.

Face Recognition Based Attendance Management System

The goal of the "Face Recognition Based Attendance Management System" project is to create an automated attendance system that efficiently and reliably marks attendance using facial recognition technology. As people enter a specified region, the technology would take pictures of them and compare them to a database of people who have already registered. The system would record the person's attendance and save the data for further use after a successful identification. By completing this project, manual attendance taking—which may be laborious and error-prone—would no longer be necessary. Additionally, it would give administrators access to real-time attendance data, which would simplify the tracking of attendance and enable data-driven decision-making . Large volumes of data would be handled by the system, which would be developed using machine learning and computer vision techniques. All things considered, this project has the power to completely change how attendance is kept track of in a variety of places, including companies, colleges, and schools. It provides a very precise and effective solution that can lower expenses, save time, and boost output all around. Keyword- Attendance Management System, Face Recognition, Camera, NumPy, OpenCV.

ERSegDiff: a diffusion-based model for edge reshaping in medical image segmentation

Medical image segmentation is a crucial field of computer vision. Obtaining correct pathological areas can help clinicians analyze patient conditions more precisely. We have observed that both CNN-based and attention-based neural networks often produce rough segmentation results around the edges of the regions of interest. This significantly impacts the accuracy of obtaining the pathological areas. Without altering the original data and model architecture, further refining the initial segmentation outcomes can effectively address this issue and lead to more satisfactory results. Recently, diffusion models have demonstrated outstanding results in image generation, showcasing their powerful ability to model distributions. We believe that this ability can greatly enhance the accuracy of the reshaping results. This research proposes ERSegDiff, a neural network based on the diffusion model for reshaping segmentation borders. The diffusion model is trained to fit the distribution of the target edge area and is then used to modify the segmentation edge to produce more accurate segmentation results. By incorporating prior knowledge into the diffusion model, we can help it more accurately simulate the edge probability distribution of the samples. Moreover, we introduce the edge concern module, which leverages attention mechanisms to produce feature weights and further refine the segmentation outcomes. To validate our approach, we employed the COVID-19 and ISIC-2018 datasets for lung segmentation and skin cancer segmentation tasks, respectively. Compared with the baseline model, ERSegDiff improved the dice score by 3%–4% and 2%–4%, respectively, and achieved state-of-the-art scores compared to several mainstream neural networks, such as swinUNETR.

Deep Learning based Semantic Segmentation for Buildings Detection from Remote Sensing Images

Building extraction from remote sensing images is the process of automatically identifying and extracting the boundaries of buildings from high-resolution aerial or satellite images. The extracted building footprints can be used for a variety of applications, such as urban planning, disaster management, city development, land management, environmental monitoring, and 3D modeling. The results of building extraction from remote sensing images depend on several factors, such as the quality and resolution of the image and the choice of algorithm.The process of building extraction from remote sensing images typically involves a series of steps, including image pre-processing, feature extraction, and classification. Building extraction from remote sensing images can be challenging due to factors such as varying building sizes and shapes, shadows, and occlusions. However, recent advances in deep learning and computer vision techniques have led to significant improvements in the accuracy and efficiency of building extraction methods. This research presents a deep learning semantic segmentation architecture-based model for developing building detection from high resolution remote sensing images. The open-source Massachusetts dataset is used to train the suggested UNet architecture. The model is optimized using the RMSProp algorithm with a learning rate of 0.0001 for 100 epochs. After 1.52 hours of training on Google Colab the model achieved an 83.55% F1 score, which indicates strong precision and recall.

EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training.

The superior performance of modern computer vision backbones (e.g., vision Transformers learned on ImageNet-1K/22K) usually comes with a costly training procedure. This study contributes to this issue by generalizing the idea of curriculum learning beyond its original formulation, i.e., training models using easier-to-harder data. Specifically, we reformulate the training curriculum as a soft-selection function, which uncovers progressively more difficult patterns within each example during training, instead of performing easier-to-harder sample selection. Our work is inspired by an intriguing observation on the learning dynamics of visual backbones: during the earlier stages of training, the model predominantly learns to recognize some 'easier-to-learn' discriminative patterns in the data. These patterns, when observed through frequency and spatial domains, incorporate lower-frequency components, and the natural image contents without distortion or data augmentation. Motivated by these findings, we propose a curriculum where the model always leverages all the training data at every learning stage, yet the exposure to the 'easier-to-learn' patterns of each example is initiated first, with harder patterns gradually introduced as training progresses. To implement this idea in a computationally efficient way, we introduce a cropping operation in the Fourier spectrum of the inputs, enabling the model to learn from only the lower-frequency components. Then we show that exposing the contents of natural images can be readily achieved by modulating the intensity of data augmentation. Finally, we integrate these two aspects and design curriculum learning schedules by proposing tailored searching algorithms. Moreover, we present useful techniques for deploying our approach efficiently in challenging practical scenarios, such as large-scale parallel training, and limited input/output or data pre-processing speed. The resulting method, EfficientTrain++, is simple, general, yet surprisingly effective. As an off-the-shelf approach, it reduces the training time of various popular models (e.g., ResNet, ConvNeXt, DeiT, PVT, Swin, CSWin, and CAFormer) by [Formula: see text] on ImageNet-1K/22K without sacrificing accuracy. It also demonstrates efficacy in self-supervised learning (e.g., MAE). Code is available at: https://github.com/LeapLabTHU/EfficientTrain.

YOLO-Based Phenotyping of Apple Blotch Disease (Diplocarpon coronariae) in Genetic Resources after Artificial Inoculation

YOLO-Based Phenotyping of Apple Blotch Disease (Diplocarpon coronariae) in Genetic Resources after Artificial Inoculation

Measuring Rotational and Translational Movements in Rotating Machines Using a Computer Vision Approach

Measuring Rotational and Translational Movements in Rotating Machines Using a Computer Vision Approach

DMS: Low-overlap Registration of 3D Point Clouds with Double-layer Multi-scale Star-graph.

Registering 3D point clouds with low overlap is challenging in 3D computer vision, primarily due to difficulties in identifying small overlap regions and removing correspondence outliers. We observe that the neighborhood similarity can be utilized to detect point correspondence, and the consistent neighborhood correspondence can be used as a criterion to detect robust overlapping regions. So that a Double-layer Multi-scale Star-graph (DMS) structure is proposed to detect robust correspondences using two different types of multi-scale star-graphs. The first-layer Multi-scale Neighbor Feature Star-graphs (MNFS) takes each point as the center and its multi-scale nearest neighbors as the leaves. The MNFS enables to establish the initial correspondence candidate set between the two point clouds based on multi-scale neighborhood topology and feature similarity. Subsequently, each pair of corresponding points find their nearest neighbors within the correspondence sets to construct a Multi-scale Matching Star-graphs (MMS) on each side, so the mutual correspondence relationships between the MMS vertices are identified. These identified mutual correspondences are treated as vertices to construct the Multi-scale Correspondence Star-graphs (MCS), that indicate the relationships among the correspondences. We design edge weight and vertex weight criterion in MCS to detect only the robust correspondence set that has strong neighborhood consistency, so as to reject the outliers. Finally, the point cloud registration is conducted based on the detected robust correspondence. The experimental results demonstrate clearly that the proposed DMS method exhibits superior robustness when compared to existing state-of-the-art registration algorithms. The code of this study will be available at https://github.com/HualongCao/DMS.