Advanced Computer Vision Research Articles

MODVORTEx: Computer Vision-Driven Automation for Magnetic Domain Wall Velocity Analysis

Abstract This paper presents Magneto Optic Domain Velocity Observation and Real-Time Extraction (MODVORTEx), a comprehensive software solution for automating domain wall (DW) velocity measurements in magnetic materials using magneto-optic Kerr effect (MOKE) microscopy. Building upon our previous work on bubble domain structures, we introduce a versatile graphical user interface (GUI) that accommodates arbitrary domain shapes and employs advanced computer vision techniques. The software provides different methods for DW detection and velocity calculation, catering to various domain structures of arbitrary shape. Our approach significantly reduces the time and effort required for data extraction, transforming a process that previously took days of manual work into a task completable within minutes. We provide the details of the algorithmic implementation which is organized into pre-processing, DW detection, displacement measurement, and velocity extraction. This tool is applicable to a wide range of scenarios, including bubble domain dynamics, Dzyaloshinskii-Moriya interaction studies in perpendicular magnetic anisotropy systems, current-driven DW motion in patterned strips etc. By providing both GUI and Application Programing Interface (API), our software offers flexibility for integration into existing measurement systems and adaptability for specific research needs. This automation promises to accelerate research in spintronics and magnetic materials, enabling more comprehensive and accurate studies of DW dynamics.

Improved lane detection for autonomous vehicles using deep learning, semantic segmentation, edge detection and multi-sensor data fusion

Automated driving has gained significant attention because it can eliminate severe driving risks in real time. While autonomous vehicles rely heavily on sensors for lane detection, obstacle identification, and environmental awareness, accurate lane recognition remains a persistent challenge due to factors such as noise from shadows, poor lane markings, and obstructed views. Despite advances in computer vision, this problem is yet to be fully resolved, presenting a gap in the current literature. The primary objective of this research is to address these challenges by developing an enhanced lane-detection system. To achieve this, the study integrates advanced techniques, including semantic segmentation, edge detection, and deep learning, coupled with multi-sensor data fusion from cameras, LIDAR, and radar. By employing this methodology, the research examines various lane-detection methods and benchmarks the proposed model against existing systems in terms of accuracy, specificity and processing speed. Initial findings demonstrate that the combination of semantic segmentation and multi-sensor fusion improves lane detection in real-time scenarios. The proposed model achieved a lane detection accuracy of 97.8%, a specificity of 99.28%, and an average processing time of 0.0047 seconds per epoch. This study not only addresses the limitations of existing lane detection systems but also offers insights into improving road safety for autonomous vehicles.

Application and Integration of Computer Vision Technologies for Automated Recognition and Recording of Chemical Experiments

Abstract Advancements in computer vision have significantly improved motion and object recognition accuracy. These advancements should aid the automatic recognition of chemical experiments, potentially contributing to the recording of experiments. Creating an electronic laboratory notebook from experiment filming enhances convenience and allows more detailed information storage compared to traditional manual recording methods. Our previous research focused on employing object detection and action recognition to automate the recognition of chemical experiments. This paper presents a novel system that combines object detection, action recognition, multiple object tracking, and barcode recognition to automatically generate experimental flowcharts. We implemented our system as a graphical user interface-based application for laboratory use that successfully constructs flowcharts from videos of chemical experiments, including simple chemical manipulations.

Fine-art recognition using convolutional transformers

Digital image processing is a constantly evolving field encompassing a wide range of techniques and applications. Researchers worldwide are continually developing various algorithms across multiple fields to achieve accurate image classification. Advanced computer vision algorithms are crucial for architectural and artistic analysis. The digitalization of art has significantly enhanced the accessibility and conservation of fine-art paintings, yet the risk of art theft remains a significant challenge. Improving art security necessitates the precise identification of fine-art paintings. Although current recognition systems have shown potential, there is significant scope for enhancing their efficiency. We developed an improved recognition system for categorizing fine-art paintings using convolutional transformers, specified by an attention mechanism to enhance focused learning on the data. As part of the most advanced architectures in the deep learning family, transformers are empowered by a multi-head attention mechanism, thus improving learning efficiency. To assess the performance of our model, we compared it with those developed using four pre-trained networks: ResNet50, VGG16, AlexNet, and ViT. Each pre-trained network was integrated into a corresponding state-of-the-art model as the first processing blocks. These four state-of-the-art models were constructed under the transfer learning strategy, one of the most commonly used approaches in this field. The experimental results showed that our proposed system outperformed the other models. Our study also highlighted the effectiveness of using convolutional transformers for learning image features.

Early Detection of Lumpy Skin Disease in Cattle Using Deep Learning-A Comparative Analysis of Pretrained Models.

Lumpy Skin Disease (LSD) poses a significant threat to agricultural economies, particularly in livestock-dependent countries like India, due to its high transmission rate leading to severe morbidity and mortality among cattle. This underscores the urgent need for early and accurate detection to effectively manage and mitigate outbreaks. Leveraging advancements in computer vision and artificial intelligence, our research develops an automated system for LSD detection in cattle using deep learning techniques. We utilized two publicly available datasets comprising images of healthy cattle and those with LSD, including additional images of cattle affected by other diseases to enhance specificity and ensure the model detects LSD specifically rather than general illness signs. Our methodology involved preprocessing the images, applying data augmentation, and balancing the datasets to improve model generalizability. We evaluated over ten pretrained deep learning models-Xception, VGG16, VGG19, ResNet152V2, InceptionV3, MobileNetV2, DenseNet201, NASNetMobile, NASNetLarge, and EfficientNetV2S-using transfer learning. The models were rigorously trained and tested under diverse conditions, with performance assessed using metrics such as accuracy, sensitivity, specificity, precision, F1-score, and AUC-ROC. Notably, VGG16 and MobileNetV2 emerged as the most effective, achieving accuracies of 96.07% and 96.39%, sensitivities of 93.75% and 98.57%, and specificities of 97.14% and 94.59%, respectively. Our study critically highlights the strengths and limitations of each model, demonstrating that while high accuracy is achievable, sensitivity and specificity are crucial for clinical applicability. By meticulously detailing the performance characteristics and including images of cattle with other diseases, we ensured the robustness and reliability of the models. This comprehensive comparative analysis enriches our understanding of deep learning applications in veterinary diagnostics and makes a substantial contribution to the field of automated disease recognition in livestock farming. Our findings suggest that adopting such AI-driven diagnostic tools can enhance the early detection and control of LSD, ultimately benefiting animal health and the agricultural economy.

Vision transformer introduces a new vitality to the classification of renal pathology

Recent advancements in computer vision within the field of artificial intelligence (AI) have made significant inroads into the medical domain. However, the application of AI for classifying renal pathology remains challenging due to the subtle variations in multiple renal pathological classifications. Vision Transformers (ViT), an adaptation of the Transformer model for image recognition, have demonstrated superior capabilities in capturing global features and providing greater explainability. In our study, we developed a ViT model using a diverse set of stained renal histopathology images to evaluate its effectiveness in classifying renal pathology. A total of 1861 whole slide images (WSI) stained with HE, MASSON, PAS, and PASM were collected from 635 patients. Renal tissue images were then extracted, tiled, and categorized into 14 classes on the basis of renal pathology. We employed the classic ViT model from the Timm library, utilizing images sized 384 × 384 pixels with 16 × 16 pixel patches, to train the classification model. A comparative analysis was conducted to evaluate the performance of the ViT model against traditional convolutional neural network (CNN) models. The results indicated that the ViT model demonstrated superior recognition ability (accuracy: 0.96–0.99). Furthermore, we visualized the identification process of the ViT models to investigate potentially significant pathological ultrastructures. Our study demonstrated that ViT models outperformed CNN models in accurately classifying renal pathology. Additionally, ViT models are able to focus on specific, significant structures within renal histopathology, which could be crucial for identifying novel and meaningful pathological features in the diagnosis and treatment of renal disease.

Pose‐to‐Motion: Cross‐Domain Motion Retargeting with Pose Prior

AbstractCreating plausible motions for a diverse range of characters is a long‐standing goal in computer graphics. Current learning‐based motion synthesis methods rely on large‐scale motion datasets, which are often difficult if not impossible to acquire. On the other hand, pose data is more accessible, since static posed characters are easier to create and can even be extracted from images using recent advancements in computer vision. In this paper, we tap into this alternative data source and introduce a neural motion synthesis approach through retargeting, which generates plausible motion of various characters that only have pose data by transferring motion from one single existing motion capture dataset of another drastically different characters. Our experiments show that our method effectively combines the motion features of the source character with the pose features of the target character, and performs robustly with small or noisy pose data sets, ranging from a few artist‐created poses to noisy poses estimated directly from images. Additionally, a conducted user study indicated that a majority of participants found our retargeted motion to be more enjoyable to watch, more lifelike in appearance, and exhibiting fewer artifacts. Our code and dataset can be accessed here.

Research on Animation Character Expression Generation Based on Attention Conditioned Cyclegan

With advancements in computer vision and AI, deep learning is applied in animation and game design to enhance facial expression realism. This study proposes ACCGAN, an innovative method for automated animated character expression generation and transfer using deep learning. By integrating cycle consistency, conditional generation, and attention mechanisms, ACCGAN optimizes GANs in capturing complex expression features, maintaining naturalness, and accurately expressing details. ACCGAN utilizes multi-level attention mechanisms, dynamically adjusting focus on key facial features, enhancing capture and expression.

Empowering Brain Tumor Diagnosis through Explainable Deep Learning

Brain tumors are among the most lethal diseases, and early detection is crucial for improving patient outcomes. Currently, magnetic resonance imaging (MRI) is the most effective method for early brain tumor detection due to its superior imaging quality for soft tissues. However, manual analysis of brain MRI scans is prone to errors, largely influenced by the radiologists’ experience and fatigue. To address these challenges, computer-aided diagnosis (CAD) systems are more significant. These advanced computer vision techniques such as deep learning provide accurate predictions based on medical images, enhancing diagnostic precision and reliability. This paper presents a novel CAD framework for multi-class brain tumor classification. The framework employs six pre-trained deep learning models as the base and incorporates comprehensive data preprocessing and augmentation strategies to enhance computational efficiency. To address issues related to transparency and interpretability in deep learning models, Gradient-weighted Class Activation Mapping (Grad-CAM) is utilized to visualize the decision-making processes involved in tumor classification from MRI scans. Additionally, a user-friendly Brain Tumor Detection System has been developed using Streamlit, demonstrating its practical applicability in real-world settings and providing a valuable tool for clinicians. All simulation results are derived from a public benchmark dataset, showing that the proposed framework achieves state-of-the-art performance, with accuracy approaching 99% in ResNet-50, Xception, and InceptionV3 models.

Kidney medicine meets computer vision: a bibliometric analysis.

Rapid advances in computer vision (CV) have the potential to facilitate the examination, diagnosis, and treatment of diseases of the kidney. The bibliometric study aims to explore the research landscape and evolving research focus of the application of CV in kidney medicine research. The Web of Science Core Collection was utilized to identify publications related to the research or applications of CV technology in the field of kidney medicine from January 1, 1900, to December 31, 2022. We analyzed emerging research trends, highly influential publications and journals, prolific researchers, countries/regions, research institutions, co-authorship networks, and co-occurrence networks. Bibliographic information was analyzed and visualized using Python, Matplotlib, Seaborn, HistCite, and Vosviewer. There was an increasing trend in the number of publications on CV-based kidney medicine research. These publications mainly focused on medical image processing, surgical procedures, medical image analysis/diagnosis, as well as the application and innovation of CV technology in medical imaging. The United States is currently the leading country in terms of the quantities of published articles and international collaborations, followed by China. Deep learning-based segmentation and machine learning-based texture analysis are the most commonly used techniques in this field. Regarding research hotspot trends, CV algorithms are shifting toward artificial intelligence, and research objects are expanding to encompass a wider range of kidney-related objects, with data dimensions used in research transitioning from 2D to 3D while simultaneously incorporating more diverse data modalities. The present study provides a scientometric overview of the current progress in the research and application of CV technology in kidney medicine research. Through the use of bibliometric analysis and network visualization, we elucidate emerging trends, key sources, leading institutions, and popular topics. Our findings and analysis are expected to provide valuable insights for future research on the use of CV in kidney medicine research.

U-Net architecture for high-resolution thermal image segmentation: Enhancing CFD models in smart HVAC design

This paper presents an enhanced U-Net architecture for accurately segmenting high-resolution thermal images to facilitate the design of intelligent Heating, Ventilation, and Air Conditioning (HVAC) systems using Computational Fluid Dynamics (CFD) models. Indoor thermal comfort for occupants will improve under the same energy load by imposing comfort demands. Segmented data is fed into CFD simulations using a U-Net variant for thermal imagery. Integrating the segmented thermal data into the CFD simulations increases the model’s accuracy by 15 % for segmentation and 20% for prediction when compared to a traditional U-Net architecture. Experimental findings demonstrate the potential to achieve energy savings of up to 25 % in the simulated scenario by using the trained and validated model on a dataset of 1,500 high-resolution thermal images of various building styles. Compared to existing methods, the proposed method saves 30% in total simulation time for complex building models. Deep learning enables building the next generation of HVAC systems by pushing the boundaries of energy-efficient and environmentally friendly buildings. This study shows that advanced computer vision can be leveraged to create more innovative and sustainable built environments, leading to drastic improvements in energy efficiency.

Deep-millet: a deep learning model for pearl millet disease identification to envisage precision agriculture

Abstract Plants are integral to human sustenance, serving as fundamental sources of sustenance, materials, and energy, crucial for economic prosperity. However, their productivity and yield are increasingly threatened by pests and diseases, exacerbated by shifting climatic conditions. Pearl millet, a vital crop in Africa and Asia, is particularly susceptible to a range of diseases including downy mildew, rust, ergot, smut, and blast, posing significant risks to crop yield and quality. Timely and accurate disease identification is paramount for effective management strategies. Traditional methods of disease detection relying on visual identification are laborious, costly, and often require specialized expertise, presenting formidable challenges for farmers. In this study, we propose a novel mobile application integrating a robust Deep Learning (DL) model for the automated identification of pearl millet leaf diseases, employing advanced computer vision techniques. A Convolutional Neural Network (CNN) architecture, named Deep Millet, was trained on a comprehensive dataset comprising 3441 field images depicting pearl millet leaves in both healthy and diseased states. It consists of fewer but more effective layers, which are optimized to extract the most pertinent features from the RGB images Comparative analysis against pre-trained models, including AlexNet, ResNet50, InceptionV3, Xception, NasNet mobile, VGG16, and VGG19, was conducted to evaluate the performance of the proposed model. Results demonstrate that Deep Millet achieved superior accuracy, completing training in a mere 240 s and yielding an impressive accuracy rating of 98.86%, surpassing current state-of-the-art models.

AI-Driven Personalized Shopping: Enhancing Retail Experiences through Body Type Analysis and Real-Time Inventory Integration

Abstract: This article introduces an innovative AI-driven personalized shopping system integrating body type analysis, real-time inventory management, and smart recommendations to transform the retail experience. The system significantly improves fit satisfaction, shopping efficiency, and inventory optimization by leveraging advanced computer vision, machine learning, and deep learning technologies. User trials involving 5,000 participants show a 40.3% increase in fit and style satisfaction, 37% reduction in shopping time, and 28% increase in conversion rates compared to traditional methods. The system's ability to provide highly accurate, personalized recommendations at scale addresses key challenges in e-commerce, potentially revolutionizing the retail industry by enhancing customer satisfaction, reducing returns, and optimizing inventory management.

Survey on machine vision-based intelligent water quality monitoring techniques in water treatment plant: Fish activity behavior recognition-based schemes and applications

Abstract Water is a vital resource essential to the survival and development of all creatures. With the rapid growth of industry and agriculture, people face a severe threat of ecological destruction and environmental pollution while living earthly lives. Water pollution, in particular, harms people’s health the most. As a result, water supply security has become a top priority. As a critical point in water supply safety, monitoring water quality effectively and forecasting sudden water contamination on time has become a research hotspot worldwide. With the rapid development and wide applications of artificial intelligence and computer vision technologies, biological activity identification-based intelligent water quality monitoring methods have drawn widespread attention. They were taking fish activities as the water-quality indicator has gained extensive attention by introducing advanced computer vision and artificial intelligence technologies with low cost and ease of carrying. This article comprehensively reviews recent progress in the research and applications of machine vision-based intelligent water quality monitoring and early warning techniques based on fish activity behavior recognition. In detail, it addresses water quality-oriented fish detection and tracking, activity recognition, and abnormal behavior recognition-based intelligent water quality monitoring. It analyzes and compares the performance and their favorite application conditions. Finally, it summarizes and discusses the difficulties and hotspots of water quality monitoring based on the fish’s abnormal behavior recognition and their future development trends.

Monocular 3D Multi-Person Pose Estimation for On-Site Joint Flexion Assessment: A Case of Extreme Knee Flexion Detection.

Work-related musculoskeletal disorders (WMSDs) represent a significant health challenge for workers in construction environments, often arising from prolonged exposure to ergonomic risks associated with manual labor, awkward postures, and repetitive motions. These conditions not only lead to diminished worker productivity but also incur substantial economic costs for employers and healthcare systems alike. Thus, there is an urgent need for effective tools to assess and mitigate these ergonomic risks. This study proposes a novel monocular 3D multi-person pose estimation method designed to enhance ergonomic risk assessments in construction environments. Leveraging advanced computer vision and deep learning techniques, this approach accurately captures and analyzes the spatial dynamics of workers' postures, with a focus on detecting extreme knee flexion, a critical indicator of work-related musculoskeletal disorders (WMSDs). A pilot study conducted on an actual construction site demonstrated the method's feasibility and effectiveness, achieving an accurate detection rate for extreme flexion incidents that closely aligned with supervisory observations and worker self-reports. The proposed monocular approach enables universal applicability and enhances ergonomic analysis through 3D pose estimation and group pose recognition for timely interventions. Future efforts will focus on improving robustness and integration with health monitoring to reduce WMSDs and promote worker health.

Optimization of building thermal environment and VR industrial heritage landscape design enhanced by computer vision algorithms

With the acceleration of urbanization, the optimization of thermal environment of buildings has become an important topic to improve energy efficiency and environmental sustainability. This paper aims to enhance the optimization of building thermal environment through computer vision algorithm, and explore its application in VR industrial heritage landscape design, so as to achieve the dual goals of building energy conservation and cultural inheritance. In this study, advanced computer vision technology is used to monitor and analyze the thermal performance of buildings in real time. Through the construction of thermal environment model, combined with deep learning algorithm to extract building thermal energy data, and use VR technology for visual display, so as to provide scientific basis for design decisions. The research objects include several representative industrial heritage buildings, and analyze their thermal energy consumption characteristics and optimization potential. The results show that the application of the computer vision algorithm reduces the heat loss of the building and significantly improves the energy efficiency. At the same time, the VR visual model intuitively shows the design effect after thermal optimization, enhancing the communication and understanding between designers and stakeholders. Therefore, the combination of computer vision and VR technology can effectively improve the optimization ability of building thermal environment and promote the sustainable design and operation of traditional industrial heritage.

SBD-Net: Incorporating Multi-Level Features for an Efficient Detection Network of Student Behavior in Smart Classrooms

Detecting student behavior in smart classrooms is a critical area of research in educational technology that significantly enhances teaching quality and student engagement. This paper introduces an innovative approach using advanced computer vision and artificial intelligence technologies to monitor and analyze student behavior in real time. Such monitoring assists educators in adjusting their teaching strategies effectively, thereby optimizing classroom instruction. However, the application of this technology faces substantial challenges, including the variability in student sizes, the diversity of behaviors, and occlusions among students in complex classroom settings. Additionally, the uneven distribution of student behaviors presents a significant hurdle. To overcome these challenges, we propose Student Behavior Detection Network (SBD-Net), a lightweight target detection model enhanced by the Focal Modulation module for robust multi-level feature fusion, which augments feature extraction capabilities. Furthermore, the model incorporates the ESLoss function to address the imbalance in behavior sample detection effectively. The innovation continues with the Dyhead detection head, which integrates three-dimensional attention mechanisms, enhancing behavioral representation without escalating computational demands. This balance achieves both a high detection accuracy and manageable computational complexity. Empirical results from our bespoke student behavior dataset, Student Classroom Behavior (SCBehavior), demonstrate that SBD-Net achieves a mean Average Precision (mAP) of 0.824 with a low computational complexity of just 9.8 G. These figures represent a 4.3% improvement in accuracy and a 3.8% increase in recall compared to the baseline model. These advancements underscore the capability of SBD-Net to handle the skewed distribution of student behaviors and to perform high-precision detection in dynamically challenging classroom environments.

Computer vision quickly identifies radio signals with unlimited accuracy

A seminal component of systems thinking is the application of an advanced technology in one domain to solve a challenging problem in a different domain. This article introduces a method of using advanced computer vision to solve the challenging signal processing problem of specific emitter identification. A one-dimensional signal is sampled; those samples are transformed into to two-dimensional images by computing a bispectrum; those images are evaluated using advanced computer vision; and the results are statistically combined until any user-selected level of classification accuracy is obtained. In testing on a published DARPA challenge dataset, for every eight additional signal samples taken from a candidate signal (out of many thousands), classification error decreases by an entire order of magnitude.

Cross-pollination of knowledge for object detection in domain adaptation for industrial automation

AbstractArtificial Intelligence is revolutionizing industries by enhancing efficiency through real-time Object Detection (OD) applications. Utilizing advanced computer vision techniques, OD systems automate processes, analyze complex visual data, and facilitate data-driven decisions, thus increasing productivity. Domain Adaptation for OD has recently gained prominence for its ability to recognize target objects without annotations. Innovative approaches that merge traditional cross-disciplinary domain modeling with cutting-edge deep learning have become essential in addressing complex AI challenges in real-time scenarios. Unlike traditional methods, this study proposes a novel, effective Cross-Pollination of Knowledge (CPK) strategy for domain adaptation inspired by botanical processes. The CPK approach involves merging target samples with source samples at the input stage. By incorporating a random and unique selection of a few target samples, the merging process enhances object detection results efficiently in domain adaptation, supporting detectors in aligning and generalizing features with the source domain. Additionally, this work presents the new Planeat digit recognition dataset, which includes 231 images. To ensure robust comparison, we employ a self-supervised Domain Adaptation (UDA) method that simultaneously trains target and source domains using unsupervised techniques. UDA method leverages target data to identify high-confidence regions, which are then cropped and augmented, adapting UDA for effective OD. The proposed CPK approach significantly outperforms existing UDA techniques, improving mean Average Precision (mAP) by 10.9% through rigorous testing on five diverse datasets across different conditions- cross-weather, cross-camera, and synthetic-to-real. Our code is publicly available https://github.com/anwaar0/CPK-Object-Detection

Automated fish counting system based on instance segmentation in aquaculture

Non-invasive methods of fish counting are crucial for the effective management of aquaculture and aquatic resources. However, accurate and reliable estimation of fish populations in real-time is a significant challenge due to severe occlusion, body deformation, and rapid movement of fish. Recent advancements in computer vision and deep learning algorithms offer substantial potential to revolutionize fish counting methodologies, thereby enhancing conservation efforts and promoting the sustainable management of aquatic resources. This paper presents a comprehensive dataset of 256,580 fish instances annotated with mask information at the pixel level. To address the challenges associated with geometric variations in fish size, pose, perspective, and body shape, we propose a lightweight instance segmentation model based on YOLOv8, which integrates the CSP bottleneck with dual convolutions and the deformable convolutional networks (DConv). In addition, an efficient feature fusion network incorporating multi-scale information was introduced to improve segmentation accuracy by enhancing the representation of feature information, and a decoupled head with attention mechanisms was designed to detect and segment fish bodies in real time. Our proposed approach achieved an outstanding mAP50 of 98.1 % at 60.6 FPS, with model parameters and floating point operations per second (FLOPS) amounting to 25.4 M and 96.9 G, respectively. The instance segmentation-based counting system exhibited impressive performance metrics, with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and R-Square (R2) values of 1.19, 1.65, and 0.992, respectively. The proposed method demonstrates strong potential for practical application as an intelligent solution for biomass estimation in recirculating aquaculture systems.