Computer Vision Research Articles

Information disclosure and funding success of green crowdfunding campaigns: a study on GoFundMe

AbstractCrowdfunding has become important in increasing financial support for the development of green technologies. Self-disclosed information significantly affects supporters’ decisions and is important for the success of green project funding. However, current studies still lack investigations into the impact of information disclosure on green crowdfunding performance. This research aims to fill this knowledge gap by exploring eight information disclosure-relevant factors in green crowdfunding performance. Applying machine learning techniques (e.g., Natural Language Processing and Computer Vision) and logistic regression, this study investigates 720 green crowdfunding campaigns on GoFundMe and empirically finds that the duration, length of campaign introductions, and length of the title influence fundraising outcomes. However, no evidence supports the impact of goal size, emotion of campaign introduction, or image content on funding success. This study clarifies the information disclosure-related data that green crowdfunding campaigns should consider and provides founders with a constructive guide to smoothly raise money for a green crowdfunding campaign. This study also contributes to data processing methods by providing future studies with an approach for transferring unstructured data to structured data.

Enhanced breast cancer diagnosis through integration of computer vision with fusion based joint transfer learning using multi modality medical images

Enhanced breast cancer diagnosis through integration of computer vision with fusion based joint transfer learning using multi modality medical images

Evaluasi Trade-off Akurasi dan Kecepatan YOLOv5 dalam Deteksi Kebakaran pada Edge Devices

Real-time object detection using the YOLO (You Only Look Once) algorithm has shown promising performance in various computer vision applications. However, its application on devices with limited resources is still a challenge due to its high computational requirements. This study aims to optimize the YOLOv5 model for fire and smoke detection on Orange Pi Zero 3 devices using quantization techniques. Using a dataset of 2247 fire and smoke images, this study applies static quantization techniques to improve model efficiency. The methodology includes training of standard YOLOv5 models, conversion to ONNX format, and application of static quantization. Results show a significant improvement in computational efficiency, with a 42.2% reduction in model size and a 65.21% increase in inference speed. Despite a decrease in the mAP value by 25.6%, the optimized model was still able to perform object detection at a significantly higher speed. In conclusion, the quantization technique is effective in optimizing the YOLOv5 model for deployment on edge computing devices, despite the trade-off between speed and accuracy.

Deep Learning Approach for Suspicious Activity Detection from Surveillance Video

The present generation seeks ways to live their lives free of fear. This often prompts the need for enhanced law enforcement and security systems. Suspicion activity recognition in surveillance footage has become an essential component in modern security systems, focused around the problem of recognizing inappropriate behaviors without too much human time. The advent of deep learning has made an impact in computer vision tasks within such a way the real-time detection by the evaluation of large amounts of video footage becomes realistic. In this context the paper presents a model in which deep learning, particularly Convolutional Neural Networks (CNNs), are utilized for the behavioral pattern recognition from security footage. In addition, it also improves the efficacy of detection and reduces the shortcomings associated with these systems where only the post-event analysis is performed manually or automatically. In this paper, we demonstrate a novel IP model that analyzes live video and performs normal-suspicious–threat triage in real-time. Results indicate the validity of the method in different environmental and illumination conditions. An important objective that this system is designed for is to change the way security is practiced and delivered in today’s intelligent systems providing actionable intelligence and the ability to prevent threats rather than just respond to them making this a high growth opportunity for marketing high-risk and sensitive areas.

Crack Detection and Cost Analyzer Using AI

Crack Vision is an innovative AI-powered Android application developed to detect and analyze wall cracks, providing users with real-time, automated repair cost estimates. By leveraging advanced object detection models such as YOLOv7 and Mask R-CNN, Crack Vision enhances both the accuracy and efficiency of crack detection, enabling precise measurement of crack dimensions, severity assessment, and quick identification of structural issues. This application simplifies the inspection and budgeting process for wall repairs, making it an essential tool for homeowners, contractors, and structural engineers who seek to minimize reliance on manual inspection methods. The core functionality of Crack Vision combines high-resolution image processing with state-of-the-art machine learning algorithms, ensuring reliable detection across various wall types and crack textures. Using computer vision and deep learning techniques, the app can differentiate between structural and non-structural cracks, thereby offering a comprehensive solution for repair prioritization. Furthermore, Crack Vision integrates with a robust backend that stores user data, allowing users to track the condition of their walls over time. This feature aids in preventive maintenance by identifying trends in crack expansion and deterioration, potentially mitigating costly repairs. Additionally, the application provides a detailed breakdown of estimated repair costs based on crack size, depth, and type, empowering users to make informed decisions about necessary repairs. Crack Vision is designed with user-friendliness in mind, featuring an intuitive interface that guides users through the inspection process. The app also supports multilingual options and customizable settings to cater to diverse user needs. In sum, Crack Vision redefines wall crack inspection, offering a rapid, accurate, and accessible solution for both casual users and industry professionals, ultimately contributing to improved building safety and maintenance efficiency

Prediction of Salicornia europaea L. biomass using a computer vision system to distinguish different salt-tolerant populations.

Salicornia europaea L. is emerging as a versatile crop halophyte, requiring a low-cost, non-destructive method for salt tolerance classification to aid selective breeding. We propose using a computer vision system (CVS) with multivariate analysis to classify S. europaea based on morphometric and colour traits to predict plant biomass and the salinity in their substrate. A trial and validation set of 96 and 24 plants from 2 populations confirmedthe efficacy. CVS and multivariate analysis evaluated the plants by morphometric traits and CIELab colour variability. Through Pearson analysis, the strongest correlations were between biomass fresh weight (FW) vs. projected area (PA) (0.91) and anatomical cross-section (ACS) vs. shoot diameter (Sd) (0.94). The PA and FW correlation retrieved different equation fits between lower and higher salt-tolerant populations (R2 = 0.93 for linear and 0.90 for 2nd-degree polynomial), respectively. The higher salt-tolerant reached a maximum biomass PA at 400 mM NaCl, while the lower salt-tolerant produced less under 200 and 400 mM. A second Pearson correlation and PCA described sample variability with 80% reliability using only morphometric-colour parameters. Multivariate discriminant analysis (MDA) demonstrated that the method correctly classifies plants (90%) depending on their salinity level and tolerance, which was validated with 100% effectiveness. Through multiple linear regression, a predictive model successfully estimated biomass production by PA, and a second model predicted the salinity substrate (Sal.s.) where the plants thrive. Plants' Sd and height influenced PA prediction, while Sd and colour difference (ΔE1) influenced Sal.s. Models validation of actual vs. predicted values showed a R2 of 0.97 and 0.90 for PA, and 0.95 and 0.97 for Sal.s. for lower and higher salt-tolerant, respectively. This outcome confirms the method as a cost-effective tool for managing S. europaea breeding. The CVS effectively extracted morphological and colour features from S. europaea cultivated at different salinity levels, enabling classification and plant sorting through image and multivariate analysis. Biomass and salinity substrate were accurately predicted by modelling non-destructive parameters. Enhanced by AI, machine learning and smartphone technology, this method shows great potential in ecology, bio-agriculture, and industry.

Image-processing-based model for surface roughness evaluation in titanium based alloys using dual tree complex wavelet transform and radial basis function neural networks.

In this study, we examine the assessment of surface roughness on turned surfaces of Ti 6Al 4V using a computer vision system. We utilize the Dual-Tree Complex Wavelet Transform (DTCWT) to break down the images of the turned surface into sub-images oriented in directions. Three different methods of feature generation have been compared, i.e., the use of Gray-Level Co-Occurrence Matrix (GLCM) and DTCWT-based extraction of second-order statistical features, DTCWT Image fusion, and the use of GLCM for feature extraction, and DTCWT image fusion using Particle Swarm Optimization (PSO) based GLCM features. Principal Component Analysis (PCA) was utilized to identify and select features. The model was developed using a Radial Basis Function Neural Network (RBFNN). Accordingly, six models were designed based on the three feature generation methods, considering all features and features selected using PCA. The RBFNN model, which incorporates DTCWT Image fusion and utilizes PSO with PCA features, achieved a training data prediction accuracy of 100% and a test data prediction accuracy of 99.13%.

Automated Geographic-Information-System-Based Framework for Detecting Crosswalk Changes from Bi-Temporal High-Resolution Aerial Images

Identifying changes in pavement markings has become crucial for infrastructure monitoring, maintenance, development, traffic management, and safety. Automated extraction of the roadway geometry is critical in helping with this, given the increasing availability of high-resolution images and advancements in computer vision and object detection. Specifically, because of the substantial volume of satellite and high-resolution aerial images captured at different time instances, change detection has become a viable solution. In this study, an automated framework is developed to detect changes in crosswalks in Orange, Osceola, and Seminole counties in Florida, utilizing data extracted from high-resolution images obtained at various time intervals. Specifically, for Orange County, crosswalk changes between 2019 and 2021 were manually extracted, verified, and categorized as either new or modified crosswalks. For Seminole County, the developed model was used to automatically extract crosswalk changes between 2018 and 2021, while for Osceola County, changes between 2019 and 2020 were extracted. Findings indicate that Orange County witnessed approximately 2,094 crosswalk changes, with 314 occurring on state roads. In Seminole and Osceola counties, on the other hand, 1,040 and 1,402 crosswalk changes were observed on both local and state roads, respectively. Among these, 340 and 344 were identified on state roads in Seminole and Osceola, respectively. Spatiotemporal changes observed in crosswalks can be utilized to regularly update the existing crosswalk inventories, which is essential for agencies engaged in traffic and safety studies. Data extracted from these crosswalk changes can be combined with traffic and crash data to provide valuable insights to policymakers.

Enhancing Text-to-Image Generation: Integrating CLIP and Diffusion Models for Improved Visual Accuracy and Semantic Consistency

Abstract. Text-to-Image (T2I) generation focuses on producing images that precisely match given textual descriptions by combining techniques from computer vision and natural language processing (NLP). Existing studies have shown an innovative approach to enhance T2I generation by integrating Contrastive Language-Image Pretraining (CLIP) embeddings with a Diffusion Model (DM). The method involves initially extracting rich and meaningful text embeddings using CLIP, which are then transformed into corresponding images. These images are progressively refined through an iterative denoising process enabled by diffusion models. Comprehensive experiments conducted on the MS-COCO dataset validate the proposed method, demonstrating significant improvements in image fidelity and the alignment between text and images. When compared to traditional models like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), which often struggle with maintaining both visual quality and semantic accuracy, this hybrid model shows superior performance. Future research could explore optimizing hybrid models further and applying T2I technology to specialized fields, such as medical imaging and scientific visualization, expanding its potential use cases.

Automated estimation of cementitious sorptivity via computer vision.

Monitoring water uptake in cementitious systems is crucial to assess their durability against corrosion, salt attack, and freeze-thaw damage. However, gauging absorption currently relies on labor-intensive and infrequent weight measurements, as outlined in ASTM C1585. To address this issue, we introduce a custom computer vision model trained on 6234 images, consisting of 4000 real and 2234 synthetic, that automatically detects the water level in prismatic samples absorbing water. This model provides accurate and frequent estimations of water penetration values every minute. After training the model on 1440 unique data points, including 15 paste mixtures with varying water-to-cement ratios from 0.4 to 0.8 and curing periods of 1 to 7 days, we can now predict initial and secondary sorptivities in real time with high confidence, achieving R² > 0.9. Finally, we demonstrate its application on mortar and concrete systems, opening a pathway toward low-cost and automated durability assessment of construction materials.

Same same but different: The graded influence of vowel quality and prosodic prominence on letter detection.

This study investigates the impact of phonetic realisation and prosodic prominence on visual letter identification, focusing on the letter <e> in German bisyllabic words. Building upon previous research, a computerised letter search task was conducted with 78 skilled adult readers. Words featuring different phonetic realisations of <e > (/eː, ɛ, e, l̩, n̩, ɐ/) in stressed and unstressed first and second syllables were systematically included. Analyses of error rates and response times revealed a graded pattern in the detection of <e>, with the closed long (/eː/) and closed short (/e/) realisations being easiest to detect, open (/ɛ/) and near-open central (/ɐ/) vowels becoming incrementally harder, and silent vowels in syllabic consonants (/n̩/) being the most challenging. Results divided by position and stress of the syllable containing the target letter further indicated influences of prosodic prominence. The findings contribute to understanding the intricate interplay of grapheme-phoneme correspondences and prosodic structure in skilled readers' visual letter recognition.

You Only Look Once Version 5 and Deep Simple Online and Real-Time Tracking Algorithms for Real-Time Customer Behavior Tracking and Retail Optimization

The speedy progress of computer vision and machine learning engineering has inaugurated novel means for improving the purchasing experiment in brick-and-mortar stores. This paper examines the utilization of YOLOv (You Only Look Once) and DeepSORT (Deep Simple Online and Real-Time Tracking) algorithms for the real-time detection and analysis of the purchasing penchant in brick-and-mortar market surroundings. By leveraging these algorithms, stores can track customer behavior, identify popular products, and monitor high-traffic areas, enabling businesses to adapt quickly to customer preferences and optimize store layout and inventory management. The methodology involves the integration of YOLOv5 for accurate and rapid object detection combined with DeepSORT for the effective tracking of customer movements and interactions with products. Information collected in in-store cameras and sensors is handled to detect tendencies in customer behavior, like repeatedly inspected products, periods expended in specific intervals, and product handling. The results indicate a modest improvement in customer engagement, with conversion rates increasing by approximately 3 percentage points, and a decline in inventory waste levels, from 88% to 75%, after system implementation. This study provides essential insights into the further integration of algorithm technology in physical retail locations and demonstrates the revolutionary potential of real-time behavior tracking in the retail industry. This research determines the foundation for future developments in functional strategies and customer experience optimization by offering a solid framework for creating intelligent retail systems.

Exploring the Role of Transformer Models in Vision, Multi-Modal, and Orbital Data for Autonomous Driving

Abstract. Autonomous driving aims to reduce human error in driving, improve traffic efficiency, and provide a more comfortable driving experience. The integration of computer vision, advanced sensors, and machine learning has been pivotal in this advancement. The introduction of Transformer models has particularly revolutionized the field by offering a novel approach to processing data through attention mechanisms, which is crucial for tasks involving complex relationships between data elements. The paper categorizes research into three main approaches based on input data types: camera-based perception, multi-modal data fusion, and orbital data integration. As autonomous driving technology progresses towards higher levels of autonomy, with L2+ systems becoming standard, challenges remain in accurately interpreting complex environments, handling edge cases, and navigating legal and regulatory landscapes. The paper concludes that while Artificial Intelligence (AI) and deep learning advancements have brought autonomous driving closer to full realization, further research is necessary to address current limitations and ensure safe and reliable autonomous vehicle operation.

A tactile perception method with flexible grating structural color

Abstract Affordable high-resolution cameras and state-of-the-art computer vision techniques have led to the emergence of various vision-based tactile sensors. However, current vision-based tactile sensors mainly depend on geometric optics or marker tracking for tactile assessments, resulting in limited performance. To solve this dilemma, we introduce optical interference patterns as the visual representation of tactile information for flexible tactile sensors. We propose a novel tactile perception method and its corresponding sensor combining structural colors from flexible blazed gratings with deep learning. The richer structural colors and finer data processing foster the tactile estimation performance. The proposed sensor has an overall normal force magnitude accuracy of 6 mN, a planar resolution of 79 μm, and a contact-depth resolution of 25 μm. This work presents a promising tactile method that combines wave optics, soft materials, and machine learning. It demonstrates well in tactile measurement, and can be expanded into multiple sensing fields.

Backdoor Attacks and Defenses Targeting Multi-Domain AI Models: A Comprehensive Review

Since the emergence of security concerns in artificial intelligence (AI), there has been significant attention devoted to the examination of backdoor attacks. Attackers can utilize backdoor attacks to manipulate model predictions, leading to significant potential harm. However, current research on backdoor attacks and defenses in both theoretical and practical fields still has many shortcomings. To systematically analyze these shortcomings and address the lack of comprehensive reviews, this paper presents a comprehensive and systematic summary of both backdoor attacks and defenses targeting multi-domain AI models. Simultaneously, based on the design principles and shared characteristics of triggers in different domains and the implementation stages of backdoor defense, this paper proposes a new classification method for backdoor attacks and defenses. We use this method to extensively review backdoor attacks in the fields of computer vision and natural language processing, and also examine the current applications of backdoor attacks in audio recognition, video action recognition, multimodal tasks, time series tasks, generative learning, and reinforcement learning, while critically analyzing the open problems of various backdoor attack techniques and defense strategies. Finally, this paper builds upon the analysis of the current state of AI security to further explore potential future research directions for backdoor attacks and defenses.

Method to Estimate Dislocation Densities from Images of α‐Ga2O3‐Based Corundum Oxides Using the Computer Vision YOLO Algorithm

This work applies the computer vision “You only look once” (YOLO) algorithm to extract bounding boxes around dislocations in weak‐beam dark‐field transmission electron microscopy (WBDF TEM) images of semiconductor thin films. A formula is derived to relate the sum of the relative heights of the bounding boxes to the dislocation densities in the films. WBDF TEM images reported in the literature and taken from our α‐Ga2O3 samples are divided into train, evaluation, and test datasets. Different models are trained using the train dataset and evaluated using the evaluation dataset to find the best confidence values, which are used to select the best model based on the performance against the test data set. For α‐Ga2O3 thin films, dislocation density output by this model is on average ≈58% of those estimated by the traditional Ham method. A factor of 4/π may contribute to the systematic underestimation of the model versus the Ham method.

Mapping Computer Vision Syndrome: An Engineering Problem in Human–Computer Interaction

Mapping Computer Vision Syndrome: An Engineering Problem in Human–Computer Interaction

Human Operator Mental Fatigue Assessment Based on Video: ML-Driven Approach and Its Application to HFAVD Dataset

The detection of the human mental fatigue state holds immense significance due to its direct impact on work efficiency, specifically in system operation control. Numerous approaches have been proposed to address the challenge of fatigue detection, aiming to identify signs of fatigue and alert the individual. This paper introduces an approach to human mental fatigue assessment based on the application of machine learning techniques to the video of a working operator. For validation purposes, the approach was applied to a dataset, “Human Fatigue Assessment Based on Video Data” (HFAVD) integrating video data with features computed by using our computer vision deep learning models. The incorporated features encompass head movements represented by Euler angles (roll, pitch, and yaw), vital signs (blood pressure, heart rate, oxygen saturation, and respiratory rate), and eye and mouth states (blinking and yawning). The integration of these features eliminates the need for the manual calculation or detection of these parameters, and it obviates the requirement for sensors and external devices, which are commonly employed in existing datasets. The main objective of our work is to advance research in fatigue detection, particularly in work and academic settings. For this reason, we conducted a series of experiments by utilizing machine learning techniques to analyze the dataset and assess the fatigue state based on the features predicted by our models. The results reveal that the random forest technique consistently achieved the highest accuracy and F1-score across all experiments, predominantly exceeding 90%. These findings suggest that random forest is a highly promising technique for this task and prove the strong connection and association among the predicted features used to annotate the videos and the state of fatigue.

Movement Recognition and Injury Risk Assessment of Wushu Athletes Based on Computer Vision

This paper aims to study a method of movement recognition and injury risk assessment for Wushu athletes based on computer vision. Studying the optimization of Wushu athletes’ movement trajectory can effectively improve the athletes’ movement quality. Based on a hybrid real-time synchronization algorithm, the arm motion trajectory model of martial arts athletes is studied. A dynamic and static arm recognition algorithm is proposed, and the motion feature extraction method in Wushu movement is studied. Dynamic arm recognition typically relies on the collection of video or image sequences. These sequences contain the position, shape, and motion trajectory of the arm at different time points. Static arm recognition mainly relies on the acquisition of a single image or image frame. The dynamic arm recognition algorithm captures the motion trajectory and changes of the arm by processing video or image sequences, while the static arm recognition algorithm only processes individual images or image frames. On this basis, the design of the computer software architecture of the digital site system is completed. The real-time collection, analysis, display and storage of motion information and assembly configuration are realized. The simulation results show that the method has high accuracy and provides a powerful scientific basis for improving athletes’ movement injuries. To predict the risk of injury an athlete may suffer during training or competition.

MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images

Semantic segmentation of remote sensing images is a fundamental task in computer vision, holding substantial relevance in applications such as land cover surveys, environmental protection, and urban building planning. In recent years, multi-modal fusion-based models have garnered considerable attention, exhibiting superior segmentation performance when compared with traditional single-modal techniques. Nonetheless, the majority of these multi-modal models, which rely on Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs) for feature fusion, face limitations in terms of remote modeling capabilities or computational complexity. This paper presents a novel Mamba-based multi-modal fusion network called MFMamba for semantic segmentation of remote sensing images. Specifically, the network employs a dual-branch encoding structure, consisting of a CNN-based main encoder for extracting local features from high-resolution remote sensing images (HRRSIs) and of a Mamba-based auxiliary encoder for capturing global features on its corresponding digital surface model (DSM). To capitalize on the distinct attributes of the multi-modal remote sensing data from both branches, a feature fusion block (FFB) is designed to synergistically enhance and integrate the features extracted from the dual-branch structure at each stage. Extensive experiments on the Vaihingen and the Potsdam datasets have verified the effectiveness and superiority of MFMamba in semantic segmentation of remote sensing images. Compared with state-of-the-art methods, MFMamba achieves higher overall accuracy (OA) and a higher mean F1 score (mF1) and mean intersection over union (mIoU), while maintaining low computational complexity.