Computer Vision Methods Research Articles

Can ChatGPT4-vision identify radiologic progression of multiple sclerosis on brain MRI?

The large language model ChatGPT can now accept image input with the GPT4-vision (GPT4V) version. We aimed to compare the performance of GPT4V to pretrained U-Net and vision transformer (ViT) models for the identification of the progression of multiple sclerosis (MS) on magnetic resonance imaging (MRI). Paired coregistered MR images with and without progression were provided as input to ChatGPT4V in a zero-shot experiment to identify radiologic progression. Its performance was compared to pretrained U-Net and ViT models. Accuracy was the primary evaluation metric and 95% confidence interval (CIs) were calculated by bootstrapping. We included 170 patients with MS (50 males, 120 females), aged 21-74 years (mean 42.3), imaged at a single institution from 2019 to 2021, each with 2-5 MRI studies (496 in total). One hundred seventy patients were included, 110 for training, 30 for tuning, and 30 for testing; 100 unseen paired images were randomly selected from the test set for evaluation. Both U-Net and ViT had 94% (95% CI: 89-98%) accuracy while GPT4V had 85% (77-91%). GPT4V gave cautious nonanswers in six cases. GPT4V had precision (specificity), recall (sensitivity), and F1 score of 89% (75-93%), 92% (82-98%), 91 (82-97%) compared to 100% (100-100%), 88 (78-96%), and 0.94 (88-98%) for U-Net and 94% (87-100%), 94 (88-100%), and 94 (89-98%) for ViT. The performance of GPT4V combined with its accessibility suggests has the potential to impact AI radiology research. However, misclassified cases and overly cautious non-answers confirm that it is not yet ready for clinical use. GPT4V can identify the radiologic progression of MS in a simplified experimental setting. However, GPT4V is not a medical device, and its widespread availability highlights the need for caution and education for lay users, especially those with limited access to expert healthcare. Without fine-tuning or the need for prior coding experience, GPT4V can perform a zero-shot radiologic change detection task with reasonable accuracy. However, in absolute terms, in a simplified "spot the difference" medical imaging task, GPT4V was inferior to state-of-the-art computer vision methods. GPT4V's performance metrics were more similar to the ViT than the U-net. This is an exploratory experimental study and GPT4V is not intended for use as a medical device.

Characterization of dynamic features in the walking videos of patients with adolescent idiopathic scoliosis based on moving entropy

Adolescent idiopathic scoliosis (AIS), which typically occurs in patients between the ages of 10 and 18, can be caused by a variety of reasons, and no definitive cause has been found. Early diagnosis of AIS or timely recognition of progression is crucial for the prevention of spinal deformity and the reduction of the risk of surgery or postponement. However, it remains a significant challenge. The purpose of this study is to develop an easy-to-use, non-invasive, and portable method for early diagnosis of AIS. A new framework of moving entropy-based computer vision method is presented, which can determine the severity of AIS by analyzing patients’ walking videos. First, Alphapose system and direct linear transformation method are employed to estimate 3D keypoint coordinates. Then, the joint angle-based and joint distance-based dynamic network are constructed. Based on these works, the new measures called moving angle entropy and moving edge-weighted graph entropy are proposed and fused using canonical correlation analysis. Finally, the power spectral exponents of entropy sequences are calculated and used in recognizing the severity of AIS. A comparison with healthy subjects and statistical analysis for entropy values can provide effective information for quantifying AIS. The recognized results of our proposed method were also comparable with the clinical diagnosis of Cobb angle from imaging by a certified clinician.

An intelligent model approach for leakage detection of modified atmosphere pillow bags

Modified atmosphere pillow bags have been widely used to package various food products due to their advantages for preservation and shipment. Sealing defects are statistically inevitable, although modern packaging machinery and manual inspection utilized by manufacturers continue reducing the leakage probability. Hence the bag contents may spoil if the seal is broken. Instead of manual inspection and various destructive methods utilized by factories, this study introduces non-destructive leakage detection using deep learning methods. Firstly, a squeezing method is developed to aggravate the feature difference between positive samples and negative samples without destroying the bag content, thus 2160 images of three different pillow bags are acquired to establish dataset. Secondly, the deep learning model Vision Transformer (ViT) is deployed and studied so that feasibility of computer vision method is verified. Then the Semantic segmentation and Contour Extraction model combining ViT (SCE-ViT) is proposed and improved to the Multi-dimensional Fusion model (SCE-MdF). The accuracies of SCE-MdF reached 97.5%, 97.5%, and 97.5%, respectively. The F1-scores of SCE-MdF reached 97.6%, 97.6%, and 97.4%, respectively. Compared to averaged accuracies of SCE-ViT, accuracies introduced in the ultimate model SCE-MdF improved by 19.17%, 5.84%, and 11.67%, respectively. Therefore, combination of unique squeezing method and Semantic segmentation Contour Extraction with Multi-dimensional Fused ViT, is eventually validated viable on leakage detection of modified atmosphere pillow bags. Hence a cost-effective, efficient and non-destructive leakage detection method for modified atmosphere pillow bags in relevant industry is introduced, filling a gap between artificial intelligence and food packaging industry.

Real-Time Detection of Customer-Induced Damage in Printed Circuit Boards Using Mobile Devices and YOLO Detectors

The identification of Consumer-induced damage is essential for electronics manufacturers’ warranty programs. Consumer Induced Damage (CID) is any damage caused by an unauthorized person, including the consumer. The product warranty does not cover these damages, avoiding expenses in the manufacturer’s revenue. The consumer-induced damage warranty process is usually done manually by technically trained people. However, this task demands a lot of attention to detail, can be time-consuming, and is susceptible to human errors. With this in mind, this work presents an object detection model for low-computational-cost devices that uses computer vision and deep learning methods with YOLO detectors embedded in mobile devices to identify consumer-induced damages on printed circuit boards (PCB). We conducted sixteen experiments with four YOLO neural network architectures and successfully developed a mobile application for CID detection. Our best model achieved a mAP@0.5 of 33.1% and an average of 5.7 FPS on real mobile devices.

Intelligent Medical System for Diagnosis of Intervertebral Disc Deformation

The article examines the application of semi-supervised learning and computer vision methods for segmentation of spine MR images and diagnosis of deformation of intervertebral discs. Existing neural network architectures for spine MR image segmentation and semi-supervised learning methods used in medical image segmentation are reviewed. A system for diagnosis deformation of intervertebral discs is proposed, which consists of two modules: a segmentation module and a diagnostic module. The implementation of the proposed system using two convolutional neural networks is presented: U-Net for segmentation of spine MR images and ResNet for classification of the degree of deformation of each intervertebral disc based on the Pfirrmann classification of degenerative changes. The software implementation of the medical diagnosis system was developed in the Python programming language using the PyTorch library. Neural networks were trained on an open dataset of spine MR images using a variant of the Mean Teacher semi-supervised learning method. As a result of the verification, it was found that the system is capable of performing segmentation with high accuracy. It was found that the exact prediction of the grade of degenerative changes according to Pfirrmann remains a difficult task, but the introduction of another classification made it possible to increase the accuracy of diagnosis of deformation of intervertebral discs. The proposed medical system involves the addition of new diagnostic modules, which makes it possible to use it for the comprehensive analysis of various spine diseases

Methods for improving the functional efficiency of UAVS

The integration of innovative solutions into all levels of the UAV system can significantly improve the efficiency of their operation in a variety of conditions and tasks. The main challenge in using drones is efficient flight route planning. The imperfection of route planning models and methods leads to the loss of unmanned aerial vehicles and a decrease in the efficiency of performing tasks. This review article discusses methods that automate route planning processes and reduce the influence of the human factor on control. An important issue is detecting attacks on the UAV navigation system, as such attacks directly affect the ability of drones to perform their functions, so the article discusses methods for detecting GPS spoofing and a way to automatically return the UAV to the launch area when external cyberattacks are detected. The article also outlines promising areas of artificial intelligence application in drones. Innovative approaches, such as generative design, swarm control algorithms, computer vision, and information-extreme learning methods, are considered, which help to expand the functionality of UAVs.

Optimizing Quality Control: A Comprehensive Analysis of Computer Vision Methods for Assessing Vegetables and Fruits

Efficient quality control in the agriculture sector, particularly regarding the inspection of vegetables and fruits, stands as a critical necessity in today's health-focused industry. Conventional fruit grading methods, ill-suited for large-scale production, demand an automated, non-invasive, and economically feasible substitute. Computer vision emerges as a promising avenue, leveraging image analysis and machine learning algorithms to evaluate the quality of produce. The convergence of computer vision and image processing technologies in contemporary agriculture has brought about a substantial transformation in quality assessment methodologies. This paper conducts an in-depth exploration of the amalgamation of computer vision and image processing techniques for the evaluation of agricultural produce quality. Through a comprehensive review, this scientific analysis investigates the integration of computer vision and image processing techniques in agricultural quality assessment. It scrutinizes key studies, their practical implementations, outcomes, and the research voids they reveal. Technological progressions within the agricultural domain have the potential to amplify productivity and curtail the circulation of flawed or substandard products. Moreover, this study deliberates on the forthcoming trends in computer vision technology applications, accentuating their prospective influence on the vegetables and fruits industry.

Additive Manufacturing of a Frost-Detection Resistive Sensor for Optimizing Demand Defrost in Refrigeration Systems.

This article presents the development of a resistive frost-detection sensor fabricated using Fused Filament Fabrication (FFF) with a conductive filament. This sensor was designed to enhance demand-defrost control in industrial refrigeration systems. Frost accumulation on evaporator surfaces blocks airflow and creates a thermal insulating barrier that reduces heat exchange efficiency, increasing energy consumption and operational costs. Traditional timed defrosting control methods can mitigate these effects but often lead to inefficiencies due to their inability to align with actual frost accumulation, which can vary according to system and environmental conditions. Frost-detection sensors aim to solve this problem by acting as a tool to support defrosting control. A series of tests were conducted to evaluate the sensor's performance in detecting frost under controlled conditions on a heat exchanger (HX). The sensor reliably detected frost in all cases, demonstrating its effectiveness in real-time frost detection. The sensor measurements were validated by comparison with results obtained through a computer vision method, confirming its reliability. It was also found that the sensor can detect temperature changes. This advancement in sensor technology highlights the potential of additive manufacturing to provide cost-effective, customizable, replicable, and compact sensor designs, contributing to improved system performance and energy efficiency in refrigeration systems.

Design and Application of Clerical Style Recognition System Based on Data Mining Algorithm

With the advancements in high-definition imaging and parallel computing hardware, the analysis of massive visual data has become a key focus in pattern recognition and artificial intelligence. Chinese calligraphy, an integral part of traditional culture, has seen digitization of numerous works stored in digital libraries. However, current automatic calligraphy character recognition technology is limited, necessitating the development of efficient computer vision methods for recognizing calligraphy styles. Data mining, crucial in artificial intelligence, involves extracting valuable knowledge from vast and noisy datasets. Recent simulation results show promising recognition rates for Chinese text images, with an average recognition time of 5 seconds per 100 words. This system significantly improves handwriting recognition accuracy compared to existing algorithms, though further refinement and expansion are needed for optimal functionality.

Novel Building Detection and Location Intelligence in Aerial Satellite Imagery

The accurate detection and extraction of building information from aerial imagery is of paramount importance in urban planning, land use analysis, and disaster management. This study presents a comprehensive investigation into the development of a robust and efficient methodology for building detection in satellite imagery utilizing state-of-the-art deep learning techniques. We conducted a comparative analysis of three distinct semantic segmentation models based on the U-Net architecture: a baseline U-Net trained from scratch, a U-Net incorporating a pre-trained ResNet34 encoder, and a U-Net with custom architectural enhancements. Our methodological approach encompassed data augmentation strategies, transfer learning techniques, and ensemble methods to optimize model performance. The Inria Aerial Image Labelling Dataset served as the primary source for model training and validation. We explored the efficacy of various loss functions, including dice loss, focal loss, and weighted cross-entropy, to address class imbalance and enhance segmentation accuracy. Model performance was rigorously evaluated using a comprehensive set of metrics, including pixel-wise accuracy, Intersection over Union (IoU), and F1-score. Our highest-performing individual model achieved a dice score of 92 percent on the validation set, while the implementation of ensemble techniques further improved detection accuracy to 93 percent on the heldout test set. Post-processing algorithms, incorporating traditional computer vision methods, were applied to refine building polygon delineation. This research demonstrates the efficacy of deep learning-based segmentation approaches for building detection in aerial imagery and offers valuable insights into potential applications across various domains, including urban planning, construction monitoring, and disaster response. Future research directions may explore building classification, change detection analysis, and model optimization for real-time applications in dynamic urban environments.

Optimization of Cocoa Pods Maturity Classification Using Stacking and Voting with Ensemble Learning Methods in RGB and LAB Spaces.

Determining the maturity of cocoa pods early is not just about guaranteeing harvest quality and optimizing yield. It is also about efficient resource management. Rapid identification of the stage of maturity helps avoid losses linked to a premature or late harvest, improving productivity. Early determination of cocoa pod maturity ensures both the quality and quantity of the harvest, as immature or overripe pods cannot produce premium cocoa beans. Our innovative research harnesses artificial intelligence and computer vision technologies to revolutionize the cocoa industry, offering precise and advanced tools for accurately assessing cocoa pod maturity. Providing an objective and rapid assessment enables farmers to make informed decisions about the optimal time to harvest, helping to maximize the yield of their plantations. Furthermore, by automating this process, these technologies reduce the margins for human error and improve the management of agricultural resources. With this in mind, our study proposes to exploit a computer vision method based on the GLCM (gray level co-occurrence matrix) algorithm to extract the characteristics of images in the RGB (red, green, blue) and LAB (luminance, axis between red and green, axis between yellow and blue) color spaces. This approach allows for in-depth image analysis, which is essential for capturing the nuances of cocoa pod maturity. Next, we apply classification algorithms to identify the best performers. These algorithms are then combined via stacking and voting techniques, allowing our model to be optimized by taking advantage of the strengths of each method, thus guaranteeing more robust and precise results. The results demonstrated that the combination of algorithms produced superior performance, especially in the LAB color space, where voting scored 98.49% and stacking 98.71%. In comparison, in the RGB color space, voting scored 96.59% and stacking 97.06%. These results surpass those generally reported in the literature, showing the increased effectiveness of combined approaches in improving the accuracy of classification models. This highlights the importance of exploring ensemble techniques to maximize performance in complex contexts such as cocoa pod maturity classification.

Computer-Integrated Surface Image Processing of Hydrogen-Saturated Steel Wear Products

This paper briefly describes the conceptual direction of the application of computer vision (CV) methods that involve controlling the morphology of hydrogenated wear particles (WPs). During long-term operation, in the surface and subsurface layers of the materials of parts in the tribo-joint, changes in the micromechanisms of fracture occur, which change the morphology of WPs. It has been shown that the developed computer program (CP) can be used to monitor the fractography of the surface of wear particles, and, accordingly, it is possible to control changes in the surface morphology. Therefore, it is possible to predict the operational stability and durability of the tribo-joint. A conceptual scheme for determining the performance of a tribotechnical assembly depending on the determined parameters of WPs is presented. The modes marked on it, including normal operation, transient, run-in, and catastrophic failure, can be evaluated by robotics approaches.

A Review of Passenger Counting in Public Transport Concepts with Solution Proposal Based on Image Processing and Machine Learning

The accurate counting of passengers in public transport systems is crucial for optimizing operations, improving service quality, and planning infrastructure. It can also contribute to reducing the number of public transport lines where a high number of vehicles is not needed in certain periods during the year, but also by increasing the number of lines where the need is increased. This paper provides a comprehensive review of current methodologies and technologies used for passenger counting, without the actual implementation of the automatic passenger counting system (APC), but with a proposal based on image processing and machine learning techniques and concepts, since it represents one of the most used approaches. The research explores various technologies and algorithms, like card swiping, infrared, weight and ultrasonic sensors, RFID, Wi-Fi, Bluetooth, LiDAR, thermos cameras, including CCTV cameras and traditional computer vision methods, and advanced deep learning approaches, highlighting their strengths and limitations. By analyzing recent advancements and case studies, this review aims to offer insights into the effectiveness, scalability, and practicality of different passenger counting solutions and offers a solution proposal. The research also analyzed the current General Data Protection Regulation (GDPR) that applies to the European Union and how it affects the use of systems like this. Future research directions and potential areas for technological innovation are also discussed to guide further developments in this field.

Graph convolutional network for predicting abnormal grain growth in Monte Carlo simulations of microstructural evolution

Recent developments in graph neural networks show promise for predicting the occurrence of abnormal grain growth, which has been a particularly challenging area of research due to its apparent stochastic nature. In this study, we generate a large dataset of Monte Carlo simulations of abnormal grain growth. We train simple graph convolution networks to predict which initial microstructures will exhibit abnormal grain growth, and compare the results to a standard computer vision approach for the same task. The graph neural network outperformed the computer vision method and achieved 73% prediction accuracy and fewer false positives. It also provided some physical insight into feature importance and the relevant length scale required to maximize predictive performance. Analysis of the uncertainty in the Monte Carlo simulations provides additional insights for ongoing work in this area.

Efficient Masked Autoencoders With Self-Consistency.

Inspired by the masked language modeling (MLM) in natural language processing tasks, the masked image modeling (MIM) has been recognized as a strong self-supervised pre-training method in computer vision. However, the high random mask ratio of MIM results in two serious problems: 1) the inadequate data utilization of images within each iteration brings prolonged pre-training, and 2) the high inconsistency of predictions results in unreliable generations, i.e., the prediction of the identical patch may be inconsistent in different mask rounds, leading to divergent semantics in the ultimately generated outcomes. To tackle these problems, we propose the efficient masked autoencoders with self-consistency (EMAE) to improve the pre-training efficiency and increase the consistency of MIM. In particular, we present a parallel mask strategy that divides the image into K non-overlapping parts, each of which is generated by a random mask with the same mask ratio. Then the MIM task is conducted parallelly on all parts in an iteration and the model minimizes the loss between the predictions and the masked patches. Besides, we design the self-consistency learning to further maintain the consistency of predictions of overlapping masked patches among parts. Overall, our method is able to exploit the data more efficiently and obtains reliable representations. Experiments on ImageNet show that EMAE achieves the best performance on ViT-Large with only 13% of MAE pre-training time using NVIDIA A100 GPUs. After pre-training on diverse datasets, EMAE consistently obtains state-of-the-art transfer ability on a variety of downstream tasks, such as image classification, object detection, and semantic segmentation.

A computer vision and RFID fusion-based method for measuring individual feed intake and its application for detecting individual differences in feed efficiency of large yellow croaker (Larimichthys crocea)

A computer vision and RFID fusion-based method for measuring individual feed intake and its application for detecting individual differences in feed efficiency of large yellow croaker (Larimichthys crocea)

Extensible portal frame bridge synthetic dataset for structural semantic segmentation

A number of bridges have collapsed around the world over the past years, with detrimental consequences on safety and traffic. To a large extend, such failures can be prevented by regular bridge inspections and maintenance, tasks that fall in the general category of structural health monitoring (SHM). Those procedures are time and labor consuming, which partly accounts for their neglect. Computer vision and artificial intelligence (AI) methods have the potential to ease this burden, by fully or partially automating bridge monitoring. A critical step in this automation is the identification of a bridge’s structural components. In this work, we propose an extensible synthetic dataset for structural component semantic segmentation of portal frame bridges (PFBridge). We first create a 3 dimensional (3D) generic mesh representing the bridge geometry, while respecting a set of rules. The definition of new, or the extension of the existing rules can adjust the dataset to specific needs. We then add textures and other realistic elements to the model, and create an automatically annotated synthetic dataset. The synthetic dataset is used in order to train a deep semantic segmentation model to identify bridge components on bridge images. The amount of available real images is not sufficient to entirely train such a model, but is used to refined the model trained on the synthetic data. We evaluate the contribution of the dataset to semantic segmentation by training several segmentation models on almost 2,000 synthetic images and then finetuning with 88 real images. The results show an increase of 28% on the F1-score when the synthetic dataset is used. To demonstrate a potential use case, the model is integrated in a 3D point cloud capturing system, producing an annotated point cloud where each point is associated with a semantic category (structural component). Such a point cloud can then be used in order to facilitate the generation of a bridge’s digital twin.

Development of Software Interface for AI-Driven Weed Control in Robotic Vehicles, with Time-Based Evaluation in Indoor and Field Settings

Herbicide blanket application is widely practiced by farmers to chemically control weeds in the field, thereby enhancing productivity and improving crop quality. However, their repetitive use also has caused several negative issues on the environment and human health. To overcome these negative issues precision site-specific weed control can be used to reduce the herbicide application by targeted application on only weed areas. Hence, a robotic platform utilizing NVIDIA Jetson embedded device as control and processing unit was developed and tested on lab and field conditions for weed control. The average time taken for data acquisition and artificial intelligence-based computer vision tasks was tested under both lab and field conditions. Moreover, a grid map creation algorithm using YOLOv4 deep learning algorithm was evaluated to control the nozzles of the robotic platform. Integrating all these components together enabled real-time weed management approaches, enhancing the precision and efficiency of herbicide application Independent paired t-tests reveal that there is no significant difference in computational time between lab and field testing. Conversely, independent paired t-test revels that there is significant difference in image size between lab and field testing. The reduction of herbicide application based on grid map was obtained 79 to 80 %, which does not include YOLOv4 algorithm failure and system synchronization failure. The study suggests the importance of field testing for real-time applications of the robotic platform by using deep learning computer vision methods for weed control.

IoT Sensor-Based Convolutional Neural Network System for Concealed Weapon Detector for Security Enhancement

Security has been a major concern in our societies due to the rise in crime rate, most especially in a crowded area. Concealed weapons have been posing a significant threat to government, law enforcement, security agencies, and civilians. Existing weapons detection systems seem to be not culpable of detecting concealed weapons without the cooperation of the person being searched. There remains a need for a weapons detector that can detect and identify concealed weapons for security enhancement in Nigeria. For this purpose, computer vision methods and a deep learning approach were applied for the identification of a weapon from captured images downloaded from the internet as a prototype for the study. Recent work in deep learning and machine learning using convolutional neural networks has shown considerable progress in the areas of object detection and recognition. The CNN algorithms are trained on the collected datasets to identify and differentiate between weapons and non-weapons. We built a concealed weapon detection system prototype and conducted a series of experiments to test the system's accuracy, precision, and false positives. The models were compared by evaluating their average values of sensitivity, specificity, F1 score, accuracy, and the area under the receiver operating characteristic curve (AUC). The experimental findings clearly demonstrated that the ResNet-50 model performed better than the VGG-16 and Alex Net models in terms of sensitivity, specificity, and accuracy.

Validation studies of the FLASH-TV system to passively measure children’s TV viewing

TV viewing is associated with health risks, but existing measures of TV viewing are imprecise due to relying on self-report. We developed the Family Level Assessment of Screen use in the Home (FLASH)-TV, a machine learning pipeline with state-of-the-art computer vision methods to measure children’s TV viewing. In three studies, lab pilot (n = 10), lab validation (n = 30), and home validation (n = 20), we tested the validity of FLASH-TV 3.0 in task-based protocols which included video observations of children for 60 min. To establish a gold-standard to compare FLASH-TV output, the videos were labeled by trained staff at 5-second epochs for whenever the child watched TV. For the combined sample with valid data (n = 59), FLASH-TV 3.0 provided a mean 85% (SD 8%) accuracy, 80% (SD 17%) sensitivity, 86% (SD 8%) specificity, and 0.71 (SD 0.15) kappa, compared to gold-standard. The mean intra-class correlation (ICC) of child’s TV viewing durations of FLASH-TV 3.0 to gold-standard was 0.86. Overall, FLASH-TV 3.0 correlated well with the gold standard across a diverse sample of children, but with higher variability among Black children than others. FLASH-TV provides a tool to estimate children’s TV viewing and increase the precision of research on TV viewing’s impact on children’s health.