Combines Computer Vision Research Articles

An automated nondestructive testing system for the surface of pressure pipeline welds

AbstractPressure pipelines are widely used in hydropower generation, oil and gas transmission, and other fields. After years of operation, a pressure pipeline needs regular maintenance to ensure its safety. At present, manual detection methods are unable to meet this demand. An automatic pressure pipeline detection technology is urgently needed to achieve improved efficiency and accuracy. On the basis of the above requirements, a wall‐climbing robot is designed for automatic pressure pipe inspection and maintenance tasks. Moreover, rapid nondestructive testing of welds on the inner surface of pressure pipelines was performed, and a weld tracking function was developed for wall‐climbing robots. We propose an algorithm framework for weld recognition and centerline extraction by combining computer vision technology with traditional image processing technology using visual images. The experimental verification of the wall‐climbing robot designed in this paper and the algorithm framework for weld recognition and centerline extraction were performed based on actual pressure pipelines. The results show that the algorithm framework developed based on the wall‐climbing robot equipped with an industrial camera for pressure pipeline weld detection can achieve greatly improved efficiency, and the actual weld identification accuracy can exceed 90%, which is very meaningful for practical applications.

Yoga Pose Trainer and Analyser: Machine Learning (TensorFlow and Open CV)

The "Yoga Pose Trainer and Analyzer" is a sophisticated software system designed to enhance the yoga experience for practitioners of all levels. This project combines computer vision, machine learning, and user-friendly interfaces to provide valuable insights and guidance during yoga sessions. Users can interact with the system, receiving real-time feedback on their yoga poses and personalized recommendations to improve their practice. The core functionalities of the system include pose recognition and classification. Using advanced computer vision techniques, the system identifies key points on the user's body and classifies their pose, accurately determining which yoga asana (pose) they are performing. Additionally, users can specify their goals or desired poses, and the system tailors their practice accordingly. The software provides a comprehensive library of yoga poses, offering detailed instructions and highlighting the benefits of each pose. Users can track their progress over time, set goals, and receive guidance on how to achieve their objectives. Safety and privacy considerations are paramount, with the system promoting proper form to prevent injury and ensuring user data security.

Exploiting image logs to reduce drilling hazards: an innovative Artificial Intelligence methodology applied in East Africa

SUMMARY Image logs are crucial for petrophysical and structural characterization of drilled formations. Logging while drilling (LWD) image logs allow near real-time characterization of formations during drilling. Interpretation time required by humans can become critical, as structural interpretation provides the most crucial information for decision-making steps with respect to drilling operations. An example of this occurred in a deepwater system of reservoirs in East Africa. During the drilling of some wells, drilling was interrupted by unexpected borehole stability issues observed in a mass transport deposit. Those problems can detrimentally affect operations by impacting the efficiency of drilling. Therefore, a procedure for the automation of LWD data analysis was developed aimed at reducing drilling risk associated with borehole stability. A workflow is proposed based on an innovative assortment of AI methodologies for automatic real-time interpretation of LWD image log, applied to density image logs. The goal is to mimic manual human dip picking, combining computer vision, numerical series analysis and machine learning. Computer vision techniques are applied to detect the main density contrasts, representing the most likely surfaces. Sharp contrasts in the image are then investigated by dynamic time warping for similarity analysis. Finally, the geological plane is defined as a regression plane passing through the most similar contrasts, and a surface confidence criterion is defined based on the homogeneity of contrast along the curve suggested by the regression. Feature identification by a geologist was used as benchmark for comparison and metrics. Accuracy is referred to the difference between apparent dip and azimuth of geological surfaces coming from human and automated workflows, following the idea that two similar geometric results would eventually lead to equally similar geological considerations, and thus to similar operational measures. In general, a prediction accuracy of 60 per cent for the tested wells is observed, considering the total set of surfaces coming from the model. Nevertheless, the results suggest a positive correlation between the apparent dip of the discontinuities and the model error and a general less confidence for the surfaces with greater dip. By isolating the subset of low dipping features generated by the model, that is up to 35°, the accuracy appears to increase up to more than 70 per cent, suggesting that correlation is more difficult for high dipping geological planes. This class of curves also shows a general lower confidence and much of this latter class of surfaces are more likely to be found in the depth interval in which borehole stability issues occurred.

Understanding Policy and Technical Aspects of AI-enabled Smart Video Surveillance to Address Public Safety

Recent advancements in artificial intelligence (AI) have seen the emergence of smart video surveillance (SVS) in many practical applications, particularly for building safer and more secure communities in our urban environments. Cognitive tasks, such as identifying objects, recognizing actions, and detecting anomalous behaviors, can produce data capable of providing valuable insights to the community through statistical and analytical tools. However, artificially intelligent surveillance systems design requires special considerations for ethical challenges and concerns. The use and storage of personally identifiable information (PII) commonly pose an increased risk to personal privacy. To address these issues, this paper identifies the privacy concerns and requirements needed to address when designing AI-enabled smart video surveillance. Further, we propose the first end-to-end AI-enabled privacy-preserving smart video surveillance system that holistically combines computer vision analytics, statistical data analytics, cloud-native services, and end-user applications. Finally, we propose quantitative and qualitative metrics to evaluate intelligent video surveillance systems. The system shows the 17.8 frame-per-second (FPS) processing in extreme video scenes. However, considering privacy in designing such a system results in preferring the pose-based algorithm to the pixel-based one. This choice resulted in dropping accuracy in both action and anomaly detection tasks. The results drop from 97.48% to 73.72% in anomaly detection and 96% to 83.07% in the action detection task. On average, the latency of the end-to-end system is 36.1 seconds.

In-situ TEM investigation of void swelling in nickel under irradiation with analysis aided by computer vision

Understanding the stability of irradiation-induced voids in materials is important for engineering material's swelling behavior under irradiation. In-situ TEM offers a spatial and temporal resolution that is suitable for investigating the evolution of voids under irradiation. However, the in-situ videos have often been too large to be analyzed manually, leaving the valuable data underutilized. We developed a deep learning-based semantic segmentation model to consistently study the growth and shrinkage of voids in nickel under 1 MeV krypton ion irradiation at various temperatures from 525 °C to 650 °C. With a foil thickness near 100 nm and ion flux of 6.3 × 1011 ions⋅ cm−2⋅s−1, the pre-existing voids, which were created beforehand by irradiation at 600 °C to 0.5 dpa, shrank at low temperatures and grew at high temperatures under irradiation, where the transition occurred at 575 °C (∼0.5 TM). The observed stability transition provided new insight for the shrinkage mechanism of voids under irradiation. In addition, an annealing experiment on nickel, previously irradiated at 600 °C to 3 dpa, was performed sequentially at 650 °C to 720 °C to reveal the shrinkage rate of void as a function of temperature and void size. The advantage of combining computer vision and in-situ TEM to obtain comprehensive void evolution was demonstrated.

Analyzing architectural diversity in maize plants using the skeleton-image-based method

Shoot architecture in maize is critical since it determines resource use, impacts wind and rain damage tolerance, and affects yield stability. Quantifying the diversity among inbred lines in heterosis breeding is essential, especially when describing germplasm resources. However, traditional geometric description methods oversimplify shoot architecture and ignore the plant’s overall architecture, making it difficult to reflect and illustrate diversity. This study presents a new method to describe maize shoot architecture and quantifies its diversity by combining computer vision algorithms and persistent homology. Our results reveal that persistent homology can capture key characteristics of shoot architecture in maize and other details often overlooked by traditional geometric analysis. Based on this method, the morphological diversity of shoot architecture can be mined (quantified), and the main shoot architecture types can be obtained. Consequently, this method can easily describe the diversity of shoot architecture in many maize materials.

Gesture Recognition Algorithm Based on Computer Vision

With the advent of the era of artificial intelligence, computer vision and machine learning are developing rapidly, and human-computer interaction technology has become one of the important research directions. This paper mainly combines computer vision and other related technologies to analyze and judge the gestures, and predict the corresponding models according to the different matching of gestures, so as to recognize the corresponding gesture information. By converting the image into HSV space, HSV channel separation is realized, and then filtering and skin color segmentation are carried out to carry out morphological operation. Finally, shape matching is carried out according to the contour information of the image to judge the information of gesture. The experimental results show that the gesture recognition algorithm proposed in this paper can effectively judge the gesture information contained in the image, has good operation effect and strong practicability.

Vision Gloves for Blinds

For people who are visually impaired finding things in an indoor setting that are present in a typical environment might be a difficult chore in a visually impaired person's daily routine. They must be assisted by one or the other to find an object, so to overcome this, an idea has been proposed to develop a glove that can be used by visually impaired individuals to detect objects in an indoor environment and will also function as an artificial eye for those with visual impairments. The glove will have a camera in the hand that will feed real-time footage to the Raspberry Pi so that it can process it, a voice recognition module that will be attached to the Raspberry Pi to identify the object, and a micro vibrating motor to indicate the location of the object. The visually impaired person uses the speech module's built-in microphone to issue voice commands. An approach that uses object detection to find the object is used to locate it. By giving visually impaired people a hands-free means to find objects in enclosed places, the proposed smart glove has the potential to greatly improve their daily life. Hence an efficient and user-friendly system is planned to build by combining computer vision, voice recognition, and haptic feedback technology.

A review on the application of computer vision and machine learning in the tea industry

Tea is rich in polyphenols, vitamins, and protein, which is good for health and tastes great. As a result, tea is very popular and has become the second most popular beverage in the world after water. For this reason, it is essential to improve the yield and quality of tea. In this paper, we review the application of computer vision and machine learning in the tea industry in the last decade, covering three crucial stages: cultivation, harvesting, and processing of tea. We found that many advanced artificial intelligence algorithms and sensor technologies have been used in tea, resulting in some vision-based tea harvesting equipment and disease detection methods. However, these applications focus on the identification of tea buds, the detection of several common diseases, and the classification of tea products. Clearly, the current applications have limitations and are insufficient for the intelligent and sustainable development of the tea field. The current fruitful developments in technologies related to UAVs, vision navigation, soft robotics, and sensors have the potential to provide new opportunities for vision-based tea harvesting machines, intelligent tea garden management, and multimodal-based tea processing monitoring. Therefore, research and development combining computer vision and machine learning is undoubtedly a future trend in the tea industry.

A combined computer vision and image processing method for surface coverage measurement of shot peen forming

Coverage is an essential monitored parameter during the shot peen forming process. However, the conventional manual visual inspection methods suffer from low efficiency and poor accuracy. In this paper, a combined computer vision and image processing method is applied to realize the automatic measurement of shot peening surface coverage, and the critical issues involved, such as shot peening image pre-processing, indentation light spots recognition, shot-peened area segmentation, etc., are systematically investigated. The results indicate that the influence of machining streaks on the subsequent image processing is effectively reduced by adjusting the illumination conditions and adopting the median filtering algorithm. The number and location of indentations in the shot peening surface image are easily obtained by treating the central light spot as an identifier for the indentation. The features of the shot-peened area can be extracted effectively by segmenting the image reconstructed based on the light spots. The effectiveness of the coverage measurement method is verified by shot peen forming experiments, and the accuracy of the measurement results can be over 97 %. The influence of coverage on shot peening deformation is further examined using the results of algorithm processing, which is reflected in that the radius of curvature decreases with the coverage.

TagFocus: Towards Fine-Grained Multi-Object Identification in RFID-based Systems with Visual Aids

Obtaining fine-grained spatial information is of practical importance in Radio Frequency Identification (RFID)-based systems for enabling multi-object identification. However, as high-precision positioning remains impractical in commercial-off-the-shelf (COTS)-RFID systems, researchers propose to combine computer vision (CV) with RFID and turn the positioning problem into a matching problem. Promising though it seems, current methods fuse CV and RFID through converting traces of tagged objects extracted from videos by CV into phase sequences for matching, which is a dimension-reduced procedure causing loss of spatial resolution. Consequently, they fail in harsh conditions like small tag intervals and low reading rates. To address the limitation, we propose TagFocus to achieve fine-grained multi-object identification with visual aids in RFID systems. The key observation is that traces generated through different methods shall be compatible if they are of one identical object. Accordingly, a Transformer-based sequence-to-sequence (seq2seq) model is trained to generate a simulated trace for each candidate tag-object pair. And the trace of the right pair shall best match the observed trace directly extracted by CV. A prototype of TagFocus is implemented and extensively assessed in lab environments. Experimental results show that our system maintains a matching accuracy of over 91% in harsh conditions, outperforming state-of-the-art schemes by 27%.

A Framework for Understanding Unstructured Financial Documents Using RPA and Multimodal Approach

The financial business process worldwide suffers from huge dependencies upon labor and written documents, thus making it tedious and time-consuming. In order to solve this problem, traditional robotic process automation (RPA) has recently been developed into a hyper-automation solution by combining computer vision (CV) and natural language processing (NLP) methods. These solutions are capable of image analysis, such as key information extraction and document classification. However, they could improve on text-rich document images and require much training data for processing multilingual documents. This study proposes a multimodal approach-based intelligent document processing framework that combines a pre-trained deep learning model with traditional RPA used in banks to automate business processes from real-world financial document images. The proposed framework can perform classification and key information extraction on a small amount of training data and analyze multilingual documents. In order to evaluate the effectiveness of the proposed framework, extensive experiments were conducted using Korean financial document images. The experimental results show the superiority of the multimodal approach for understanding financial documents and demonstrate that adequate labeling can improve performance by up to about 15%.

Two-in-one system and behavior-specific brain synchrony during goal-free cooperative creation: an analytical approach combining automated behavioral classification and the event-related generalized linear model.

In hyperscanning studies of natural social interactions, behavioral coding is usually necessary to extract brain synchronizations specific to a particular behavior. The more natural the task is, the heavier the coding effort is. We propose an analytical approach to resolve this dilemma, providing insights and avenues for future work in interactive social neuroscience. The objective is to solve the laborious coding problem for naturalistic hyperscanning by proposing a convenient analytical approach and to uncover brain synchronization mechanisms related to human cooperative behavior when the ultimate goal is highly free and creative. This functional near-infrared spectroscopy hyperscanning study challenged a cooperative goal-free creative game in which dyads can communicate freely without time constraints and developed an analytical approach that combines automated behavior classification (computer vision) with a generalized linear model (GLM) in an event-related manner. Thirty-nine dyads participated in this study. Conventional wavelet-transformed coherence (WTC) analysis showed that joint play induced robust between-brain synchronization (BBS) among the hub-like superior and middle temporal regions and the frontopolar and dorsomedial/dorsolateral prefrontal cortex (PFC) in the right hemisphere, in contrast to sparse within-brain synchronization (WBS). Contrarily, similar regions within a single brain showed strong WBS with similar connection patterns during independent play. These findings indicate a two-in-one system for performing creative problem-solving tasks. Further, WTC-GLM analysis combined with computer vision successfully extracted BBS, which was specific to the events when one of the participants raised his/her face to the other. This brain-to-brain synchrony between the right dorsolateral PFC and the right temporo-parietal junction suggests joint functioning of these areas when mentalization is necessary under situations with restricted social signals. Our proposed analytical approach combining computer vision and WTC-GLM can be applied to extract inter-brain synchrony associated with social behaviors of interest.

Automatic Damage Detection and Diagnosis for Hydraulic Structures Using Drones and Artificial Intelligence Techniques

Large-volume hydraulic concrete structures, such as concrete dams, often suffer from damage due to the influence of alternating loads and material aging during the service process. The occurrence and further expansion of cracks will affect the integrity, impermeability, and durability of the dam concrete. Therefore, monitoring the changing status of cracks in hydraulic concrete structures is very important for the health service of hydraulic engineering. This study combines computer vision and artificial intelligence methods to propose an automatic damage detection and diagnosis method for hydraulic structures. Specifically, to improve the crack feature extraction effect, the Xception backbone network, which has fewer parameters than the ResNet backbone network, is adopted. With the aim of addressing the problem of premature loss of image detail information and small target information of tiny cracks in hydraulic concrete structures, an adaptive attention mechanism image semantic segmentation algorithm based on Deeplab V3+ network architecture is proposed. Crack images collected from concrete structures of different types of hydraulic structures were used to develop crack datasets. The experimental results show that the proposed method can realize high-precision crack identification, and the identification results have been obtained in the test set, achieving 90.537% Intersection over Union (IOU), 91.227% Precision, 91.301% Recall, and 91.264% F1_score. In addition, the proposed method has been verified on different types of cracks in actual hydraulic concrete structures, further illustrating the effectiveness of the method.

Oil Palm Fresh Fruit Bunch Ripeness Detection Methods: A Systematic Review

The increasing severity of the labour shortage problem in the Malaysian palm oil industry has created a need to explore other avenues for harvesting oil palm fresh fruit bunches (FFBs) such as through autonomous robots’ deployment. However, the first step in using an autonomous system to harvest FFBs is to identify which FFBs have become ripe and are ready to be harvested. In this work, we reviewed previous and current methods of identifying the maturity of fresh fruit bunches as found in the literature. The different methods were then compared in terms of the types of sample data used, sensor modalities, and types of classifiers used with a particular focus on the feasibility of each method for on-field application. From the 51 papers reviewed, which include a total of 11 unique approaches, it was found that the most feasible method for detecting ripe FFBs in the field is a combination of computer vision and deep learning. This system has the advantages of being a noncontact approach that is low cost while also being able to operate in real time with high accuracy.

Advancing diagnostic efficacy using a computer vision-assisted lateral flow assay for influenza and SARS-CoV-2 detection.

Lateral flow assays (LFAs) have emerged as indispensable tools for point-of-care testing during the pandemic era. However, the interpretation of results through unassisted visual inspection by untrained individuals poses inherent limitations. In our study, we propose a novel approach that combines computer vision (CV) and lightweight machine learning (ML) to overcome these limitations and significantly enhance the performance of LFAs. By incorporating CV-assisted analysis into the LFA assay, we achieved a remarkable three-fold improvement in analytical sensitivity for detecting Influenza A and for SARS-CoV-2 detection. The obtained R2 values reached approximately 0.95, respectively, demonstrating the effectiveness of our approach. Moreover, the integration of CV techniques with LFAs resulted in a substantial amplification of the colorimetric signal specifically for COVID-19 positive patient samples. Our proposed approach, which incorporates a simple machine learning algorithm, provides substantial enhancements in assay sensitivity, improving diagnostic efficacy and accessibility of point-of-care testing without requiring significant additional resources. Moreover, the simplicity of the machine learning algorithm enables its standalone use on a mobile phone, further enhancing its practicality for point-of-care testing.

Nondestructive Testing of Bridge Stay Cable Surface Defects Based on Computer Vision

The automatically defect detection method using vision inspection is a promising direction. In this paper, an efficient defect detection method for detecting surface damage to cables on a cable-stayed bridge automatically is developed. A mechanism design method for the protective layer of cables of a bridge based on vision inspection and diameter measurement is proposed by combining computer vision and diameter measurement techniques. A detection system for the surface damages of cables is de-signed. Images of cable surfaces are then enhanced and subjected to threshold segmentation by utilizing the improved local grey contrast enhancement method and the improved maximum correlation method. Afterwards, the data obtained through diameter measurement are mined by employing the moving average method. Image enhancement, threshold segmentation, and diameter measurement methods are separately validated experimentally. The experimental test results show that the system delivers recall ratios for type-I and II surface defects of cables reaching 80.4% and 85.2% respectively, which accurately detects bulges on cable surfaces.

Comparing pre-trained algorithms for facial emotion recognition: An analysis

Facial emotion recognition is an evolving field that aims to detect and interpret human emotions based on facial expressions. This technology combines computer vision, machine learning, and artificial intelligence techniques to analyze facial features and accurately classify emotions such as happiness, sadness, anger, fear, surprise, and disgust. The human face is a powerful channel for emotional communication, and facial emotion recognition seeks to replicate this ability using computational algorithms. By capturing and analyzing facial cues such as eyebrow movements, eye widening, lip curvature, and changes in skin color, these systems can infer emotional states in real-time. The applications of facial emotion recognition are diverse and impactful. In psychology, it enables researchers to study emotional responses and behavior, providing insights into mental health conditions and aiding therapists in understanding their clients' emotional states. In human-computer interaction, this technology allows for more intuitive and personalized interactions, enhancing user experiences. In marketing and advertising, facial emotion recognition helps companies understand consumer emotional engagement and tailor their strategies accordingly. In healthcare, it can aid in the diagnosis and monitoring of mental health conditions, enabling personalized treatment options.

Sustainable Palm Tree Farming: Leveraging IoT and Multi-Modal Data for Early Detection and Mapping of Red Palm Weevil

The Red Palm Weevil (RPW) is a highly destructive insect causing economic losses and impacting palm tree farming worldwide. This paper proposes an innovative approach for sustainable palm tree farming by utilizing advanced technologies for early detection and management of RPW. Our approach combines computer vision, deep learning (DL), the Internet of Things (IoT), and geospatial data to effectively detect and classify RPW-infested palm trees. The main phases include; (1) DL Classification using sound data from IoT devices, (2) palm tree detection using YOLOv8 on UAV images, and (3) RPW mapping using geospatial data. Our custom DL model achieves 100% precision and recall in detecting and localizing infested palm trees. The integration of geospatial data enables the creation of a comprehensive RPW distribution map for Efficient monitoring and targeted management strategies. This technology-driven approach benefits agricultural authorities, farmers, and researchers in managing RPW infestations, safeguarding palm tree plantations’ productivity.

Performance Analysis of Human Activity

This project aims to develop an AI-powered gym assistant using Jupyter Notebook and MediaPipe, a popular computer vision library, to count the repetitions of three joint exercises: curls, squats, and sit-ups. The system will provide real-time feedback and monitoring, allowing users to track their progress and improve performance. The proposed method utilizes MediaPipe, which offers pre-trained machine-learning models for human pose estimation and hand tracking. These models will accurately detect and track critical body joints and hand movements during the exercises. The system will then analyze the detected poses to identify the repetitions of each activity based on predefined movement patterns and pose thresholds. Jupyter Notebook will be used as the development environment for coding and testing the system. Python programming language and MediaPipe’s Python API will be employed to implement the pose estimation and repetition counting algorithms. The system will also incorporate a user-friendly interface, allowing users to interact with the gym assistant and receive feedback on their exercise performance. The completed project will provide an AI-powered gym assistant that can accurately count the repetitions of curls, squats, and sit-ups in real-time. Additionally, this project will contribute to the advancement of the field of fitness technology by showcasing the potential of combining computer vision and artificial intelligence techniques for gym monitoring and performance tracking. The results of this project have the potential to benefit fitness enthusiasts, trainers, and researchers alike, providing contributions to the field of fitness technology.