Machine Vision Detection Research Articles

A machine vision‐based intelligent segmentation method for dam underwater cracks using swarm optimization algorithm and deep learning

AbstractEnsuring the safety of water networks is a research hotspot in the current water conservancy industry, and dams are an important part. However, over time, the dam is prone to varying degrees of aging and disease, most of which are structural cracks. If they cannot be discovered and repaired in time, the normal operation of the dam will be affected, and even catastrophic accidents such as dam failure will occur. However, complex backgrounds and blurred images can easily lead to misjudgments by machine vision detection models, and high‐efficiency and accurate detection and evaluation technology are urgently needed. This paper combines the deep semantic segmentation network and the model hyperparameters optimization algorithm to propose a data‐intelligent perception method of dam underwater cracks driven by knowledge coupling. Taking the underwater detection of a concrete face rockfill dam as an example, the effectiveness of the model is verified by using the underwater vehicle as the carrier. Experimental results indicate that the developed method achieves an intersection‐union ratio of 0.9301, a precision rate of 0.9678, a precision rate of 0.9472, and a recall rate of 0.9577 in the test set. This shows that the constructed method has a high crack fine detection performance. In addition, the developed method has better segmentation performance in different complex underwater crack scenes, which further illustrates the high performance of the developed method.

Visual guidance technology of flying cars based on multilevel markers and depth

Split-type flying car will play an important role in the future transportation. This paper adopts a guidance method that couples visual information and depth information, and improves the docking accuracy through the mutual cooperation of the drone and the vehicle. Firstly, a multilevel docking marker is designed to achieve adaptive target matching within different distances during the docking process. The marker has strong robustness and can adapt to complex scenes such as occlusion, strong light, and large angle tilting, providing the redundant corner points required for machine vision detection pose information accurately. Secondly, a three-dimensional pose estimation algorithm is proposed, which can introduce depth information to correct the homography matrix. The algorithm combines the advantages of strong robustness to multilevel marker detection and high accuracy of depth information, and can output millimeter-level precision pose information in different environments, different inclination angles, and different occlusions. Finally, a flying car model experiment was carried out, and the results showed that the guidance technology can obtain millimeter-level precise pose information during the entire process of long distance-near distance-completion of docking, thus realizing precise docking.

Surface roughness measurement using microscopic vision and deep learning

Due to the self-affine property of the grinding surface, the sample images with different roughness captured by the micron-scale camera exhibit certain similarities. This similarity affects the prediction accuracy of the deep learning model. In this paper, we propose an illumination method that can mitigate the impact of self-affinity using the two-scale fractal theory as a foundation. This is followed by the establishment of a machine vision detection method that integrates a neural network and correlation function. Initially, a neural network is employed to categorize and forecast the microscopic image of the workpiece surface, thereby determining its roughness category. Subsequently, the corresponding correlation function is determined in accordance with the established roughness category. Finally, the surface roughness of the workpiece was calculated based on the correlation function. The experimental results demonstrate that images obtained using this lighting method exhibit significantly enhanced accuracy in neural network classification. In comparison to traditional lighting methods, the accuracy of this method on the micrometer scale has been found to have significantly increased from approximately 50% to over 95%. Concurrently, the mean squared error (MSE) of the surface roughness calculated by the proposed method does not exceed 0.003, and the mean relative error (MRE) does not exceed 5%. The two-scale fractal geometry offers a novel approach to image processing and machine learning, with significant potential for advancement.

Research on intelligent auxiliary assembly technology based on deep learning

Background Auxiliary assembly refers to guiding and prompting the assembly process to help operators complete complex assembly operations. Due to the complex structure of products, the similar shape of parts and human factors, the misassembly and missing assembly of parts still occur in the process of product assembly, so it is of great significance to detect the assembly correctness of complex products. Methods Aiming at the problem that manual inspection is inefficient and depends heavily on the level of inspectors in the process of complex product assembly inspection, this paper proposes an assembly correctness detection method based on deep learning. Through the three steps of view transformation, semantic segmentation and template matching, the automatic judgment of assembly errors such as wrong assembly, missing assembly and redundancy is realized, and the method is verified by the computer motherboard. Results Taking the computer motherboard as the verification object to test the correctness of assembly, the experimental re sults show that the perspective adjustment of the image after homography transformation is very obvious. The evaluation index of the semantic segmentation network detection object is calculated, and each accuracy meets the requirements of assembly correctness detection. A visualization module is also used to visually display the results of assembly correctness detection based on template matching. Conclusions The assembly correctness detection method can provide a guarantee for the manual assembly process and reduce the error rate of assembly. The machine vision detection technology can be used for automatic detection of assembly quality to improve the efficiency and automation level of detection.

Strategies for improving the detection accuracy of computerized machine vision considering spatial applications

In this paper, strategies in image preprocessing, hardware composition and detection methods are considered to improve computerized machine vision detection accuracy. First, image preprocessing and image enhancement are performed to improve the quality of the input image. Second, the hardware composition of the computer vision online inspection system is optimized by focusing on the light source selection and the performance of the image acquisition card in spatial applications. Combined with spatial application calculations, methods such as frequency domain method and Canny operator are used in order to improve the accuracy of machine vision detection. Finally, in the same test environment, the machine vision detection requires only 400MB and the detection accuracy ranges from 85.13% to 99.42%. With these comprehensive strategies, this paper provides a comprehensive and effective approach for computerized machine vision detection in spatial applications to improve detection accuracy and meet demanding application scenarios.

Automatic detection system for steel skeleton size based on machine vision.

Currently, most buildings are constructed using prefabricated concrete slabs supported by a steel skeleton that is generally tied and welded manually. However, if the overall size of the skeleton is incorrect and this error is not noticed before the concrete is poured, then a huge waste is incurred by having to scrap all the prefabricated slabs. Therefore, we propose an automatic system for measuring the frame size needed for prefabricated slabs made of reinforced concrete using LabVIEW and the NI Vision library of machine-vision functions to perform image processing and machine-vision detection. A charge-coupled device camera obtained an overall image, and then, LabVIEW processed the image and extracted its features. The system compared the measured size with the design size and issued a warning if the two differ by more than 0.5cm. From experimental comparison and verification, the maximum measurement error between the test results and the actual size was 2.7% and the root mean square error was 0.66%. This meets the requirements of actual site construction and provides a scientific reference for realizing the automatic detection of the skeleton size for reinforced-concrete precast slabs.

MBA-DNet: A mask block attention-based foreign matter detection network for tobacco packages.

Foreign matter, such as varia and mildew in the cutaway view of tobacco packages, can be detected using machine vision detection technology. However, mainstream object detection algorithms have poor detection ability for small targets when applied to foreign matter detection in the cutaway view of tobacco packages. To solve this problem, this study proposes Mask Block Attention (MBA) and introduces it into the feature extraction network to improve the global modeling ability of the object detection network, further enhancing its ability to detect foreign matter in the cutaway view of tobacco packages. Meanwhile, this study establishes a K-fold packet slicing defect dataset called K-PSDDS (K-fold packet slicing defect dataset) for foreign matter in the cutaway view of tobacco packages and conducts numerous experiments on K-PSDDS. The experimental results indicate that the AP50 and APbbox of DINO (DETR with an improved denoising anchor box for end-to-end target detection) + MBA reached 94.9% and 47.7%, respectively, showing an improvement of 0.3% and 0.9% over the baseline network DINO. Meanwhile, it achieves better performance and detection capabilities than fast region-based convolutional neural networks and other detection algorithms.

Research on intelligent auxiliary assembly technology based on deep learning

Background Auxiliary assembly refers to guiding and prompting the assembly process to help operators complete complex assembly operations. Due to the complex structure of products, the similar shape of parts and human factors, the misassembly and missing assembly of parts still occur in the process of product assembly, so it is of great significance to detect the assembly correctness of complex products. Methods Aiming at the problem that manual inspection is inefficient and depends heavily on the level of inspectors in the process of complex product assembly inspection, this paper proposes an assembly correctness detection method based on deep learning. Through the three steps of view transformation, semantic segmentation and template matching, the automatic judgment of assembly errors such as wrong assembly, missing assembly and redundancy is realized, and the method is verified by the computer motherboard. Results Taking the computer motherboard as the verification object to test the correctness of assembly, the experimental re sults show that the perspective adjustment of the image after homography transformation is very obvious. The evaluation index of the semantic segmentation network detection object is calculated, and each accuracy meets the requirements of assembly correctness detection. A visualization module is also used to visually display the results of assembly correctness detection based on template matching. Conclusions The assembly correctness detection method can provide a guarantee for the manual assembly process and reduce the error rate of assembly. The machine vision detection technology can be used for automatic detection of assembly quality to improve the efficiency and automation level of detection.

Panoramic image stitching technology and its application in the field of autonomous driving

Image processing technology is an indispensable part of the development of many fields, in which panoramic image stitching technology is one of the most widely used technologies in image processing, which is not only applied in the frontier fields of technology such as unmanned aerial vehicles and machine vision detection, but is also closely related to people's lives, for example, the panoramic camera shooting function of the smart phone. The driverless car is also a widely concerned field, its development means the liberation of manpower, the realization and improvement of self-driving car technology can alleviate traffic congestion, the use of transportation for people without the ability to drive and many other social problems. This paper will firstly introduce the panoramic collage technology, including its principle, system architecture and application prospect. Secondly, it focuses on the application of panoramic collage technology in the field of self-driving cars, which is divided into two aspects: sensor hardware and collocation algorithm, and analyzes the applicability of different algorithmic techniques in this field based on the demand characteristics of self-driving cars. The research in this paper points out the optimization direction of panoramic collage technology required by self-driving cars, which will be of great value to the development and application of panoramic collage technology in the field of self-driving cars.

Improved foreign object tracking algorithm in coal for belt conveyor gangue selection robot with YOLOv7 and DeepSORT

Given the indistinct dissimilarities between foreign matter and coal in terms of their physical characteristics, the utilization of machine vision detection technology for foreign matter tracking yields suboptimal accuracy and precision, thereby failing to satisfy the exigencies of coal mine production. In this paper, we proffer an enhanced you only look once version 7 (YOLOv7) and simple online and realtime tracking with a deep association metric (DeepSORT) algorithm for the purpose of tracking foreign entities in the coal domain. The YOLOv7 network model undergoes enhancements through the reduction of Backbone convolutional layers, the introduction of the context overlap and transition network (COTN) module, and the incorporation of a compact target detection layer. Concurrently, the DeepSORT tracking algorithm is refined by substituting the re-recognition network structure of DeepSORT with the machine translation interface (MTL) framework and replacing the DeepSORT foreign object tracking algorithm with the occlusion-aware spatial attention (OSA) module. Empirical findings substantiate the efficacy of the proposed algorithm, as it successfully achieves foreign object detection and tracking. Specifically, the algorithm attains a foreign object detection accuracy of 91.3% and a recall rate of 90.6%. In addition, it achieves a tracking accuracy of 76.1% for multiple object tracking accuracy (MOTA), and a precision rate of 80.6% for multiple object tracking precision (MOTP). Notably, in comparison to the DeepSORT tracking algorithm, the proposed algorithm exhibits a significant improvement of 6 percentage points in MOTA and 3.9 percentage points in MOTP.

Rapid dataset generation methods for stacked construction solid waste based on machine vision and deep learning.

The development of urbanization has brought convenience to people, but it has also brought a lot of harmful construction solid waste. The machine vision detection algorithm is the crucial technology for finely sorting solid waste, which is faster and more stable than traditional methods. However, accurate identification relies on large datasets, while the datasets from the field working conditions are scarce, and the manual annotation cost of datasets is high. To rapidly and automatically generate datasets for stacked construction waste, an acquisition and detection platform was built to automatically collect different groups of RGB-D images for instances labeling. Then, based on the distribution points generation theory and data augmentation algorithm, a rapid-generation method for synthetic construction solid waste datasets was proposed. Additionally, two automatic annotation methods for real stacked construction solid waste datasets based on semi-supervised self-training and RGB-D fusion edge detection were proposed, and datasets under real-world conditions yield better models training results. Finally, two different working conditions were designed to validate these methods. Under the simple working condition, the generated dataset achieved an F1-score of 95.98, higher than 94.81 for the manually labeled dataset. In the complicated working condition, the F1-score obtained by the rapid generation method reached 97.74. In contrast, the F1-score of the dataset obtained manually labeled was only 85.97, which demonstrates the effectiveness of proposed approaches.

Quantitative analysis of the effect of illumination variations on image processing in machine vision inspection

Abstract When unavoidable light, age, expression, and gesture changes occur, the machine vision detection accuracy will be greatly reduced, especially the light change impact on image processing. Combining the reflection equation and frequency domain analysis to construct a light change model based on spherical harmonic function and process the image from four directions, namely, digital image conversion, median filtering, sharpening processing, and image global segmentation. Aiming at the shortcomings of traditional image processing algorithms, the light compensation based on the improved pulse-coupled neural network model is proposed, and the model is used to analyze the impact of light changes on image processing. Despite the changes in light conditions, this algorithm maintains above 0.95 in each subset and has the best overall performance, as shown by the results. When the light intensity is less than 41.88kLux, using the optimal threshold for background segmentation of adult green peppers with respect to a fixed threshold T=0.3 results in a significant reduction of the background segmentation error. When the light intensity is greater than 41.88kLux, the superiority of the optimal threshold is not obvious. The results of this research not only have a significant role in promoting the development of the field of computer vision but also can provide people with convenient and efficient services in all aspects of social life.

Cotton Gin Stand Machine-Vision Inspection and Removal System for Plastic Contamination: Hand Intrusion Sensor Design

Plastic contamination in cotton lint poses significant challenges to the U.S. cotton industry, with plastic wrap from John Deere round module harvesters being a primary contaminant. Despite efforts to manually remove this plastic during module unwrapping, some inevitably enters the cotton gin’s processing system. To address this, a machine-vision detection and removal system has been developed. This system uses inexpensive color cameras to identify plastic on the gin stand feeder apron, triggering a mechanism that expels the plastic from the cotton stream. However, the system, composed of 30–50 Linux-based ARM computers, requires substantial effort for calibration and tuning and presents a technological barrier for typical cotton gin workers. This research aims to transition the system to a more user-friendly, plug-and-play model by implementing an auto-calibration function. The proposed function dynamically tracks cotton colors while excluding plastic images that could hinder performance. A critical component of this auto-calibration algorithm is the hand intrusion detector, or “HID”, which is discussed in this paper. In the normal operation of a cotton gin, the gin personnel periodically have to clear the machine, which entails running a stick or their arm/hand under the detection cameras. This results in the system capturing a false positive, which interferes with the ability of auto-calibration algorithms to function correctly. Hence, there is a critical need for an HID to remove these false positives from the record. The anticipated benefits of the auto-calibration function include reduced setup and maintenance overhead, less reliance on skilled personnel, and enhanced adoption of the plastic removal system within the cotton ginning industry.

Research on the operational efficiency and mechanism of integrated photocatalytic/coagulation system for the removal of natural organic matter (NOM)

Natural organic matter (NOM) widely exists in natural water bodies and threat drinking water safety. This study aims to construct an integrated photocatalytic/coagulation system and explore its treatment efficiency for NOM in source water. Morphological characteristics of the floc complex were studied based on the machine vision detection system. In addition, structural change characteristics of the floc complex in the integrated photocatalytic/coagulation system were revealed from microscopic perspective to clarify the mechanism of its effect in source water treatment process. And recycling and recycling method of photocatalyst was optimized. The results showed that the integrated photocatalytic/coagulation system can effectively reduce generation of DBPs at concentration level of μg·L−1, and the formed petal-like BiFeO3-core floc complex can achieve rapid sedimentation and had excellent recycling performance. The construction of integrated photocatalytic/coagulation system in this study is expected to provide a theoretical basis for achieving low consumption and high efficiency improvement of source water quality.

Cascaded detection method for surface defects of lead frame based on high-resolution detection images

In the field of semiconductor production and manufacturing, the detection of defects on lead frame surfaces is a vital process. This process plays a key role in ensuring the quality of the final product. Using high-resolution detection images to detect multi-scale tiny surface defects is necessary, but this amplifies the impact of environmental noise. Therefore, suppressing both the false negative rate and false positive rate in practical detection scenarios is a challenge that needs to be overcome. Current research on lead frame surface defect detection is mostly concentrated on the downloaded standard original images, which limits its application in actual production lines. This paper presents a cascaded detection method for surface defects of lead frame based on high-resolution detection images. Firstly, this study presents the unit cell extraction module to convert the detection object from high-resolution image to hundreds of unit cells. The proposed module can handle real-time detection images in the production pipeline, especially addressing situations such as lighting imbalances and tilted detection images. Subsequently, this study proposes a lead frame surface defect detection network (LDD-net), which takes unit cells as inputs and can effectively detect multi-scale defects. Compared to other models, LDD-net can effectively capture the features of subtle defects. Additionally, this paper introduces the deviation in the central width direction into the CIoU localization loss, enhancing the accuracy of defect localization in LDD-net. The data set is constructed using the machine vision detection system and conducts training and testing. Specifically, experiments of LDD-net on the data set obtained 85.01% mean average precision (mAP) and 37 ms of inference time, respectively. The detection accuracy exceeds 95%, and the false negative rate can be controlled below 6%. This approach will assist manual monitoring personnel in evaluating product quality.

Deep learning based weed detection and target spraying robot system at seedling stage of cotton field

The precision spraying robot dispensing herbicides only on unwanted plants based on machine vision detection is the most appropriate approach to ensure the sustainable agro-ecosystem and the minimum impact of nuisance weeds. However, the coexistence of crops and weeds, the similarities of plants and the multi-scale attribute of weeds make reliable detection difficult, leading to serious limitations in the application of deep learning method to target spraying in the field environment. In this paper, 4694 representative images are acquired from cotton field scenario as the data basis for deep learning model. A novel weed detection model is constructed by employing CBAM module, BiFPN structure and Bilinear interpolation algorithm. The proposed network can effectively learn the deep information and distinguish weeds from cotton seedlings in various complicated growth states. Evaluation experiments on our constructed dataset indicate that the proposed method reaches an mAP of 98.43% with faster inference speed than Faster R-CNN. Our proposed weed detection model is also deployed in spraying robot that we developed ourselves, and field trials are conducted for detection and spraying, which could maintain the excellent performance with mAP of 97.42% and effective spraying rate of 98.93%. The ability to successfully execute the weed detection and herbicide spraying management in the field lays foundation for targeted spraying in precision weed control, which has an excellent impact on cotton cultivation and growth.

Research on Lightweight Model for Rapid Identification of Chunky Food Based on Machine Vision

To meet the demands of the food industry for automatic sorting of block-shaped foods using DELTA robots, a machine vision detection method capable of quickly identifying such foods needs to be studied. This paper proposes a lightweight model that incorporates the CBAM attention mechanism into the YOLOv5 model, replaces ordinary convolution with ghost convolution, and replaces the position loss function with SIoU loss. The resulting YOLOv5-GCS model achieves a mAP increase from 95.4% to 97.4%, and a reduction in parameter volume from 7.0 M to 6.2 M, compared to the YOLOv5 model. Furthermore, the first 17 layers of the MobileNetv3-large network are replaced with the CSPDarkNet53 network in YOLOv5-GCS, resulting in the YOLOv5-MGCS lightweight model, with a high FPS of 83, which is capable of fast identification of block-shaped foods.

Intelligent Automation Manufacturing for Betula Solid Timber Based on Machine Vision Detection and Optimization Grading System Applied to Building Materials

Wood material is the foundation of wood structure architecture, and its production technology and equipment technology decide the development and upgrading of modern wood structure architecture. Aiming at the problems of low automation degree, low material utilization rate, low production efficiency and high labor costs in the process of traditional wood processing, we explore the integration and innovation of the traditional wood processing industry and modern industrial Internet information technology on the basis of studying the properties of Betula (Betula costata) solid wood materials, wood comprehensive utilization rate, wood structure component development and processing technology requirements, and form an intelligent, automatic and industrial production mode for building materials. Through technology and methods such as mechanical design, automation technology, machine vision, deep learning, optimization algorithm, electronic design automation, computer aided manufacturing, etc., the key technologies of intelligent automatic optimization of wood materials were studied, and intelligent automatic production lines of Betula species identification, log optimization sawing, solid timber longitudinal multiblade sawing, sawn timber quality detection and solid timber optimizing cross-cuts are built. Based on the machine vision method, features are extracted; a tree species, defect classification and recognition model database is established; an image processing algorithm with high recognition accuracy, as well as fast processing speed and high robustness are studied; non-destructive testing and classification methods of machine vision are optimized; key problems of online rapid classification, detection and optimization of sawing are solved and production quality and processing efficiency are improved. Finally, the timber defect detection accuracy and Betula timber yield are analyzed, and the comprehensive utilization value of optimized sawing timber is compared with the comprehensive utilization value of manually marking sawing timber. The processing cost and efficiency of Betula sawing timber with an intelligent automatic production line are calculated. The test results show that the average detection accuracy of timber defect type, size and location is 89.69%, 89.69%, 92.25% and 82.29%, respectively, and the detection stability is high. By adopting intelligent automatic detection, classification and optimization sawing production line of wood, the comprehensive utilization value of optimized sawing timber is 14.13% higher than that of manual marking sawing timber, and 16,089.29 m3 more building materials can be processed annually. In the process of intelligent automatic wood processing, the intelligent detection system is used to detect defects, improve production performance and production efficiency and reduce labor costs. Compared with the traditional wood processing process, the method studied in this paper is improved to optimize the production line processing performance and processing technology. The research and development of an intelligent automatic production system for solid wood can promote the application and development of an automatic industrial production mode for sawn timber for the wood structure construction industry, deepen the integration of artificial intelligence technology, Internet technology and the whole wood processing industry and lead the upgrading of building materials for wood structures to an intelligent manufacturing production mode.

Screw Absence Classification on Aluminum Plate via Features Based Transfer Learning Models

Screw is one of the important elements in every industry. Present of screw play an important role in which it holds the product in its own position and prevent loosen or collision with the case which will cause the small components or compartment fall off from its original position and lead to product failure. With the rise of revolution 4.0 in the industry, it helps to reduce the labor cost and human error. The main purpose of this study is to create a robust classification model used for machine vision detection – absence and present of screw, which could be adapted into respective robotics application system. 6 degree of freedom UR robot, Universal Robot is used to collect the custom dataset in TT Vision Technologies Sdn Bhd. The collected dataset is then classified into two categories, named as absent and present. Pretrained dataset, ImageNet is used to ease the training process in this research. Transfer learning model is used to extract the features which used to feed into different machine learning models. Each machine learning models undergoes hyperparameters tunning to achieve best classification accuracy. Samling ratio of 60:20:20 is used to separate the data in training, validation and testing respectively before fed into different ml models

Preface

The 2022 2nd International Conference on Electronic Communication, Computer Science and Technology (ECCST 2022) was successfully held during November 11st to 13rd, 2022 in Nanchang, China (virtual conference). The conference is an international forum for scientists, engineers, and practitioners to present their latest research results and inspirations in all areas of electronic communication, computer science and technology.The success of ECCST 2022 promoted an information exchange of recent and advanced technologies among scientists and researchers in the fields of energy environment and green development. The conference also promoted the communication and collaboration between experts, scholars, and those engaged in the development of practical technology in related areas, strengthened the academic research and discussion, and facilitated future business or research contacts among all the participants.The scope of the conference covered all areas of research in energy environment and green development, including: Circuits and Systems, Physical Electronics and Optoelectronics, Information and Communication System, Information Technology, Computer Programming, etc. The conference brought together about 80 participants, primarily from China and UK, but also from USA, Italy, and other countries.A three-day program of the conference was divided into three parts, including keynote speeches, oral presentations, and academic investigation. Each keynote speaker was given 30-40 minutes for keynote speeches and every presenter was given 10-15 minutes. Among them, Prof. Wei Tao from Shanghai Jiaotong University, China performed a keynote speech on Machine Vision Detection Methods and Applications Using Laser. She is committed to exploring the ultimate measurement accuracy, researching high-precision visual measurement methods and solving the bottleneck problem in the research and development of optical sensor products and her research results have been successfully applied in high-speed rail, satellite, intelligent manufacturing, life and health and other fields. Besides, she has presided over more than 30 projects entrusted by the National Key R&D Program, Natural Science Foundation of China, provincial and ministerial level, enterprises and institutions, and published more than 50 high-level papers. The dramatic and insightful speeches of each keynote speakers triggered heated discussions and a number of informal talks between all the participants.We would like to thank all the participants in the 2022 2nd International Conference on Electronic Communication, Computer Science and Technology. We would also like to thank the Academic Committees and all the Committee members, for the work they put in to organize this event and make it a great success. Finally, our special thanks go to the Journal of Physics: Conference Series, for the endeavor of all its colleagues in publishing this paper volume. We deeply appreciate their efforts in making the conference successful.The Committee of ECCST 2022List of Committee member is available in this pdf.