Workpiece Recognition Research Articles

Target reconstruction and process parameter decision-making for bolt intelligent assembly based on robot and multi-camera

Bolt assembly is widely used in modern manufacturing. And with the development of industrial automation, there is a growing demand for automatic assembly. To form a fully automatic and highly intelligent system for bolt assembly, a novel strategy based robot and multi-camera is proposed. In this strategy, a high assurance components reconstruction scheme is designed, wherein the Mask R-CNN is used for the identification and segmentation of the workpiece, then the shape, size and pose of the workpiece are determined by the local point cloud data and its envelope using basic geometries. The results show that the accuracy of workpiece recognition exceeds 96 %, the maximum positioning accuracy of the hole is 0.334 mm based on multi-camera system. Based on the historical fastening process data, an improved artificial jellyfish search optimizer (AJSO) BP neural network is proposed to automatically determine the optimal values for the bolt fastening speed and torque. The BP neural network optimized by AJSO reduces the maximum prediction error of process parameters from 0.117 mm/s to 0.051 mm/s for speed and from 0.052 N·m to 0.027 N·m for torque. Finally, the feasibility of the proposed strategy is demonstrated by an experimental case, which used a dual robot unit for the bearing seat assembly.

The-Application-of-Servo-Control-Technology-in-Robot-Positioning-and-Tracking-System-via-Heuristic-Algorithm

Based on the visual servo technology, this paper focuses on the visual tracking algorithm of moving objects and the dynamic grasping control method of robots, and realizes the automatic loading and unloading of moving workpieces to improve production efficiency. Firstly, aiming at the difficulties in the selection of high-dimensional features extracted by visual servo, this paper proposes a training method of generation countermeasure network based on heuristic algorithm by using the efficient search ability of heuristic algorithm. Secondly, we use image processing technology to realize real-time recognition and location of workpieces under complex background. According to the positioning results, an adaptive dual rate unscented Kalman filter visual tracking algorithm is proposed to solve the problem of delay and multi sampling rate in visual servo, and realize visual tracking of moving objects. The experimental results show that the proposed visual tracking algorithm has better stability and real-time performance.

High-Frequency Workpiece Image Recognition Model Integrating Multi-Level Network Structure.

High-frequency workpieces have the characteristics of complex intra-class textures and small differences between classes, leading to the problem of low recognition rates when existing models are applied to the recognition of high-frequency workpiece images. We propose in this paper a novel high-frequency workpiece image recognition model that uses EfficientNet-B1 as the basic network and integrates multi-level network structures, designated as ML-EfficientNet-B1. Specifically, a lightweight mixed attention module is first introduced to extract global workpiece image features with strong illumination robustness, and the global recognition results are obtained through the backbone network. Then, the weakly supervised area detection module is used to locate the locally important areas of the workpiece and is introduced into the branch network to obtain local recognition results. Finally, the global and local recognition results are combined in the branch fusion module to achieve the final recognition of high-frequency workpiece images. Experimental results show that compared with various image recognition models, the proposed ML-EfficientNet-B1 model has stronger adaptability to illumination changes, significantly improves the performance of high-frequency workpiece recognition, and the recognition accuracy reaches 98.3%.

An Improved SSD Model for Small Size Work-pieces Recognition in Automatic Production Line

<p>Aiming at the problems of slow recognition speed and low recognition accuracy of arbitrarily placed workpiece by machine vision in traditional automated production lines, a workpiece recognition algorithm based on improved SSD is proposed. Firstly, the improved DarkNet53 is used to replace the backbone network in the original SSD network framework, and the network enhancement is used in the backbone network to solve the defect of small target missed detection. Then, channel attention module and deep semantic feature fusion module are added, in order to improve the recognition ability and detection accuracy of the small target features. Lastly, the loss function was optimized, and the problem caused by sample imbalance was solved by changing the weight distribution of positive and negative samples. In the experiment, image datasets of typical bolts, nuts, and connecting plates were constructed for the network training, the experimental results showed that, the recognition accuracy and speed have been optimized and meet the requirements of automatic work-piece detection in actual production, compared with traditional YOLOv4 and the original SSD algorithm in the work-piece recognition task.</p> <p> </p>

Research on visual recognition and positioning of industrial robots based on big data technology

Abstract This paper proposes a fast recognition and positioning algorithm based on deep learning for the problems of slow recognition of complex workpieces, low accuracy, and inaccurate positioning of industrial robots. The image grayscale and parameter calibration are performed by building an industrial high-precision vision system on the target image information. The target image is then localized and segmented by boundary pixel detection, and the trained improved SSD algorithm is used to identify the target, obtain the coordinates of the target location and the category to which it belongs, and realize the industrial robot sorting. The results show that the target recognition algorithm based on the improved SSD algorithm has an error of less than 0.5 mm, the fastest recognition speed of 0.045 sec/each, and the recognition accuracy can be maintained above 98% in the experimental environment, and the distance error between the real point and the calculated point is 8.09 mm on average, indicating that the algorithm has good accuracy and stability. Building a prototype system based on the improved SSD algorithm for industrial robots with complex processes is expected to provide an automated robot identification and positioning solution for production lines.

Convolutional Neural Networks Combined with Machine Vision for Mechanical Compressor Defect Detection

The study aims to detect the defects in the production line of compressor, promote the development of convolution neural network (CNN) in defect diagnosis and recognition, and expand the application of intelligent algorithm tools in the detection and recognition of defect and fault. The detection and recognition of the defects in the compressor workpiece were discussed based on the optimization of CNN. First, the active learning under the background of machine learning (ML) was introduced into the selection of sample training model for the annotation of massive data sets in diagnosing the compressor defect, and compared with the random selection method. Second, aiming at the limitation of deep CNN, through the introduction of deep separable convolution and reverse residual structure, an improved space-separable residual CNN model was proposed, and its training process was observed and analyzed. Finally, the improved space-separable residual CNN model was applied to the defect detection and recognition of compressor workpiece, and the effect of defect recognition was evaluated. The results showed that the accuracy of the active learning was higher than that of the random selection, and it can save about 18.76% of the cost of manual annotation data. The recognition accuracy of the improved CNN model is more than 90%, and the accuracy curve of the training set and the test set basically coincided in the later training period. The average recognition accuracy of the compressor defect detection model was 96.87%, and the normal workpiece could be fully recognized. The combination of improved and optimized CNN and ML has great potential in the detection of compressor defect.

Online Workpieces Recognition for the Robotic Spray-Painting Production Line With a Low-Cost RGB-D Camera

Workpiece recognition plays a significant role in automatic painting and provides a basis for the selection path of robot spraying. However, it is not trivial to quickly and accurately recognize the multiple types of complex workpieces densely arranged in the production line. To solve this issue, we propose an online recognition approach for robotic spray-painting applications. Firstly, with a consumer RGB-D camera, the RGB images and depth images only containing a single workpiece are obtained from the densely arranged workpieces by the proposed event-triggered collection method. Secondly, a coarse-to-fine recognition method is proposed to efficiently and accurately identify the workpieces. Specifically, such method applies the S-Mask R-CNN network for quickly coarse classification and segmentation of the workpieces based on the RGB images. Then, incorporating the strong distinctiveness of the FPFH (Fast Point Feature Histogram) feature and the affine invariance in the 4PCS (4-Points Congruent Sets) method, the workpieces are recognized finely based on the point clouds, which is obtained by the depth images and the 2D segmentation results. To evaluate the system performance, over 1500 workpieces from 34 different categories on the spray-painting production line had been tested. According to our experiments, the proposed approach can accurately recognize different types of workpieces within 1.5s, and the recognition accuracy can reach 99.32%.

A fast workpiece detection method based on multi-feature fused SSD

PurposeWorkpiece sorting is a key link in industrial production lines. The vision-based workpiece sorting system is non-contact and widely applicable. The detection and recognition of workpieces are the key technologies of the workpiece sorting system. To introduce deep learning algorithms into workpiece detection and improve detection accuracy, this paper aims to propose a workpiece detection algorithm based on the single-shot multi-box detector (SSD).Design/methodology/approachPropose a multi-feature fused SSD network for fast workpiece detection. First, the multi-view CAD rendering images of the workpiece are used as deep learning data sets. Second, the visual geometry group network was trained for workpiece recognition to identify the category of the workpiece. Third, this study designs a multi-level feature fusion method to improve the detection accuracy of SSD (especially for small objects); specifically, a feature fusion module is added, which uses “element-wise sum” and “concatenation operation” to combine the information of shallow features and deep features.FindingsExperimental results show that the actual workpiece detection accuracy of the method can reach 96% and the speed can reach 41 frames per second. Compared with the original SSD, the method improves the accuracy by 7% and improves the detection performance of small objects.Originality/valueThis paper innovatively introduces the SSD detection algorithm into workpiece detection in industrial scenarios and improves it. A feature fusion module has been added to combine the information of shallow features and deep features. The multi-feature fused SSD network proves the feasibility and practicality of introducing deep learning algorithms into workpiece sorting.

Design of workpiece recognition and sorting system based on deep learning

Aiming at the actual needs of industrial robots in the handling process of sorting and picking according to the types of objects, a design scheme of visual inspection and workpiece recognition based on deep learning is proposed. The solution uses deep learning to model the object recognition system, builds the mapping relationship between the image and the object category, and realizes the effective recognition of the object image information. After the analysis is completed, the industrial robot classifies the object according to the given information. Experiments show that the use of deep learning image detection mode can realize the classification and picking operation of industrial robots, which can meet the needs of some actual scene operations.

Neural Network Based on Work Piece Recognition and Robot Intelligent Capture in Complex Environments

In today’s rapid development of science and technology, the manual work in the factory has been gradually replaced by machines, and the process of industrial intelligence has been further deepened. Workpiece recognition is the use of machine learning, computer vision and other technologies to identify the target workpiece, and the robot intelligent capture is a higher level of operation of the workpiece recognition, which is the key to realize the intelligentization of industrial robots. Due to the complex environmental factors and the diversity of the shape and size of the objects to be grasped, the accuracy and efficiency of the workpiece recognition are not ideal at this stage, and intelligent crawling is even more difficult to talk about. Aiming at the above problems, this paper builds a crawling planning model based on the convolutional neural network and the grasping pose mapping rules. Based on the established crawling planning model, the sampling candidate grabbing algorithm is designed and the migration learning method is adopted. The pretrained convolutional neural network for image recognition on the ImageNet dataset was migrated to the capture detection task of the Carnegie Mellon dataset. Experiments show that the network model proposed in this paper performs well, and its correct crawl rate is as high as 81.27%. This is to achieve a more stable and reliable identification and intelligent crawling work.

Pick-and-Place Task Implementation Using Visual Open-Loop Control

Abstract In the ever increasing number of robotic system applications in the industry, the robust and fast visual recognition and pose estimation of workpieces are of utmost importance. One of the ubiquitous tasks in industrial settings is the pick-and-place task where the object recognition is often important. In this paper, we present a new implementation of a work-piece sorting system using a template matching method for recognizing and estimating the position of planar workpieces with sparse visual features. The proposed framework is able to distinguish between the types of objects presented by the user and control a serial manipulator equipped with parallel finger gripper to grasp and sort them automatically. The system is furthermore enhanced with a feature that optimizes the visual processing time by automatically adjusting the template scales. We test the proposed system in a real-world setup equipped with a UR5 manipulator and provide experimental results documenting the performance of our approach.

An Industrial Micro-Defect Diagnosis System via Intelligent Segmentation Region.

In the field of machine vision defect detection for a micro workpiece, it is very important to make the neural network realize the integrity of the mask in analyte segmentation regions. In the process of the recognition of small workpieces, fatal defects are always contained in borderline areas that are difficult to demarcate. The non-maximum suppression (NMS) of intersection over union (IOU) will lose crucial texture information especially in the clutter and occlusion detection areas. In this paper, simple linear iterative clustering (SLIC) is used to augment the mask as well as calibrate the score of the mask. We propose an SLIC head of object instance segmentation in proposal regions (Mask R-CNN) containing a network block to learn the quality of the predict masks. It is found that parallel K-means in the limited region mechanism in the SLIC head improved the confidence of the mask score, in the context of our workpiece. A continuous fine-tune mechanism was utilized to continuously improve the model robustness in a large-scale production line. We established a detection system, which included an optical fiber locator, telecentric lens system, matrix stereoscopic light, a rotating platform, and a neural network with an SLIC head. The accuracy of defect detection is effectively improved for micro workpieces with clutter and borderline areas.

PolishNet-2d and PolishNet-3d: Deep Learning-Based Workpiece Recognition

In recent years, there have been many deep learning research projects based on two-dimensional object detection and three-dimensional point cloud recognition. However, relatively few of these combine the two, and the number of projects based on a three-dimensional workpiece recognition method for the industrial field is even fewer. In this paper, to perform the recognition task for polishing workpieces in manufacturing, we use RGB-D images as input, and propose our designed PolishNet-2d for two-dimensional workpiece detection and our designed PolishNet-3d for three-dimensional workpiece recognition. The two deep neural networks are employed together in series to first detect and then recognize polishing workpieces in an industrial environment. For this paper, a large number of experiments have been carried out on deep learning datasets of polishing workpieces. These datasets were created by us to contain diverse combinations of real and simulated workpieces in real and simulated industrial environments. The contributions of this paper are as follows: (1) A rotation parameter learning network is proposed and a two-dimensional workpiece detection neural network, named PolishNet-2d, which was constructed by integrating our designed algorithms with the backbone networks ResNet101 and region proposal network (RPN), is introduced; (2) A hierarchical feature extraction network is proposed and a three-dimensional workpiece recognition neural network, named PolishNet-3d, which was constructed by integrating our designed algorithms with the backbone network PointNet, is introduced; (3) PolishNet-2d and PolishNet-3d are employed in series, with the detection output of PolishNet-2d being used as the input for PolishNet-3d for its recognition tasks: the workpiece regions are detected in the RGB image; the workpiece point cloud is segmented in the corresponding regions in the depth image, and lastly the segmented point cloud is placed into PolishNet-3d to identify the workpiece; (4) For the experiments in this paper, datasets containing rich and diverse data types of polishing workpieces for industrial fields have been constructed; (5) Numerous experimental results on polishing workpiece datasets show that the conjunction of PolishNet-2d and PolishNet-3d can achieve exemplary recognition results on polishing workpiece datasets.

Semantic recognition of workpiece using computer vision for shape feature extraction and classification based on learning databases

In this study, the Knowledge Discovery Database (KDD) was used to aid the computer to visualize, recognize and classify workpieces automatically through intelligent identification. In order to acquire initial statistical shape features, smooth and segment workpieces images were accurately taken and the shape features were stored into a database. Rough Set and Relevancy Analysis was used to reduce the shape features in order to generate the minimum characteristic vector. Finally, the generated classification rules were used to guide semantic recognition as prior knowledge. The algorithm is verified on four different types of workpieces, and the results showed that the obtained shape features are suitable for object classification and recognition semantically. This paper provides a new technique of image processing of machine vision and has great significance in the future study.

Optimal Selection of the Workpiece Recognition Parameters by Minimizing the Total Error Cost

Workpiece recognition is crucial in many flexible assembly cells, in automatic unmanned workstations and in robotic cells for human-robot collaborative scenarios. It allows for variability in the workpiece location & pose, and timing of the various operations. Automatic object recognition systems often rely on supervised machine learning methods, their parameters have to be chosen before training and cannot be changed later in runtime. In many machine learning applications the parameters are chosen in order to optimize the measures of relevance: accuracy, precision and recall. Usually these conflicting performance metrics are optimized independently from production costs. The innovation of present study is the introduction of a new metric to be optimized, a dimensionless total cost. It is a linear combination of the false positive and the false negative rates which are directly proportional to the real-life costs of errors. The presented case study, an object recognition system for reflective workpieces, is applied the proposed parameter selection to achieve optimal results with minimum production costs.

Automating the Process of Work-Piece Recognition and Location for a Pick-and-Place Robot in a SFMS

This paper reports the development of a vision system to automatically classify work-pieces with respect to their shape and color together with determining their location for manipulation by an in-house developed pick-and-place robot from its work-plane. The vision- based pick-and-place robot has been developed as part of a smart flexible manufacturing system for unloading work-pieces for drilling operations at a drilling workstation from an automatic guided vehicle designed to transport the work-pieces in the manufacturing work- cell. Work-pieces with three different shapes and five different colors are scattered on the work-plane of the robot and manipulated based on the shape and color specification by the user through a graphical user interface. The number of corners and the hue, saturation, and value of the colors are used for shape and color recognition respectively in this work. Due to the distinct nature of the feature vectors for the fifteen work-piece classes, all work-pieces were successfully classified using minimum distance classification during repeated experimentations with work-pieces scattered randomly on the work-plane.

Research on Image Preprocessing of Workpiece Recognition and Positioning System

Research on Image Preprocessing of Workpiece Recognition and Positioning System

An Effective Method for Fast Hub Location Based on Design-Template

In order to solve the hub workpiece recogni tion, location and precision grabbing problem before fine processing, a method of hub workpiece recognition and lo cation based on design-template is proposed. In this method, create design-template according to the design size and external shape of the hub workpiece, extract some invariants as features, obtain similar ity according to the corresponding features rec ognized and extracted from target image, adjust and align the image in small range based on the smallest similarity criter ia, then achieve target location. Experiments show that this method can precisely locate the key pa rts of the hub workpiece an d the recognition accuracy is greater than 98%, error accuracy of the position and angle, resp ectively, are within 2 pixels and 0.08 degrees, moreover it has less amount of calculation. It could satisfy the actual need of production line.

Work piece recognition based on the permutation neural classifier technique

This article describes a permutation neural classifier technique for the object recognition problem. Our research is aimed to help the automation of micromanufacturing and microassembly processes. In this article, we describe an object recognition system based on permutation of codes and neural classifier technique. This approach is called permutation code neural classifier (PCNC). In this work, we describe our experiments and results applying the PCNC in the recognition of micro work pieces. Two databases with different images were used for the experiments. The authors have published these databases and encourage the community to compare results. The best recognition rate obtained for the PCNC was of 97%.

Neural classifier for micro work piece recognition

The aim of this article, is to describe a technical vision system for automation of micromanufacturing and microassembly processes. One of the principal problems is connected with the recognition of work pieces and detection of their positions. For this purpose we use neural classifier named limited receptive area (LIRA) grey scale. This classifier was developed for wide range of image recognition tasks. A special software was designed. We describe some experiments and results of application of LIRA in the recognition of micro work pieces and their positions for automated handling system. The best recognition rate obtained was 94%.