Robotic Sorting Research Articles

Fruit sorting robot based on color and size for an agricultural product packaging system

Indonesia's location in the equator gives an ideal condition for agriculture. However, agriculture suffers the issue of old farming due to a lack of youth interest working in this sector. This problem can be overcome by applying digital farming methods, in which one of them is by employing robots. Robotics technology is suitable for handling the harvested product, such as a sorting robot. This paper presents the application of a 4DOF fruit sorting robot based on color and size in a packaging system. The sorting is made possible by image processing where color is recognized by HSV analysis, and the diameter is known in the grayscale image and setting the thresholding. The fruit to be sorted is red and green tomatoes and red and green grapes. The experiments were conducted to show the effectiveness of the proposed method. The time requires for the robot to accomplish the task is 11.91s for red tomatoes, 11.76s for green tomatoes, 12.56s for red grapes, and 12.92s for green grapes. The time difference is due to the position of the boxes for the sorted fruit. The experimental results show that the arm robot manipulator is applicable for a sorting robot using the proposed method.

Optimal Sequence Planning for Robotic Sorting of Recyclables From Source-Segregated Municipal Solid Waste

Abstract Sorting of recyclables from source-segregated municipal solid waste (MSW) stream is an essential step in the recycling chain in a material recovery facility (MRF) for waste management. Manual sorting of recyclables in an MRF is a highly hazardous operation for human health as well as time-consuming. Application of robotics for automated waste sorting can alleviate these problems to a large extent. The total sorting time depends upon the pick-and-place (PAP) sequence used in a robotic sorting system. In this context, the generation of optimal PAP sequence plan is a key challenge considering that it cannot be solved by an exhaustive search due to the combinatorial explosion of the search space. This paper reports an approach for generating optimal PAP sequence plan for robotic sorting of recyclables from source-segregated MSW stream in a system equipped with thermal-imaging technique. The PAP sequence generation is formulated as an optimization problem wherein the objective is to minimize the total sorting time. The formulated problem has been solved using a genetic algorithm (GA)-based approach. Numerical simulations as well as physical experiments using a 6 degrees-of-freedom (DOF) articulated manipulator have been performed to test and validate the developed optimal sequence generation algorithm. Results revealed an improvement of up to 4.28% speedup in total sorting time over that of randomly generated sequences. It is envisaged that the developed approach can substantially improve the sorting performance in an MRF.

Multi-target visual recognition and positioning methods for sorting robots

Aiming at the problems of complex shape and random placement of sorting robots, this paper proposes a multi-objective visual recognition and positioning methods for sorting robots. A sorting robot system with visual perception was constructed,which based on the four-degree-of-freedom DOBOT robot. What’s more, the visual system acquires images of multiple targets on the conveyor belt, the image preprocessing algorithm improves the contrast of the sorting targets, thus circular and linear structural elements are constructed in order to fill and smooth the multi-target regions segmented by the largest inter-class variance method. Besides, Establishing centroid coordinates based on image information to identify and locate connected domains of multi-target regions. The experimental results show that the method can achieve statistics on the number of multi-targets. On the basis of the hand-eye calibration of the sorting robot, the multi-objects with complex shapes and random positions are identified and positioned, and the positional information provides the autonomous grasping and sorting for the sorting robot.

Anti-corrosion wood automatic sorting robot system based on near-infrared imaging technology

To implement the discovery of discarded anti-corrosion wood, an automatic sorting robot system was built. Three kinds of commonly used wood were selected as the research object, which uses hyperspectral imaging technology to achieve the identification. In the range of 900–1700 nm (230 bands), the infrared spectra of three kinds of anti-corrosion wood were collected, and then the characteristic information was obtained through the analysis of MATLAB to distinguish them. Among them, three kinds of preservative woods are Scots pine (add CCA treatment), Pseudotsuga menziesii (high-temperature carbonization treatment) and Incense Cedar (pressurized treatment). After the pretreatment by the Savitzky-Golay method, spectral data were conducted by principal component analysis (PCA), and the contribution rate of the first three principal components reached 99.902 %. Besides, through the loading coefficients of the first three principal components that were plotted on the wavelength, we obtained five characteristic wavelengths and corresponding reflectance information, simultaneously; this set up a typical discriminant analysis model. Then, the model was validated by the validation set, and the accuracy rate of the prediction set was 98.89 %. This method can effectively identify and classify three kinds of anti-corrosion wood, which can provide a scientific method and basis for a solid waste sorting system.

Optimization of Grasping Efficiency of a Robot Used for Sorting Construction and Demolition Waste

The recycling of construction and demolition waste (CDW) remains an urgent problem to be solved. In the industry, raw CDW needs to be manually sorted. To achieve high efficiency and avoid the risks of manual sorting, a sorting robot can be designed to grasp and sort CDW on a conveyor belt. But dynamic grasping on the conveyor belt is a challenge. We collected location information with a three-dimensional camera and then evaluated the method of dynamic robotic grasping. This paper discusses the grasping strategy of rough processed CDW on the conveyor belt, and implements the function of grasping and sorting on the recycling line. Furthermore, two new mathematical models for a robotic locating system are established, the accuracy of the model is tested with Matlab, and the selected model is applied to actual working conditions to verify the sorting accuracy. Finally, the robot kinematics parameters are optimized to improve the sorting efficiency through experiments in a real system, and it was concluded that when the conveyor speed was kept at around 0.25 m/s, better sorting results could be achieved. Increasing the speed and shortening the acceleration/deceleration time would reach the maximum efficiency when the load would allow it. Currently, the sorting efficiency reached approximately 2 000 pieces per hour, showing a high accuracy.

Image positioning and identification method and system for coal and gangue sorting robot

ABSTRACT Accurately positioning and identifying coal and gangue under the heterogeneous and complex scenarios of coal production is a key and difficult problem for coal and gangue sorting robot. This paper proposes an image processing based method for the problem and further develops a positioning and identification system. The coal and gangue samples are selected from Hancheng Coal Mine, China’s Shaanxi province. Three automatic image threshold segmentation methods are used for image binarization and the comparative analysis shows that the clustering method outperforms the others. Particle analysis of coal and gangue samples is carried out by using the morphological corrosion and expansion methods and a complete, clean target sample is obtained; the center of mass is extracted by using the center of mass method. The features of the grayscale and texture of the samples are analyzed under the light of 300lux. It is found that coal and gangue have higher discrimination degrees in gray variance, skewness, texture contrast and entropy than other parameters. LS-SVM (Least squares support vector machines) is applied as the image classifier. Three classifiers are trained by using the feature of grayscale, texture, and the joint feature combining skewness with contrast respectively. The experimental results show that the classifier using the joint feature outperforms the others. Furthermore, we develop a positioning and identification system and further evaluate the system by using the images of goal and gangue which are randomly picked from the production environment. The results show that the average coordinate errors in x and y directions are 2.73% and 2.72%, the identification accuracy of coal and gangue samples is 88.3% and 90.0% respectively, and the sum of time of identification, positioning and opening the camera for a single sample in average is 0.130 s.

The Brownian and Flow-Driven Rotational Dynamics of a Multicomponent DNA Origami-Based Rotor.

Nanomechanical devices are becoming increasingly popular due to the very diverse field of potential applications, including nanocomputing, robotics, and drug delivery. DNA is one of the most promising building materials to realize complex 3D structures at the nanoscale level. Several mechanical DNA origami structures have already been designed capable of simple operations such as a DNA box with a controllable lid, bipedal walkers, and cargo sorting robots. However, the nanomechanical properties of mechanically interlinked DNA nanostructures that are in general highly deformable have yet to be extensively experimentally evaluated. In this work, a multicomponent DNA origami-based rotor is created and fully characterized by electron microscopy under negative stain and cryo preparations. The nanodevice is further immobilized on a microfluidic chamber and its Brownian and flow-driven rotational behaviors are analyzed in real time by single-molecule fluorescence microscopy. The rotation in previous DNA rotors based either on strand displacement, electric field or Brownian motion. This study is the first to attempt to manipulate the dynamics of an artificial nanodevice with fluidic flow as a natural force.

Binocular vision mechanical arm system based on salient region target recognition

Robotic arms have been widely used in the sorting of manufacturing lines. During the sorting process of the robot arm, the captured speed and accuracy are core problems. In order to improve that, this paper proposes a binocular vision sorting robot arm system based on the target identification of significant regions. First, establish the target artifact training set and the Faster R-CNN algorithm is imported. A binocular camera is used to take pictures of the workpiece in the tray, and then deep learning algorithm is used to extract the significant area where the object is located. Image processing is performed on this region. Then, the three-dimensional coordinates of the workpiece centroid are calculated and passed to the manipulator for grasping and sorting. Because the significance area is extracted, the amount of calculation by the image processing algorithm is greatly reduced, and the speed is significantly improved. Finally, the picking and sorting experiment was designed. The results show that the system has high accuracy, strong robustness and strong portability.

Design of Vision Servo Sorting Robot System Based on SVM

Aiming at the problems of the traditional industrial waste disposal, such as heavy workload, low efficiency and poor safety, a single visual servo sorting robot based on SVM is designed. These include intelligent identification and classification systems, and wireless communication systems. The robot adopts a rectangular coordinate robot structure. After collecting picture information, the robot interacts with the upper computer. The SVM algorithm is used to identify and classify the waste material to realize autonomous sorting and transportation of waste material. When the system is applied in practice, it can effectively reduce labor input, improve recovery efficiency, and reduce production costs.

Vision system for detection of defects on apples using hyperspectral imaging coupled with neural network and Haar cascade algorithm

The article is devoted to the development of algorithms for detecting defective apples transported on a roller conveyor using a vision system. In developing the algorithms, the possibility of classifying various regions of interest (intact and damaged by rot, scab, codling moth, as well as the conveyor) by the principal component method was investigated. When choosing the optimal spectral region for cluster analysis, spectrograms obtained in various spectral ranges, including Vis-NIR (400–1000 nm), NIR (780–1000 nm), and Vis (400–780 nm) were used. The PCA method showed that for the successful classification of the conveyor area, intact, decayed and damaged by the codling moth, it is necessary to use spectrograms in the Vis-NIR range. To classify these ROIs, it was proposed to use a direct distribution neural network with two hidden layers of 128 and 64 layers, respectively, the “relu” activation function in the hidden layers and the “softmax” activation function in the output layer. The optimal network configuration was determined experimentally. This configuration showed a classification accuracy of 0.847 on a test sample of 6,000 apples. Since the samples of spectrograms of scab and stem regions do not differ, for their classification in parallel with the neural network, it was proposed to use the Haar cascade classifier trained on 2000 two-dimensional images of apples in the visible region containing scab and stem regions. The classification accuracy was at least 0.95. The developed algorithm is intended for use in the robotic sorting of apples.

Optimization and comparison of models for prediction of soluble solids content in apple by online Vis/NIR transmission coupled with diameter correction method

The online system can achieve high efficiency towards fruit quality determination in postharvest period. Thus, developing online and nondestructive technology for inspecting and grading fruit is meaningful and profitable in the existing robotic sorting systems. In this study, the effect of fruit diameter differences on online prediction of soluble solids content (SSC) of ‘Fuji’ apples based on visible and near-infrared (Vis/NIR) spectroscopy was studied. Partial least square (PLS) regression was employed to establish calibration models based on three wavelength regions (675–1025, 710–980, 750–1025 ​nm) and two fruit orientations (stem-calyx axis vertical with stem upward (T1) and stem-calyx axis horizontal with stem towards light source (T2)), respectively. A novel diameter correction method was proposed to reduce the effect of fruit diameter differences on original spectra. Combined with pretreatment and effective wavelength (EWs) selection methods, models were optimized and compared to determine the best calibration strategy. Diffuse transmission spectra in 710–980 ​nm and diameter correction method with calculated attenuation coefficient were testified much better than other corresponding regions and correction methods, respectively. Baseline offset correction (BOC) and 7-point Savitzky-Golay smoothing (SGS) of pretreatments and competitive adaptive reweighted sampling (CARS) of EWs selection methods were proved to be outstanding among other methods. 59 and 63 ​EWs achieved the best detection accuracies with correlation coefficient of prediction (rp) and root mean square error of prediction (RMSEP) of 0.92 and 0.50 °Brix, 0.89 and 0.56 °Brix for T1 and T2, respectively. The overall results indicated that online Vis/NIR transmission spectra after BOC and 7-SGS with proposed diameter correction method can make the variation of fruit diameters a small interference for SSC determination, and CARS-PLS would be effective to simplify models and promote computing efficiency to make this nondestructive detection technique promisingly applied.

Prioritizing robotic grasping of stacked fruit clusters based on stalk location in RGB-D images

Determining the priority order for grasping stacked fruit clusters such as those of grape and lychee is a challenging problem in robotic manipulation based on RGB-D images. Grasping fruit cluster stalks at the top of a stack can improve robotic grasping success and reduce fruit damage. Hence, stalk location is an important factor affecting grasping priority. Because of the disordered arrangement of fruit clusters, stalks are not neatly arranged in horizontal planes, and this makes it difficult to determine grasping priority by stalk vertical location. Therefore, this paper proposes a prioritization method for grasping stacked fruit clusters by extracting and classifying “stalk depth sets“ in RGB-D images. For extracting small stalks with multiple background elements in RGB-D images, a convolutional neural network (CNN) is constructed to reduce the variation in resolution across the feature map and to retain details and local features. To reduce background interference, a coarse-to-fine cascaded Faster R-CNN based on multi-scale feature maps is proposed. A transfer learning scheme is used to improve the network’s learning capability for stalk depth sets. Using grape cluster experiments with a robotic sorting system, the proposed method was verified and compared with the existing LeNet-5, AlexNet, VGG16, and Faster R-CNN methods. Compared with existing networks, the average precisions of cascaded Faster R-CNN for extracting stalk depth sets and constructed CNN trained by improved transfer learning for grasping prioritization increased by more than 8.01% and 3.79%, respectively. The experimental results demonstrate that the proposed method can accurately extract and classify stalk depth sets and improve grasping prioritization for stacked fruit clusters.

Robotic technology towards industry 4.0: Automatic object sorting robot arm using kinect sensor

As one of the most commonly employed robots in manufacturing business, robot arm has to develop into a technology capable of successing the industry 4.0 revolution. A robot arm that can interact well with humans to solve a problem becomes one of the encouraging research topics. It requires the ability to behave like humans which causes a robot arm should be utilizing intelligent peripherals. This study aims to create a robot arm design that uses a Kinect sensor as a smart peripheral to approach the human ability to see an object. As a case study, a color-based object sorting simulation was conducted. The automatic movement that the robot used is based on inverse kinematics. The experimental test shows that average percentage errors for end-effector’s position in x, y and z coordinates are 5.83%, 5.89% and 8.59%, respectively. This result has led the robot arm equipped with the Kinect sensor managed well to sort and move objects based on the color.

The Development of Waste Sorting Robot to Improve Environmental Care Attitudes at MTsN 1 Banda Aceh

One of the waste management efforts is to sort waste. Waste handling in Indonesia, especially in Aceh, is still conventional. Based on these facts, a new breakthrough is needed during the era of the industrial revolution 4.0 by making a waste sorting robot. The purpose of this study was to determine the process of developing a waste sorting robot and to determine the environmental care attitude at MTsN 1 Banda Aceh by developing a waste sorting robot. This research method is Research and Development (RD 2) data collection; 3) Product design; 4) Design validation; 5) Revised design; 6) Product trials; 7) Product revision; 8) Trial use; 9) Product revision; 10) Mass production. Research and development carried out by researchers only from steps 1-9. In the tenth step (10) of mass production, researchers did not do it because of time and cost constraints. The research location was conducted at MTsN 1 Banda Aceh. The research was carried out from October to November 2020. The research instruments were interview guides, a waste sorting robot product feasibility assessment questionnaire, and an environmental care attitude questionnaire. The data analysis technique used descriptive statistics with percentages. The results of the validation of the feasibility design of the waste sorting robot by 2 machine design experts and 1 science teacher at MTsN 1 Banda Aceh, an environmental material expert, obtained a score of 17 with a value of 100% in the less feasible category. Furthermore, in the trial, the score was 29 with a value of 100% in the very feasible category. The conclusion of this study is that the waste sorting robot is very feasible to use. The development of a waste sorting robot can improve environmental awareness at MTsN 1 Banda Aceh.

Development of a method of detection and classification of waste objects on a conveyor for a robotic sorting system

Currently used recycling technologies have limitations on the composition of recyclable waste, which makes them specialized. Thus, the preliminary sorting of municipal solid waste is a necessary step, increasing the efficiency of using municipal solid waste as a resource. To sort municipal solid waste we developed a method for detecting and classifying waste on a conveyor line using neural network image processing. Images from a camera are fed to a neural network input, which determines the position and type of detected objects. To train the neural network a database of more than 13,000 municipal solid waste images was created. Mean-Average Precision for the neural network model was 64%.

Recycling and Re-Use of Lithium-Ion Batteries (ReLiB)

Newcastle University is part of the Recycling and Re-use of Lithium-ion Batteries (ReLiB) project that aims to facilitate the creation of a UK re-use/recycling industry to recycle > 90% of automotive batteries. It is in order to make better use of global resources, increased decarbonisation and ultimately dedicate to mitigate the climate change issues. The project tackles the most demanding technical challenges in sensing, gateway testing, robotic sorting, re-use and recycling of batteries coming from EVs. The project is extremely wide-ranging covering everything from lifecycle, technical and economic assessment and legal liability issues, to tackling the most demanding technical challenges in sensing, gateway testing, robotic sorting, re-use and recycling of automotive batteries. The £9M ReLiB project is led by Birmingham University and includes a total of eight universities and 14 industrial partners. ReLiB was funded by the UK’s flagship Faraday Institution that supports research, training, and analysis in the electrochemical energy storage, science and technology. At Newcastle University, the team is assessing risk, identifying safe management methods and developing response protocols for handling used batteries. Abuse of Lithium ion batteries by overcharging or short circuit can lead to fire or even explosion, and one of the tasks of the Newcastle University team in close collaboration with researchers from the Extreme Robotics Lab of the University of Birmingham is to trains robots to disassemble the battery packs. This will be achieved by the use of various 3-D analyses and imaging techniques with the aim of removing . Ultimately, that will remove any risk to human health when disassembling very high voltage and high energy automotive batteries. The Newcastle team is investigating energy management applications and developing non-destructive testing techniques (NDT) to predict and certify the second lifetime. A set of NDTs (e.g. Electrochemical Impedance Spectroscopy and thermal imaging) are being employed to assess the state of health (SOH) of batteries in order to make a decision regarding whether a module or cell should be re-used or recycled, in a so-called ‘gateway’ test for re-use or materials recovery. Results so obtained will be compared to analytical data from teardown at the University of Birmingham and material analyses at the University of Leicester and with the aim of identifying potentially dangerous ageing mechanisms such as lithium plating. Using gateway sensing, the NU team will develop new assessments for low carbon transport strategies and characterise the performance of second-life battery packs operating in power and energy ancillary service markets. The suitability of second life batteries for various applications will be explored in terms of technical performance, life-cycle costing and the challenges of integration into existing and future infrastructure systems. This understanding will feed directly into economic models for reuse. In addition, new business models and regulatory frameworks will be investigated in conjunction with the complete, full-cycle value chain. A detailed understanding of second life battery pack lifetime will hence be developed and demonstrated. Other academic partners are investigating various processes, approaches and techniques with the aim of achieving the maximum value for the components and minimising the cost of processing. These include: physical processing, upgrading of battery components, pyrometallurgical recovery of metals and novel biological recovery routes. The re-use/recycling processes will be subjected to Life Cycle Assessment in order to evaluate their environmental impact and pick the one with the smallest footprint. Such a comprehensive approach is designed to give the collaborating recycling companies a competitive edge in the growing market. The increasing application of LiBs in electric vehicles and high energy storage systems has led the Newcastle team to realise there is an urgent need for the education of the public, businesses, government organisations and the emergency services in the safe handling of LiBs and hazards associated with these devices when involved in fires. To this end, the ReLiB team is advising a range of organisations including the Fire and Rescue Service, the Department for Transport and the CBI, and is actively seeking to disseminate specialist help and advice to a broader range of stakeholders. The team is due to carry out fire safety tests on LiBs in collaboration with the Tyne and Wear Fire Service in order to aid them in refining their firefighting procedures for fires involving an electric vehicle. Figure 1

Research on multi-manipulator coal and gangue sorting robot system based on machine visio

Research on multi-manipulator coal and gangue sorting robot system based on machine visio

Vision-based robot sorting system

In view of the problems of inefficiency, quite expensively traditional manual sorting in the existing logistics industry, a vision-based robot sorting system is proposed. At the same time, in intelligent logistics, the use of machine vision for intelligent sorting tasks becomes more and more obvious. The automatic sorting robot uses vision to identify the target, locate the target, grab the target, and sort the target. After the robot vision technology is combined, the sorting is more efficient, stable and safe. The lack of sorting exists. Therefore, the vision-based robot sorting system has high application value. In this paper, for the hardware and software parts, STM32F4 is used as the main controller, and the CCD image sensor is used to establish the plane vision coordinate system. The experimental results show that the feasibility of the system is verified. The sorting system realizes the automatic sorting function of the object and has a good application prospect.

A Receiver-Controlled Coupler for Multiple Output Wireless Power Transfer Applications

Multiple output wireless power transfer (WPT) system has a great potential to be used in applications where multiple receivers need to be powered simultaneously, including wireless office table, logistic sorting robots, and magnetic resonant imaging (MRI) equipment. For the sake of reducing no-load losses and avoiding leakage magnetic fields, it is desirable to only switch on those transmitter coils that are covered by receivers and switch off the rests. To realize this, however, bidirectional power switches, sensors and control logic are needed, which are expensive, bloated, and lossy. This paper proposed a novel receiver-controlled coupler (RC-Coupler) that can realize switching on/off of transmitter coils without using bidirectional switches, sensors, and control. The structure, design considerations, and leakage fields of the proposed RC-coupler are researched, and its feasibility has been confirmed with a GaN-based 120 W, 6.78 MHz prototype. A power density of 2.78 W/cm3 is reached benefiting from the receiver-controlled characteristics.

An intelligent machine vision system for detecting surface defects on packing boxes based on support vector machine

Defects in product packaging are one of the key factors that affect product sales. Traditional defect detection depends primarily on artificial vision detection. With the rapid development of machine vision, image processing, pattern recognition, and other technologies, industrial automation detection has become an inevitable trend because machine vision technology can greatly improve accuracy and efficiency; therefore, it is of great practical value to study automatic detection technology of the surface defects encountered in packaging boxes. In this study, machine vision and machine learning were combined to examine a surface defect detection method based on support vector machine where defective products are eliminated by a sorting robot system. After testing, the support vector machine training model using radial basis function kernel detects three kinds of defects at the same time under the ideal condition of parameter selection, and the effective detection rate is 98.0296%.