Intel RealSense Research Articles

Compensation for Vanadium Oxide Temperature with Stereo Vision on Long-Wave Infrared Light Measurement.

In this paper, using automated optical inspection equipment and a thermal imager, the position and the temperature of the heat source or measured object can effectively be grasped. The high-resolution depth camera is with the stereo vision distance measurement and the low-resolution thermal imager is with the long-wave infrared measurement. Based on Planck's black body radiation law and Stefan-Boltzmann law, the binocular stereo calibration of the two cameras was calculated. In order to improve the measured temperature error at different distances, equipped with Intel Real Sense Depth Camera D435, a compensator is proposed to ensure that the measured temperature of the heat source is correct and accurate. From the results, it can be clearly seen that the actual measured temperature at each distance is proportional to the temperature of the thermal image vanadium oxide, while the actual measured temperature is inversely proportional to the distance of the test object. By the proposed compensation function, the compensation temperature at varying vanadium oxide temperatures can be obtained. The errors between the average temperature at each distance and the constant temperature of the test object at 39 °C are all less than 0.1%.

PREFACE

PREFACE

Obstacle Avoidance for Omnidirectional Mobile Robot Using SLAM

Abstract In the field of mobile robotics, simultaneous localization and mapping (SLAM) is an algorithmic approach to the computational problem of creating and updating a map of an environment while simultaneously keeping track of where the robot is within the environment. Applications of a SLAM algorithm are important for autonomous mobile systems to traverse an environment while avoiding obstacles and accurately achieving designated goal destinations. This paper presents the design of a SLAM-driven controller for a semi-autonomous omnidirectional mobile robot. Input to the system comes from a brain computer interface in the form of simple driving commands or a goal destination as decided by the user. Due to latency issues of reacting and responding in real-time, the system must safely navigate following the last given commands until it runs out of free space, reaches a goal designation, or receives a new command. The robotic system utilizes a three-wheeled robot kit with an upgraded sensor system. The Intel realsense depth camera D435 and two lidar sensors are utilized to construct a full 360 deg field of view. The SLAM algorithm and system controllers are developed using the robot operating system (ROS). The controllers are developed and tested within Gazebo, which is a physics simulation engine utilized for rapid prototyping. Testing was performed to validate controller performance when given varying commands as well as performing long distance path planning and obstacle avoidance. The system was often capable of achieving its goal destinations with a small error of around 3% or less though the error was found to increase with the more commands the system processed.

Accuracy of Fully Automated 3D Imaging System for Child Anthropometry in a Low-Resource Setting: Effectiveness Evaluation in Malakal, South Sudan.

Adoption of 3D imaging systems in humanitarian settings requires accuracy comparable with manual measurement notwithstanding additional constraints associated with austere settings. This study aimed to evaluate the accuracy of child stature and mid-upper arm circumference (MUAC) measurements produced by the AutoAnthro 3D imaging system (third generation) developed by Body Surface Translations Inc. A study of device accuracy was embedded within a 2-stage cluster survey at the Malakal Protection of Civilians site in South Sudan conducted between September 2021 and October 2021. All children aged 6 to 59 months within selected households were eligible. For each child, manual measurements were obtained by 2 anthropometrists following the protocol used in the 2006 World Health Organization Child Growth Standards study. Scans were then captured by a different enumerator using a Samsung Galaxy 8 phone loaded with a custom software, AutoAnthro, and an Intel RealSense 3D scanner. The scans were processed using a fully automated algorithm. A multivariate logistic regression model was fit to evaluate the adjusted odds of achieving a successful scan. The accuracy of the measurements was visually assessed using Bland-Altman plots and quantified using average bias, limits of agreement (LoAs), and the 95% precision interval for individual differences. Key informant interviews were conducted remotely with survey enumerators and Body Surface Translations Inc developers to understand challenges in beta testing, training, data acquisition and transmission. Manual measurements were obtained for 539 eligible children, and scan-derived measurements were successfully processed for 234 (43.4%) of them. Caregivers of at least 10.4% (56/539) of the children refused consent for scan capture; additional scans were unsuccessfully transmitted to the server. Neither the demographic characteristics of the children (age and sex), stature, nor MUAC were associated with availability of scan-derived measurements; team was significantly associated (P<.001). The average bias of scan-derived measurements in cm was -0.5 (95% CI -2.0 to 1.0) for stature and 0.7 (95% CI 0.4-1.0) for MUAC. For stature, the 95% LoA was -23.9 cm to 22.9 cm. For MUAC, the 95% LoA was -4.0 cm to 5.4 cm. All accuracy metrics varied considerably by team. The COVID-19 pandemic-related physical distancing and travel policies limited testing to validate the device algorithm and prevented developers from conducting in-person training and field oversight, negatively affecting the quality of scan capture, processing, and transmission. Scan-derived measurements were not sufficiently accurate for the widespread adoption of the current technology. Although the software shows promise, further investments in the software algorithms are needed to address issues with scan transmission and extreme field contexts as well as to enable improved field supervision. Differences in accuracy by team provide evidence that investment in training may also improve performance.

Negative Ulnar Variance Lessens DRUJ Instability After DRUJ Disruption: A Biomechanical Analysis.

The purpose of this study was to perform a biomechanical investigation on the effect of ulnar variance (UV) on the stability of the distal radioulnar joint (DRUJ) prior to and after DRUJ sectioning. Ten cadaveric forearm specimens were included in the study and baseline UV was assessed radiographically. Radial motion relative to the ulna was evaluated using Intel real sense cameras and a custom developed program. The forearms were dissected, and a radial osteotomy was performed. Using a custom-made plate, radial stability was assessed with an UV of + 4, 0, and -4 mm by measuring the maximum and minimum radial position relative to the ulna during a simulated Shuck test. The volar radioulnar ligaments and triangular fibrocartilage complex (TFCC) were then sectioned, and testing was repeated at each UV state. Sectioning significantly increased radial translation at neutral (P = .008), +4 mm UV (P = .008), and -4 mm UV (P = .018). There were no significant differences in translation between the 3 UV groups with the DRUJ intact (P = .124). The ulnar negative (-4 mm) state had significantly lower translation compared to the positive (+4 mm) (P < .001) and the neutral (0 mm) (P = .026) UV states. There were no significant differences between the positive and neutral UV groups with the DRUJ sectioned. Fixating the radius in -4 mm of ulnar negativity significantly decreased radial translation after sectioning the volar radioulnar ligament and TFCC. Ulnar variance had no effect on stability with an intact DRUJ. Biomechanical Study.

Characterization of infants’ general movements based on a single RGB-Depth camera: a feasibility study

Movements in infants have age-specific characteristics which could be quantified by extracting specific metrics based on the kinematics of General Movements (GM). For this purpose, a number of metrics have been proposed [1,2]. However the latter have been often obtained using complex and expensive experimental setups. The aim of this study was to evaluate if a simple experimental setup associated (RGB-D camera) with a suitable signal processing (a purposely modified markerless method - MLM) can be used to characterize the GM with the above-mentioned metrics. Data from three typically developing infants sitting in a baby seat were acquired at 3, 4 and 5 months from birth with a single RGB-D camera with depth obtained from a stereoscopic view of two infrared sensors (Intel RealSense D435). Each acquisition lasted three minutes. First, the video and depth data alignment was guaranteed using a purposely developed method, then an automatic tracking algorithm (DeepLabCut [3]) was used to estimate the locations of six points of interest (PoI) for both left and right upper body sides (shoulder, elbow and wrist). The 3D position of PoIs tracked in the RGB images was obtained by exploiting the depth data. Erroneous values were found when one or more of the following situations occurred: 1) a residual spatial misalignment between RGB and depth images, 2) a ‘black’ area in the depth images caused by the different points of view of the two infrared sensors and 3) PoIs were occluded by a body segment. Such erroneous values were replaced by applying a cubic spline interpolation to obtain the 3D trajectories of the PoIs. The periodicity index of the wrist trajectories, the area in which the wrist velocity profiles fall outside the standard deviation band of their moving average and the skewness of the wrist velocity were then determined [1]. Results are shown in Table 1. This work has shown that the infants’ GM could be characterized with a simple experimental setup, therefore giving parents the possibility of performing such evaluations at home with a higher frequency than currently possible in clinical facilities. This would allow an earlier detection of the appearance of GM anomalies. Additional work is needed to streamline the process and reduce the processing time while developing protocols for improving the acceptance and reliability of parents’ participation.

An Unsupervised Learning Approach for Road Anomaly Segmentation Using RGB-D Sensor for Advanced Driver Assistance System

Condition monitoring of road surfaces has acquired a lot of attention in the field of computer vision throughout the years. It is due to two main reasons; firstly, it produces safety and comfort to the community, and secondly, it causes less damage to the vehicles for an advanced driver assistance system (ADAS). To this extent, this article aims to present a real-time vision-based approach that automatically segments the road anomalies from the drivable area. An Intel RealSense D435 depth camera has been employed to capture RGB and depth (RGB-D) images of the road surface. An unsupervised learning method based on diffusion process has been employed to learn the affinity matrix of the RGB-D data and spectral clustering has been applied on the updated affinity matrix to cluster the road images. Image multiplex visibility graphs of the input sensor data are diffused by regularized diffusion process (RDP) to update the affinity matrix followed by generation of saliency map of the road surfaces. The prime motive to employ RDP is to use the graph Laplacian as a tool for similarity measurement for preserving the manifold structure. Qualitative and quantitative results reveal the efficacy of the proposed system with state-of-the-art methods on our RGB-D dataset. Benchmark datasets (KITTI and Cityscapes) are also used to validate the proposed method for segmentation of the drivable area for an intelligent transportation system.

An Experimental Assessment of Depth Estimation in Transparent and Translucent Scenes for Intel RealSense D415, SR305 and L515.

RGB-D cameras have become common in many research fields since these inexpensive devices provide dense 3D information from the observed scene. Over the past few years, the RealSense™ range from Intel® has introduced new, cost-effective RGB-D sensors with different technologies, more sophisticated in both hardware and software. Models D415, SR305, and L515 are examples of successful cameras launched by Intel® RealSense™ between 2018 and 2020. These three cameras are different since they have distinct operating principles. Then, their behavior concerning depth estimation while in the presence of many error sources will also be specific. For instance, semi-transparent and scattering media are expected error sources for an RGB-D sensor. The main new contribution of this paper is a full evaluation and comparison between the three Intel RealSense cameras in scenarios with transparency and translucency. We propose an experimental setup involving an aquarium and liquids. The evaluation, based on repeatability/precision and statistical distribution of the acquired depth, allows us to compare the three cameras and conclude that Intel RealSense D415 has overall the best behavior namely in what concerns the statistical variability (also known as precision or repeatability) and also in what concerns valid measurements.

Real-Time Appearance Based Mapping using Visual Sensor for Unknown Environment

Underwater environment is one of the most challenging areas when considering scene reconstruction and mapping, feature extraction of environment etc. GPS usage is denied in underwater environment and a waterproof lidar is also not a cost-effective solution to map and localize. Lidar-based simultaneous localization and mapping (SLAM) are also not preferred in budget-constrained applications. So, a visual-slam (V-SLAM) approach called Real-Time Appearance Based Mapping (RTAB-Map) is used to reconstruct the view in underwater scenario, i.e., map and localize the camera in the environment. In this work, Intel Real Sense D455 depth camera with inbuilt inertial measurement unit (IMU) is used for scene reconstruction. In order to get better visualization, it needs high performance embedded computer. Here, Jetson Nano embedded computer with robot operating system is used. Experiment is conducted for 3-D reconstruction map with the help of RTAB mapping technique and inbuilt IMU localized the camera in unknown environment. Then various depth, point clouds, odometry of a camera, visual object information is extracted in the indoor environment.

Driver awareness collision/proximity detection system for heavy vehicles based on deep neural network

Heavy vehicles are essentially used to perform several critical tasks in construction industry. These heavy vehicles are different in shape and structure according to their capabilities and needs on the construction sites. Bigger size and different design of these vehicles result in extended blind spot compare to the normal vehicles, thus risking the equipment and workers operating around them. Each year, a large number of incidents are reported involving human injuries or loss of life due to improper handling of blind spots in heavy vehicles. In this paper we developed a deep learning based collision detection and driver assistance methodology by removing the blind spots in haul truck. In the proposed methodology, vision sensor data is used for automatic detection and classification of objects, employing the YOLO-v5 network architecture. For distance measurement of the objects we used stereo imaging camera Intel RealSense D455. The proposed algorithm detects and classify the object around the vehicle and provide the captioning to describe the nature of object to the driver. In addition, the algorithm also measures the distance of objects from the vehicle. The experimental results validate the performance of our proposed methodology in real time.

Estimation of Greenhouse Lettuce Growth Indices Based on a Two-Stage CNN Using RGB-D Images.

Growth indices can quantify crop productivity and establish optimal environmental, nutritional, and irrigation control strategies. A convolutional neural network (CNN)-based model is presented for estimating various growth indices (i.e., fresh weight, dry weight, height, leaf area, and diameter) of four varieties of greenhouse lettuce using red, green, blue, and depth (RGB-D) data obtained using a stereo camera. Data from an online autonomous greenhouse challenge (Wageningen University, June 2021) were employed in this study. The data were collected using an Intel RealSense D415 camera. The developed model has a two-stage CNN architecture based on ResNet50V2 layers. The developed model provided coefficients of determination from 0.88 to 0.95, with normalized root mean square errors of 6.09%, 6.30%, 7.65%, 7.92%, and 5.62% for fresh weight, dry weight, height, diameter, and leaf area, respectively, on unknown lettuce images. Using red, green, blue (RGB) and depth data employed in the CNN improved the determination accuracy for all five lettuce growth indices due to the ability of the stereo camera to extract height information on lettuce. The average time for processing each lettuce image using the developed CNN model run on a Jetson SUB mini-PC with a Jetson Xavier NX was 0.83 s, indicating the potential for the model in fast real-time sensing of lettuce growth indices.

Predicting ribeye area and circularity in live calves through 3D image analyses of body surface.

The use of sexed semen at dairy farms has improved heifer replacement over the last decade by allowing greater control over the number of retained females and enabling the selection of dams with superior genetics. Alternatively, beef semen can be used in genetically inferior dairy cows to produce crossbred (beef x dairy) animals that can be sold at a higher price. Although crossbreeding became profitable for dairy farmers, meat cuts from beef x dairy crosses often lack quality and shape uniformity. Technologies for quickly predicting carcass traits for animal grouping before harvest may improve meat cut uniformity in crossbred cattle. Our objective was to develop a deep learning approach for predicting ribeye area and circularity of live animals through 3D body surface images using two neural networks: 1) nested Pyramid Scene Parsing Network (nPSPNet) for extracting features and 2) Convolutional Neural Network (CNN) for estimating ribeye area and circularity from these features. A group of 56 calves were imaged using an Intel RealSense D435 camera. A total of 327 depth images were captured from 30 calves and labeled with masks outlining the calf body to train the nPSPNet for feature extraction. Additional 42,536 depth images were taken from the remaining 26 calves along with three ultrasound images collected for each calf from the 12/13th ribs. The ultrasound images (three by calf) were manually segmented to calculate the average ribeye area and circularity and then paired with the depth images for CNN training. We implemented a nested cross-validation approach, in which all images for one calf were removed (leave-one-out, LOO), and the remaining calves were further divided into training (70%) and validation (30%) sets within each LOO iteration. The proposed model predicted ribeye area with an average coefficient of determination (R2) of 0.74% and 7.3% mean absolute error of prediction (MAEP) and the ribeye circularity with an average R2 of 0.87% and 2.4% MAEP. Our results indicate that computer vision systems could be used to predict ribeye area and circularity in live animals, allowing optimal management decisions toward smart animal grouping in beef x dairy crosses and purebred.

“Introduction of a low-cost and automated four-dimensional assessment system of the face.”

Existing automated objective grading systems either fail to consider the face's complex three-dimensional morphology or have poor feasibility and usability. Consumer-based red, green, and blue depth sensors or smartphone integrated three-dimensional hardware can inexpensively collect detailed four-dimensional facial data in real time but are yet to be incorporated into a practical system. This study aimed to evaluate the feasibility of a proof-of-concept automated four-dimensional facial assessment system using a red/green/blue depth sensor (OpenFAS) for use in a standard clinical environment. This study was performed on healthy adult volunteers and patients with facial nerve palsy. The setup consists of the Intel (Santa Clara, Calif.) RealSense SR300 connected to a laptop running the OpenFAS application. The subject sequentially mimics the facial expressions shown on screen. Each frame is landmarked and automatic anthropometric calculations are performed. Any errors during each session were noted. Landmarking accuracy was estimated by comparing the ground-truth position of landmarks annotated manually with those placed automatically. Eighteen participants were included in the study, nine healthy participants and nine patients with facial nerve palsy. Each session was standardized at approximately 106 seconds. A total of 61.8 percent of landmarks were automatically annotated within approximately 1.575 mm of their ground-truth locations. The findings support that OpenFAS is usable and feasible in routine settings, laying down the critical groundwork for a facial assessment system that addresses the shortcomings of existing tools. The iteration of OpenFAS presented in this study is nascent; future work, including improvements to landmarking accuracy, analyses components, and red/green/blue depth technology, is required before clinical application.

Reliability of 3D Depth Motion Sensors for Capturing Upper Body Motions and Assessing the Quality of Wheelchair Transfers.

Wheelchair users must use proper technique when performing sitting-pivot-transfers (SPTs) to prevent upper extremity pain and discomfort. Current methods to analyze the quality of SPTs include the TransKinect, a combination of machine learning (ML) models, and the Transfer Assessment Instrument (TAI), to automatically score the quality of a transfer using Microsoft Kinect V2. With the discontinuation of the V2, there is a necessity to determine the compatibility of other commercial sensors. The Intel RealSense D435 and the Microsoft Kinect Azure were compared against the V2 for inter- and intra-sensor reliability. A secondary analysis with the Azure was also performed to analyze its performance with the existing ML models used to predict transfer quality. The intra- and inter-sensor reliability was higher for the Azure and V2 (n = 7; ICC = 0.63 to 0.92) than the RealSense and V2 (n = 30; ICC = 0.13 to 0.7) for four key features. Additionally, the V2 and the Azure both showed high agreement with each other on the ML outcomes but not against a ground truth. Therefore, the ML models may need to be retrained ideally with the Azure, as it was found to be a more reliable and robust sensor for tracking wheelchair transfers in comparison to the V2.

Construction and benchmark of an autonomous tracked mobile robot system

Robots require a certain set of skills to perceive and analyse the environment and act accordingly. For tracked mobile robots getting good odometry data from sensory information is a challenging key prerequisite to perform in an unstructured dynamic environment, thus an essential issue in the tracked mobile robotics domain. In this article, we construct a ROS-based tracked mobile robot system taking the Jaguar V4 mobile robot as the base platform. On which several visual odometry solutions based on different cameras and methods (Intel RealSense T265, Zed camera, RTAB-Map RGBD) are integrated and benchmark comparison is performed. Analysis of new challenges faced by different methods while applied on a tracked vehicle as well as recommendations and conclusions are presented. Intel RealSense T265 solution proved to perform well in uncertain conditions which involves bounded vibrations and low lighting conditions with low latency, which result in good map generation. Further evaluations with a path planning algorithm and Intel RealSense T265 were conducted to test the effect of the robot’s motion profiles on odometry data accuracy.

A Study on Long-Close Distance Coordination Control Strategy for Litchi Picking

For the automated robotic picking of bunch-type fruit, the strategy is to roughly determine the location of the bunches, plan the picking route from a remote location, and then locate the picking point precisely at a more appropriate, closer location. The latter can reduce the amount of information to be processed and obtain more precise and detailed features, thus improving the accuracy of the vision system. In this study, a long-close distance coordination control strategy for a litchi picking robot was proposed based on an Intel Realsense D435i camera combined with a point cloud map collected by the camera. The YOLOv5 object detection network and DBSCAN point cloud clustering method were used to determine the location of bunch fruits at a long distance to then deduce the sequence of picking. After reaching the close-distance position, the Mask RCNN instance segmentation method was used to segment the more distinctive bifurcate stems in the field of view. By processing segmentation masks, a dual reference model of “Point + Line” was proposed, which guided picking by the robotic arm. Compared with existing studies, this strategy took into account the advantages and disadvantages of depth cameras. By experimenting with the complete process, the density-clustering approach in long distance was able to classify different bunches at a closer distance, while a success rate of 88.46% was achieved during fruit-bearing branch locating. This was an exploratory work that provided a theoretical and technical reference for future research on fruit-picking robots.

Low-damage transplanting method for leafy vegetable seedlings based on machine vision

To solve the high damage rate in the seedling transplantation of horticultural facilities, a low-damage transplanting method of leafy vegetable seedlings based on machine vision and image processing was proposed. An Intel Realsense D415 camera was used to obtain the side image of single-row seedlings. Then, image processing based on Python-Opencv was performed to obtain the seedlings’ height and extreme edge points. The pixel coordinates in the RGB image were then obtained, and the depth image was aligned to the RGB image to acquire the depth information of the corresponding extreme points. Then, the path planning of the end effector was carried out according to the coordinated information to realise the low-damage transplantation of seedlings. The calibration accuracy of the low-damage transplanting method for seedling edge extreme points was verified experimentally. Under the experimental conditions, the calibration success rate was 98%, and the deviation ratio was within 2%. The seedling transplantation experiment compared (a) the low-damage transplanting method with (b) the fixed-path and (c) the vertical-horizon motion transplanting method. The success rates of the three methods were 94.90%, 88.89%, and 93.52%, and the average single transplanting time was 4.895 s, 4.907 s and 5.627 s. It was proved that the low-damage transplanting method for leafy vegetable seedlings based on machine vision could significantly reduce the seedling damage rate and increase the success rate of seedling transplantation.

Accuracy of Fully Automated 3D Imaging System for Child Anthropometry in a Low-Resource Setting: An Effectiveness Evaluation in South Sudan

ObjectivesTo evaluate accuracy of child stature (height/length) and mid-upper arm circumference (MUAC) measurements produced by the AutoAnthro 3D imaging system developed by Body Surface Technology Inc following improvements to the software algorithm to improve accuracy and support automated processing, and hardware changes aimed to reduce cost.MethodsA two-stage cluster survey in Malakal Protection of Civilians (PoC) in South Sudan between September 27 and October 2, 2021. All children aged 6–59 months within selected households were eligible. For each child, manual measurements were obtained by two anthropometrists following the protocol used for the 2006 WHO Child Growth Standards (CGS) study. Scans were then captured by a different enumerator using a Samsung Galaxy 8 phone loaded with a custom software, AutoAnthro, and an Intel RealSense 3D scanner. Scans were processed using a fully automated algorithm. A multivariate logistic regression was fit to evaluate adjusted odds of achieving a successful scan. Accuracy of measurements were visually assessed using Bland-Altman (BA) plots and quantified using average bias, technical error of measurement (TEM), limits of agreement (LoA), and the 95% precision interval for individual differences.ResultsManual measurements were obtained for 539 age eligible children, from which scan derived measurements were successfully processed for 234 (43.4%) of children. Caregivers for at least 56 children (10.4%) refused consent for scan capture; additional scans were unsuccessfully transmitted to the server. Neither demographic characteristics of the children (age and sex), stature, nor MUAC were associated with availability of scan derived measurements (P > 0.05); team was significantly associated (P < 0.001). The average bias of measurements in cm was −0.5 (95% confidence interval (CI): −2.0, 1.0) for stature and + 0.7 (CI: 0.4, 1.0) for MUAC. For stature, 95% LoA was −23.9 to 22.9 cm. For MUAC, the 95% LoA was −4.0 to 5.4 cm. The TEM was 8.4 cm for stature and 1.8 cm for MUAC. All metrics of accuracy varied considerably by team.ConclusionsScan derived measurements were not of sufficient accuracy for widespread adoption. Differences in accuracy by team provide evidence that investments in training may be able to improve performance.Funding SourcesUSAID's Bureau for Humanitarian Affairs and Grand Challenges Canada.

Edge recognition and reduced transplantation loss of leafy vegetable seedlings with Intel RealsSense D415 depth camera

Seedling transplantation is the key link in the automation of protected horticulture, and the transplantation quality will directly affect the crop yield. In the previous research, our research group developed a prototype of a low-loss transplantation robot for plug seedlings based on machine vision. On this basis, a method of planning seedling path by integrating seedling edge recognition technology and end effector of transplanting manipulator is proposed to reduce the loss of seedling transplantation. Firstly, the RGB image and depth image of the whole row of seedlings were obtained from the side of the plug seedling with Intel RealSense D415 depth camera. Then, the extreme point coordinates E (Xcb, Yca) of seedling edge were obtained by edge recognition algorithm, and the coordinate value of z-axis was known. Taking point E as the starting point the transplanting manipulator picked up seedlings from the E point in an “L” path, which can reduce the damage of transplanting manipulator to seedling stems and leaves during seedling taking, thereby reducing the transplanting loss. Through three-factor four-level orthogonal test, it was determined that the distance between the depth camera and the plug tray was 600 mm, the height from the horizontal plane was 135 mm, and the light intensity was level 7. Then, a calibration test of extreme points on the edge of plug seedlings was carried out, and the calibration success rate was 98.4%. The deviation of X coordinate was within 5 mm, and the average ratio of deviation was 12.8%. The deviation of Y coordinate was within 4 mm, and the average ratio of deviation was 3%. The quality of image and the accuracy of depth information were the main causes of the deviation. Then, the comparative experiments of routine transplantation group, fixed transplantation group and machine vision transplantation group were carried out. The results showed that compared with the conventional transplantation group, the injury rate of machine vision group decreased by 11.11%, and the average time of single transplantation increased by 0.029 s. Compared with the fixed transplantation group, the injury rate increased by 0.46%, and the average transplantation time decreased by 0.238 s. Because the edge recognition time and the transplantation time intersected and did not interfere, only the time for the first seedling edge recognition was increased in the pipeline operation. Therefore, this method can reduce the damage rate while ensuring the efficiency. This study can provide a reference for reducing the injury rate of leafy vegetable seedlings transplantation.

Poisson disk sampling with randomized satellite points for projected texture stereo

A stereo camera is fundamental for 3D sensing. We can only use simple stereo algorithms for real-time applications; however, it has low accuracy. Adding texture by projection is a solution to the problem. The system is called projected texture stereo. Projecting light patterns is essential and deeply related to the matching algorithm in the system. This article proposes a projected pattern and a new texture stereo camera system with a suitable stereo matching algorithm. Experimental results show that our system is superior to Intel RealSense, the commercially successful projected texture stereo camera.