Trinocular Stereo Research Articles

Trinocular 360-degree stereo for accurate all-round 3D reconstruction considering uncertainty

This research proposes a method for accurate all-round 3D reconstruction of an indoor environment in one-shot using a system of trinocular 360-degree cameras. Binocular 360-degree stereo is unable to reconstruct in all directions due to lack of disparity along epipolar directions. Thus, a third camera along a perpendicular epipolar direction is introduced to cover for this, making the system trinocular. However, previous works with trinocular stereo did not adequately take into account the uncertainty of disparity estimation and geometric constraints around 3D reconstruction. Therefore, we propose a geometric optimization scheme considering disparity estimation uncertainty and show that this results in both higher accuracy and lesser outliers along epipolar directions, in both simulated and real environments.

Anatomic and morphometric examination of auditory ossicles in sheep.

It is important to know the morphometry of the auditory ossicles for middle ear surgical applications. The present study aims to investigate the morphometric measurements of sheep auditory ossicles and the relationship between these ossicles. In this study, 100 malleus, incus and stapes of 50 Akkaraman sheep were examined using at trinocular stereo microscope and a total of 19 morphometric measurements were taken. The mean values of the morphometric measurements were obtained, and correlation analysis was performed between each part of each auditory ossicle. Similarities were found between the measurements of the lengths of the malleus, manubrium mallei and incus, and the width of the base of the stapes with morphometric measurements in human auditory ossicles. A significant positive correlation was found between the length of the malleus with the length of the manubrium mallei, between the length of the incus with the lengths of the long crus and corpus incudis, between the length of the stapes with the lengths of the rostral crus, caudal crus, caput stapedis, and the length and width of the intercrural foramen. Due to the anatomic similarities between sheep and human auditory ossicles, it was concluded that the auditory ossicles of sheep are suitable for use in the training of human ear surgery applications.

High-through counting of Chinese cabbage trichomes based on deep learning and trinocular stereo microscope

The trichome trait is one of the important phenotypes for variety classification and breeding improvement of Chinese cabbage (Brassica campestris L. syn. B. rapa). However, obtaining the number of trichomes per unit area on leaves is a time-consuming and laborious detection work, especially when hundreds of germplasm resources need to be evaluated. Therefore, this study constructed the first diverse Chinese cabbage trichome dataset called CCTD with10,955 RGB images and proposed a deep learning model for trichome detection called TRI-YOLOv8. By adding the RepVGG module in the Backbone, adding a new detection layer in the Neck and replacing the loss function with Normalized Gaussian Wasserstein Distance Loss, the detection performance of the model for small trichomes was effectively improved. At the same time, Ghost convolution was used to reduce memory consumption and speed up inference. The experimental results showed that TRI-YOLOv8 outperformed other classical detection models. AP50 was as high as 94.4%, which was 3.8% higher than YOLOv8n. Furthermore, the number of trichomes per unit area was obtained by TRI-YOLOv8 and combined with genome-wide association study and selective sweep analysis, the candidate gene BraA03g029740.3.5C (STP7) was screened out. Overall, this study achieved the accurate detection and counting of trichomes, and provided a feasible plan for breeders to digitally analyze phenotypes, automatically identify and screen Chinese cabbage germplasm resources.

Synthesis and Characterization of Hydrogels Derived from Cellulose of Water Hyacinth (<i>Eichornia crassipes</i>) and Chitosan Using the Freeze-Thaw Method

This study aims to determine the absorption and ability of hydrogels to hold water. The hydrogel was synthesized using the freeze-thaw method, then the absorption ability and water retention time in the hydrogel were tested and characterized by FTIR and trinocular stereo microscope. The FTIR results showed that the resulting hydrogel had N–H, O–H, aliphatic C–H, bend N–H, C–O, and C–N functional groups. The highest absorption of the hydrogel with a ratio of cellulose:chitosan:EDTA variations of 2:2.25:0.25 (g) respectively, which was 287.46% and the appearance of the hydrogel under a microscope showed that the structure of the hydrogel was rather hollow, so that it affected its absorption.

Dental Implant Navigation System Based on Trinocular Stereo Vision.

Traditional dental implant navigation systems (DINS) based on binocular stereo vision (BSV) have limitations, for example, weak anti-occlusion abilities, as well as problems with feature point mismatching. These shortcomings limit the operators’ operation scope, and the instruments may even cause damage to the adjacent important blood vessels, nerves, and other anatomical structures. Trinocular stereo vision (TSV) is introduced to DINS to improve the accuracy and safety of dental implants in this study. High positioning accuracy is provided by adding cameras. When one of the cameras is blocked, spatial positioning can still be achieved, and doctors can adjust to system tips; thus, the continuity and safety of the surgery is significantly improved. Some key technologies of DINS have also been updated. A bipolar line constraint algorithm based on TSV is proposed to eliminate the feature point mismatching problem. A reference template with active optical markers attached to the jaw measures head movement. A T-type template with active optical markers is used to obtain the position and direction of surgery instruments. The calibration algorithms of endpoint, axis, and drill are proposed for 3D display of the surgical instrument in real time. With the preoperative path planning of implant navigation software, implant surgery can be carried out. Phantom experiments are carried out based on the system to assess the feasibility and accuracy. The results show that the mean entry deviation, exit deviation, and angle deviation are 0.55 mm, 0.88 mm, and 2.23 degrees, respectively.

High Precision Optical Tracking System Based on near Infrared Trinocular Stereo Vision.

A high precision optical tracking system (OTS) based on near infrared (NIR) trinocular stereo vision (TSV) is presented in this paper. Compared with the traditional OTS on the basis of binocular stereo vision (BSV), hardware and software are improved. In the hardware aspect, a NIR TSV platform is built, and a new active tool is designed. Imaging markers of the tool are uniform and complete with large measurement angle (>60°). In the software aspect, the deployment of extra camera brings high computational complexity. To reduce the computational burden, a fast nearest neighbor feature point extraction algorithm (FNNF) is proposed. The proposed method increases the speed of feature points extraction by hundreds of times over the traditional pixel-by-pixel searching method. The modified NIR multi-camera calibration method and 3D reconstruction algorithm further improve the tracking accuracy. Experimental results show that the calibration accuracy of the NIR camera can reach 0.02%, positioning accuracy of markers can reach 0.0240 mm, and dynamic tracking accuracy can reach 0.0938 mm. OTS can be adopted in high-precision dynamic tracking.

Validating a Low-Fidelity Model for Microsurgical Anastomosis Training.

Background:With cost, size, and availability in mind, we developed a low-fidelity microsurgery anastomosis model for mastery of the tool skills needed to execute microsurgical procedures. The model combined the use of a cannulated Konnyaku Shirataki (KS) noodle with a low-cost, industrial inspection, trinocular stereo (IITS) microscope. The purpose of this study was to establish face and construct validity of this novel “combined” microsurgery training tool.Methods:Fifteen participants, divided into 3 groups based on microsurgery experience, attempted microsurgical anastomoses of a cannulated KS noodle using the IITS microscope. Participants were asked to (1) manipulate the noodle ends adjacent to each other, (2) place a single 7-0 nylon suture through the opposed ends, and (3) complete the anastomosis. To determine construct validity, the performance of the microsurgical repair (maximum score 53 points) and time-to-anastomosis was assessed. To determine face validity, microsurgeons were given a 25-item, 5-point scale survey rating their experience with the model.Results:Participants included 5 microsurgeons, 5 experienced trainees, and 5 novices. The microsurgeons judged the IITS microscope to be a close analog to an operating microscope (4.6/5 points), the combined model to have high educational value (4.7/5 points), and somewhat technical similarity with microsurgery in the operating room (OR) (3.7/5 points). The median technical score was 50 among microsurgeons, 40 among experienced trainees, and 27 among novices. Increased training level was associated with greater technical score among all 3 groups (p=0.002). The median time-to-anastomosis was 5.88 minutes for microsurgeons, 8.37 minutes for experienced trainees, and 17.10 minutes for novices. Increased training level was associated with shorter time-to-anastomosis (p=0.003).Conclusion:The use of the KS noodle with a benchtop stereo microscope is a novel approach to microsurgical training. It is inexpensive, available, conducive to high-repetition training, and suited to many learning environments. Microsurgeons found that this combined model was representative of microsurgery in the OR, and we concluded face validity. Furthermore, an association was demonstrated between training level and performance on the model, suggesting construct validity.

空间非合作目标的三目立体视觉测量方法

空间非合作目标的三目立体视觉测量方法

Effects of Ground Manifold Modeling on the Accuracy of Stixel Calculations

This paper highlights the role of ground manifold modeling for stixel calculations; stixels are medium-level data representations used for the development of computer vision modules for self-driving cars. By using single-disparity maps and simplifying ground manifold models, calculated stixels may suffer from noise, inconsistency, and false-detection rates for obstacles, especially in challenging datasets. Stixel calculations can be improved with respect to accuracy and robustness by using more adaptive ground manifold approximations. A comparative study of stixel results, obtained for different ground-manifold models (e.g., plane-fitting, line-fitting in $v$ -disparities or polynomial approximation, and graph cut), defines the main part of this paper. This paper also considers the use of trinocular stereo vision and shows that this provides options to enhance stixel results, compared with the binocular recording. Comprehensive experiments are performed on two publicly available challenging datasets. We also use a novel way for comparing calculated stixels with ground truth. We compare depth information, as given by extracted stixels, with ground-truth depth, provided by depth measurements using a highly accurate LiDAR range sensor (as available in one of the public datasets). We evaluate the accuracy of four different ground-manifold methods. The experimental results also include quantitative evaluations of the tradeoff between accuracy and run time. As a result, the proposed trinocular recording together with graph-cut estimation of ground manifolds appears to be a recommended way, also considering challenging weather and lighting conditions.

A Robot System for Pruning Grape Vines

This paper describes a robot system for the automatic pruning of grape vines. A mobile platform straddles the row of vines, and it images them with trinocular stereo cameras as it moves. A computer vision system builds a three‐dimensional (3D) model of the vines, an artificial intelligence (AI) system decides which canes to prune, and a six degree‐of‐freedom robot arm makes the required cuts. The system is demonstrated cutting vines in the vineyard. The main contributions of this paper are the computer vision system that builds 3D vine models, and the test of the complete‐integrated system. The vine models capture the structure of the plants so that the AI system can decide where to prune, and they are accurate enough that the robot arm can reach the required cuts. Vine models are reconstructed by matching features between images, triangulating feature matches to give a 3D model, then optimizing the model and the robot's trajectory jointly (incremental bundle adjustment). Trajectories are estimated online at 0.25 m/s, and they have errors below 1% when modeling a 96 m row of 59 vines. Pruning each vine requires the robot arm to cut an average of 8.4 canes. A collision‐free trajectory for the arm is planned in intervals of 1.5 s/vine with a rapidly exploring random tree motion planner. The total time to prune one vine is 2 min in field trials, which is similar to human pruners, and it could be greatly reduced with a faster arm. Trials also show that the long chain of interdependent components limits reliability. A commercially feasible pruning robot should stop and prune each vine in turn.

The Master Manipulator Control System in a Master-Slave Robot Based on Trinocular Stereo Vision

In this paper, a master manipulator control system in a master-slave robot which uses trinocular stereo vision is presented. The system utilizes three real-time cameras to identify and track the movements of the master handle, while also measuring the master handle position. The system follows a basic four step process. First, the master handle in the image is identified using the contrast between the handle and the background. Second, the speed and direction of the next position for the master handle is predicted in real-time by using the Kalman filter. Third, the position of the master handle is measured in real-time using stereo matching, which utilizes the method of combining feature matching and region gray matching to consistently identify the markers on the master handle and eliminate error matching. The experimental results in this study conclude that the master-handle manipulator control system is fast and exact.

Trinocular stereo system with object space oriented correlation for inner pipe inspection

Geometric inspection of pipes is a fundamental task in the oil industry. This work shows the development of a system to measure the geometry of a pipe’s inner wall using a trinocular stereo system. Natural pipe texture is used to produce a point cloud intrinsically organized in a regular mesh in a cylindrical coordinate system. The main novelty in the proposed algorithm is the definition of a regular kernel in object space, which is mathematically projected to each image plane using the inverse triangulation approach. The use of a triple correlation coefficient estimator led to simultaneous detection of homologous points. Experiments depict measurements of pipes with real and simulated textures, and results on a calibrated pipe of approximately 150mm in diameter showed a mean error of 0.3mm with a standard deviation of 0.2mm.

Observations of surface waves interacting with ice using stereo imaging

AbstractA powerful Automated Trinocular Stereo Imaging System (ATSIS) is used to remotely measure waves interacting with three distinct ice types: brash, frazil, and pancake. ATSIS is improved with a phase‐only correlation matching algorithm and parallel computation to provide high spatial and temporal resolution 3‐D profiles of the water/ice surface, from which the wavelength, frequency, and energy flux are calculated. Alongshore spatial frequency distributions show that pancake and frazil ices differentially attenuate at a greater rate for higher‐frequency waves, causing a decrease in mean frequency. In contrast, wave propagation through brash ice causes a rapid increase in the dominant wave frequency, which may be caused by nonlinear energy transfer to higher frequencies due to collisions between the brash ice particles. Consistent to the results in frequency, the wavelengths in pancake and frazil ices increase but decrease in brash ice. The total wave energy fluxes decrease exponentially in both pancake and frazil ice, whereas the overall energy flux remain constant in the brash ice due to thin layer thickness. The spatial energy flux distributions also reveal that wave reflection occurs at the boundary of each ice layer, with reflection coefficient decaying exponentially away from the ice interface. Reflection is the strongest at the pancake/ice‐free and frazil/brash interfaces and the weakest at the brash/ice‐free interface. These high resolution observations measured by ATSIS demonstrate the spatially variable nature of waves propagating through ice.

3D Heterogeneous Multi-sensor Global Registration

This manuscript presents a deterministic model to register heterogeneous 3D data arising from a ring of eight ultrasonic sonar, one high data density LiDAR (light detection and ranging), and a semi-ring of three visual sensors. The three visual sensors are arranged in a cylindrical ring, and although they provide 2D colour images, a radial multi-stereo geometric model is proposed to yield 3D data. All deployed sensors are geometrically placed on-board a wheeled mobile robot platform, and data registration is carried out navigating indoors. The sensor devices in discussion are coordinated and synchronized by a home-made distributed sensor suite system. Mathematical deterministic formulation for data registration is used to obtain experimental and numerical results on global mapping. Data registration relies on a geometric model to compute depth information from a semi- circular trinocular stereo sensor that is proposed to rectify and calibrate three image frames with different orientations and positions, but with same projection point.

A Trinocular Stereo-Matching Algorithm Based on Adaptive Support-Weight for Mobile Service Robot

A trinocular stereo-matching algorithm based on adaptive support-weight and disparity adjustment has been proposed to improve the accuracy of trinocular vision. Adaptive support-weight which is an efficient window selection method in binocular stereo is applied in the trinocular stereo. A robust target image selection method is also proposed for using the information of different target images in trinocular stereo-vision system efficiently. So occlusion can be handled well by this means. The vision system is tested in the project Family Life Support Multi-Robot Systems.

Research on ROI Image Processing Technology of Teleoperation Construction Robot Based on Trinocular Stereo Vision

Digital image has a large quantity of image data and long time for transmitting. It affects the real-time of the teleoperation robot system. According to the basic principle of human eye identifying objects and image blurry processing, a new image processing method of simulating human eye range of interest (ROI) is proposed. The method uses the calibration algorithm of three-dimensional stereo target and the Gauss blurred principle. The non-ROI region is blurred to hierarchy for extracting the feature and measurement to finish the image processing tasks. The experimental results showed that the quality of the images was assured and the transmission time was shorted. The real-time of the teleoperation robot system was also guaranteed.

An FPGA-based RGBD imager

This paper describes a trinocular stereo vision system using a single chip of FPGA to generate the composite color (RGB) and disparity data stream at video rate, called the RGBD imager. The system uses the triangular configuration of three cameras for synchronous image capture and the trinocular adaptive cooperative algorithm based on local aggregation for smooth and accurate dense disparity mapping. We design a fine-grain parallel and pipelining architecture in FPGA for implementation to achieve a high computational and real-time throughput. A binary floating-point format is customized for data representation to satisfy the wide data range and high computation precision demands in the disparity calculation. Memory management and data bit-width control are applied in the system to reduce the hardware resource consumption and accelerate the processing speed. The system is able to produce dense disparity maps with 320 × 240 pixels in a disparity search range of 64 pixels at the rate of 30 frames per second.

Efeito da usinagem na rugosidade da superfície da madeira de Eucalyptus benthamii

The objective of this research was to evaluate the effect of planing and sanding on the surface roughness of Eucalyptus benthamii Maiden et Cambage wood. Surface roughness measurements were carried out using a Surftest SJ – 301 Profilometer. Surface images obtained by a trinocular stereo microscope were also analyzed. The E. benthamii surface roughness was not affected by the machining method (planing or sanding). However, it was possible to observe through the images that the sanded surface presented fibrillation and scratches, while the planed surface showed open vessel cells.

Tele-operation Construction Robot Control System with Virtual Reality Technology

In this paper, a tele-operation construction robot (TOCR) control system with virtual reality is discussed. The system consists of TOCR driven by hydraulic servo system, two joysticks for controlling the TOCR, and 2-dimensional working environment created by virtual reality technology. And the operator performs the remote operation of the construction robot by manipulating the graphic robot directly in virtual environment using the joysticks. The position and shape of the task objects in virtual world are updated in real-time on basis of image information obtained by a trinocular stereo vision camera fixed in the remote site. To improve the efficiency as well as security of tele-operation, and overcome the shortcomings of conventional visual display, the methods of auto point of view (APV) and semi-transparent object (STO) are introduced in this paper. Finally, the experiments confirmed that one-screen visual display combining APV and STO is superior in operability, safety, and reduction of stress than the conventional visual display.

Morphological and behavioral development of the piracanjuba larvae

The objective of this work was to study the morphologic development and the swimming and feeding behaviors of piracanjuba larvae, Brycon orbignyanus Valenciennes (1849) (Characiformes, Characidae, Bryconinae), during the period from zero to 172 hours after hatching (standard length = 3.62 - 11.94 mm). The morphological analyses were accomplished by using a trinocular stereo microscope, while the behavioral analyses were performed through periodic observations. In 28 hours after hatching, the larvae (standard length = 6.25 ± 0.13 mm) showed the following structural and behavioral characteristics that made them become active predators able to overcome a larval critical phase, the beginning of exogenous feeding: presence of pigmented eyes, terminal and wide mouth, developed oral dentition, developing digestive tube, yolk sac reduction, fins and swim bladder formation, horizontal swimming, cannibalism, and predation. Intense cannibalism among larvae was verified from 26 to 72 hours. At the end of the metamorphosis - 172 hours after hatching - the larvae measuring 11.94 + 0.80 mm in standard length presented a flexed notochord, caudal fin bifurcation, dorsal and anal fin formation, synchronized movements, and formation of shoals, characteristics that together allow enhanced perception and locomotio in exploration of the environment, determining the best moment for transfering to the fishponds. New studies can contribute to commercial fish farming by improving feeding management, performance, survival, and productivity of this species.