Binocular Robot Research Articles

High Value Bit Model Reconstruction and Damage Identification Based on Binocular Robot

In order to achieve high value bit repair, bit model reconstruction and damage identification must be carried out. In this paper, the binocular camera and the industrial robot were combined to realize the point cloud acquisition, model reconstruction and damage identification of the bit. The bit damage data obtained by this method was applied to the repair experiment of the bit and the repaired bit met the requirements of re-use.

Multilevel Cross-Aware RGBD Indoor Semantic Segmentation for Bionic Binocular Robot

Scene understanding is one of the foundations for robots to achieve true artificial intelligence. Semantic segmentation that imitates the mechanism of human visual system can effectively promote the correctness of scene understanding. It conforms to the basic principle of human environment perception and enables robots to better serve human society. In this paper, we propose a multilevel cross-aware network (MCA-Net) for RGBD semantic segmentation. It utilizes basic residual structure to encode texture information and depth geometric information respectively. In view of the flow pattern of different visual features in the visual pathway of human brain, our MCA-Net attempts to jointly reason about 2D appearance and depth geometric information. Therefore, multilevel cross-aware fusion modules are designed to fuse multi-scale complementary features extracted from RGB and depth images. As the depth and color information are independent, the reasonable combination of depth and RGB images can improve the quality of semantic labeling. Experiments conducted on ScanNetv2 dataset show that the proposed network produces high quality segmentation results of RGBD images and outperforms state-of-the-art methods. Furthermore, the semantic labeling results on the bionic binocular robot in real-office scenes further demonstrate the effectiveness of the proposed MCA-Net.

VCA Protocol-Based Multilevel Flexible Architecture on Embedded PLCs for Visual Servo Control

Visual, motion control, and programmable logic controller (PLC) systems are becoming increasingly inseparable and important in manufacturing. However, the types of programs and hardware used in these systems are numerous, and there are many communication protocols and mixed development methods among the systems; meanwhile, the requirements and complexities of these systems are ever growing. All these factors hinder application implementation for users, and very few reported studies have researched the integration of these three systems to ease complexity. Most of these studies have focused on individual applications. In this paper, we propose a multilevel flexible architecture based on a vision control algorithm (VCA) protocol. This multilevel architecture includes a flexible layer, a control layer, and an algorithm layer. The flexible layer is used to seamlessly integrate the visual system and the embedded PLC (ePLC). The VCA protocol is designed for data interactions between the layers. Correspondingly, customized hardware, memory allocation, and Petri-net-based multithreading structure of the proposed architecture are described in detail. Then, two cases are implemented: a winding machine with a visual system that obtains a regular winding effect by correcting $\theta$ , and a binocular catching robot uses cameras to track the trajectory and successfully catch a ball by adjusting speed and position. The results indicate that the proposed VCA-based architecture could easily be applied to two very different scenarios.

RBF Adaptive Control Strategy based on Sub-block Approximation Algorithm for Binocular Vision Robot

Intelligent robot not only can realize reservation on conduct and actions, but also can be understand the characteristics of the unknown environment and adapt to changes in the environment through their own sensory system, robots get outside information mainly by vision, this paper take the joint robot of binocular robot vision system as control plant, pointing on the problem of the uncertainties that existed in the dynamic model of binocular vision robot may cause instability. This paper has proposed a sliding model control scheme with RBF neural network adaptive control strategy based on sub-block approximation algorithm, in this control method, sliding model control was used to control trajectory of the joints of robot, and utilize the RBF neural network to approximate the each uncertain in the dynamic model of robot. The simulation results show that compared with the RBF neural network adaptive control strategy based on integral approximation for uncertainness, the proposed control method has features with good position tracking.

Development of a binocular vision-based catcher robot system using DSP platform

This study adopted the digital signal processing system of Texas instruments as an embedded processing system for a catcher robot. Two cameras were used to capture images, and image processing techniques were applied to calculate the centroid of the target object image. The 3-D coordinates of the target object in space were then calculated according to the principles of binocular vision. The error coming from visual calculation was less than the size of the target object. Next, the Lagrange interpolation formula and a linear equation were used to calculate (in real time) the trajectory of the ball in space and the location to catch the ball. Finally, inverse kinematics was used to control the 2° of freedom robotic arm to catch balls pitched by humans from a distance of 4 m. Combining the advantages of the platform and the hardware efficacy with a processing rate of 30 frames/s, a low-cost rapid robot visual tracking system resembling a human is established.

2P1-F06 衝動性眼球運動・滑動性眼球運動・前庭動眼反射を統合的に実現可能な両眼視覚制御システム(バイオロボティス(2))

In recent years, expectations for mobile robots and humanoid robots which work in the human living environment is growing. Such a robot needs vision system to obtain information about the outside world. However, in traditional vision system, it is difficult to obtain information stably. On the other hand, humans solve this problem by information processing and eye movement characteristics of retina. We tried to realize a behavior of human eye movement by making binocular control model based on physiology and anatomy. However, in traditional model, the accuracy of Saccade is insufficient and there is a possibility of runaway. The purpose of this study is to propose a new method of Saccade and a binocular control model integrating Saccade, Smooth Pursuit and VOR. In addition, by implementing proposed model in binocular robot, we confirm the superiority of proposal model.

Real-time visuomotor update of an active binocular head

In order for a binocular head to perform optimal 3D tracking, it should be able to verge its cameras actively, while maintaining geometric calibration. In this work we introduce a calibration update procedure, which allows a robotic head to simultaneously fixate, track, and reconstruct a moving object in real-time. The update method is based on a mapping from motor-based to image-based estimates of the camera orientations, estimated in an offline stage. Following this, a fast online procedure is presented to update the calibration of an active binocular camera pair. The proposed approach is ideal for active vision applications because no image-processing is needed at runtime for the scope of calibrating the system or for maintaining the calibration parameters during camera vergence. We show that this homography-based technique allows an active binocular robot to fixate and track an object, whilst performing 3D reconstruction concurrently in real-time.

Multiple welding seam video transmission and monitoring of welding robot in local area network

A method of transmission and monitoring in local area network (LAN) for the welding seam video information of binocular vision welding robot is proposed. This paper uses MPEG-4 video CODEC functions of DivX, video for Windows (VFW) video capture technology, multi-threaded network communication idea and WinSock Socket in Visual C++ 6.0. In the end, client/server (C/S) communication model based on TCP/IP protocol has been developed. Experimental results show that the method can achieve the multiple welding seam video’s real-time transmission in LAN. Moreover, the Canny operator is used to realize the edge feature extraction of the seam image which is captured by the client, so that the edge of the seam extracted is continuous and clear.

Binocular Robot Vision with Active Viewpoint for Sphere Measurement

A binocular robot vision system reconstructs 3D scenes from right and left images by using triangulation. However when using triangulation, the corresponding problems have to be resolved. To resolve the corresponding problems easily, authors have proposed a binocular robot vision system with an autonomously moving active viewpoint. By using this active viewpoint, the system constructs the correspondence between images of feature points of an object on the right and left images and calculates the spatial coordinates of the feature points. However the system cannot achieve correspondence between right and left images for smooth surface without feature points like corners. In this paper, we propose a method to extract feature points on a smooth surface virtually. To extract the feature points on the smooth surface of an object virtually, we use an active viewpoint of our system and vergence motion of the right and left cameras. In this system, the right and left cameras viewpoints are corresponded mechanically by the active viewpoint and vergence motion of both cameras. As the virtual feature point, a contact point between the smooth surface and tangential line from the epipole on the baseline of right and left cameras is extracted from the camera images. Because the right and left cameras viewpoints are fixed by the active viewpoint, the active viewpoint becomes the constraint for the extraction of the virtual feature points. The information of the virtual feature points on the smooth surface of an object is used for the calculation of the spatial coordinates of the object. The effect of the proposed method to extract the feature points virtually is shown experimentally by using a sphere as an object.

Early perception-action cycles in binocular robot vision

Frontiers Events is a rapidly growing calendar management system dedicated to the scheduling of academic events. This includes announcements and invitations, participant listings and search functionality, abstract handling and publication, related events and post-event exchanges. Whether an organizer or participant, make your event a Frontiers Event!

2A2-M14 眼球運動下における視覚細胞の時空間応答エミュレーションのためのロボットビジョン

Many computational models have been developed to explain response properties and functions of the visual cortical reurons. To elucidate the functions of neurons using computational models, however, one needs to study how these models respond to realistic visual inputs, e.g., natural scenes, during eye movements. We describe a binocular robot vision system, which consists of two neuromorphic silicon retinas, field programmable gate array (FPGA), and PC for emulating visual computational models in real-time.

2A1-N10 視差エネルギー計算を用いた両眼ロボットビジョンの輻輳運動制御(脳科学・神経科学とロボティクス)

We describe a binocular vision system to emulate a disparity computation in the primary visual cortex (V1). The system consists of two silicon retinas, simple cell chips, and FPGA. The silicon retinas emulate a Laplacian-Gaussian (▽^2G)-like receptive field of the vertebrate retina. The simple cell chips generate an orientatino-selective receptive field by aggregating multiple pixels of the silicon retina, mimicking the Hubel and Wiesel type feed-forward model. The FPGA receives the outputs from the two simple cell chips corresponding to left and right eyes and calculates responses of the complex cells based on the disparity energy model. The system provides complex cell outputs tuned to five disparities and a disparity map in real time. We controlled the vergence eye movement of the binocular robot vision based on the disparity information.

Self-alignment of a binocular robot

We describe a novel method by which a four-axis binocular head platform can autonomously align its cameras so that their optic axes are parallel to each other and to the forward direction of the robot. The method uses controlled pans and elevations of the robot while viewing an unstructured scene to determine lines on the plane at infinity, whose intersection we prove to be the forward direction of the robot. The alignment is completed by fixating the projections of this point in both cameras. We summarize the underlying theory, and present results from a fully autonomous implementation of the algorithm.

A LOW-COST ROBOT CAMERA HEAD

Active vision involving the exploitation of controllable cameras and camera heads is an area which has received increased attention over the last few years. At LIA/AUC a binocular robot camera head has been constructed for use in geometric modelling and interpretation. In this manuscript the basic design of the head is outlined and a first prototype is described in some detail. Detailed specifications for the components used are provided together with a section on lessons learned from construction and initial use of this prototype.

Gaze controls cooperating through prediction

Animate vision systems, biological or robotic, employ gaze control systems to acquire and stabilize objects on their retinae, and in some animals to foveate (centre) objects. Our goal is to guild robust gaze control behaviour from cooperating lower-level visual reflexes. Predictive control strategies can cope with time delayed, multi-rate and interacting controls. Solutions are explored through simulation incorporating ten primitive gaze control capabilities, more or less comparable to subsystems in primate gaze and head control. Versions of several of the subsystems have been implemented on a binocular robot. Smith prediction is the basic control strategy, using kinematic simulation of agent and optimal filtering to predict world state.