High-speed Vision System Research Articles

Experimental analysis on relationship between roughness parameters and texture features of 6082T6 sandblasted components

Abstract In the present investigation, feasibility assessment has been deployed on 6082T6 alloy sand blasted components to correlate surface roughness parameters and surface attributes. A non-contact type high-speed machine vision system has been adopted to acquire the images of sandblasted work pieces. The surface profilometer has been used to quantify surface roughness parameters of different sandblasted work pieces. The measurement has been taken for various process conditions such as blasting pressure, the distance between work piece and nozzle, nozzle size and type of abrasive. The discrete wavelet transform has been adopted to enhance captured images in such a way that interfering noise can be eliminated. Gray level co-occurrence matrix (GLCM) algorithm has been used to extract texture features from pre-processed images of machined components. The result showed high correlation factors between texture feature contrast and surface roughness parameters, which confirms that proposed approach can be useful for roughness parameters prediction of high precision.

High-speed Vision System for Dynamic Structural Distributed Displacement Analysis

Structural health monitoring (SHM) has become prominent topic in the field of civil engineering in detecting structural damage, estimating remaining service life of structure, optimizing decision making process on maintenance, and aiding performance-based structure design methodology. Dynamics measurement is one of important techniques to diagnose deterioration of structures by inspecting their dynamics properties, mainly derived from displacement time-series from multiple measurement points. Various conventional distributed displacement measurement techniques of civil structures currently in place have been limited due to various constraints: inadequate accuracy, overly-localized measurement point, measurement drift, robustness issue on field usage, structural obstruction, and reliability for long-term installation. Optical technique utilizing LED markers has been proposed as an alternative for dynamic distributed displacement measurement, providing visual cue and and easily-identifiable marker position. The technique incorporates high-speed camera system and telephoto lens to capture both lateral and vertical displacement of bridge members, LED markers to cope with the insufficient incident light over long distance and small camera aperture, and computer vision system to calculate multiple markers’ displacement simultaneously. Further analysis was conducted to evaluate structure natural vibration frequencies using FFT (Fast Fourier Transporm) peak-picking method.

A High-Speed, Real-Time Vision System for Texture Tracking and Thread Counting

In garment manufacturing, an automatic sewing machine is desirable to reduce cost. To accomplish this, a high speed vision system is required to track fabric motions and recognize repetitive weave patterns with high accuracy, from a micro perspective near a sewing zone. In this paper, we present an innovative framework for real-time texture tracking and weave pattern recognition. Our framework includes a module for motion estimation using blob detection and feature matching. It also includes a module for lattice detection to facilitate the weave pattern recognition. Our lattice detection algorithm utilizes blob detection and template matching to assess pair-wise similarity in blobs' appearance. In addition, it extracts information of dominant orientations to obtain a global constraint in the topology. By incorporating both constraints in the appearance similarity and the global topology, the algorithm determines a lattice that characterizes the topological structure of the repetitive weave pattern, thus allowing for thread counting. In our experiments, the proposed thread-based texture tracking system is capable of tracking denim fabric with high accuracy (e.g., 0.03 degree rotation and 0.02 weave-thread' translation errors) and high speed (3 frames per second), demonstrating its high potential for automatic real-time textile manufacturing.

A High-Speed Vision System with Multithread Automatic Exposure Control for High-Dynamic-Range Imaging

In this study, we develop a real-time high-frame-rate vision system with frame-by-frame automatic exposure (AE) control that can simultaneously synthesize multiple images with different exposure times into a high-dynamic-range (HDR) image for scenarios with dynamic change in illumination. By accelerating the video capture and processing for time-division multithread AE control at the millisecond level, the proposed system can virtually function as multiple AE cameras with different exposure times. This system can capture color HDR images of 512 × 512 pixels in real time at 500 fps by synthesizing four 8-bit color images with different exposure times at consecutive frames, captured at an interval of 2 ms, with pixel-level parallel processing accelerated by a GPU (Graphic Processing Unit) board. Several experimental results for scenarios with a large change in illumination are demonstrated to confirm the performance of the proposed system for real-time HDR imaging.

Vibration Sensing of a Bridge Model Using a Multithread Active Vision System

This study proposes a concept for multithread active vision sensing that can measure dynamically changing displacement and vibration at multiple points on civil engineering structures. In multithread active vision sensing, a high-speed camera can function virtually as multiple tracking cameras by accelerating its measurement, computation, and actuation with ultrafast viewpoint switching at millisecond level. This enables simultaneous measurement of small vibrations distributed across a wide range, which cannot be observed using a single camera system, because its pixel accuracy is generally incompatible with its wide angle of view. We developed a galvanomirror-based high-speed multithread active vision system that can switch 500 different views in a second; it functioned as 15 virtual cameras each operating at 33.3 fps to observe multiple scenes in completely different views. The experimental results for a 4-m-long truss-structure bridge model to which 15 markers were attached show that a single active vision system can observe the deformation of the bridge structure and estimate modal parameters, such as resonant frequencies and mode shapes, at a frequency on the order of dozens of hertz.

Dynamic compensation robot with a new high-speed vision system for flexible manufacturing

This paper aims to enable an industrial robot to realize real-time adaptation to uncertainty, which is generally associated with the issue of machine flexibility of a flexible manufacturing system (FMS). A new robotic scheme called dynamic compensation robot with a new high-speed vision system is presented. Under the proposed scheme, a traditional multi-joint industrial robot is designated for fast and coarse global motion, whereas a direct-driven add-on module is realizing local compensation of accumulated uncertainty. 1000 fps image sensing and processing is realized within the new high-speed vision system. Overall latency of high-speed visual feedback was measured to be within 3.0 ms. As an early stage showcase towards flexible manufacturing application, contour tracing of planar target with uncertainty was evaluated. Effectiveness of the proposed method as well as the new high-speed vision was validated by experimental results.

Development of impact point control system of the golf swing using high speed vision system

Development of impact point control system of the golf swing using high speed vision system

Flower Stick Manipulation Based on the High-speed Visual Feedback Control of a Dual Robotic Arm

Abstract This paper describes a manipulation strategy for a flower stick based on the highspeed visual feedback control of a dual robotic arm. Previously, we achieved propeller motion of a flower stick by using a robotic arm with high-speed visual feedback; this motion was a dynamic continuous rotational motion. This work aims to develop a dual robotic arm system with high-speed visual feedback and achieve dynamic dexterous manipulation. We focused on the “idling motion” of a flower stick—a juggling task—generated by using two arms as a fundamental manipulation of a dual robotic arm with high-speed visual feedback. We developed a dual robotic arm system with a high-speed vision system and proposed a visual feedback control strategy for idling motion. The developed system and proposed control strategy are demonstrated through an experiment.

Real-time Vibration Spectrum Imaging Using High-speed Vision

This paper describes a real-time vibration spectrum imaging system with pixel-level shortterm Fourier Transform (STFT); the pixel-level STFT function for HFR images at 1000 fps or more is accelerated with its parallel-implementation on a GPU-based high-speed vision system, and power vibration spectrum images of 512×512 pixels can be obtained in real time at 30 fps By showing experimental results for vibrating objects such as a multicopter and guitar strings, the effectiveness of real-time vibration spectrum imaging was verified on our system.

Multimodal sensor-based weight drop spinal cord impact system for large animals

Background ContextA conventional weight drop spinal cord (SC) impact system for large animals is composed of a high-speed video camera, a vision system, and other things. However, a camera with high speed at over 5,000 frames per second (FPS) is very expensive. In addition, the use of the vision system involves complex pattern recognition algorithms and accurate arrangement of the camera and the target. PurposeThe purpose of this study was to develop a large animal spinal cord injury (SCI) modeling system using a multimodal sensor instead of a high-speed video camera and vision system. Another objective of this study was to demonstrate the possibility of the developed system to measure the impact parameters in the experiments using different stiffness materials and an in vivo porcine SC. Study DesignA multimodal sensor-based SCI impact system was developed for large animals. The experiments to measure SC impact parameters were then performed using three different stiffness materials and a Yucatan miniature pig to verify the performance of system developed. MethodsA comparative experiment was performed using three different stiffness materials such as high-density (HD) sponge, rubber, and clay to demonstrate the system and perform measurement for impact parameters such as impact velocity, impulsive force, and maximally compressed displacement reflecting physical properties of materials. In the animal experiment, a female Yucatan miniature pig of 60-kg weight was used. Impact conditions for all experiments were fixed at freefalling object mass of 50 g and height of 20 cm. ResultsIn the impact test, measured impact velocities were almost the same for the three different stiffness materials at 1.84±0.0153 m/s. Impulsive forces for the three materials of rubber, HD sponge, and clay were 50.88 N, 32.35 N, and 6.68 N, respectively. Maximally compressed displacements for rubber, HD sponge, and clay were 1.93 mm, 3.35 mm, and 15.01 mm, respectively. In the pig experiment, impact velocity, impulsive force, and maximally compressed dural displacement were measured at 1.84 m/s, 13.35 N, and 3.04 mm, respectively. After 3 days from the experiment, paralysis was confirmed for the lower half body of the experimental pig. ConclusionsThrough experiments, it was verified that our proposed system could be used to measure the SC impact parameters and induce SCI for large animals.

A Vector-Quantization Compression Circuit With On-Chip Learning Ability for High-Speed Image Sensor

As the fundamental technology of autonomous vehicles and high-speed tracking, high-speed vision always suffers from the bottlenecks of on-chip bandwidth and storage due to the resource constraints. To improve the resource efficiency, we propose a hardware-efficient image compression circuit based on the vector quantization for a high-speed image sensor. In this circuit, a self-organizing map is implemented for the on-chip learning of codebook to flexibly satisfy the requirements of different applications. To reduce the hardware resources, we present a reconfigurable complete-binary-adder-tree, where the arithmetic units are reused completely. In addition, a mechanism of partial vector-component storage is adapted to make the compression ratio adjustable. Finally, a parallel-elementary-stream design ensures a high processing speed. The proposed circuit has been implemented on the field-programmable gate araray and also applied in a high-speed object tracking system. The experimental results indicate that it achieves an encoding speed of 722 frames/s with 128 weight vectors when working at 79.8 MHz, and the worst tracking error caused by the proposed circuit is merely 9 pixels. These results evince that our proposed circuit can be completely integrated with a high-speed image sensor and used in high-speed vision systems.

用于机械系统瞬时目标的双视角高速视觉检测系统

用于机械系统瞬时目标的双视角高速视觉检测系统

Review of some advances and applications in real-time high-speed vision: Our views and experiences

The frame rate of conventional vision systems is restricted to the video signal formats (e.g., NTSC 30 fps and PAL 25 fps) that are designed on the basis of the characteristics of the human eye, which implies that the processing speed of these systems is limited to the recognition speed of the human eye. However, there is a strong demand for real-time high-speed vision sensors in many application fields, such as factory automation, biomedicine, and robotics, where high-speed operations are carried out. These high-speed operations can be tracked and inspected by using high-speed vision systems with intelligent sensors that work at hundreds of Hertz or more, especially when the operation is difficult to observe with the human eye. This paper reviews advances in developing real-time high speed vision systems and their applications in various fields, such as intelligent logging systems, vibration dynamics sensing, vision-based mechanical control, three-dimensional measurement/automated visual inspection, vision-based human interface, and biomedical applications.

A Filter Formulation for Computing Real Time Optical Flow

Dense optical flow is a crucial visual cue for obstacle avoidance and motion control for robotic systems functioning in complex unstructured environments. In order to compute image motion for modern highly dynamic robots, it is necessary to use high speed vision systems. To perceive small and thin objects such as tree branches, fence poles, and other similar objects, high resolution vision systems must be used. The data stream from a high-resolution, high-speed vision system will saturate the onboard data bus and overload the onboard processor of almost all existing robotic systems. To implement such a vision system, we believe it is necessary to process the data in situ at the camera using dedicated processing hardware, a vision processing paradigm that we term rapid embedded vision (REV) systems. Even then, the task of processing multiple hundred hertz of dense high-resolution images is not possible using classical algorithms and classical computing hardware. A new processing paradigm that actively exploits the high frame rate of the image sequences, and high performance computing hardware such as GPU or FPGA must be employed. This letter proposes a filtering algorithm for the computation of dense optical flow fields in real time. The filter is designed as a pyramidal structure of update and propagation loops, where an optical flow state is constantly refined with new image data from the camera. The computational properties of the proposed algorithm makes it well suited for implementation in GPU and FPGA systems. We present results from a GPU implementation achieving frame rates in the order of 800 Hz at VGA resolution. Experimental validation and comparison is provided for both synthetic and real life high speed video sequences at frame rates of 300 Hz and 1016 × 544 pixel resolution.

Visual encoder: robust and precise measurement method of rotation angle via high-speed RGB vision.

In this paper, a new measurement method of the rotation angle of a rotor, which is named the 'visual encoder,' was proposed. This method is based on the principles of the vision-based method and the optical encoder, and realized by using a high-speed vision system. The visual encoder shows advantageous features such as non-contact, high-resolution and robustness against the free motion and the fluctuation of the rotation axis. A high resolution method to increase the measurement resolution was also suggested. The accuracy and the robustness of the visual encoder were confirmed through the experimental verifications, and the operation was possible at 6,000 rpm even under the fluctuation of rotation axis.

Equi-biaxial fatigue testing of EPM utilising bubble inflation

This paper describes an equi-biaxial tension fatigue test system which utilises the bubble inflation method to subject elastomers to equi-biaxial fatigue loading between user-defined limits of pressure, volume, stretch ratio or stress. The test system integrates a hydraulic inflation system, a high speed vision system and a control system. The high-speed vision system allows the stretch ratio and stress acting on the test specimen to be evaluated in real-time during testing. This in turn allows either stretch ratio or stress to be used as a direct control limit. In this research, constant maximum engineering stress control tests have been carried out to evaluate the suitability of the developed dynamic bubble inflation system for equi-biaxial fatigue testing of elastomers. The resulting test data was used to produce Wöhler (S/N) curves and stress/stretch ratio plots for ethylene-propylene rubber (EPM).

ヒトとのコミュニケーションを意識したロボットハンド機構の研究

In this paper we develop a janken(stone-scissors-paper)robot as one example of human-machine interface systems. Human being plays one of stone, paper and scissors at the timing of one, two, three. According the timing, the robot hand plays one of stone, paper and scissors. In this system we must develop 1) the high-speed vision system of recognition about position and shape of human hand, 2) the high-speed robot hand playing as same speed as human hand. But a high speed vision system already have develop in few previous study. In this paper we develop robot hand as noted above 2). We have already developed a lower arm reported in 2014 at this conference. In this report we discuss total janken robot system with upper arm. From some experiments, the preproduction janken robot system can play one of stone, paper and scissors in 75[ms].

ハンドアームロボットによる動力学を考慮したボールジャグリング動作

In this paper, we study ball juggling as a dexterous and accurate high-speed manipulation like a human. We have already achieved two-ball juggling with our high-speed robotic handarm system which has three general-purpose fingers and high-speed vision system. However, the number of times of ball-catching is not so many because the trajectory of the motion of the robot is not optimized. In this paper, we propose a trajectory generation method using smoothing splines. The motion of throwing a ball is optimized so as to minimize the torques of the joint angles. The motion of catching is controlled by visual feedback. As a result, stable throwing motion is achieved so that the success rate of ball-catching is about 90%.

高速ロボットハンドシステムの人間機械協調動作への応用

We demonstrated a high-speed human-robot cooperation task using a high-speed robot hand system consisting of a high-speed vision system, an image processing PC, a high-speed robot hand, a high-speed tactile sensor (only for measurement) and a real-time controller. In particular, our goal was to achieve a concrete task in which a board grasped by both a human subject and a robot hand is always kept horizontal by controlling the robot hand based on high-speed visual feedback using the vision system. We propose a basic strategy for appropriately achieving the task and discuss an image processing procedure for detecting the state of the board (position and angle). Also, we describe the inverse kinematics of the robot hand so as to keep the board horizontal, and we set limitations for the robot hand joint angles for avoiding an error action due to human error. Finally, we show experimental results for the human-robot cooperation task achieved using the high-speed robot system and our proposed method, and we compare the experimental results between low frame rate and high frame rate in terms of response time and reaction force detected by a high-speed tactile sensor.

Motion Generation for a Sword-Fighting Robot Based on Quick Detection of Opposite Player’s Initial Motions

<div class=""abs_img""> <img src=""[disp_template_path]/JRM/abst-image/00270005/11.jpg"" width=""300"" /> Defensive motion against attack</div> In this paper, we discuss how to generate defensive motions for a sword-fighting robot based on quick detection of the opposite player’s initial motions. Our sword-fighting robot system, which has a stereo high-speed vision system, recognizes both the position of a human player and that of the sword grasped by the robot’s hand. Further, it detects the moment when the human player initiates a move using ChangeFinder, which is a method of detecting change points. Next, using least squares method, it predicts the possible trajectories of the sword of the human player from the moment when the attack starts. Finally, it judges the type of the attack and generates an appropriate defensive motion. The effectiveness of the proposed algorithm is verified by experimental results. </span>