Real-time Manipulation Research Articles

Lights, nano, action! New plasmonic materials and methods to probe nanoscale phenomena

Abstract

Optical assembly of bio-hybrid micro-robots.

The combination of micro synthetic structures with bacterial flagella motors represents an actual trend for the construction of self-propelled micro-robots. The development of methods for fabrication of these bacteria-based robots is a first crucial step towards the realization of functional miniature and autonomous moving robots. We present a novel scheme based on optical trapping to fabricate living micro-robots. By using holographic optical tweezers that allow three-dimensional manipulation in real time, we are able to arrange the building blocks that constitute the micro-robot in a defined way. We demonstrate exemplarily that our method enables the controlled assembly of living micro-robots consisting of a rod-shaped prokaryotic bacterium and a single elongated zeolite L crystal, which are used as model of the biological and abiotic components, respectively. We present different proof-of-principle approaches for the site-selective attachment of the bacteria on the particle surface. The propulsion of the optically assembled micro-robot demonstrates the potential of the proposed method as a powerful strategy for the fabrication of bio-hybrid micro-robots.

Adaptive hidden Markov model with anomaly States for price manipulation detection.

Price manipulation refers to the activities of those traders who use carefully designed trading behaviors to manually push up or down the underlying equity prices for making profits. With increasing volumes and frequency of trading, price manipulation can be extremely damaging to the proper functioning and integrity of capital markets. The existing literature focuses on either empirical studies of market abuse cases or analysis of particular manipulation types based on certain assumptions. Effective approaches for analyzing and detecting price manipulation in real time are yet to be developed. This paper proposes a novel approach, called adaptive hidden Markov model with anomaly states (AHMMAS) for modeling and detecting price manipulation activities. Together with wavelet transformations and gradients as the feature extraction methods, the AHMMAS model caters to price manipulation detection and basic manipulation type recognition. The evaluation experiments conducted on seven stock tick data from NASDAQ and the London Stock Exchange and 10 simulated stock prices by stochastic differential equation show that the proposed AHMMAS model can effectively detect price manipulation patterns and outperforms the selected benchmark models.

Controlling assembly of colloidal particles into structured objects: Basic strategy and a case study

A computational study is presented in which real-time manipulation of the interaction potential between particles in a colloidal system is used to control their assembly into a close-packed crystalline object. The basic model used throughout the study is a high-fidelity representation of a real experimental system in which 32 colloidal silica particles are suspended in aqueous solution with polymer hydrogel providing a temperature-tunable attractive force between the particles. Diffusion mapping is used to determine a set of coarse variables that provide an appropriate low-dimensional representation of this system at four discrete values of the attraction strength. In this case the diffusion mapping process identified two dimensions; one correlates well with the radius of gyration of the entire set of particles and the other correlates well with the average distance between distinct clusters of particles. Two different stochastic models are then built in the two-dimensional (2D) space of these variables, using data from a large number of short Brownian dynamics simulations of the full 32-particle system. The first 2D model is based on a Smoluchowski framework and is used to characterize the overall equilibrium and diffusive properties of the system. The second 2D model is based on a transition rate matrix and is used for process control. A control policy based on an infinite-horizon Markov decision process is developed using the four different attraction strengths as the input variables. The resulting policy is non-trivial; rather than simply selecting the strongest level of attraction, some mix of weak and strong attractions generally provides the optimal approach to the target close-packed state. This study, while focused on the particular mechanism of tunable depletion attraction, suggests a general strategy that could be adapted to different mechanisms of actuating colloidal assembly.

Reversible thermo-pneumatic valves on centrifugal microfluidic platforms.

Centrifugal microfluidic systems utilize a conventional spindle motor to automate parallel biochemical assays on a single microfluidic disk. The integration of complex, sequential microfluidic procedures on these platforms relies on robust valving techniques that allow for the precise control and manipulation of fluid flow. The ability of valves to consistently return to their former conditions after each actuation plays a significant role in the real-time manipulation of fluidic operations. In this paper, we introduce an active valving technique that operates based on the deflection of a latex film with the potential for real-time flow manipulation in a wide range of operational spinning speeds. The reversible thermo-pneumatic valve (RTPV) seals or reopens an inlet when a trapped air volume is heated or cooled, respectively. The RTPV is a gas-impermeable valve composed of an air chamber enclosed by a latex membrane and a specially designed liquid transition chamber that enables the efficient usage of the applied thermal energy. Inputting thermo-pneumatic (TP) energy into the air chamber deflects the membrane into the liquid transition chamber against an inlet, sealing it and thus preventing fluid flow. From this point, a centrifugal pressure higher than the induced TP pressure in the air chamber reopens the fluid pathway. The behaviour of this newly introduced reversible valving system on a microfluidic disk is studied experimentally and theoretically over a range of rotational frequencies from 700 RPM to 2500 RPM. Furthermore, adding a physical component (e.g., a hemispherical rubber element) to induce initial flow resistance shifts the operational range of rotational frequencies of the RTPV to more than 6000 RPM. An analytical solution for the cooling of a heated RTPV on a spinning disk is also presented, which highlights the need for the future development of time-programmable RTPVs. Moreover, the reversibility and gas impermeability of the RTPV in the microfluidic networks are validated on a microfluidic disk designed for performing liquid circulation. Finally, an array of RTPVs is integrated into a microfluidic cartridge to enable sequential aliquoting for the conversion of dengue virus RNA to cDNA and the preparation of PCR reaction mixtures.

Creating and maintaining chemical artificial life by robotic symbiosis.

We present a robotic platform based on the open source RepRap 3D printer that can print and maintain chemical artificial life in the form of a dynamic, chemical droplet. The robot uses computer vision, a self-organizing map, and a learning program to automatically categorize the behavior of the droplet that it creates. The robot can then use this categorization to autonomously detect the current state of the droplet and respond. The robot is programmed to visually track the droplet and either inject more chemical fuel to sustain a motile state or introduce a new chemical component that results in a state change (e.g., division). Coupling inexpensive open source hardware with sensing and feedback allows for replicable real-time manipulation and monitoring of nonequilibrium systems that would be otherwise tedious, expensive, and error-prone. This system is a first step towards the practical confluence of chemical, artificial intelligence, and robotic approaches to artificial life.

Visualization and interaction in the atlas of the human brain, head and neck

Our ultimate objective is to create a holistic and reference atlas of the whole adult human brain along with the head and neck. Several techniques have been employed to create atlases. Here we discuss the atlas design and use from a point of view of two key techniques, visualization and interaction. For visualization, surface rendering of a geometrical model of the brain, head and neck is employed. Geometrical model ensures anatomic parcellability, high (sub-pixel) resolution, editibility, extendibility, structure separability, structure-to-structure conflict detection, and integration a knowledge-based content with the atlas. Interaction allows the user to create and explore any region of interest along with its surroundings just with a few clicks, taking into account that the atlas provides a rich set of functions and the number of atlas components is about 3,000. There are seven types of interaction enabling to: select and deselect tissue classes/groups/individual structures, do real-time manipulation, do virtual dissections, select and scroll the original scans, query a structure to get its label or location, get stereotactic coordinates and measure distances, and support other functionality. This design of visualization and interaction provides a fast and easy to use solution, and allows the atlas to run on desktop and mobile iPad and Android-based platforms.

Coherent versus Measurement Feedback: Linear Systems Theory for Quantum Information

To control a quantum system via feedback, we generally have two options in choosing control scheme. One is the coherent feedback, which feeds the output field of the system, through a fully quantum device, back to manipulate the system without involving any measurement process. The other one is the measurement-based feedback, which measures the output field and performs a real-time manipulation on the system based on the measurement results. Both schemes have advantages/disadvantages, depending on the system and the control goal, hence their comparison in several situation is important. This paper considers a general open linear quantum system with the following specific control goals; back-action evasion (BAE), generation of a quantum non-demolished (QND) variable, and generation of a decoherence-free subsystem (DFS), all of which have important roles in quantum information science. Then some no-go theorems are proven, clarifying that those goals cannot be achieved by any measurement-based feedback control. On the other hand it is shown that, for each control goal, there exists a coherent feedback controller accomplishing the task. The key idea to obtain all the results is system theoretic characterizations of BAE, QND, and DFS in terms of controllability and observability properties or transfer functions of linear systems, which are consistent with their standard definitions.

Uncertainty-Aware Sensor Data Management and Early Warning for Monitoring Industrial Infrastructures

In several industrial applications, monitoring large-scale infrastructures in order to provide notifications for abnormal behavior is of high significance. For this purpose, the deployment of large-scale sensor networks is the current trend. However, this results in handling vast amounts of low-level, and often unreliable, data, while an efficient and real-time data manipulation is a strong demand. In this paper, the authors propose an uncertainty-aware data management system capable of monitoring interrelations between large and heterogeneous sensor data streams in real-time. To this end, an efficient similarity function is employed instead of the typical correlation coefficient to monitor dynamic phenomena for timely alerting notifications, and to guarantee the validity of detected extreme events. Experimental evaluation with a set of real data recorded by distinct sensors in an industrial water desalination plant reveals a superior performance of our proposed approach in terms of achieving significantly reduced execution times, along with increased accuracy in detecting extreme events and highly correlated pairs of sensor data streams, when compared with state-of-the-art data stream processing techniques.

Remote Control of Tissue Interactions via Engineered Photo-switchable Cell Surfaces

We report a general cell surface molecular engineering strategy via liposome fusion delivery to create a dual photo-active and bio-orthogonal cell surface for remote controlled spatial and temporal manipulation of microtissue assembly and disassembly. Cell surface tailoring of chemoselective functional groups was achieved by a liposome fusion delivery method and quantified by flow cytometry and characterized by a new cell surface lipid pull down mass spectrometry strategy. Dynamic co-culture spheroid tissue assembly in solution and co-culture tissue multilayer assembly on materials was demonstrated by an intercellular photo-oxime ligation that could be remotely cleaved and disassembled on demand. Spatial and temporal control of microtissue structures containing multiple cell types was demonstrated by the generation of patterned multilayers for controlling stem cell differentiation. Remote control of cell interactions via cell surface engineering that allows for real-time manipulation of tissue dynamics may provide tools with the scope to answer fundamental questions of cell communication and initiate new biotechnologies ranging from imaging probes to drug delivery vehicles to regenerative medicine, inexpensive bioreactor technology and tissue engineering therapies.

Motion planning for a rapid mobile manipulator using model-based ZMP stabilization

SUMMARYThis paper introduces a novel approach to motion planning for a rapid mobile manipulator using inverted pendulum models. Our aim was to realize an actual rapid mobile manipulator with high acceleration and speed performance for an object's delivery. In our research, we developed an actual rapid mobile manipulator called KDMR-1. We proposed simple motion planning methods using a single inverted pendulum model (SIPM) and a double inverted pendulum model (DIPM), which are easily adaptable to a real-time system with only a small computational burden. The SIPM was useful for basic movement but did not provide object carrying capability. For that, a DIPM was proposed. In both models, we designed linear quadratic optimal controllers to stabilize the Zero Moment Point (ZMP). Two kinds of ZMP stabilization strategies were proposed, fixed ZMP and relaxed ZMP. Using these strategies, we realized optimal ZMP stabilizations for a real-time rapid mobile manipulator. For decoupled forward and rotational linear DIPM, we designed a centrifugal acceleration compensation model in the manner of feedback linearization. The experimental results showed high acceleration and speed performances during rapid object delivery.

Real-time transrectal ultrasonography-guided hands-free technique for focal cryoablation of the prostate.

To describe, step-by-step, our hands-free technique for focal cryoablation of prostate cancer. After detailed discussion of its limitations and benefits, consent was obtained to perform focal cryoablation in patients with biopsy-proven unilateral low- to intermediate-risk prostate cancer. The procedure was performed transperineally, using a hands-free technique (without an external grid template) under real-time bi-plane transrectal ultrasonography (TRUS) guidance, using an argon/helium-gas-based third generation cryoablation system. Follow-up consisted of validated questionnaires, physical examination, PSA measures, multiparametric TRUS and/or magnetic resonance imaging (MRI) and mandatory biopsy. The important steps for achieving safety, satisfactory oncological and functional outcomes included: patient selection, including TRUS/MRI fusion target biopsy; thermocouple and cryoprobe placement with a hands-free technique, allowing delivery in unrestricted angulations according to the prostatic contour, the course of the neurovascular bundle and the rectal wall angle; and hands-free bi-plane TRUS probe manipulation to facilitate real-time monitoring of anatomical landmarks at the ideal angle of the image plane. To achieve a lethal temperature in the known cancer area, while preserving the urinary sphincter, neurovascular bundle, urethra and rectal wall, continuous intraoperative control of the thermocouple temperatures was necessary, as were real-time TRUS monitoring of ice-ball size, control of the energy delivered and the use of a warming urethral catheter. We have described step-by-step the focal cryoablation of prostate cancer using a hands-free technique. This technique facilitates the effective delivery of cryoprobes and the intra-operative real-time quick manipulation of the TRUS probe.

Real-time interactive treatment planning

The goal of this work is to develop an interactive treatment planning platform that permits real-time manipulation of dose distributions including DVHs and other dose metrics. The hypothesis underlying the approach proposed here is that the process of evaluating potential dose distribution options and deciding on the best clinical trade-offs may be separated from the derivation of the actual delivery parameters used for the patient’s treatment. For this purpose a novel algorithm for deriving an Achievable Dose Estimate (ADE) was developed. The ADE algorithm is computationally efficient so as to update dose distributions in effectively real-time while accurately incorporating the limits of what can be achieved in practice. The resulting system is a software environment for interactive real-time manipulation of dose that permits the clinician to rapidly develop a fully customized 3D dose distribution. Graphical navigation of dose distributions is achieved by a sophisticated method of identifying contributing fluence elements, modifying those elements and re-computing the entire dose distribution. 3D dose distributions are calculated in ~2–20 ms. Including graphics processing overhead, clinicians may visually interact with the dose distribution (e.g. ‘drag’ a DVH) and display updates of the dose distribution at a rate of more than 20 times per second. Preliminary testing on various sites shows that interactive planning may be completed in ~1–5 min, depending on the complexity of the case (number of targets and OARs). Final DVHs are derived through a separate plan optimization step using a conventional VMAT planning system and were shown to be achievable within 2% and 4% in high and low dose regions respectively. With real-time interactive planning trade-offs between Target(s) and OARs may be evaluated efficiently providing a better understanding of the dosimetric options available to each patient in static or adaptive RT.

Real-time manipulation with magnetically tunable structures.

Magnetically tunable micropillar arrays with uniform, continuous and extreme tilt angles for real-time manipulation are reported. We experimentally show uniform tilt angles ranging from 0° to 57°, and develop a model to accurately capture the behavior. Furthermore, we demonstrate that the flexible uniform responsive microstructures (μFUR) can dynamically manipulate liquid spreading directionality, control fluid drag, and tune optical transmittance over a large range.

Two-photon-responsive supramolecular hydrogel for controlling materials motion in micrometer space.

Spatiotemporal control of fluidity inside a soft matrix by external stimuli allows real-time manipulation of nano/micromaterials. In this study, we report a two-photon-responsive peptide-based supramolecular hydrogel, the fluidity of which was dramatically controlled with high spatial resolution (10 μm×10 μm×10 μm). The off-on switching of the Brownian motion of nanobeads and chemotaxis of bacteria by two-photon excitation was successfully demonstrated.

Some aspects of formation and tribological properties of silver nanodumbbells.

In this paper, metal nanodumbbells (NDs) formed by laser-induced melting of Ag nanowires (NWs) on an oxidized silicon substrate and their tribological properties are investigated. The mechanism of ND formation is proposed and illustrated with finite element method simulations. Tribological measurements consist in controllable real-time manipulation of NDs inside a scanning electron microscope (SEM) with simultaneous force registration. The geometry of NDs enables to distinguish between different types of motion, i.e. rolling, sliding and rotation. Real contact areas are calculated from the traces left after the displacement of NDs and compared to the contact areas predicted by the contact mechanics and frozen droplet models.PACS81.07.-b; 62.25.-g; 62.23.Hj

Generating harmonic surfaces for interactive design

A method is given for generating harmonic tensor product Bézier surfaces and the explicit expression of each point in the control net is provided as a linear combination of prescribed boundary control points. The matrix of scalar coefficients of these combinations works like a mould for harmonic surfaces. Thus, real-time manipulation of the resulting surfaces subject to modification of prescribed information is possible.

Electrostatic pick-and-place micro/nanomanipulation under the electron beam

Here we present a method of particle pick-and-place manipulation, which is based on electrostatic interaction between objects charged under the beam of a scanning electron microscope. This method allows well-controlled pickup, transport and placement of micro- and nanoparticles as well as visualization of the process and the result of manipulation in real time on electron images. Manipulation of Al2O3, WO3 and tungsten particles under the influence of the electric field proposed by a charged metalized tip was experimentally demonstrated. We also created a theoretical model, which is based on the assumption that the dielectrophoretic force created by the charged metallic tip pulls a particle whereas the van der Waals force retains it on a substrate.

Vision-based control of robotic manipulator for citrus harvesting

The main contribution of this paper is in the development of vision-based estimation and control system for robotic fruit harvesting and rigorous stability analysis to guarantee performance of the closed-loop system. The presented cooperative visual servo controller benefits from the large field-of-view of a fixed camera and the accuracy of a camera-in-hand (CiH). Computationally inexpensive perspective transformation-based range estimation method obtains 3D fruit position using a monocular camera to enable real-time manipulator control. A rotation controller is developed to orient the robot such that the target fruit selected by the fixed camera can be viewed by the CiH attached to the end-effector. Subsequently, the end-effector can be servoed to the target fruit location using the presented pursuit guidance based hybrid translation controller. Lyapunov-based stability analysis guarantees global exponential regulation of the end-effector. Numerical simulations verify the feasibility of the developed controller while the performance is evaluated on a seven degrees-of-freedom kinematically redundant manipulator using an artificial citrus tree. The position of the fruit was randomly selected, and the closed-loop visual servo control experiment was performed 21 times to analyze the repeatability and accuracy of the developed controller. With 95% confidence level the expected position of the robot end-effector is observed to lie within the confidence ellipsoid. The accuracy of the controller was observed to be about 15mm, thus making the system suitable for harvesting medium and large varieties of citrus fruit but may limit operation for small varieties such as page and blood oranges.

プロジェクタカメラ系を用いた光沢感と透明感の実時間操作

プロジェクタカメラ系を用いた光沢感と透明感の実時間操作