Real-time Manipulation Research Articles

Attosecond Science and Technology

The quest for how quantum systems evolve, and eventually, how to induce such quantum systems to behave as desired has been growing during the last two decades in a new paradigm, “Control Age” of the 21st century science. The past two decades have witnessed the remarkable advance in the new metrology for ultrafast electron dynamics, which allows one to control material processes at electron level and study dynamics far away from equilibrium, to which the 2023 Nobel Prize in Physics was awarded. This article reviews recent progress in the generation of isolated attosecond pulses and their applications for characterization and real-time measurement and manipulation of electron dynamics in atoms, and condensed matters, providing the audience with the new insights and perspective that these tools can provide in aspects of both fundamental science and future technology.

Three-Dimensional (3D) Microcatheter Shaping Using Touch Screen Devices for Cerebral Aneurysm Coil Embolization

In cerebral aneurysm coil embolization, proper microcatheter shaping is crucial to reduce complications and achieve sufficient embolization. Shaping a microcatheter in 3 dimensions (3D) is often required but can be challenging. We assessed the usefulness of a novel shaping on screen (SOS) method that displays real-size 3D rotational angiography (RA) images on a touch screen device during cerebral aneurysm embolization to facilitate 3D microcatheter shaping. In this study, 18 patients with cerebral aneurysm treated with this technique were included. Real-size 3D-RA images obtained during the embolization procedure were displayed on the touch screen device, which allowed for real-time manipulation. The shape of the microcatheter was adjusted to conform to the curvature of the vessel by swiping the touch screen device and bending the mandrel accordingly. We assessed the clinical and angiographic results, along with the accuracy and stability of the microcatheter. No procedure-related complications were observed. The mean packing density was 41%±12%. In all but 1 case, microcatheters were inserted into the aneurysms without guidewire assistance. After coiling, all microcatheter forms were stable. Three-dimensional (3D) microcatheter shaping using touch screen devices during cerebral aneurysm coil embolization may be simple and safe and can achieve high packing density of aneurysms.

Investigation of a new magnetorheological elastomer metamaterial plate with continuous programmable properties for vibration manipulation

Metamaterial plates have attracted considerable attention due to their unprecedented elastic wave manipulation abilities. Currently, the band gap and other properties of traditional metamaterial plates are fixed, and the material parameters of most metamaterials cannot be dynamically and continuously adjusted. To address this issue, a two-dimensional (2D) magnetorheological elastomer (MRE) programmable metamaterial plate (MREPMP) with local resonance was proposed. The proposed MREPMP was composed of a thin steel plate and a periodic array of resonators, consisting of MREs and electromagnetic mass blocks. It not only has a compact structural design, but also enables real-time vibration control and programming wave manipulation by altering the amplitude of the current. The dispersion relation of the proposed MREPMP was theoretically calculated based on the plane wave expansion (PWE) method and the simulated transmissibility was numerically calculated using the finite element method in COMSOL. Subsequently, a prototype of the MREPMP was fabricated, and vibration tests were conducted to assess its programmable vibration manipulation functionalities. Two different types of defect paths were used as excitations and the corresponding real-time vibration manipulation performance was analyzed. The simulation results agree with the experimental results, showing that the MREPMP can dynamically tune the band gap and activate different waveguides through continuous online programming. This paper opens up a new idea for tunable manipulation of wave propagation.

Towards Industry 4.0: Color-Based Object Sorting Using a Robot Arm and Real-Time Object Detection

With the introduction of Industry 4.0, automation and robotics have made great strides, enabling 
 enterprises to improve their manufacturing processes for increased productivity and efficiency. This project 
 introduces a novel method for implementing Industry 4.0 concepts through color-based object sorting employing 
 a robot arm with real-time object identification capabilities. Creating a reliable and effective system that can 
 automatically categorize items based on their color properties is the main goal of this project. To enable seamless 
 object recognition and manipulation in real time, the suggested system integrates robotic manipulation with 
 computer vision algorithms. The system makes use of a convolutional neural network (CNN) for precise object 
 detection, using recent advancements in deep learning and image processing, allowing the robot arm to interact 
 with a variety of items effectively. The training phase and the sorting phase are the two key phases of the 
 approach. The CNN model is trained on a sizable dataset of labeled objects during the training phase to recognize 
 various colors and forms. In order for the robotic arm to recognize things as they go along the conveyor belt and 
 sort them into predetermined bins according to their respective colors, the trained model must be integrated with 
 the robotic arm during the sorting phase. Several experiments are carried out with various lighting setups and 
 object arrangements to evaluate the performance of the suggested system. The outcomes show how well the 
 system performs in terms of exact object detection and reliable sorting. The system's capacity to effectively 
 handle a variety of objects and adapt to changing environmental conditions further emphasizes its suitability for 
 use in actual industrial scenarios. This project has important ramifications for the manufacturing sector, enabling 
 improved automation capabilities and cost-efficiency. An important step towards implementing Industry 4.0 
 principles is the seamless integration of color-based object sorting and real-time object detection using a robotic 
 arm. This will allow industries to optimize their production processes, minimize human intervention, and 
 increase overall productivity. Further developments in robotics and computer vision are anticipated to push the 
 limits of automation and open the door for more advanced and intelligent industrial systems as technology 
 develops.

First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery.

The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre-operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real-time de-occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to the authors' best knowledge, the first-in-human on edge deployment of a real-time binary segmentation pipeline during three robot-assisted surgeries: partial nephrectomy, migrated endovascular stent removal, and liver metastasectomy. To this end, a state-of-the-art real-time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real-time binary segmentation of 37 non-organic surgical items, which are never occluded during AR. The application features real-time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics, and acceptance of AR in minimally invasive surgery.

Development of x-ray beam wavefront sensors for Advanced Photon Source upgrade.

Next-generation synchrotron radiation facilities, such as the Advanced Photon Source Upgrade (APS-U), bring significant advancements in scientific research capabilities, necessitating advanced diagnostic tools. Central to these diagnostics are x-ray wavefront sensors, crucial for preserving beam properties, including brightness, coherence, and stability. This paper presents two novel wavefront sensor prototypes developed at the APS using the coded-mask-based technique. The first is a compact design tailored for specific conditions and adaptability to diverse beamline configurations. The second, an adjustable zoom version, offers flexibility to accommodate a wide range of beam conditions. Both prototypes underwent rigorous testing at the APS 28-ID-B beamline and demonstrated their effectiveness in both absolute wavefront sensing and relative metrology modes. These results highlight their promise in beamline diagnostics, potentially enabling applications such as beamline auto-alignment and real-time wavefront manipulation.

A 3D-Printed Micro-Optofluidic Chamber for Fluid Characterization and Microparticle Velocity Detection.

This work proposes a multi-objective polydimethylsiloxane (PDMS) micro-optofluidic (MoF) device suitably designed and manufactured through a 3D-printed-based master-slave approach. It exploits optical detection techniques to characterize immiscible fluids or microparticles in suspension inside a compartment specifically designed at the core of the device referred to as the MoF chamber. In addition, we show our novel, fast, and cost-effective methodology, dual-slit particle signal velocimetry (DPSV), for fluids and microparticle velocity detection. Different from the standard state-of-the-art approaches, the methodology focuses on signal processing rather than image processing. This alternative has several advantages, including the ability to circumvent the requirement of complex and extensive setups and cost reduction. Additionally, its rapid processing speed allows for real-time sample manipulations in ongoing image-based analyses. For our specific design, optical signals have been detected from the micro-optics components placed in two slots designed ad hoc in the device. To show the devices' multipurpose capabilities, the device has been tested with fluids of various colors and densities and the inclusion of synthetic microparticles. Additionally, several experiments have been conducted to prove the effectiveness of the DPSV approach in estimating microparticle velocities. A digital particle image velocimetry (DPIV)-based approach has been used as a baseline against which the outcomes of our methods have been evaluated. The combination of the suitability of the micro-optical components for integration, along with the MoF chamber device and the DPSV approach, demonstrates a proof of concept towards the challenge of real-time total-on-chip analysis.

A Method for Precise Reduction of Facial Bone Fractures Using Intraoperative Ultrasound With a Solid Gel Pad as a Coupling Medium.

This study introduces a method to overcome technical challenges in using intraoperative ultrasound for the closed reduction of nasal bone and zygomatic arch fractures. The curvature of the face makes it difficult to apply an ultrasound probe to the facial area. We used a solid gel pad as a coupling medium during surgery to improve the scanning of facial bone fractures. The results show that the fracture sites observed on preoperative computed tomography scans can be easily visualized using intraoperative ultrasound, and real-time manipulation confirms successful reduction. The solid gel pad is light, malleable, easy to use, and provides accurate images. Overall, the use of ultrasound with a solid gel pad enhances the accuracy of closed reduction in facial bone fracture surgeries, confirming fracture patterns and ensuring precise reduction.

Deep learning approaches for improving robustness in real-time 3D-object positioning and manipulation in severe lighting conditions

Deep learning approaches for improving robustness in real-time 3D-object positioning and manipulation in severe lighting conditions

Using end-to-end learning and PyAutoGUI to apply gesture recognition for human-computer interaction

Contact-less human-machine interaction is becoming increasingly important due to the growing number of special environmental needs and accessibility situations. Gesture recognition has also been a hot topic in computer vision and machine learning in recent years. In this paper, a real-time computer manipulation system based on hand gesture recognition is studied and deployed. A relatively mature end-to-end target recognition model, the YOLOv5 model, is trained in this paper to achieve real-time detection and recognition of hand gestures. According to the result of the recognition, it is translated into the corresponding operation on the computer according to a set of rules, and then PyAutoGUI is used to actually control the computer. At the end of the research, the trained YOLOv5 model exhibited excellent performance and verified the feasibility and scalability of the solution. This is a good inspiration for developing a more convenient and efficient related software.

In Situ Twistronics: A New Platform Based on Superlubricity.

Twistronics, an emerging field focused on exploring the unique electrical properties induced by twist interface in graphene multilayers, has garnered significant attention in recent years. The general manipulation of twist angle depends on the assembly of van der Waals (vdW) layered materials, which has led to the discovery of unconventional superconductivity, ferroelectricity, and nonlinear optics, thereby expanding the realm of twistronics. Recently, in situ tuning of interlayer conductivity in vdW layered materials has been achieved based on scanning probe microscope. In this Perspective, the advancements in in situ twistronics are focused on by reviewing the state-of-the-art in situ manipulating technology, discussing the underlying mechanism based on the concept of structural superlubricity, and exploiting the real-time twistronic tests under scanning electron microscope (SEM). It is shown that the real-time manipulation under SEM allows for visualizing and monitoring the interface status during in situ twistronic testing. By harnessing the unique tribological properties of vdW layered materials, this novel platform not only enhances the fabrication of twistronic devices but also facilitates the fundamental understanding of interface phenomena in vdW layered materials. Moreover, this platform holds great promise for the application of twistronic-mechanical systems, providing avenues for the integration of twistronics into various mechanical frameworks.

Optimized hologram generation method for real-time spontaneous manipulation

Aided by computer generated holography, holographic optical tweezers enable manipulation of particles and objects with exceptional versatility. The responsiveness of the manipulation is often hindered by the speed of holograph generation, especially when the number of manipulated objects is high. Here, we propose an optimized hologram generation method with an improved iterative algorithm utilizing parallel computation with graphic processing units. The algorithm requires fewer iterations to produce high-quality holograms than established methods, such as weighted Gerchberg–Saxton algorithm, leading to a responsive and stable micromanipulation. This method expands the capabilities of holographic optical tweezers and provides more responsive traps in micro-manipulation.

Training Scene Construction and Motion Realization of Unmanned Craft based on Unreal Engine

Aiming at the problems of the traditional training system, such as the unrealistic simulation of 3D animation effects and the inability to meet the demand of real-time manipulation response of simulation training based on the original mechanical calculation, this paper puts forward a method of unmanned craft training scene construction and motion realization based on Unreal Engine. The method makes full use of Unreal Engine's powerful graphic rendering ability and physics engine function, and combines 3D MAX to realize the rapid construction of unmanned craft training scene, environment simulation, and unmanned craft motion simulation and control. The developed unmanned craft training software has the characteristics of smooth and clear picture, realistic and delicate scene, strong training immersion, and real manipulation experience, which can effectively improve the training efficiency and training safety, reduce the training cost, and have significant economic benefits.

Screen space shape manipulation by global structural optimization

Existing structure-aware shape manipulation frameworks have greatly facilitated the shape manipulation procedure. However, they still require users to possess the necessary 3D modeling experience since the manipulation, e.g., positioning shape handles, is performed in 3D space. For novice users without such experience, this procedure remains tedious. On the other hand, performing manipulation in screen space is more natural to novice users, especially with 2D input and display devices. This paper presents a novel interactive framework for manipulating 3D man-made shapes in screen space. Users only need to edit abstract proxies of input shapes in 2D to obtain desired deformed shapes without considering their 3D geometries. This is achieved by first analyzing the structures of the shapes and then estimating the deformed shapes with a structure-preserving optimization method. Our efficient algorithm can produce real-time shape manipulation results as previews, thus avoiding a trial-and-error editing procedure. To verify the effectiveness of our framework, we conducted a user study that contained four shape manipulation tasks. In this study, we compared our framework with a baseline framework that requires users to edit shape’s geometry in 3D space and does not provide previews for the manipulation results. The statistics and the user feedback confirm that our framework is more intuitive and efficient for shape manipulation.

An Overview of Active Control Techniques for Vortex Rope Mitigation in Hydraulic Turbines

This review addresses the current state of research into active control and suppression of vortex rope in hydroturbines under off-design operating conditions. Only active control methods that can be “switched on” when required under off-design operating conditions are considered in this work. The review focuses on air addition into the flow, as well as various auxiliary fluid jets. It includes all the best practices for vortex rope suppression in numerical and experimental studies. It can be inferred from the review that a modern flow control system should be comprehensive, designed for a specific hydroturbine geometry, and obtain feedback from the flow. Injecting ~2% of air from the impeller fairing cone appears optimal for suppressing pressure pulsations without significant efficiency loss. The cost of air injection is rarely estimated, but the use of an automatic venting system can minimize overheads and potentially improve efficiencies at low gas contents. Fluid jets ranging from 3% to 12% of the main flow rate can efficiently suppress pressure pulsations, but their high energy requirements limit their use. Azimuthal perturbation of the flow appears promising as it does not require significant energy loss, but practical implementation remains challenging as one needs to accurately know the system dynamics and be capable of real-time manipulation of the flow.

Eye accommodation-inspired neuro-metasurface focusing

The human eye, which relies on a flexible and controllable lens to focus light onto the retina, has inspired many scientific researchers to understand better and imitate the biological vision system. However, real-time environmental adaptability presents an enormous challenge for artificial eye-like focusing systems. Inspired by the mechanism of eye accommodation, we propose a supervised-evolving learning algorithm and design a neuro-metasurface focusing system. Driven by on-site learning, the system exhibits a rapid response to ever-changing incident waves and surrounding environments without any human intervention. Adaptive focusing is achieved in several scenarios with multiple incident wave sources and scattering obstacles. Our work demonstrates the unprecedented potential for real-time, fast, and complex electromagnetic (EM) wave manipulation for various purposes, such as achromatic, beam shaping, 6 G communication, and intelligent imaging.

COVID-19 Vaccine Tracking Geospatial Application using GIS and GPS

One of the problems in addressing the COVID -19 epidemic is that the coverage of COVID -19 vaccination data in each area cannot be immediately displayed. In addition, the COVID-19 vaccination data presented do not include spatial data, which means that the data cannot be used for decision making to address the problem. Although COVID -19 vaccination data are presented through digital platforms. However, a current limitation is that the platforms used to manipulate COVID-19 vaccination data have not supported real-time data manipulation, data processing, or spatial data visualization. As a result, the distribution of vaccinated individuals in each area cannot be tracked. Therefore, this research developed a geospatial data application for real-time data processing and visualization of COVID -19 vaccination using a Geographic Information System (GIS) and Global Positioning System (GPS) for data processing to plan and support decision making by people and officials so that COVID -19 operations can be conducted efficiently and data collection time can be reduced. The software used to develop this platform includes PHP, MySQL, Google Maps API and Leaflet. The results show that the developed platform can be used to monitor and track vaccination data in real time COVID-19.

Electronically reconfigurable transmitarray for fully independent beam manipulation in two divided frequency bands.

Electronically reconfigurable transmitarray (ERTA) combines the advantages of optic theory and coding metasurface mechanism with the characteristic of low-loss spatial feed and real-time beam manipulation. Designing a dual-band ERTA is challenging due to multiple factors, including large mutual coupling generated by dual-band operation and separate phase control in each band. In this paper, a dual-band ERTA is demonstrated with the capability of fully independent beam manipulation in two divided bands. This dual-band ERTA is constructed by two kinds of orthogonally polarized reconfigurable elements which share the aperture in an interleaved way. The low coupling is achieved by utilizing polarization isolation and a backed cavity connected to the ground. To separately control the 1-bit phase in each band, a hierarchical bias method is elaborately presented. As proof of concept, a dual-band ERTA prototype composed of 15 × 15 upper-band elements and 16 × 16 lower-band elements is designed, fabricated, and measured. Experimental results verify that fully independent beam manipulation with orthogonal polarization is implemented in 8.2-8.8 GHz and 11.1-11.4 GHz. The proposed dual-band ERTA may be a suitable candidate for space-based synthetic aperture radar imaging.

Efficient High-Refractive-Index Azobenzene Dendrimers Based on a Hierarchical Supramolecular Approach.

Real-time manipulation of light in a diffractive optical element made with an azomaterial, through the light-induced reconfiguration of its surface based on mass transport, is an ambitious goal that may enable new applications and technologies. The speed and the control over photopatterning/reconfiguration of such devices are critically dependent on the photoresponsiveness of the material to the structuring light pattern and on the required extent of mass transport. In this regard, the higher the refractive index (RI) of the optical medium, the lower the total thickness and inscription time can be. In this work, we explore a flexible design of photopatternable azomaterials based on hierarchically ordered supramolecular interactions, used to construct dendrimer-like structures by mixing specially designed sulfur-rich, high-refractive-index photoactive and photopassive components in solution. We demonstrate that thioglycolic-type carboxylic acid groups can be selectively used as part of a supramolecular synthon based on hydrogen bonding or readily converted to carboxylate and participate in a Zn(II)-carboxylate interaction to modify the structure of the material and fine-tune the quality and efficiency of photoinduced mass transport. Compared with a conventional azopolymer, we demonstrate that it is possible to fabricate high-quality, thinner flat diffractive optical elements to reach the desired diffraction efficiency by increasing the RI of the material, achieved by maximizing the content of high molar refraction groups in the chemical structure of the monomers.

Auxiliary Circuit Free Maximum Power Efficiency Tracking Scheme for Wireless Motor System With Source-Load Coupling

This paper proposes a real-time maximum power efficiency tracking (MPET) scheme for a series-parallel compensated wireless motor (SP-WM) system, where the system efficiency is maximized based on real-time manipulation of the dc-link voltage. The transient performance of the motor and stability of the dc-link voltage can be improved by series-parallel compensation with higher current overload capacity compared to the series-series one. Compared with the conventional resistive load, the motor drive can be considered as a constant power load (CPL), serving as an adjustable equivalent load resistance. The efficiency curve and resistance adjusting capability of the SP-WM system are investigated under different conditions, which demonstrates the source-load coupling between the motor drive and magnetic coupler. Besides, a dc-link voltage determination (DVBD) method is proposed to ensure the safe operation of the system. The perturbation and observation (P&O) and DVBD are integrated and implemented iteratively in the proposed MPET scheme. The maximum system efficiency can be achieved without an auxiliary circuit under different mutual inductance conditions, which reduces the cost and volume of the system. Finally, the feasibility and effectiveness of the proposed MPET scheme are verified by the experiments. The system efficiency can be increased by up to 4.51%.