Remote Operation Research Articles

Research Progress on Pole-Climbing Robots: A Review

Background: Pole-climbing robots are a type of application robot commonly used for quality inspection, cleaning, and maintenance of utility poles, high voltage lines, bridge cables, transmission pipelines, and other important facilities with a certain height. The working conditions are often harsh and dangerous, making manual participation unsafe. The pole-climbing robot can provide a platform for maintenance and cleaning work. Remote control operation of the pole-climbing robot is achieved through communication and control technology. It has been created to improve efficiency and reduce personnel accidents greatly. At the same time, the continuous development of science and technology has led to the expansion of robot design concepts. It has also brought about great changes and growth in the development of pole-climbing robots. Objective: The purpose of this paper is to report the latest progress in the research of pole-climbing robots from bionic and non-bionic perspectives and to provide a research reference for researchers in this field. Method: By analyzing academic theses and published patents, this paper presents a new classification of pole-climbing robots from the perspective of bionic and non-bionic. By summarizing the literature, the structural characteristics of various pole-climbing robots and their differences and applications are summarized. Results: The performance of the pole-climbing robots is analyzed from the viewpoint of structural characteristics and action execution methods. In turn, the characteristics of various types of pole-climbing robots are summarized. Finally, based on the above discussion, the future problems and development directions of pole-climbing robots are predicted. Conclusion: The pole-climbing robots can be divided into two categories, bionic and non-bionic, based on the analysis of design features. Both bionic and non-bionic pole-climbing robots have good development and applications in their respective directions. Also, bionic and non-bionic pole-climbing robots have different detailed classifications based on material, structure, and other perspectives. They have different advantages and disadvantages in terms of the performance of pole-climbing action execution. It provides a research reference for future researchers in this field.

A 4D Soil-Structure Interaction Model Testing Apparatus

Abstract A new three-axis loading frame has been developed to enable real-time visualization of in-situ soil and rock structure interactions via X-ray tomography during small-scale model testing. The constructed frame is capable of performing a wide range of small-scale 1g tests and can accommodate monotonic and cyclic actuation under both load and displacement control. The compact size of the system enables remote multi-axis operation from within an X-ray cone-beam scanning bay, a capability which is owed to a comprehensive design process. Design and fabrication involved a blend of physical and numerical experiments to assess suitable construction materials and performance. In this scope, the new equipment is discussed and its capability is showcased.

Fault Detection in Transmission Line

Abstract: This project focuses on the design and implementation of an advanced fault detection system for overhead transmission lines, utilizing key components such as the ESP 8266 microcontroller, current sensors, a GPS module, and IoT integration. The system is capable of detecting and classifying four major fault types: single line to ground (L-G), line to line (LL), double line to ground (L-L-G), and three-phase faults (L-L-LG). By placing current sensors at the transmission line's sending end, the system identifies faults and accurately determines their location through GPS coordinates .Simulations using protect us software were conducted to verify the system’s performance prior to building a hardware prototype. Upon fault detection, the system provides alerts via an LCD display and a buzzer, while real-time fault data is transmitted to an Android app and web server through the arduino IoT cloud. Testing confirmed the system’s precision in fault identification and location tracking, ensuring timely notifications for control room and remote device operators. This successful implementation highlights the system's potential for improving safety and efficiency in power transmission networks. the project lays the groundwork for further exploration into advanced fault detection and real-time monitoring solutions in electrical engineering.

Innovative Application of Optoelectronic Control Engineering for Safety Assurance of Surgical Robots

As an emerging technology, optoelectronic control engineering has demonstrated a wide range of application potential in surgical robot safety and security. Through a literature review, this paper provides an in-depth discussion on the innovative applications of optoelectronics technology in enhancing the positioning accuracy, environment sensing, supple control and remote operation of surgical robots. At the same time, this paper summarizes the current research hotspots and future research directions, with a view to providing a reference for promoting the technological progress in this field. The current research focuses on improving the operating precision of surgical robots, enhancing the environment sensing ability and the application of force feedback system, while the future research will mainly focus on interdisciplinary cooperation and new technology development.

Feasibility of novel robot-assisted, remotely-performed echocardiographic examination

Abstract Background Access to echocardiography services outside metropolitan areas is persistently limited. We sought to determine the feasibility and efficiency of performing remotely-controlled, semi-automated robot-assisted echocardiographic examinations (robot-assisted echo) by cardiac sonographers as a potential avenue to provide standard echocardiographic specialist services to remote and rural areas as well as permit expedited diagnoses enabling earlier management of disease. Methods Two accredited cardiac sonographers (Cardiac Investigations Unit) undertook 6 hours virtual training, and 10 hours hands-on robot-assisted echo simulation training; using a novel, semi-automated robotic arm system to remotely manipulate an ultrasound probe to acquire echocardiographic images. Sonographers then performed remotely-performed robot-assisted echo examinations on staff volunteers, and consenting inpatients in whom echocardiography was clinically indicated. Cameras, probe pressure sensors and an ultrasound platform remote console were utilised to support remote operation. Traditional echocardiography (manipulation of probe by hand) was also performed as a reference. Feasibility to obtain a comprehensive robot-assisted echo was compared to traditional echocardiography. Sonographer learning curve and efficiency (image acquisition time and offline measurement time) was also investigated. Results Total cohort was n=78 (age 51±15yrs; 57% male) with 63% patients. All participants had robot-assisted echo and traditional echocardiograms performed within 72hrs. Average feasibility (%) of robot-assisted echo as (a) 11 averaged assessments and (b) 38 averaged measures was 92±9 (a) and 86±11 (b), compared to (a) 99±1 and (b) 97±7 by traditional examination (all respectively). Sonographer learning curve was identified via 1:1 block subcohort analysis, with averaged assessment feasibility (n=33:31) being 86±19 (first block) to 98±3 (last block) and averaged measurement feasibility (n=32:31) being 77±11 (first block) to 92±7 (last block), all respectively. In the staff participant cohort (n=31), average image acquisition efficiency in robot-assisted echo was poorer than that of traditional examination (47+9 minutes compared to 19+2 minutes; p<0.0001), respectively. Average efficiency of offline measurement time was 13+7 minutes by robot-assisted echo; comparable (p=0.33) to traditional examination at 12+6 minutes. There was no learning curve effect on image acquisition efficiency; 42+10 vs 51+7 minutes (1:1; n=16:15; p=0.007). Conclusion Remotely-performed, robot-assisted echo using a commercially-available ultrasound platform performed by briefly-trained, cardiac sonographers is feasible, with reasonable efficiency even though less efficient in image acquisition compared to traditional echocardiograms. Further investigation of performance as a remote service is warranted in larger, clinically-indicated populations.

Simultaneous Remote Monitoring of Multiple Automated Vehicles

As vehicle automation continues to become a more prevalent part of everyday life, remote operators may be able to contribute to making this technology safer and more efficient. Such operators could be expected to monitor vehicles until it is necessary for them to intervene. In recent years several countries have passed legislation mandating the implementation of remote operation, making it a plausible scenario. For many practical reasons, operators may monitor more than one vehicle. This study provides an initial look into how remote operators might perform when asked to attend to multiple concurrent displays using simulated automated vehicles. The initial results indicate that participants retained a comparable level of accuracy in detecting failures in automation as the number of vehicles they monitored rose from one to two, albeit with a higher workload. The results and the methodology provide insights into potential task designs for remote operators of automated vehicles.

Stable Droplet Manipulation in Multiple Directions on a Magnetically Responsive Micropillar Array Surface

AbstractDroplets constitute an essential discrete form of liquids and are prevalent in both natural and industrial fields. Droplet manipulation technologies have broad application prospects in various fields, including biomedical devices, chemical engineering, water collection, and so on. The manipulation of surfaces with magnetic responsiveness has emerged as a pivotal technique in droplet control owing to its flexibility and precision in remote operations. In this study, the dynamic rolling behavior of low‐surface‐tension working fluids and water droplets on a superhydrophobic magnetically responsive micropillar array (SMMA) is investigated, with a focus on the mechanisms that generate instabilities during the directional rolling motion of droplets. The factors influencing these instabilities are studied using data analysis. Moreover, techniques for capturing and releasing droplets are investigated, offering significant insights into the efficient handling of minute amounts of liquids in experiments and the automation of intricate chemical processes.

A Comprehensive Review of State-of-the-art Optical Methods for Methane Gas Detection

Methane (CH<sub>₄</sub>), a potent greenhouse gas, significantly contributes to climate change and global warming. Its impact over 100 years surpasses carbon dioxide (CO<sub>₂</sub>) by 28 times. Addressing methane emissions, particularly from oil and gas production activities such as transmission pipelines, is imperative. One promising avenue is the development of reliable sensors to detect and mitigate methane leaks and prevent hazardous issues. Optical-based methods present notable advantages, including versatility and remote operation, making them pivotal in this endeavor. This review article provides a concise overview of optical-based methane identification technologies, encompassing sensing materials, absorption spectra, operational mechanisms, and recent advancements. Potential perspectives are explored, and inferences from this assessment are also derived. Emphasizing the significance of optical fiber-based methane detection methods, the authors advocate for further research to support ongoing efforts and foster innovation in this critical area.

A Bio-Inspired Magnetic Soft Robotic Fish for Efficient Solar-Energy Driven Water Purification.

Solar-driven water evaporation is a promising solution for global water scarcity but is still facing challenges due to its substantial energy requirements. Here, a magnetic soft robotic bionic fish is developed by combining magnetic nanoparticles (Fe3O4), poly(N-isopropylacrylamide), and carboxymethyl chitosan. This bionic fish can release liquid water through hydrophilic/hydrophobic phase transition and dramatically reduce energy consumption. The introduced Fe3O4 nanoparticles endow the bionic fish with magnetic actuation capability, allowing for remote operation and recovery. Additionally, the magnetic actuation process accelerates the water absorption rate of the bionic fish as confirmed by the finite element simulations. The results demonstrate that bionic fish can effectively remove not only organic molecular dyes dissolved in water but also harmful microbes and insoluble microparticles from natural lakes. Moreover, the bionic fish maintains a good purification efficiency even after five recycling cycles. Furthermore, the bionic fish possesses other functions, such as salt purification and salt rejection. Finally, the mechanism of water purification is explained in conjunction with molecular dynamics calculations. This work provides a new approach for efficient solar-energy water purification by phase transition behavior in soft robotics.

A multi-level rescheduling approach for a dynamic remote operations scheduling problem

ABSTRACT In this paper, we tackle a dynamic scheduling problem faced by a large international company. The problem involves assigning installation projects arriving over time to specialised technicians who execute them remotely. Each project consists of several tasks having processing times, release dates, and execution deadlines. The company needs to assign projects to technicians and schedule tasks complying with technicians' skills, precedence constraints between tasks, and tasks requiring multiple technicians simultaneously. The problem is dynamic as new projects and tasks become available over time, requiring their allocation to technicians. We formulate the offline problem as a mixed integer linear program that minimises the makespan, and we address the dynamic version solving restricted problems within a rolling horizon framework. The approach systematically implements different levels of schedule adjustment to incorporate new information. To study scalability, we validate our algorithm by using both real-world and synthetic simulations demonstrating its efficiency and effectiveness. Additionally, we provide interesting managerial insights for the company.

Quantifying marine growth on pipelines with archival remotely operated vehicle footage – a case study

Abstract. Marine growth can substantially affect the hydrodynamic properties of subsea infrastructure. The ability to accurately quantify this growth is essential for designing and maintaining subsea structures. Traditionally, quantifying marine growth has been done by applying simple ratio equations putting in relationship the known distance of an object and its length observed in the image to the length of an unknown object only observed in an image. This method is very sensitive if measurements are not taken in the same plane. Hence, this study aims to establish a method for creating a 3D model of a subsea pipeline segment based on Structure from Motion (SfM). The proof of concept is done using a case study. For this study, Remotely Operated Vehicle (ROV) footage from a 2019 inspection survey is utilised. The scenario in which the procedure is tested represents a difficult, but realistic test case. Despite the lack of metadata, video quality and obstructions (e.g. watermarks), meaningful results were produced. The SfM workflow included masking obstructions (e.g. watermarks), and an iterative camera alignment procedure to generate 3D models, enabling the measurement of marine growth using either the models or ortho images created. While the resulting models were sufficiently accurate to provide meaningful measurements for harder growth types, the method faced challenges in accurately modelling softer growth types, suggesting the need for further refinement. Nevertheless, SfM was proven to be a more reliable method of measuring biofouling than the ratio approach that been traditionally used.

Improving Underwater Photogrammetric 3D Reconstruction Processing of Shipwreck Sites

Abstract. Applying image processing algorithms to enhance the clarity of underwater images can significantly assist in the visual interpretation of subsea environments. Taking advantage of techniques to maximise the information extracted from the captured input data to produce the highest quality output ensures a higher utilisation of the valuable recorded content which may have been collected at the high cost of a laborious diver survey or an expensive ROV (Remotely Operated Vehicle) expedition. Adopting photogrammetric 3D reconstruction processing in underwater imaging has enabled the creation of high-fidelity digital 3D models which have been used to map subsea infrastructure for maintenance and repair, coral reefs for ecological monitoring, and in the primary context of this article, shipwrecks sites for maritime archaeology and accident investigation. Alongside exploring algorithms to reduce the characteristic haze of underwater photography, this article illustrates the advantages of optimising the processes utilised at distinct stages of the photogrammetric 3D reconstruction workflow to improve the photorealism of the digital 3D models.

In situ monitoring of defects in steel structures using a robot‐assisted ultrasonic inspection technique

AbstractIncidents in extreme environments can bring unprecedented consequences that would endanger the lives of many humans. Real‐time and early detection of cracks and defects in the steel used for the construction of industrial sites can bring many benefits to remote operators and site managers. Although different methods and systems have been proposed in the past, they are not suitable for real‐time fault detection or long‐term operation in harsh environments. Therefore, this paper proposes a low‐power robotic platform with a magnetic actuator mechanism which is capable of wireless inspection of steel structures using the ultrasonic method. The proposed magnetic motion uses the inertia force generated by the piston vibration and the difference in frictional force during movement. The paper evaluates the performance of the steel structure defect detection system from four different perspectives: sensing principle, wireless communication network, driving part, and power supply.

Combined Method Comprising Low Burden Physiological Measurements with Dry Electrodes and Machine Learning for Classification of Visually Induced Motion Sickness in Remote-Controlled Excavator

The construction industry is actively developing remote-controlled excavators to address labor shortages and improve work safety. However, visually induced motion sickness (VIMS) remains a concern in the remote operation of construction machinery. To predict the occurrence and severity of VIMS, we developed a prototype system that acquires multiple physiological signals with different mechanisms under a low burden and detects VIMS from the collected data. Signals during VIMS were recorded from nine healthy adult males operating excavator simulators equipped with multiple displays and a head-mounted display. Light gradient-boosting machine-based VIMS detection binary classification models were constructed using approximately 30,000 s of time-series data, comprising 23 features derived from the physiological signals. These models were validated using leave-one-out cross-validation on seven participants who experienced severe VIMS and evaluated through area under the curve (AUC) scores. The mean receiver operating characteristic curve AUC score was 0.84, and the mean precision–recall curve AUC score was 0.71. All features were incorporated into the models, with saccade frequency and skin conductance response identified as particularly important. These trends aligned with subjective assessments of VIMS severity. This study contributes to advancing the use of remote-controlled machinery by addressing a critical challenge to operator performance and safety.

Synthesis-in-place of V2O5 nanobelts for wide range humidity detection

Resistive type humidity sensors offer advantages in simplicity, low cost, compact size, and remote operation. Due to the benefits, they have great potential for use in a variety of environments and future applications, including the Internet of Everything (IoE). However, resistive humidity sensors have limitations in detecting extreme humidity levels below 10 % or above 90 % relative humidity (RH) and are also vulnerable to high temperatures, which hinders their use in harsh environments applications. For broader applications, humidity sensors with reliable performance across the full range of humidity levels and high stability at elevated temperatures need to be developed. Here, we report high performance resistive humidity sensors based on V2O5 nanobelts operating at high temperatures. The V2O5 nanobelts are directly deposited between Pt electrodes by O2-assisted physical vapor transport (PVT) method. The V2O5 nanobelt humidity sensor exhibits reliable humidity sensing properties over a wide range from 1 % to 100 % RH. Furthermore, long-term stability and reproducibility of V2O5 nanobelts are demonstrated through 25 consecutive humid air pulses and measurements obtained after 4 months of storage. The high performance of the V2O5 nanobelt humidity sensor is advantageous for practical use in applications ranging from IoE environments to harsh industrial conditions.

Boredom, vigilance, and repetition during remote control room work for autonomous ships

Abstract Introduction: Remote Operating Centers (ROCs) enable human oversight of remotely operated and autonomous ships. However, the “human-in-the-loop” model of automated control oversight implies prolonged periods of inactivity punctuated by safety-critical action. This paper investigates the extent to which this role is aligned with humans’ innate attentional resources in terms of boredom, vigilance, and repetition. Method: A simulator study was completed with ROC operators (volunteer participants with relevant expertise). One group completed a short scenario (5 minutes; N = 16); the other completed a long scenario (30 minutes; N = 16). The duration marked the onset of a stimulus (an alarm), prompting manual takeover. In a mixed method approach, four sources of data were collected, consisting of pupillometry, questionnaire, interview, and video data. Results: Pupillometry data showed larger pupillary constrictions in the 30-minute group, indicative of decreased cognitive activity (effect size 0.25 mm). The questionnaire showed that operators in the 30-minute scenario were more likely to experience their tasks as unchallenging, understimulating, and lacking in meaning. Finally, a relationship was found between repetitive content and varying perceptions of time duration. Moreover, operators perceived scenarios as longer when they detected an “oddball” in the repetitive series (a gorilla passenger). Conclusion: ROC work as currently imagined is prone to boredom, vigilance decrement, and suppression of repetition. The study adds to the empirical research about boredom and its associated cognitive states in control room work. The study also contributes to design considerations impacting safety and organizational aspects like operator training and retention.

Wavelet-based spatiotemporal sparse quaternion dictionary learning for reconstruction of multi-channel vibration data

Accurate reconstruction of multi-source data information plays an essential role in remote intelligent operation, and prognostic and health maintenance (RIO-PHM) of industrial systems. However, traditional signal reconstruction methods such as compressed sensing fail to capture the spatio-temporal characteristics and coupling nature of the multi-source or multi-channel data. To alleviate this bottleneck, in this paper, a new wavelet-based spatiotemporal sparse quaternion dictionary learning (WSTS-QDL) algorithm is formulated for the reconstruction of multi-channel vibration data for the first time. Specifically, the wavelet-based spatiotemporal sparse low-rank matrix (SLRM) algorithm is elaborated and the sparse coefficient matrix of the multi-channel signals can be estimated using the split Bregman iteration (SBI) technique. Then, quaternion-based dictionary learning is introduced for multi-channel data reconstruction according to the updated sparse coefficient matrix during quaternion dictionary learning. Eventually, two equipmental scenarios including run-to-failure data of rolling bearing in bearing test bench and vibration signal of the failured bearing in corn thresher, are utilized for verification. The experimental results demonstrate that the highest recovery accuracy performance with scoring function and the lowest errors are achieved by the proposed method in contrast with the state-of-the-art benchmarks such as orthogonal matching pursuit (OMP) method and spatiotemporal sparse Bayesian learning (SSBL), and the waveform of phase space trajectory and points cloud distribution of the Poincare section in the original signal are similar/same to that obtained by the proposed method.

Enhancing Situational Awareness for Remote Control of Ship Cranes

Abstract Cargo handling in ship operations poses risks due to heavy loads, necessitating innovation to mitigate accidents. Automation and remote control offers safer and more efficient operations by moving people from hazardous situations. Smaller and remote ports often rely on geared ships, which are typically manually operated. A first step towards automation is to remotely control these cranes. In remote control, the operator needs to gain situational awareness through sensor inputs. Previous research has shown that there is a need to investigate the sensor configuration and placement to provide situational awareness, particularly considering the placement of cargo onboard. This study seeks to identify needed sensor configurations for situational awareness in remote control of ship cranes using a simulator framework. A modular crane simulator framework has been developed to evaluate situational awareness for different sensor configurations. The sensor configuration has been evaluated with feedback from a crane expert. This research has identified potential sensor configurations for enhancing situational awareness in remote crane operations. The developed simulator framework has demonstrated its flexibility and efficiency as a platform for testing and optimizing cargo handling operations.

Quantum yield and thermal sensitivity of SrGd2O4:Yb, Tm up-conversion nanoparticles

Luminescence thermometry is an effective method for temperature measurements using the remote operating mode. In this work, a series of Tm3+ (1 at%) and Yb3+ (2, 4, and 6 at%) ions doped SrGd2O4 up-conversion (UC) nanoparticles were synthesized by the modified sol-gel method, and fully characterized toward optimization of their composition and optical characteristic. Crystallization of single-phase nanoparticles with homogeneous doping was confirmed by XRPD, TEM/HRTEM, and STEM/EDS analysis in all samples, independently of their stoichiometry. The UC emissions recorded under 976 nm excitation revealed typical Tm3+ transitions, showing that the optimal sensitizer doping concentration is 4 at%. For this sample, a lifetime of 342 μs, and a total quantum yield of 1.12 % were determined at room temperature for the emission observed in the Vis part of up-conversion spectra. The temperature-dependent emission spectra, measured from 263 to 363 K, implied its excellent temperature-sensing capabilities. The luminescent intensity ratio (LIR) was employed for blue I479/485 and IR I795/807 Tm3+ emissions. The absolute and relative temperature sensitivity continuously decreases with the increase of the temperature, showing the values of 4.05x10−3 K-1 and 0.41 % K−1 (for blue) and 3.8x10−3 K-1 and 0.31 % K−1 (for IR) at the temperature of 300 K. The results indicate that SrGd2O4 doped with 4 % Yb3+ and 1 % Tm3+ may be employed for reliable temperature measurement over an investigated temperature range.

Convolutional Long Short-Term Memory Predictor for Collaborative Remotely Operated Vehicle Trajectory Tracking in a Leader–Follower Formation Subject to Communication and Sensor Latency in the Presence of External Disturbances

Nowadays, collaborative operations between Remotely Operated Vehicles (ROVs) face considerable challenges, particularly in leader–follower schemes. The underwater environment imposes limitations on acoustic modems, leading to reduced transmission speeds and increased latency in ROV position and speed transmission. This complicates effective communication between the ROVs. Traditional methods, such as Recursive Least Squares (RLS) predictors and the Kalman filter, have been employed to address these issues. However, these approaches have limitations in handling non-linear patterns and disturbances in underwater environments. This paper introduces a Convolutional Long Short-Term Memory (ConvLSTM) predictor designed to enhance communication and trajectory tracking between ROVs in a leader–follower scheme. The proposed ConvLSTM aims to address the shortcomings of previous methods by adapting effectively to varying conditions, including ocean currents, communication delays, and signal interruptions. Simulations were conducted to evaluate ConvLSTM’s performance and compare it with other advanced predictors, such as Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM), under different conditions. The results demonstrated that ConvLSTM achieved a 13.9% improvement in trajectory tracking, surpassing other predictors in scenarios that replicate real underwater conditions and multi-vehicle communication. These results highlight ConvLSTM’s potential to significantly enhance the performance and stability of collaborative ROV operations in dynamic underwater environments.