Remotely Operated Vehicle Research Articles

Deepdive: Leveraging Pre-trained Deep Learning for Deep-Sea ROV Biota Identification in the Great Barrier Reef

Understanding and preserving the deep sea ecosystems is paramount for marine conservation efforts. Automated object (deep-sea biota) classification can enable the creation of detailed habitat maps that not only aid in biodiversity assessments but also provide essential data to evaluate ecosystem health and resilience. Having a significant source of labelled data helps prevent overfitting and enables training deep learning models with numerous parameters. In this paper, we contribute to the establishment of a significant deep-sea remotely operated vehicle (ROV) image classification dataset with 3994 images featuring deep-sea biota belonging to 33 classes. We manually label the images through rigorous quality control with human-in-the-loop image labelling. Leveraging data from ROV equipped with advanced imaging systems, our study provides results using novel deep-learning models for image classification. We use deep learning models including ResNet, DenseNet, Inception, and Inception-ResNet to benchmark the dataset that features class imbalance with many classes. Our results show that the Inception-ResNet model provides a mean classification accuracy of 65%, with AUC scores exceeding 0.8 for each class.

Design and Implementation of an Interactive Embedded System as a Low-Cost Remotely Operated Vehicle for Underwater Applications

The underwater environment is harsh and challenging for human life, prompting companies and researchers to develop advanced technologies for exploration. Building on previous work that applied a CNN-based method for underwater object classification, this paper focuses on the design and implementation of an interactive embedded system for a compact Remotely Operated Vehicle (ROV) with specific dimensions and weight. The primary goal is to capture real-time underwater video using remote control communication via Ethernet. The ROV is powered by five brushless motors controlled by a smart PID controller. Precise pulse-width modulation signals enhance stability during movements along three axes, enabling high-resolution video capture. The system utilizes the Raspberry Pi 3's computing power for motion control, positioning, temperature monitoring, and video acquisition. Experimental results demonstrate the system's capability to process 42 frames per second. A user-friendly graphical interface allows for remote ROV control across various operating systems. With a depth rating of 100 meters and speed of 0.148 m/s. This ROV surpasses human divers' limitations and holds significant potential for applications in surveillance, operations, maintenance, and measurement tasks underwater. The proposed ROV contributes to the advancement of underwater exploration technology by combining high performance with cost-effectiveness.

First geological survey and characterisation of a giant depression in carbonate strata at the Rio Grande Rise (southwestern Atlantic)

Pockmarks are submarine depressions often considered the result of fluid (gas and oil) expulsion from the seafloor. Recent geophysical surveys at the Rio Grande Rise (RGR - SW Atlantic) revealed extensive presence of seafloor depressions, generically interpreted as pockmarks. To date, direct observations or sampling from these depressions are lacking. Moreover, hydrocarbons have never been detected at the site, questioning the possibility of pockmarks formation by fluid expulsion. Here we present the first visual inspection and rock characterisation of a giant circular depression located inside the RGR central graben. The depression was discovered during multibeam prospection and surveyed with a Remotely Operated Vehicle, with rock sampling along a vertical transect. The depression is conical (350 m wide and 80 m deep) with walls of stratified, sub-horizontal, white fossiliferous wackestone, with incipient gravitational collapse. Lithological, mineralogical, and geochemical characterisation of the wackestone rule out methane in the formation of the depression. Based on geochemical data and on the finding of anhydrite III (forming at 110–300 °C), we propose a novel perspective on the formation of this kind of depressions, invoking hypogenic karstification and collapse, due to hydrothermal fluids along deep faults. Our geochemical approach complements previous geophysical observations of circular depressions at the RGR and contribute to the general understanding of the formation mechanisms of these intriguing seafloor features.

Evaluation of hydrodynamic coefficients and sensitivity analysis of a work-class remotely operated vehicle using planar motion mechanism tests

This paper presents a comprehensive analysis of the hydrodynamic forces and moments relating to the drag on an open-frame work-class remotely operated vehicle (ROV) AUTO-1000. We describe the results of planar motion mechanism experiments conducted with the aim of modeling and simplifying the mathematical maneuverability model for such an ROV. The scale of the model is 1:4. Various experiments are performed in a nonlinear wave channel. The flow speeds in the static drag tests range from ±0.2–±0.5 m/s and the angle in the drift tests is less than 10°. The motion frequencies of the dynamic tests range from 0.25 to 3.14 rad/s. The amplitude of the translation motions is 0.03 m and the largest angular amplitude in the rolling tests is 5°. The viscous and inertial hydrodynamic coefficients are estimated. Normalized sensitivity coefficients are used to investigate the sensitivity of both the viscous and inertial hydrodynamic coefficients considering the multi-degree-of-freedom motion and velocity effect. The drag, drift, and stationary random motions are used to examine the sensitivity. A comparison of the motion simulation results given by simplified and complete models shows that a threshold normalized sensitivity coefficient value of 0.01 is suitable for filtering the ROV coefficients.

Toward a revision of the bamboo corals: Part 6, Illuminating a new candelabrum genus (Octocorallia: Keratoisididae)

Observations and collections made using remotely operated vehicles (ROV) outfit with high-definition video cameras on bathyal seamounts of the North Atlantic and North Pacific have revealed a bamboo coral (Octocorallia, Keratoisididae) with consistent and recognizable colony morphology: a long unbranched “stem” from which many vertically aligned branches arise in a single plane to produce the aspect of a tall candelabrum. Additional observations encountered colonies with only 3 branches arising from the central terminal node to produce the appearance of a standing trident. Genetic analyses suggested both colony morphologies (trident and candelabrum) to be the same species at different growth stages. Herein we formally describe this taxon, Tridentisis candelabrum n. gen. n. sp., erecting a new genus to accommodate the unique and distinctive colony morphology, and discuss morphological variation observed across the documented geographic range.

Enhancement of underwater dam crack images using multi-feature fusion

Underwater dam-crack images captured by remotely operated vehicles (ROVs) often exhibit blurred and unclear features because of the absorption and scattering of water and device shaking. To address these challenges, we propose an underwater crack image-enhancement network (UCE-CycleGAN) that outperforms existing methods on unpaired crack datasets. This network employs multi-feature fusion, incorporating edge maps and dark channels to mitigate blurring and enhance texture details. The blurriness metric and number of matched feature points in the enhanced images improved by 4.12 and 17.11 times, respectively. The mean average precision for detection and segmentation increased by 58.5 % and 39.0 %, respectively, surpassing those of other underwater image-enhancement methods. Furthermore, by integrating YOLOv8 feature recognition and a semantic segmentation network, we developed a real-time automated system (UCADS) for underwater dam-crack detection, thereby expanding the applications of enhanced underwater crack images. The effectiveness of UCADS was validated through field experiments at reservoir dams.

Enhanced trajectory tracking control algorithm for ROVs considering actuator saturation, external disturbances, and model parameter uncertainties

In this study, an augmented model-based predictive control (AMPC) algorithm was designed for a work-class remotely operated vehicles (ROV) with actuator saturation constraints, model parameter uncertainty, and external disturbances. Firstly, model parameter uncertainties and external disturbances (primarily umbilical cable forces) were considered in the kinematic model of the ROV. The ROV's actuator saturation constraints were introduced when designing the optimal control problem (OCP). Secondly, Bellman's principle of optimality was applied to the solution process of the OCP, decomposing it into multiple easily solvable sub-optimization problems to reduce computational complexity. Then, the recursive feasibility and input-to-state practical stability (ISpS) of the AMPC algorithm were proven in the presence of model parameter uncertainties and external disturbance forces. It was also demonstrated that the state convergence region of the system employing the AMPC algorithm was smaller than that of the system using the min-max MPC algorithm when reaching a steady state. Finally, numerical simulations verified that under the influence of external disturbance forces, model parameter uncertainties, and positioning system noise, the ROV using the AMPC algorithm achieved higher accuracy in tracking the desired trajectory compared to the min-max MPC algorithm.

Gaidropsarus mauritanicus (Gadiformes, Gaidropsaridae) a new three-bearded rockling from a deep-water coral ecosystem with a genetically verified biogeographical distribution of the genus and notes to its ecology and behavior.

Gaidropsarus mauritanicus sp. nov. is described from one specimen collected using a grab sample from the Tanoûdêrt Canyon (ca. 20° N) off Mauritania at a depth of 595 m. The new species was further observed during eight remotely operated vehicle (ROV) dives along the Mauritanian slope southwards down to the Tiguent Coral Mound Complex (~17° N) in a bathymetric range between 613 and 416 m. It can be distinguished from congeners by a combination of characteristics, including large eyes (38.1% head length [HL]), large head (25.8% standard length [SL]), elongated pelvic fins (35.7% SL), low number of vertebrae (44), and coloration (pinkish with a dorsal darker brownish hue and bright blotches along the dorsal-fin base). A species-delimitation analysis performed with available cytochrome c oxidase subunit 1 (COI) sequences affiliated to the genus Gaidropsarus additionally supported the validity of the new species. Video analyses showed a deep-water coral-associated and protection-seeking behavior, which may explain why the species has remained undescribed until now. Additional ROV footage from separate deep-water coral sites in the North Atlantic and Mediterranean Sea further highlights the ecological behavior and hidden diversity of bathyal three-bearded rocklings. Here, we additionally discuss the biogeographical distribution of all genetically verified Gaidropsarus spp. in combination with genetic data and morphological characters. G. mauritanicus sp. nov. is closely related to a species from Tasmania (43° S), a geographical point furthest among the studied samples, which may hint to an important influence of (paleo-) oceanography on the distributions of Gaidropsarus species.

Affordable 3D Orientation Visualization Solution for Working Class Remotely Operated Vehicles (ROV).

ROV operators often encounter challenges with orientation awareness while operating underwater, primarily due to relying solely on 2D camera feeds to manually control the ROV robot arm. This limitation in underwater visibility and orientation awareness, as observed among Malaysian ROV operators, can compromise the accuracy of arm placement, and pose a risk of tool damage if not handle with care. To address this, a 3D orientation monitoring system for ROVs has been developed, leveraging measurement sensors with nine degrees of freedom (DOF). These sensors capture crucial parameters such as roll, pitch, yaw, and heading, providing real-time data on the ROV's position along the X, Y, and Z axes to ensure precise orientation. These data are then utilized to generate and process 3D imaging and develop a corresponding 3D model of the operational ROV underwater, accurately reflecting its orientation in a visual representation by using an open-source platform. Due to constraints set by an agreement with the working class ROV operators, only short-term tests (up to 1 min) could be performed at the dockyard. A video demonstration of a working class ROV replica moving and reflecting in a 3D simulation in real-time was also presented. Despite these limitations, our findings demonstrate the feasibility and potential of a cost-effective 3D orientation visualization system for working class ROVs. With mean absolute error (MAE) error less than 2%, the results align with the performance expectations of the actual working ROV.

Design and Simulation of a Fuel Cell-based Hybrid Underwater Vehicle Propulsion System in Matlab/Simulink

Remotely Operated Vehicles (ROVs) and Autonomous Underwater Vehicles (AUVs) have many applications in underwater missions. A recurring issue is managing the power source of the vehicle, especially air-independent propulsion sources, such as batteries, accumulators, and fuel cells, to increase diving depth, underwater endurance, and range. The current article proposes the use of both fuel cells and batteries simultaneously. The subject of the study is Pluto Plus ROVs, which are mine sweeping and counter-terrorism underwater vehicles. The considered system is simulated in Matlab/Simulink. The result was a block diagram that simulates a hybrid propulsion system for submersibles in general and serves as the basis for improving the propulsion system for Pluto Plus ROVs.

Diving deep into the network: Quantifying protection effects across California's marine protected area network using a remotely operated vehicle

AbstractNetworks of marine protected areas (MPAs) are being increasingly implemented worldwide as conservation management tools. We report here on MPA effectiveness using data from a 17‐year long remotely operated vehicle (ROV) monitoring program spanning mesophotic (~ 20–130 m) depths in 26 MPAs across California's MPA network. We utilize a spatial modeling approach that includes important environmental covariates as well as spatial dependence in the data, and allows the separation of statewide and regional trends in the abundance of focal species from additional trends specific to MPAs. We demonstrate that there have been statewide and/or regional recoveries in abundance for the majority of our 10 focal demersal fish species, with all four statewide species assessments and 18 out of 22 species‐region combinations assessed displaying positive trends. We also demonstrate that MPA protection has had an additional positive effect on the abundance of the majority of these focal species inside MPAs compared with reference areas, with positive effects for all four statewide species, and 18 out of 22 statewide/regions assessed showing positive effects, four showing no statistically detectable differences, and no negative MPA effects found. Comparisons with theoretical expectations of MPA recovery for our focal species showed that 2 out of 4 statewide, and 11 out of 17 species‐region combinations assessed displayed higher mean MPA effects than expected. Our results highlight strong trajectories of increasing abundance and additional MPA effects for many of our focal species, demonstrating that MPAs are having positive effects in mesophotic depths across the network as well as at previously reported shallower depths, and that image‐based platforms such as ROVs provide an important tool to support timely reporting on the effectiveness of MPA networks.

Second-order Integral Sliding Mode Motion Control for Remotely Operated Vehicle

Second-order Integral Sliding Mode Motion Control for Remotely Operated Vehicle

Diverse and ecologically unique mesophotic coral ecosystems in the central Indian Ocean

Research on Mesophotic Coral Ecosystems (MCEs; 30–150 m) has grown exponentially in the last few decades, highlighting their rich diversity and extensive distribution. However, they are still largely underexplored compared to shallow-water coral reefs and frequently remain under-protected and under-represented in marine spatial planning. One reason for the imbalance between the high ecological value of MCEs and the limited levels of protection may be that baseline data on MCEs are largely missing to date, yet are crucial to provide evidence-based information for management actions. Here, we present data on the alpha and beta diversity of the benthic communities within MCEs in the Chagos Archipelago, Indian Ocean. Using imagery collected from Remotely Operated Vehicle surveys, benthic invertebrate megafauna were surveyed along the entire depth gradient from shallow to lower mesophotic depths (15–160 m). The diversity of the benthic communities decreased with increasing depth, from shallow water to the lower mesophotic zone. Nevertheless, the deepest parts of MCEs in the Archipelago displayed higher species richness values than several other shallow subtropical regions. In addition, the benthic communities showed high dissimilarity along the depth gradient, indicating that the key driver of community composition change with depth is species turnover (species replacement), revealing the uniqueness of MCEs. This study presents novel findings on MCEs in the central Indian Ocean, demonstrating that they host a high and unique benthic diversity, and highlighting the need to protect these ecosystems to preserve the overall biodiversity of coral reefs.

A Combination of Remote Sensing Datasets for Coastal Marine Habitat Mapping Using Random Forest Algorithm in Pistolet Bay, Canada

Marine ecosystems serve as vital indicators of biodiversity, providing habitats for diverse flora and fauna. Canada’s extensive coastal regions encompass a rich range of marine habitats, necessitating accurate mapping techniques utilizing advanced technologies, such as remote sensing (RS). This study focused on a study area in Pistolet Bay in Newfoundland and Labrador (NL), Canada, with an area of approximately 170 km2 and depths varying between 0 and −28 m. Considering the relatively large coverage and shallow depths of water of the study area, it was decided to use airborne bathymetric Light Detection and Ranging (LiDAR) data, which used green laser pulses, to map the marine habitats in this region. Along with this LiDAR data, Remotely Operated Vehicle (ROV) footage, high-resolution multispectral drone imagery, true color Google Earth (GE) imagery, and shoreline survey data were also collected. These datasets were preprocessed and categorized into five classes of Eelgrass, Rockweed, Kelp, Other vegetation, and Non-Vegetation. A marine habitat map of the study area was generated using the features extracted from LiDAR data, such as intensity, depth, slope, and canopy height, using an object-based Random Forest (RF) algorithm. Despite multiple challenges, the resulting habitat map exhibited a commendable classification accuracy of 89%. This underscores the efficacy of the developed Artificial Intelligence (AI) model for future marine habitat mapping endeavors across the country.

Detection of Flaws in Ship Hull Using Underwater Remotely Operated Vehicle

Detection of Flaws in Ship Hull Using Underwater Remotely Operated Vehicle

Hydrodynamic shape optimization of an autonomous and remotely-operated vehicle via a multi-surrogate model

The design requirements for the hydrodynamic performance of underwater vehicles vary significantly depending on the application. Optimization without an initial model results in challenges such as large design domains, nonlinear complexity, and high data requirements. To optimize the hydrodynamic performance during the design process, in this paper, a multi-surrogate model was employed to progressively design the shape of an autonomous and remotely-operated vehicle. Based on the data characteristics of different stages, this approach strikes a balance among data quantity, prediction accuracy, and multi-objective requirements. An artificial neural network prediction surrogate model was constructed based on the principle of the minimum prediction factor using the optimal Latin hypercube sampling method. During the optimization of the design domain, the optimum objective was to minimize the dimensionless force (G(x)). During the multi-objective optimization stage, a regression kriging surrogate model was constructed based on a support vector product. The optimization objectives were to maximize volume and minimize dimensionless forces. This enabled the overall design process to attain optimal Pareto solutions within the design domain while simultaneously ensuring high prediction accuracy and minimum data requirements. The results obtained are consistent with the simulation comparison, thus verifying the reliability of the entire optimization process.

Sensor platform for periodic integrity inspection of subsea structures with a permanently installed measuring system

Offshore structures are inspected with high costs and risks. In order to reduce both, the Fraunhofer IKTS has developed a sensor platform for automated periodic inspection by underwater vehicles like Remotely Operated Vehicles (ROV) or Autonomous Underwater Vehicles (AUV). The monitoring tasks are numerous. Temperature or pressure measurements are widely needed to ensure safe operation. Welded seams are a critical element of offshore infrastructure. If they fail due to high wave loads, extraordinary ice collision or due to fatigue behaviour, the entire structure becomes critical regarding the stability and safety. The Fraunhofer IKTS developed a technique for the permanent installation on subsea structures in order to enable a periodic inspection. The communication and energy supply are realized wireless by WiFi and inductive energy supply. The sensors used differ from application to application. For the welded seam inspection, shear wave laminable transducers ensure a flat design to prevent damage due to current or waves. Shear waves give a relative independency from fouling to make the interpretation of the data more comprehensive. The device has been tested in maritime environment for 2,5 years to ensure reliability. Furthermore, a modular setup with integrated transducers into the circuit board provide market relevant potential low prices on the long term. The connection unit has been designed in a way that also small ROV/AUV can operate the light-weight unit to ensure a broad applicability by almost any vehicle on the market.

Artificial Neural Network for Glider Detection in a Marine Environment by Improving a CNN Vision Encoder

Despite major economic and technological advances, much of the ocean remains unexplored, which has led to the use of remotely operated vehicles (ROVs) and gliders for surveying. ROVs and underwater gliders are essential for ocean data collection. Gliders, which control their own buoyancy, are particularly effective unmanned platforms for long-term observations. The traditional method of recovering the glider on a small boat is a risky operation and depends on the skill of the workers. Therefore, a safer, more efficient, and automated system is needed to recover them. In this study, we propose a lightweight artificial neural network for underwater glider detection that is efficient for learning and inference. In order to have a smaller parameter size and faster inference, a convolutional neural network (CNN) vision encoder in an artificial neural network splits an image of a glider into a number of elongated patches that overlap to better preserve the spatial information of the pixels in the horizontal and vertical directions. Global max-pooling, which computes the maximum over all the spatial locations of an input feature, was used to activate the most salient feature vectors at the end of the encoder. As a result of the inference of the glider detection models on the test dataset, the average precision (AP), which indicates the probability that an object is located within the predicted bounding box, shows that the proposed model achieves AP = 99.7%, while the EfficientDet-D2 model for comparison of detection performance achieves AP = 69.2% at an intersection over union (IOU) threshold of 0.5. Similarly, the proposed model achieves an AP of 78.9% and the EfficientDet-D2 model achieves an AP of 50.5% for an IOU threshold of 0.75. These results show that accurate prediction is possible within a wide range of recall for glider position inference in a real ocean environment.

Ensemble modelling to predict the distribution of vulnerable marine ecosystems indicator taxa on data‐limited seamounts of Cabo Verde (NW Africa)

AbstractAimSeamounts are conspicuous geological features with an important ecological role and can be considered vulnerable marine ecosystems (VMEs). Since many deep‐sea regions remain largely unexplored, investigating the occurrence of VME taxa on seamounts is challenging. Our study aimed to predict the distribution of four cold‐water coral (CWC) taxa, indicators for VMEs, in a region where occurrence data are scarce.LocationSeamounts around the Cabo Verde archipelago (NW Africa).MethodsWe used species presence–absence data obtained from remotely operated vehicle (ROV) footage collected during two research expeditions. Terrain variables calculated using a multiscale approach from a 100‐m‐resolution bathymetry grid, as well as physical oceanographical data from the VIKING20X model, at a native resolution of 1/20°, were used as environmental predictors. Two modelling techniques (generalized additive model and random forest) were employed and single‐model predictions were combined into a final weighted‐average ensemble model. Model performance was validated using different metrics through cross‐validation.ResultsTerrain orientation, at broad scale, presented one of the highest relative variable contributions to the distribution models of all CWC taxa, suggesting that hydrodynamic–topographic interactions on the seamounts could benefit CWCs by maximizing food supply. However, changes at finer scales in terrain morphology and bottom salinity were important for driving differences in the distribution of specific CWCs. The ensemble model predicted the presence of VME taxa on all seamounts and consistently achieved the highest performance metrics, outperforming individual models. Nonetheless, model extrapolation and uncertainty, measured as the coefficient of variation, were high, particularly, in least surveyed areas across seamounts, highlighting the need to collect more data in future surveys.Main ConclusionsOur study shows how data‐poor areas may be assessed for the likelihood of VMEs and provides important information to guide future research in Cabo Verde, which is fundamental to advise ongoing conservation planning.

Biomass mapping of fjordic cold-water coral reefs reveals distinct functional role of vertical wall habitat

Fjords provide unique habitats for large cold-water coral (CWC) reefs, typically growing on sills and vertical walls. Fjord reefs are among the most thriving CWC reefs in Norway. Yet, these reefs, especially the wall reefs, are notoriously understudied. Here, we mapped the biomass, total carbon (C) stocks and C turnover (as respiration) of the reef-building coral Lophelia pertusa (syn. Desmophyllum pertusum), and dominant, large CWC reef-associated suspension feeders (the sponges Geodia barretti and Mycale lingua, the CWC Madrepora oculata and the bivalve Acesta excavata) within the Hardangerfjord, Norway. Remotely Operated Vehicle (ROV) recorded videos from wall and sill reefs were used to estimate species-specific biomass. Coupled with high resolution terrain data (2 × 2 m), predictive maps of species biomass were produced using a random forest (RF) model. The resulting biomass data were integrated with species-specific C content and C respiration rates from literature to estimate C stocks and C turnover of wall versus sill reefs. Area-specific results from the RF models reveal that wall reefs had a higher habitat suitability for all species except L. pertusa, which was more dominant on the sills. Accordingly, the wall reefs supported an up to 11 times higher biomass, C stock, and turnover for all species, except for L. pertusa, which had two-fold higher values on the sill reef. As a result, the wall reefs showed a 1.5 to 4.8 times higher total mean C turnover by dominant suspension feeders (all studied species) compared to the sill reefs. With their high C turnover and their presumably wide distribution in Norwegian fjords and globally, benthic wall reef megafauna may have a substantial, but overlooked biomass and functional role within CWC reef systems.