Underwater Robot Research Articles

A deep learning approach for object detection of rockfish in challenging underwater environments

IntroductionPreserving the marine ecological environment and safeguarding marine species is a global priority. However, human overfishing has led to a drastic decline in fish species with longer growth cycles, disrupting the equilibrium of the marine ecosystem. To address this issue, researchers are turning to deep learning techniques and state-of-the-art underwater devices, such as underwater robots, to explore the aquatic environment and monitor the activities of endangered populations. This approach has emerged as a focal point of recent research in protecting the marine ecological environment. This study employs a deep learning-based object detection algorithm to identify fish species in complex underwater environments.MethodsThe algorithm is built upon the You Only Look Once version 7(YOLOv7) algorithm, with the addition of the attention mechanism Convolutional Block Attention Module (CBAM) in the network’s backbone. CBAM enhances the feature maps through the fusion of spatial attention and channel attention, ultimately improving the robustness and accuracy of the model’s inference by replacing the original loss function CIoU with SCYLLAIntersection over Union(SIoU). In this paper, the rockfish pictures in the dataset Label Fishes in the Wild published by the National Marine Fisheries Service are selected, and the underwater image enhancement model (UWCNN) is introduced to process the pictures.ResultThe experimental results show that the mean average precision (mAP) value of the improved model on the test set is 94.4%, which is 3.5% higher than the original YOLOv7 model, and the precision and recall rate are 99.1% and 99%, respectively. The detection performance of the algorithm in the field of complex underwater environment is improved.DiscussionThe underwater fish detection scheme proposed in this study holds significant practical value and significance in promoting the conservation of marine ecosystems and the protection of fish species.

UIESC: An Underwater Image Enhancement Framework via Self-Attention and Contrastive Learning

Low contrast, color distortion and blurred details are common problems that perplex vision-guided underwater robots. To this end, we propose an underwater image enhancement framework via self-attention and contrastive learning (UIESC) to solve these problems. In this work, local features and global dependencies are constructed through space and channel dual attention, and criss-cross attention is used to solve the high computational complexity of self-attention. Moreover, contrastive learning is introduced into network training as a loss function, and contrastive regularization ensures that the enhanced images are closer to clear positive samples and away from source negative samples. Finally, smoothed-histogram equalization is adopted for further optimization to accommodate complex and variable underwater scenes. Extensive experiments have shown that our framework outperforms state-of-the-art methods in underwater image enhancement tasks.

Underwater manipulator that imitates the movements of the human arm

In this paper, the design of a five degrees of freedom (5DoF) underwater manipulator system is presented and discussed. The forward and inverse kinematics problem are solved and shown. Additionally, a system that mimics the human arm movements using a manipulator is proposed. In order to imitate the movement of the human arm, the layout diagram of the IMU sensors and the method of first calibration were presented. The work also presents a device with a simple haptic feedback used to facilitate control of the gripper. The presented approach to the design of a robot control system is dedicated to an underwater robot with the main assumption of reducing the complexity of the system. Taking into account the authors’ knowledge, this approach distinguishes the presented system from solutions known from the literature.

Integrated force and displacement sensing in an untethered dielectric elastomer actuator with a piezoresistive element

Dielectric elastomer actuators, owing to their fully electrical control and silent operation, are becoming increasingly popular for the development of terrestrial and underwater mobile robots with versatile locomotion capabilities. It is essential to embed the ability to sense their state and external interactions in these robots to facilitate the development of future autonomous capabilities. However, sensorizing dielectric elastomer actuators for untethered robotic applications is challenging due to their use of high voltage and the nonlinear mechanics of the elastomers utilized in them. To address this challenge, we developed a novel technique based on embedded piezoresistive sensing and high voltage feedback to simultaneously estimate the actuator displacement and external force in a fully untethered actuator driven by a miniature low-cost voltage amplifier. A data-driven regression model has been developed to accurately estimate force and displacement from the measured data. Validation tests conducted on three actuators demonstrate promising results. We achieve RMSE values as low as 29.736 mN for force estimation and 0.023 mm for displacement estimation in the zero-voltage condition, where the actuator is subjected to a triangular wave with a mechanical frequency of 0.1 Hz and an amplitude of 3 mm. Additionally, we have realized fully untethered operation by employing a power source, small-size voltage amplifier, microcontroller, and wireless connectivity module embedded in a compact form-factor. This work presents a significant advancement in soft robotics, offering a reliable and cost-effective solution for future autonomous robotic systems based on high-voltage dielectric elastomer actuators.

Improving Semantic Segmentation Performance in Underwater Images

Nowadays, semantic segmentation is used increasingly often in exploration by underwater robots. For example, it is used in autonomous navigation so that the robot can recognise the elements of its environment during the mission to avoid collisions. Other applications include the search for archaeological artefacts, the inspection of underwater structures or in species monitoring. Therefore, it is necessary to improve the performance in these tasks as much as possible. To this end, we compare some methods for image quality improvement and data augmentation and test whether higher performance metrics can be achieved with both strategies. The experiments are performed with the SegNet implementation and the SUIM dataset with eight common underwater classes to compare the obtained results with the already known ones. The results obtained with both strategies show that they are beneficial and lead to better performance results by achieving a mean IoU of 56% and an increased overall accuracy of 81.8%. The result for the individual classes shows that there are five classes with an IoU value close to 60% and only one class with an IoU value less than 30%, which is a more reliable result and is easier to use in real contexts.

BotGo: Growing Pains

BotGo was started in 2007 by Ravi Panchal, an engineer, after he lost motivation to continue at a managerial role at his job. A hands-on technical person, Panchal was inspired to create an underwater tank-cleaning robot. He started BotGo by bootstrapping it with his savings and roped in his friends for key positions in the company. He also started workshops for robotics education in colleges in order to sustain the company; he called this initiative BotLearn. In 2009, BotGo was incubated and Panchal started franchises for BotLearn as part of his growth plans. This led to a crisis within the company, escalating to a point where Panchal was forced to consider options. This case highlights the importance of a product-to-market fit and examines the decision to franchise in view of the case facts. The case also points towards the mistakes in crisis management, with particular emphasis on channel management. Towards the end of the case, Panchal is faced with a dilemma on whether to continue with the franchises or close them down. The dilemma is further accentuated since Panchal's decision would ultimately affect the growth of BotGo as well as directly challenge his intention to franchise.

High speed underwater hydrogel robots with programmable motions powered by light

Stimuli-responsive shape-changing hydrogels are attractive candidates for use as underwater soft robots. The bottleneck lies in the low actuation speed inherently limited by the water diffusion between hydrogels and their surrounding environment. In addition, accessing complex motions is restricted by the material fabrication methods. Here we report a hitherto unknown mechanism to achieve high-speed and programmable actuations for a disulfide crosslinked thermally responsive hydrogel. The dynamic photo-activated disulfide bond exchange allows photo-mechanical programming to introduce spatio-selective network anisotropy. This gives rise to an actuation behavior dominated by thermally driven conformation change of the locally oriented polymer chains instead of the common mass-diffusion-based mechanism. With the incorporation of photothermal fillers, light-powered oscillation at frequencies as high as 1.7 Hz is realized. This, coupled with the versatility of the programming, allows access to robots with diverse high-speed motions including continuous swimming, step-wise walking, and rotating.

A novel miniature swimmer propelled by 36° Y-cut lithium niobate acoustic propulsion system

Self-propelled swimmers using ultrasonic transducers have been proposed and studied. The screw and bionic propulsion systems have dominated underwater robots. However, the complex structures make miniaturization difficult. The swimmers with ultrasonic transducers have no moving parts such as fins or propellers and can be driven by the nanomechanical-vibration in the surface of the transducer. Simple structure and high driving frequency make it easy to miniaturize. A 36° Y-cut lithium niobate (LN) thickness-vibration-mode transducer is used to investigate an underwater acoustic propulsion system. LN transducers are manufactured to 10 × 10 mm, 7 × 7 mm, and 4 × 4 mm square plates. The zero-speed propulsion and no-load speed are investigated to evaluate the swimmer performance. A mm-order swimmer is studied with a 4 × 4 mm LN transducer. At last, the multi-degrees-of-freedom swimmer driven by three 7 × 7 mm LN transducers is demonstrated with the advance and sway locomotion. In the future, a small-scale and agile self-propelled swimmer with an acoustic propulsion system can be expected for narrow and complex environment tasks, such as seagrass-prone areas and narrow pipelines.

MURA: a Multipurpose Underwater Robotic Arm mounted on Kalypso UUV in aquaculture

Underwater vehicles utilized in net cages at aquaculture facilities are commonly utilized for the purpose of examining the deterioration of nets and the accumulation of biofouling. The implementation of a robotic system for repairing damages has the potential to decrease the expenses associated with employing divers while reducing the risk of their injury. This study details the development, fabrication, and simulation of a cost-effective subaquatic manipulator, denoted as MURA, which can be seamlessly incorporated into submersible vehicles. The Kalypso unmanned underwater vehicle (UUV) is utilized in this study. MURA exhibits a high degree of modularity, enabling seamless alteration of the end-effector tool. Additionally, its low-cost nature renders it a viable option for integration with any underwater vehicle. Three end-effectors were subjected to testing, one designed for the purpose of disposing fish morts, another intended for removing litters from net cages in fisheries, and a third for repairing net tears. This study outlines the MURA design, including the arm’s fabrication and constituent components. In addition, the modeling of the manipulator is presented accompanied by a water flow simulation of the three manipulators. Ultimately, the experimental findings are analyzed and evaluated. These include the field experiments performed at Kefalonia fisheries, along with the duration to complete each task. For instance, the capture of fish morts typically necessitates approximately 30 s, encompassing the entire process from initial targeting to actual capture. In a similar vein, the procedure of mending tears in a net necessitates an approximate duration of 70 s on average, encompassing the stages of initial identification and subsequent detachment. The suggested design exhibits adaptability and durability while upholding affordability when utilized in aquaculture.

Manoeuvring of underwater snake robot with tail thrust using the actor-critic neural network super-twisting sliding mode control in the uncertain environment and disturbances

Manoeuvring of underwater snake robot with tail thrust using the actor-critic neural network super-twisting sliding mode control in the uncertain environment and disturbances

SwinWave-SR: Multi-scale lightweight underwater image super-resolution

The resource-limited nature of underwater vision equipment leads to poor, otherwise low-resolution information affecting the downstream underwater robotics and ocean engineering tasks. Underwater Image Enhancement (UIE) methods have emerged, particularly Super-Resolution (SR), to tackle the aforementioned challenge by restoring the corresponding low-resolution image to a high-quality counterpart. Vision Transformers (ViTs) have recently been employed for SR tasks thanks to their superior performance over mainstream convolution neural networks. The success of ViTs is largely due to their self-attention mechanism; however, they may encounter challenges in dealing with severe and unpredictable degradation in underwater imaging. In contrast, Multi-scale ViTs (MViTs) variants such as the Swin transformers have overcome that challenge by preserving long-range dependencies over multi-scale feature hierarchies through evolving channel capacity. MViTs tend to induce spatial efficiency through classical down-sampling, such as average pooling over key/values, which results in an inevitable loss of high-frequency components. To address this lack, in the current work, we propose a novel algorithm, SwinWave-SR, for efficient and accurate multi-scale SR for underwater images. Our proposed algorithm is based on Swin transformer consisting of a wavelet block to restrict the information drop by downsampling in an invertible fashion. Consequently, the key components are preserved to assist self-attention learning while reducing its computational cost simultaneously. To further complement it, we explore a prominent compression regime, namely the Lottery Ticket Hypothesis (LTH), to discover a lightweight sub-network with competitive performance to its original model by reducing computational costs up to 70.44%. Overall, SwinWave-SR improves peak signal-to-noise ratio (PSNR) by 0.95 dB ∼ 2.23 dB compared to the state-of-the-art SwinIR while reducing the number of parameters by 29.56% and the calculation cost by 18.734%. Experimental results show that the proposed SwinWave-SR method outperforms the state-of-the-art SR methods on four benchmark underwater datasets and significantly improves PSNR and structural similarity index (SSIM).

Underwater visual feature matching based on attenuation invariance

Underwater visual feature matching plays an important role in the localization and autonomous motion of underwater robots. However, feature descriptors are prone to change in degraded underwater images due to light attenuation, which leads to image color distortion, blur, and low illumination. Although the recent developments in deep learning have improved the performance of feature descriptors, the degradation of underwater images and the lack of underwater datasets have hindered the development of underwater visual feature matching and localization techniques. Herein, we propose an underwater visual feature matching method based on attenuation invariance. Our approach involves two main components. First, we study the attenuation differences between images to generate feature descriptors that are invariant to attenuation, thereby improving the accuracy of underwater visual feature matching. Second, we construct two datasets: the multiple water type (MWT) dataset, which contains over 30000 underwater images, and the underwater image feature descriptor (UIFD) evaluation dataset. These datasets are developed to compensate for the lack of evaluation datasets for underwater visual feature matching. The code and datasets are available at https://github.com/Sun-Jinghao/UIFD.

Research on a Blue–Green LED Communication System Based on an Underwater Mobile Robot

Underwater robots have been widely used in ocean exploration, deep-sea observation, seabed operations, marine scientific research, and other fields. Underwater low-latency, efficient, and safe communication modes are key to realizing the application of an underwater robot data transmission system. This paper mainly studies the optical communication between underwater mobile robots, including the large-dispersion-angle light-emitting diode (LED) design, large field of view receiving technology, weak light detector technology, etc. By designing a 120° large divergence angle underwater optical communication system in this study, the receiving field-of-view angle of the receiving end can reach 60°, which is suitable for the optical communication system of an underwater mobile platform. The high-power LED driver circuit is designed to drive the high-power LED and adopt weak light detection technology to ensure its stability and reliability. The experimental results show that, in the case of incomplete alignment between the transmitter and receiver, stable communication of underwater robots in motion is achieved through the design of a large divergence angle and a receiving field-of-view angle and the use of an underwater weak light detection technology. The communication distance is 30 m, and the communication rate remains above 10 Mbps. The information transmission content can include network data transmission, real-time video, high-definition video, high-definition images, and other data types. This equipment provides a solution for cableless data transmission of remotely operated vehicles (ROVs) and substantially enhances the application field of ROVs.

Leaping fishbot by combustion and propulsion hybrid actuation

AbstractAutonomous underwater vehicles have been extensively developed in the past decades and deployed for various ocean research activities. While the leaping‐out motions of underwater creatures have motivated the development of underwater robots with the motion ability to jump out of water, the dynamics of such jumping robots have not been fully described. In this study, we developed a leaping fishbot, a flying fish‐inspired underwater soft robot that can leap out of the water and further sail on the surface using the combustion and propulsion hybrid actuation. Specifically, the fishbot uses the transient driving method, in which a premixed oxygen–propane gas generates an instant pushing impulse to launch the robot out of the water at a transiently high speed while the attached jetting propellers provide a sustained thrust as it swims on the water surface at a stable velocity. A series of flume experiments and solid–fluid hybrid computational fluid dynamics numerical simulations were conducted to study the kinematic performance of the robot. Experimental results established a quantitative relationship between the kinematic performance, gas amount, and launch angle. The numerical results accurately predicted the water–air multiphase jumping motions, the numerical result of kinematic was significantly agreeing with the experimental results, the error in the horizontal direction is less than 15%, and that in the vertical direction is less than 10%. We further analyzed the flow field generated by the leaping fishbot to gain a better understanding of the velocity and vortex distribution around it. Our research findings provide valuable insights into bionic application of underwater robots in the future.

Trajectory Tracking and Obstacle Avoidance of Robotic Fish Based on Nonlinear Model Predictive Control.

The attainment of accurate motion control for robotic fish inside intricate underwater environments continues to be a substantial obstacle within the realm of underwater robotics. This paper presents a proposed algorithm for trajectory tracking and obstacle avoidance planning in robotic fish, utilizing nonlinear model predictive control (NMPC). This methodology facilitates the implementation of optimization-based control in real-time, utilizing the present state and environmental data to effectively regulate the movements of the robotic fish with a high degree of agility. To begin with, a dynamic model of the robotic fish, incorporating accelerations, is formulated inside the framework of the world coordinate system. The last step involves providing a detailed explanation of the NMPC algorithm and developing obstacle avoidance and objective functions for the fish in water. This will enable the design of an NMPC controller that incorporates control restrictions. In order to assess the efficacy of the proposed approach, a comparative analysis is conducted between the NMPC algorithm and the pure pursuit (PP) algorithm in terms of trajectory tracking. This comparison serves to affirm the accuracy of the NMPC algorithm in effectively tracking trajectories. Moreover, a comparative analysis between the NMPC algorithm and the dynamic window approach (DWA) method in the context of obstacle avoidance planning highlights the superior resilience of the NMPC algorithm in this domain. The proposed strategy, which utilizes NMPC, demonstrates a viable alternative for achieving precise trajectory tracking and efficient obstacle avoidance planning in the context of robotic fish motion control within intricate surroundings. This method exhibits considerable potential for practical implementation and future application.

Multi-target tracking algorithm in aquaculture monitoring based on deep learning

In order to analyze fish behavior, real-time underwater fish monitoring is essential. This work presents an underwater multi-target tracking algorithm for aquaculture monitoring, utilizing deep learning techniques. Underwater images are acquired using an underwater robot, and the images are defogged using the multi-scale Retinex algorithm based on HSV space. To enhance object detection performance, GhostNetv2 was integrated into the You Only Look Once version 5 (YOLOv5) object detection algorithm, supplemented by the addition of the Coordinate Attention (CA) module, resulting in the development of GN-YOLOv5. For the tracking algorithm, the more accurate Generalized Intersection over Union (GIoU) method was incorporated into the StrongSORT tracking algorithm. Moreover, to achieve more precise target tracking, a fish re-identification model was established. The proposed algorithms were evaluated through experiments conducted on various datasets, including VOC 2012, MOT16, and self-built datasets. The results demonstrate notable improvements: the GN-YOLOv5 model showed a 32.91% reduction in parameters and a 3.22% increase in precision. Furthermore, the enhanced StrongSORT algorithm exhibited a 3.84% increase in MOTA, a 28.00% reduction in IDS, and a speed boost of 7 FPS, leading to stable and accurate multi-target fish tracking.

Development of a Biomimetic Underwater Robot for Bottom Inspection of Marine Structures

Development of a Biomimetic Underwater Robot for Bottom Inspection of Marine Structures

Learning for Depth Control of a Robotic Penguin: A Data-Driven Model Predictive Control Approach

For bionic underwater robots, it is a great challenge for depth control due to model uncertainty and strong nonlinearity. To this end, we propose a data-driven model predictive control (MPC) approach using reinforcement learning (RL) for robotic penguin depth control. First, by imitating the underwater mode of the biological penguin, a robotic prototype with a tendon-driven head, two degrees of freedom wings, and a tendon-driven tail was designed. Then, a data-driven MPC framework is proposed considering the structure and motion properties of the robotic penguin. Specially, a data-based learning environment is constructed using a motion capture system, computational fluid dynamics, and a back-propagation neural network. Meanwhile, to maximize the benefits of the controller while ensuring safety and stability, a data-driven MPC using the RL scheme is applied to approximate the optimal policy. Combined with an appropriate reward design and periodic training, the closed-loop controller performance is significantly improved, and the validity of the proposed framework is finally tested by extensive simulations and experiments. Notably, this work will provide valuable insights into the learning-based motion control of bionic underwater robots.

Research on composite fault diagnosis technology of underwater vehicle actuator with positioning error constraint

When diagnosing the composite fault of the actuator, the characteristics of the motion force of the underwater vehicle are not analyzed, and there are diagnostic errors, resulting in the low accuracy of the diagnosis technology. In order to solve this problem and improve the operation safety of underwater vehicle actuators, this paper proposes a compound fault diagnosis technology for underwater vehicle actuators under positioning error constraints. Analyze the motion force of the underwater robot actuator, control the motion of the underwater robot actuator according to the analysis results, and extract real-time data parameters according to the control results. Under the constraint of positioning error, the composite fault features of the underwater robot actuator are divided, and the diagnosis model is built according to the deep fusion of the features to complete the fault diagnosis. The experimental results show that the technology can diagnose the composite fault data of the actuator, and the positioning error of the horizontal axis and the horizontal axis can be significantly improved, which can improve the diagnosis effect of the composite fault of the actuator and to the improvement of underwater robot running safety of actuators provide certain reference.

DEVELOPMENT OF UNDERWATER ROV (REMOTELY OPERATED VEHICLE) SYSTEMS AND MECHANISM

Remotely Operated Vehicle or abbreviated as ROV is an underwater explorer robot where the robot is controlled remotely by an operator using a remote control system. Water ecosystems certainly have various kinds of hazards that cause harm to divers, such as wild animals, waters contaminated with toxic waste, limited oxygen, and many other risks. Based on these problems, research was carried out, namely the Design and Development of Underwater ROV (Remotely Operated Vehicle) Mechanisms and Systems where the tool built is an underwater robot that uses 8 DC motors to be able to perform 12 types of movements under water. The final result obtained by the ROV is that it is able to perform all of these movements except for the right somersault and left somersault. The speed and direction of rotation of the motor is controlled by a device, namely the BTS7960 motor driver. This robot aims to measure pressure and temperature and capture images underwater to a depth of 4m. Pressure and temperature measurements were made using a BMP180 pressure sensor and a DS18B20 temperature sensor where the measurement results will be displayed on a 16x2 LCD. Based on the test results, the pressure sensor has an error rate of 3.6% and a temperature sensor of 0.34%. Images and videos are captured by a device, namely a CCTV camera where the device is connected to a laptop device as a monitor. The test produced results in the form of images and videos that were quite clear.