Objects In Complex Environments Research Articles

Small Object Detection in UAV Images Based on YOLOv8n

With the rapid development of unmanned aerial vehicle (UAV) technology, there is an urgent need for high-performance aerial object detection algorithms that are tailored for deployment on drones with limited computing capabilities. This paper proposes a series of improvements to the state-of-the-art YOLOv8 object detector to enhance its detection accuracy and speed for small, partially occluded objects in complex environments. Specifically, we introduce multi-scale feature fusion through additional detection layers, employ conditionally parameterized convolutions to increase representational capacity, and import a dynamic non-monotonic loss function named Wise-IoU to enable more effective regression of bounding boxes. Experiments conducted on the large-scale UAV benchmark dataset VisDrone demonstrate that the improved model achieves state-of-the-art accuracy of 37.6% mAP with 3 M parameters, outperforming other lightweight YOLO detectors and two-stage detectors like Faster R-CNN. The improved model also reaches 40 FPS on an embedded edge device, validating its efficiency and suitability for real-time UAV applications. Through comprehensive quantitative experiments and visual results, this work provides valuable insights and techniques to tailor object detection algorithms for robust and efficient deployment on UAVs with limited onboard computing power.

RaDro: Indoor Drone Tracking Using Millimeter Wave Radar

Core to drone design is its ability to ascertain its location by utilizing onboard inertial sensors combined with GPS data. However, GPS is not always reachable, especially in challenging environments such as indoors. This paper proposes RaDro; a system that leverages millimeter-waves (mmWave) to precisely localize and track drones in indoor environments. Unlike commonly used alternative technologies, RaDro is cost-effective and can penetrate obstacles, a bonus in non-line-of-sight (NLoS) scenarios, which enhances its reliability for tracking objects in complex environments. It does this without the need for tags or anchors to be attached to the drone, achieving 3D tracking with just a single radar point, significantly streamlining the deployment process. Comprehensive experiments are conducted in different scenarios to evaluate RaDro's performance. These include employing different drone models with different sizes to execute a range of aerial manoeuvres across different flight arenas, each with its own settings and clutter, and encountering various LoS and NLoS scenarios in dynamic environments. The experiments aimed to assess the capabilities of the system to extract coarse-grained and fine-grained information for drone detection, motion recognition, and localization. The results showcase precise localization, achieving a 50% reduction in localization error compared to the conventional baseline. This localization accuracy remains resilient even when confronted with interference from other moving sources. The results also demonstrate the system's ability to accurately localize drones in NLoS scenarios where existing state-of-the-art optical technologies cannot work.

Evaluation on Different Object Selection Visual Feedback in Collaborative Augmented Reality

Collaborative Augmented Reality (CAR) involves multiple users working together in an augmented reality setting to accomplish tasks. In CAR, transparent techniques enable users to see occluded objects in complex environments, yet they often struggle to discern which part of a complex object they have selected. Workspace awareness refers to understanding how others interact within a shared workspace and influence the choice of visualization techniques. This study investigated workspace awareness regarding different object selection visual feedback in co-located CAR. It explored four technique variations: Hi (highlight), Hi-Ray (ray emitting), Hi-View (view switching), and Hi-RV (ray emitting and view switching). Thirty-two participants were divided into 16 pairs, with an instructor demonstrating how to assemble an engine component and a student replicating the process using the four visualization techniques. Results showed the visualization techniques did not affect task completion time significantly. The Hi and Hi-Ray technique had the highest error rate, while the Hi-RV technique had the lowest. However, users preferred the Hi-View technique, which scored highest in interaction, collaboration quality, workspace awareness, and required less workload. To enhance awareness, participants pointed out the weakness in the combination of visual feedback, particularly with the ray emitting technique, which causes confusion when multiple objects are selected due to visual clutter. They suggested that integrating audio feedback could help alert users to their choices, and using color-based indicators could distinguish different actions, such as indicating correct or incorrect placement of virtual objects.

TARGET IDENTIFICATION AND TRACKING IN COMPLEX ENVIRONMENT

This study focuses on developing advanced methods for the identification and classification of objects in complex environments. Over the past two years, there has been an increase in the use of advanced technologies in various challenging scenarios. This research is centered on accurately identifying targets and tracking them. The study addresses challenges related to object detection in multi-dimensional and intricate settings, taking into account natural conditions like rain and fog, as well as technical limitations such as camera capabilities. Special emphasis is placed on data collection for training the identification model, followed by extensive data preprocessing, including cleaning, labeling, and augmentation. The research employs YOLO and Deep Sport machine learning algorithms, focusing on improving the accuracy and reliability of target recognition and increasing data processing speed to minimize misidentification risks. The integration of YOLO, known for its quick real-time object detection, with Deep Sport, which excels in detailed feature extraction and classification, forms the basis of our methodology. This fusion is a complex combination of both models' strengths, with YOLO quickly identifying relevant objects and Deep Sport performing an in-depth analysis. The experimental phase involves extensive testing of the models in varied weather conditions and settings to evaluate performance under challenging circumstances. This work aims to enhance object identification techniques in complex environments, a critical aspect for the effectiveness of various advanced operations. The findings are expected to significantly contribute to the field by enabling quicker and more accurate target identification.

Development of Annulus-Object Random Bin Picking System based on Rapid Establishment of RGB-D Images

With the development of the automation industry, robotic arms, and vision applications are no longer limited to fixed actions of the past. Production lines increasingly require the recognition and grasping of objects in complex environments, emphasizing quick setup and stability. In this paper, a rapidly constructed eye-hand system for robotic arm grasping, which enables fast and efficient object manipulation, particularly for stacking, is introduced. Initially, images were captured using a camera to generate extensive datasets from a limited number of images. Objects were subsequently segmented and categorized using deep learning networks for object detection and instance segmentation. Three-dimensional position information was obtained from an RGB-D camera. Finally, object poses were determined based on plane normal vectors, and gripping positions were manually marked. This reduced the time required for grab-point identification, model training, and pose localization. Based on experimental results, the grasping procedure proposed in this paper is suitable for various object-grasping scenarios. It achieved impressive picking success rates of 96% for unstacked annular objects and 90.86% for random bin annular objects, respectively. In the final experiment, following depth information filtering, a success rate of 95.1% was attained with random bin annular object picking.

Research on detection of transmission line corridor external force object containing random feature targets

With the objective of achieving “double carbon,” the power grid is placing greater importance on the security of transmission lines. The transmission line corridor has complex situations with external force targets and irregularly featured objects including smoke. For this reason, in this paper, the high-performance YOLOX-S model is selected for transmission line corridor external force object detection and improved to enhance model multi-object detection capability and irregular feature extraction capability. Firstly, to enhance the perception capability of external force objects in complex environment, we improve the feature output capability by adding the global context block after the output of the backbone. Then, we integrate convolutional block attention module into the feature fusion operation to enhance the recognition of objects with random features, among the external force targets by incorporating attention mechanism. Finally, we utilize EIoU to enhance the accuracy of object detection boxes, enabling the successful detection of external force targets in transmission line corridors. Through training and validating the model with the established external force dataset, the improved model demonstrates the capability to successfully detect external force objects and achieves favorable results in multi-class target detection. While there is improvement in the detection capability of external force objects with random features, the results indicate the need to enhance smoke recognition, particularly in further distinguishing targets between smoke and fog.

An Operating Stiffness Controller for the Medical Continuum Robot Based on Impedance Control.

Continuum manipulators can conform to curvilinear paths and manipulate objects in complex environments, which makes it emerging to be applied in minimally invasive surgery (MIS). However, different and controllable operating stiffness of the continuum manipulator is required during different stages of surgery to achieve safe access or stable and precise operation. This work proposes an operating stiffness controller (OSC) for the typical tendon-driven continuum manipulator based on the variable impedance control method with Lagrangian dynamic modeling. This controller can adjust the operating stiffness by modifying the driving forces along the driving tendons of the continuum manipulator without changing its material or structure. The proposed OSC converts the damping and stiffness matrices of the impedance control into variable parameters. This merit allows it to dynamically adjust the operating stiffness of the continuum manipulator according to the desired constant or time-varying stiffness. Furthermore, the OSC stability can be proven based on a Lyapunov function, and its stiffness control performances have been analyzed and evaluated in both simulations and experiments. The OSC controller generated average relevant error values of 7.82% and 3.09% for the operating stiffness control experiments with constant and time-varying desired stiffness, respectively. These experimental results indicate that the OSC has high accuracy, stability, and strong robustness in the operating stiffness control tasks.

Thresholding techniques in computer vision applications

Thresholding techniques are key pillars of image processing, especially for distinguishing objects in complex environments. This paper examines four types of thresholding strategies, each based on different theories, practical, popular, and advanced. Through a thorough literature review, the paper explains the thresholding techniques, thresholding operations, evaluation metrics, image processing techniques, and Python code for ROI of binary images in an understandable manner. The findings underscore the significance of thresholding in various applications, from object recognition to medical imaging, and highlight the importance of selecting appropriate thresholding methods based on image characteristics.

MLP-YOLOv5: A Lightweight Multi-Scale Identification Model for Lotus Pods with Scale Variation

Lotus pods in unstructured environments often present multi-scale characteristics in the captured images. As a result, it makes their automatic identification difficult and prone to missed and false detections. This study proposed a lightweight multi-scale lotus pod identification model, MLP-YOLOv5, to deal with this difficulty. The model adjusted the multi-scale detection layer and optimized the anchor box parameters to enhance the small object detection accuracy. The C3 module with transformer encoder (C3-TR) and the shuffle attention (SA) mechanism were introduced to improve the feature extraction ability and detection quality of the model. GSConv and VoVGSCSP modules were adopted to build a lightweight neck, thereby reducing model parameters and size. In addition, SIoU was utilized as the loss function of bounding box regression to achieve better accuracy and faster convergence. The experimental results on the multi-scale lotus pod test set showed that MLP-YOLOv5 achieved a mAP of 94.9%, 3% higher than the baseline. In particular, the model’s precision and recall for small-scale objects were improved by 5.5% and 7.4%, respectively. Compared with other mainstream algorithms, MLP-YOLOv5 showed more significant advantages in detection accuracy, parameters, speed, and model size. The test results verified that MLP-YOLOv5 can quickly and accurately identify multi-scale lotus pod objects in complex environments. It could effectively support the harvesting robot by accurately and automatically picking lotus pods.

Object Detection for Underwater Cultural Artifacts Based on Deep Aggregation Network with Deformation Convolution

Cultural artifacts found underwater are located in complex environments with poor imaging conditions. In addition, the artifacts themselves present challenges for automated object detection owing to variations in their shape and texture caused by breakage, stacking, and burial. To solve these problems, this paper proposes an underwater cultural object detection algorithm based on the deformable deep aggregation network model for autonomous underwater vehicle (AUV) exploration. To fully extract the object feature information of underwater objects in complex environments, this paper designs a multi-scale deep aggregation network with deformable convolutional layers. In addition, the approach also incorporates a BAM module for feature optimization, which enhances the potential feature information of the object while weakening the background interference. Finally, the object prediction is achieved through feature fusion at different scales. The proposed algorithm has been extensively validated and analyzed on the collected underwater artifact datasets, and the precision, recall, and mAP of the algorithm have reached 93.1%, 91.4%, and 92.8%, respectively. In addition, our method has been practically deployed on an AUV. In the field testing over a shipwreck site, the artifact detection frame rate reached up to 18 fps, which satisfies the real-time object detection requirement.

Dynablox: Real-Time Detection of Diverse Dynamic Objects in Complex Environments

Dynablox: Real-Time Detection of Diverse Dynamic Objects in Complex Environments

E-detector: Asynchronous Spatio-temporal for Event-based Object Detection in Intelligent Transportation System

In intelligent transportation systems, various sensors, including radar and conventional frame cameras, are used to improve system robustness in various challenging scenarios. An event camera is a novel bio-inspired sensor that has attracted the interest of several researchers. It provides a form of neuromorphic vision to capture motion information asynchronously at high speeds. Thus, it possesses advantages for intelligent transportation systems that conventional frame cameras cannot match, such as high temporal resolution, high dynamic range, as well as sparse and minimal motion blur. Therefore, this study proposes an E-detector based on event cameras that asynchronously detect moving objects. The main innovation of our framework is that the spatiotemporal domain of the event camera can be adjusted according to different velocities and scenarios. It overcomes the inherent challenges that traditional cameras face when detecting moving objects in complex environments, such as high speed, complex lighting, and motion blur. Moreover, our approach adopts filter models and transfer learning to improve the performance of event-based object detection. Experiments have shown that our method can detect high-speed moving objects better than conventional cameras using state-of-the-art detection algorithms. Thus, our proposed approach is extremely competitive and extensible, as it can be extended to other scenarios concerning high-speed moving objects. The study findings are expected to unlock the potential of event cameras in intelligent transportation system applications.

Accurate and Robust Rotation-Invariant Estimation for High-Precision Outdoor AR Geo-Registration

Geographic registration (geo-registration) is a crucial foundation for augmented reality (AR) map applications. However, existing methods encounter difficulties in aligning spatial data with the ground surface in complex outdoor scenarios. These challenges make it difficult to accurately estimate the geographic north orientation. Consequently, the accuracy and robustness of these methods are limited. To overcome these challenges, this paper proposes a rotation-invariant estimation method for high-precision geo-registration in AR maps. The method introduces several innovations. Firstly, it improves the accuracy of generating heading data from low-cost hardware by utilizing Real-Time Kinematic GPS and visual-inertial fusion. This improvement contributes to the increased stability and precise alignment of virtual objects in complex environments. Secondly, a fusion method combines the true-north direction vector and the gravity vector to eliminate alignment errors between geospatial data and the ground surface. Lastly, the proposed method dynamically combines the initial attitude relative to the geographic north direction with the motion-estimated attitude using visual-inertial fusion. This approach significantly reduces the requirements on sensor hardware quality and calibration accuracy, making it applicable to various AR precision systems such as smartphones and augmented reality glasses. The experimental results show that this method achieves AR geo-registration accuracy at the 0.1-degree level, which is about twice as high as traditional AR geo-registration methods. Additionally, it exhibits better robustness for AR applications in complex scenarios.

Implementation of a Real-Time Object Pick-and-Place System Based on a Changing Strategy for Rapidly-Exploring Random Tree

An object pick-and-place system with a camera, a six-degree-of-freedom (DOF) robot manipulator, and a two-finger gripper is implemented based on the robot operating system (ROS) in this paper. A collision-free path planning method is one of the most fundamental problems that has to be solved before the robot manipulator can autonomously pick-and-place objects in complex environments. In the implementation of the real-time pick-and-place system, the success rate and computing time of path planning by a six-DOF robot manipulator are two essential key factors. Therefore, an improved rapidly-exploring random tree (RRT) algorithm, named changing strategy RRT (CS-RRT), is proposed. Based on the method of gradually changing the sampling area based on RRT (CSA-RRT), two mechanisms are used in the proposed CS-RRT to improve the success rate and computing time. The proposed CS-RRT algorithm adopts a sampling-radius limitation mechanism, which enables the random tree to approach the goal area more efficiently each time the environment is explored. It can avoid spending a lot of time looking for valid points when it is close to the goal point, thus reducing the computing time of the improved RRT algorithm. In addition, the CS-RRT algorithm adopts a node counting mechanism, which enables the algorithm to switch to an appropriate sampling method in complex environments. It can avoid the search path being trapped in some constrained areas due to excessive exploration in the direction of the goal point, thus improving the adaptability of the proposed algorithm to various environments and increasing the success rate. Finally, an environment with four object pick-and-place tasks is established, and four simulation results are given to illustrate that the proposed CS-RRT-based collision-free path planning method has the best performance compared with the other two RRT algorithms. A practical experiment is also provided to verify that the robot manipulator can indeed complete the specified four object pick-and-place tasks successfully and effectively.

SO-YOLOv5: Small object recognition algorithm for sea cucumber in complex seabed environment

Underwater object recognition is a prerequisite for the realization of automated seafood harvesting. To solve the problems of low recognition accuracy and poor generalization ability of existing underwater small object, this paper takes sea cucumber as research objects to research the small object recognition approach in complex underwater environment. In this paper, a sea cucumber dataset is established to solve the problem of lacking sea cucumber dataset in the real seabed environment. A deep learning model SO-YOLOv5 is proposed based on YOLOv5 for underwater small object recognition, which improves the recognition ability of the algorithm for different sizes of objects in complex environment. The proposed model embeds a large-scale feature extraction layer in structure to increase the detection ability of small objects by referring to the idea of the Bi-directional Feature Pyramid Network. SO-YOLOv5 fuses the features between deep and shallow layers and balances the feature information of different scales, and integrates the Coordinate Attention mechanism to enhance the sensitivity of the algorithm to the direction and position information. The experimental results illustrate that the mAP of the proposed approach achieves 95.47% for sea cucumber recognition in the complex seabed environment, which is 3.42%, 6.79%, and 5.46% higher than the traditional Faster R-CNN, SSD, and YOLOv5 algorithms, respectively. This research not only provides an effective approach for sea cucumber recognition in complex environment but also has certain reference significance for the recognition of small objects in other complex environments. In addition, the proposed approach has practical application value to improve intelligence level in aquaculture.

Anchor-free infrared pedestrian detection based on cross-scale feature fusion and hierarchical attention mechanism

An anchor-free infrared pedestrian detection algorithm is proposed to address the problems of multiscale, partial occlusion and environmental interference in infrared images, which make it difficult for traditional algorithms to perform accurate detection. First, a cross-scale feature fusion module is devised to improve the detection performance of multiscale and partially occluded objects. This module fuses the global features of different residual layers and multiscale local region features, thereby improving the multiscale feature fusion capability of the model while expanding the feature receptive field of the backbone network. Second, the object texture features of infrared images are weak owing to the imaging mechanism, which affects the accuracy of detection model. Thus, we construct a hierarchical attention mapping module to enhance the significance of pedestrian features in complex environments while suppressing the background information. Moreover, to improve the detection performance further, a novel dual-branch head detector is designed to decouple the input features from the channel dimension, which can be used for classification and location. Finally, by introducing the anchor-free concept into the network prediction mechanism, the prediction box can learn ground truth regression parameters by itself, thereby simplifying the network structure and improving the model generalization ability. The experimental results demonstrate that the proposed algorithm can accurately detect multiscale infrared pedestrian objects in complex environments and exhibits better objective evaluation indices compared to other pedestrian detection algorithms. The average precision and recall reached 98.78% and 98.67%, respectively.

A Cross Stage Partial Network with Strengthen Matching Detector for Remote Sensing Object Detection

Remote sensing object detection is a difficult task because it often requires real-time feedback through numerous objects in complex environments. In object detection, Feature Pyramids Networks (FPN) have been widely used for better representations based on a multi-scale problem. However, the multiple level features cause detectors’ structures to be complex and makes redundant calculations that slow down the detector. This paper uses a single-layer feature to make the detection lightweight and accurate without relying on Feature Pyramid Structures. We proposed a method called the Cross Stage Partial Strengthen Matching Detector (StrMCsDet). The StrMCsDet generates a single-level feature map architecture in the backbone with a cross stage partial network. To provide an alternative way of replacing the traditional feature pyramid, a multi-scale encoder was designed to compensate the receptive field limitation. Additionally, a stronger matching strategy was proposed to make sure that various scale anchors may be equally matched. The StrMCsDet is different from the conventional full pyramid structure and fully exploits the feature map which deals with a multi-scale encoder. Methods achieved both comparable precision and speed for practical applications. Experiments conducted on the DIOR dataset and the NWPU-VHR-10 dataset achieved 65.6 and 73.5 mAP on 1080 Ti, respectively, which can match the performance of state-of-the-art works. Moreover, StrMCsDet requires less computation and achieved 38.5 FPS on the DIOR dataset.

Optimal design of robotic work-cell through hierarchical manipulability maximization

The increasing requests for flexible robotic applications involving the rapid relocation of the robot manipulator , possibly mounted on a mobile base, imposes tolerance to imprecise positioning. The high manipulability of the nominally designed poses, i.e. , the capacity to change the position and the orientation of a given robot joint configuration’s end-effector, is often considered a proxy for robustness to imprecise positioning. This work presents a method for choosing target end-effector poses to manipulate bulky objects in complex environments. The paper proposes a two-layer optimizer connected in cascade to maximize the manipulability and achieve reasonable computational time. First, using a genetic algorithm (GA) allows a global search for a satisfactory solution to the target poses of the task at the same time. Subsequently, the output of the GA becomes the initial guess for the simulated annealing (SA) algorithm, which locally maximizes each pose’s manipulability separately. The feasibility of the connecting trajectories and collisions are checked in both layers. Experiments show the method’s ability to find excellent solutions within a limited time, considering a complex problem involving manipulating large objects in a cluttered environment. The simulations of three working scenarios allowed testing of the proposed method. The final validation of the algorithm was on two relevant industrial use-cases: the manipulation of sidewalls and the manipulation of cargo panels inside an aircraft fuselage . • Optimal robotic workcell design can be a hard problem in some situations. • A tool is provided to optimize the robot position with respect to target poses. • Cascaded optimization allows for fast solutions search in complex environments. • Global optimization of the whole set of tasks followed by single task refinement.

A Stable Lightweight and Adaptive Feature Enhanced Convolution Neural Network for Efficient Railway Transit Object Detection

Obstacles in front of a train pose a significant threat to traffic safety, and many accidents happen under shunting mode when the speed of a train is below 45 km/h. The existing track object–detection algorithms encounter difficulty in balancing the detection precision and speed in shunting mode. Additionally, their accuracy is insufficient, particularly for small objects in complex environments. To address these problems, we propose a stable lightweight feature extraction and adaptive feature fusion network for real-time detection of obstacles in railway traffic scenarios to ensure driving safety. The proposed network consists of three modules. The stable bottom feature extraction module reduces the computational load and extracts more image information stably. The lightweight feature extraction module improves feature extraction using a simple and effective network. The enhanced adaptive feature fusion module fuses the image and original features, improving the multiscale detection accuracy under complex environments, particularly in the case of small objects. With a default input size of 416 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 416$ </tex-math></inline-formula> pixels (px), the proposed method achieves a detection speed of 81 FPS and a mean average precision of 94.75% for the railway traffic dataset as well as a detection speed of 78 FPS (26 FPS faster and 0.47% higher than those of YOLOv4, respectively) and a mean average precision of 42.5% for MS COCO. This indicates its potential for real-world railway object detection and other multi-target detection tasks. Additionally, the experimental results based on PASCAL VOC2007 and VOC2012 indicate that the proposed approach is considerably better than the state-of-the-art models.

Validation of Targets in Sonar Imagery Using Multispectral Analysis

The detection of underwater objects in sonar imagery is a key enabling technique, for applications ranging from mine hunting and seabed characterization to marine archaeology. Owing to the nonhomogeneity of the sonar imagery, the majority of detection approaches are geared toward the detection of features in the spatial domain to identify anomalies in the seabed’s background. Yet, when the seabed is complex and includes rocks and sand ripples, spatial features are hard to discriminate, leading to high false alarm rates. With the aim of validating the detection of man-made objects in complex environments, we utilize the expected spectral diversity of reflections. This way, we can differentiate man-made objects’ reflections from the relatively flat frequency response of natural objects’ reflections, such as rocks. Our solution merges a set of preregistered sonar images of the same scene that are obtained at a different center frequency. For low- or high-resolution sonar images, we apply the Jain’s fairness index or the Kullback–Leibler divergence, respectively, to evaluate the spectral diversity of the reflections of a given region of interest and, thus, detect anomalies across the spectrum domain. We test our algorithm over simulated data and over images collected in three designated sea experiments: a data set that we share with the community. The results show that, compared with benchmark schemes, our solution achieves lower false alarm rates while preserving high detection level.