Object Localization Algorithm Research Articles

Vision-Based Object Manipulation for Activities of Daily Living Assistance Using Assistive Robot

The increasing prevalence of upper and lower extremity (ULE) functional deficiencies presents a significant challenge, as it restricts individuals’ ability to perform daily tasks independently. Robotic devices are emerging as assistive devices to assist individuals with limited ULE functionalities in activities of daily living (ADLs). While assistive manipulators are available, manual control through traditional methods like joysticks can be cumbersome, particularly for individuals with severe hand impairments and vision limitations. Therefore, autonomous/semi-autonomous control of a robotic assistive device to perform any ADL task is open to research. This study addresses the necessity of fostering independence in ADLs by proposing a creative approach. We present a vision-based control system for a six-degrees-of-freedom (DoF) robotic manipulator designed for semi-autonomous “pick-and-place” tasks, one of the most common activities among ADLs. Our approach involves selecting and training a deep-learning-based object detection model with a dataset of 47 ADL objects, forming the base for a 3D ADL object localization algorithm. The proposed vision-based control system integrates this localization technique to identify and manipulate ADL objects (e.g., apples, oranges, capsicums, and cups) in real time, returning them to specific locations to complete the “pick-and-place” task. Experimental validation involving an xArm6 (six DoF) robot from UFACTORY in diverse settings demonstrates the system’s adaptability and effectiveness, achieving an overall 72.9% success rate in detecting, localizing, and executing ADL tasks. This research contributes to the growing field of autonomous assistive devices, enhancing independence for individuals with functional impairments.

Processing‐in‐memory based multilateration localization in wireless sensor networks using memristor crossbar arrays

SummaryMost of the multilateration algorithms for object localization rely heavily on matrix multiplication for calculating the precise location of the target from all the reference points in a 3D domain. A significant disadvantage is that performing matrix multiplication can be computationally expensive, particularly for larger matrices, leading to slower performance and longer processing times. In this article, we propose a Processing‐in‐Memory (PIM) based localization approach for multilateration using a memristor crossbar array for performing faster matrix multiplication. The experiments were conducted on real‐time data collected through ultra‐wideband (UWB) chip. The simulation results demonstrate that the proposed model is around 50% faster than the fastest known localization algorithm considered for evaluation and exhibits an average of 55% improvement in localization accuracy.

Optimizing pick-place operations: Leveraging k-means for visual object localization and decision-making in collaborative robots

This article presents an approach to object localization algorithms for pick-place operations in collaborative robots by utilizing conventional color segmentation in computer vision and k-means clustering. Adding the k-means clustering algorithm complements the color segmentation by distinguishing and grouping the sections of similar pixels; hence, object localization is more accurate. The order of pick-place operations of each cluster acquired from the proposed algorithm is prioritized based on norm. Integrating the proposed framework provides a well-structured depiction of the localized objects, which is fundamental for successful pick-place operations. The TCP/IP communication framework via socket communication is established to facilitate data transmission between the robot and the host computer. The objective is to ensure that the robot's end effector performs as directed by the host computer by obtaining information on the pick-and-place operation, including the localized coordinates, dimensions, the order of operations, and the pose of the objects of interest to the robot. In this experiment, a cobot arm is employed to autonomously pick and place objects with different shapes and colors in a workspace filled with diverse objects, requiring the robot to choose the closest objects to operate based on the data from the host computer. Our results demonstrate the effectiveness of this integration, showcasing the enhanced adaptability and efficiency of pick-place operations in collaborative robots. This study indicates 98% accuracy in pick-and-place operations with an average latency of 0.52 ± 0.1 s, indicating an improvement compared to the traditional algorithm without k-means clustering, which achieves an accuracy of 88%. Additional studies reveal that when incorporating pose estimation into the pick-place operations, the proposed algorithm's accuracy is 94%. The demonstration highlights the potential of leveraging machine learning algorithms and computer vision from the camera to perform flexible pick-place operations via socket communication.

Object Localization Algorithm Based on Meta-Reinforcement Learning

Abstract When the target localization algorithm based on reinforcement learning is trained on few-sample data sets, the accuracy of target localization is low due to the low degree of fitting. Therefore, on the basis of deep reinforcement learning target localization algorithm, this paper proposes a target localization algorithm based on meta-reinforcement learning. Firstly, during the initial training of the model, the meta-parameters were classified and stored according to the similarity of the training tasks. Then, for the new target location task, the task feature extraction was carried out and the meta parameters with the highest similarity were matched as the initial parameters of the model training. The model dynamically updated the meta parameter pool to ensure that the optimal meta parameters of multiple different types of features were saved in the meta parameter pool, so as to improve the generalization ability and recognition accuracy of multiple types of target location tasks. Experimental results show that in a variety of single target localization tasks, compared with the original reinforcement learning target localization algorithm, under the same data set size, the model converges under a small number of training steps with the meta-parameters in the matching meta-parameter pool as the initial training parameters. Moreover, the training speed of the meta-reinforcement learning method based on MAML-RL is increased by 28.2% for random initial parameters, and that of the meta-reinforcement learning method based on this paper is increased by 34.9%, indicating that the proposed algorithm effectively improves the training speed, generalization performance and localization accuracy of object detection.

Improved Discriminative Object Localization Algorithm for Safety Management of Indoor Construction.

Object localization is a sub-field of computer vision-based object recognition technology that identifies object classes and locations. Studies on safety management are still in their infancy, particularly those aimed at lowering occupational fatalities and accidents at indoor construction sites. In comparison to manual procedures, this study suggests an improved discriminative object localization (IDOL) algorithm to aid safety managers with visualization to improve indoor construction site safety management. The IDOL algorithm employs Grad-CAM visualization images from the EfficientNet-B7 classification network to automatically identify internal characteristics pertinent to the set of classes evaluated by the network model without the need for further annotation. To evaluate the performance of the presented algorithm in the study, localization accuracy in 2D coordinates and localization error in 3D coordinates of the IDOL algorithm and YOLOv5 object detection model, a leading object detection method in the current research area, are compared. The comparison findings demonstrate that the IDOL algorithm provides a higher localization accuracy with more precise coordinates than the YOLOv5 model over both 2D images and 3D point cloud coordinates. The results of the study indicate that the IDOL algorithm achieved improved localization performance over the existing YOLOv5 object detection model and, thus, is able to assist with visualization of indoor construction sites in order to enhance safety management.

Object detection and localization algorithm in agricultural scenes based on YOLOv5

Object detection and target localization are important technologies in image processing and computer vision, which have a wide range of applications in agricultural systems. By fusing the YOLOv5 algorithm and monocular vision-based method, a target localization algorithm is proposed to accurately identify and locate the various objects in agricultural scenes. The GPS information of the target object can be obtained by means of processing the attitude angle information of the unmanned aerial vehicle (UAV) at the time when it takes pictures. The superiority of the proposed method is demonstrated through the test on the agricultural scenario dataset taken by UAV, with the experimental results showing the algorithm’s satisfactory speed and accuracy.

TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge

AbstractFor decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems, like, disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground for new technology at the limits of the autonomous systems capabilities. This paper outlines the software stack and approach of the TUM Autonomous Motorsport team for their participation in the IAC, which holds two competitions: A single‐vehicle competition on the Indianapolis Motor Speedway and a passing competition at the Las Vegas Motor Speedway. Nine university teams used an identical vehicle platform: A modified Indy Lights chassis equipped with sensors, a computing platform, and actuators. All the teams developed different algorithms for object detection, localization, planning, prediction, and control of the race cars. The team from Technical University of Munich (TUM) placed first in Indianapolis and secured second place in Las Vegas. During the final of the passing competition, the TUM team reached speeds and accelerations close to the limit of the vehicle, peaking at around and . This paper will present details of the vehicle hardware platform, the developed algorithms, and the workflow to test and enhance the software applied during the 2‐year project. We derive deep insights into the autonomous vehicle's behavior at high speed and high acceleration by providing a detailed competition analysis. On the basis of this, we deduce a list of lessons learned and provide insights on promising areas of future work based on the real‐world evaluation of the displayed concepts.

An efficient regression method for 3D object localization in machine vision systems

<p>Machine vision or robot vision plays is playing an important role in many industrial systems and has a lot of potential applications in the future of automation tasks such as in-house robot managing, swarm robotics controlling, product line observing, and robot grasping. One of the most common yet challenging tasks in machine vision is 3D object localization. Although several works have been introduced and achieved good results for object localization, there is still room to further improve the object location determination. In this paper, we introduce a novel 3D object localization algorithm in which a checkerboard pattern-based method is used to initialize the object location and followed by a regression model to regularize the object location. The proposed object localization is employed in a low-cost robot grasping system where only one simple 2D camera is used. Experimental results showed that the proposed algorithm significantly improves the accuracy of the object localization when compared to the relevant works.</p>

A Convolution Location Method for Multi-Node Scheduling in Wireless Sensor Networks

The localization of continuous objects and the scheduling of resources are challenging issues in wireless sensor networks (WSNs). Due to the irregular shape of the continuous target area and the sensor deployment in WSNs, the sensor data are always discrete and sparse, and most network resources are also limited by the node energy. To achieve faster detection and tracking of continuous objects, we propose a convolution-based continuous object localization algorithm (named CCOL). Moreover, we implement the idea of greedy and dynamic programming to design an energy-saving and efficient strategy model (named MSSM) to respond to emergencies caused by multiple continuous targets in most specific WSNs. The simulation experiments demonstrate that CCOL is superior to other localization algorithms in terms of time complexity and execution performance. Furthermore, the feasibility of the multinode scheduling strategy is verified by setting different mobile nodes to respond to the target area in certain green WSNs.

Multipath Propagation of Acoustic Signal in a Swimming Pool-Source Localization Problem.

This paper explores the problem of severe multipath propagation of underwater acoustic signals in a swimming pool. The problem appeared in a study that examined a system used to signal emergency situations (i.e., pre-drowning symptoms detected by a wearable device on a pool user’s wrist) and locate the signal source. A swimming pool acoustic environment is characterized by the presence of large flat reflecting planes surrounding a small volume of water. The reflections are numerous and much stronger than in typical hydroacoustic applications. In this paper, we attempted to create a model of the swimming pool response, one that is suitable for simulation experiments with detection and localization of emergency signals. Then, we explore the possible remedies for the localization system, applied on the transmit side (waveform design) and on the receive side (receiver placement and signal processing). Finally, we present an algorithm for object localization, considering the possible reflections with a multi-hypothesis approach.

Learning-based object detection and localization for a mobile robot manipulator in SME production

Increasing research attention has been attracted to automatic production processes in small and medium-sized enterprises (SMEs) using collaborative robotic systems. In this work, we develop an object detection solution based on deep learning on 3D point clouds for a collaborative mobile robot manipulator to automate SME production. In this solution, a 3D point cloud technology is adopted to measure the shape and depth information of targeted objects in SME production, for instance, name tags production and plug-in charging. Deep learning is then employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions and the irregular distribution and structural ambiguity of point clouds. A 2D camera is employed to calibrate the relative positions of the mobile manipulator to workstations. The mobile robot manipulator is equipped with cameras, an in-house developed adaptive gripper, and a learning-based computer vision system developed in this work. The principle and procedures of the proposed 3D object detection and 2D calibration are presented in detail. The automatic name tags production and plug-in charging experiments are conducted to validate the object detection, localization algorithms, and tools developed and employed in production cases using the mobile robot manipulator.

Geospatial Data Mining, Computer Vision-based Lane Detection and Object Localization Algorithms, and Mapping and Navigation Tools in Smart Urban Mobility Systems

Geospatial Data Mining, Computer Vision-based Lane Detection and Object Localization Algorithms, and Mapping and Navigation Tools in Smart Urban Mobility Systems

Detecting sheep in UAV images

In the last decade, researchers have focused more on deep convolutional neural networks (CNNs) than other machine learning algorithms for object detection, localization, classification and segmentation. Such CNNs have achieved remarkable results in these fields and use the bounding boxes as the ground truth data. In this research article, we have used a fully connected network (FCN) for livestock detection in aerial images captured by an unmanned aerial vehicle (UAV), that used centroids as ground truth data. For performance evaluation and comparison, we have proposed a single-layered and a seven-layered CNN network in this article. These proposed networks are trained using state-of-the-art method, Region-based CNN. In addition, AlexNet, GoogLeNet, VGG16, VGG19 and ResNet50 were also fine-tuned for livestock detection. The results of the FCN and one of our proposed networks are then merged to improve the recall of the complete system from 90% to 98%.

Real-Time Object Detection and Localization for Vision-Based Robot Manipulator

Traditionally, the robot manipulators were assigned a monotonous task. Therefore, in an aim of simplifying the interactions between unstructured surrounding and robots, as well as to carry out the more complex operations, cognitive abilities such as visual perception are appended to systems for the sake of intelligent operations. Hence, this article directed towards the multistage process involved in the design and development of vision-based robot manipulator suitable for pick and place operation in real-time entailing external disturbances. The intact system development occurs in three phases: first, the robot manipulator with 3-DOF is designed, developed and also enhanced by integrating the vision source. Second, an algorithm for object detection and localization in real time is framed in which segmented object matrices are returned by the former and the position of the stabilized object is returned by the latter. Third, entire hardware–software integration is achieved to perform the desired operation. The algorithm developed in this paper proves to be good for the developed vision-based manipulator, as we achieve quite a good accuracy in object detection algorithm whereas 98.41–99.12% accuracy is achieved in the object localization algorithm.

A Low Power and Real-Time Architecture for Hough Transform Processing Integration in a Full HD-Wireless Capsule Endoscopy.

We propose a new paradigm of a smart wireless endoscopic capsule (WCE) that has the ability to select suspicious images containing a polyp before sending them outside the body. To do so, we have designed an image processing system to select images with Regions Of Interest (ROI) containing a polyp. The criterion used to select an ROI is based on the polyp's shape. We use the Hough Transform (HT), a widely used shape-based algorithm for object detection and localization, to make this selection. In this paper, we present a new algorithm to compute in real-time the Hough Transform of high definition images (1920 x 1080pixels). This algorithm has been designed to be integrated inside a WCE where there are specific constraints: a limited area and a limited amount of energy. To validate our algorithm, we have realized tests using a dataset containing synthetic images, real images, and endoscopic images with polyps. Results have shown that our algorithm is capable to detect circular shapes in synthetic and real images, but also can detect circles with an irregular contour, like that of polyps. We have implemented our architecture and validated it in a Xilinx Spartan 7 FPGA device, with an area of [Formula: see text], which is compatible with integration inside a WCE. This architecture runs at 132MHz with an estimated power consumption of 76mW and can work close to 10hours. To improve the capacity of our architecture, we have also made an ASIC estimation, that let our architecture work at 125MHz, with a power consumption of only 17.2mW and a duration of approximately 50hours.

Conditional Random Field based salient proposal set generation and its application in content aware seam carving

Most of the existing generic object localization algorithms usually give the plausible object locations without taking into consideration the saliency ordering of the proposal set. This paper presents a novel object proposal generation which ranks the key objects according to their saliency score in the proposal pool. First, we formulate a Bayesian framework for generating a probabilistic edgemap which is used to assign a saliency value to the edgelets. A conditional random field is then learnt for edge-labeling by effectively combining the edge features with the relative spatial layout of the edge segments. Lastly, we propose an objectness score for the generated proposal set by analyzing the salient object edge density completely lying within the candidate boxes. Extensive experiments on the benchmark PASCAL VOC 2007 and 2012 datasets demonstrate that the proposed method provides competitive performance against popular generic object detection techniques while using fewer number of proposals. Additionally, we demonstrate the applicability of the generated proposal set for content aware image retargeting.

Augmented Reality on Sudoku Puzzle using Computer Vision and Deep Learning

Many people try solving Sudoku puzzles everyday. These puzzles are usually found in newspapers, magazines and so on. Whenever a person is unable to solve a puzzle or is running short on time to solve the puzzle, it will be very convenient to show the solved puzzle as an augmented reality. Objectives: In this paper, we propose an optimal way of recognizing a Sudoku puzzle using computer vision and Deep Learning, and solve the puzzle using constraint programming and backtracking algorithm to display the solved puzzle as augmented reality. Also, a comparative performance analysis with the previous work is provided with this paper. Methods: In order to implement augmented reality on to the Sudoku puzzle, image classification itself won’t be sufficient as the solved puzzle has to be shown on top of the area of the unsolved puzzle in the original image. So puzzle detection has to be performed and for doing so we used CNN and Object Localization algorithms. After the detection we stored the values detected in each 9x9 cells and ran a constraint programming and backtracking algorithm to solve the puzzle and finally filled the detected empty cells with correct values of the solved puzzle. Applications/Improvements: Usually the Sudoku puzzles that we find in newspapers and magazines are surrounded by a lot of noise such as text (characters) irrelevant to the puzzle and borders of the newspaper which could be similar to a Sudoku puzzle structure. In this paper we emphasise on how to handle such disturbances and improve the performance.

Dexterous Gripper for In-Hand Manipulation with Embedded Object Localization Algorithm

Since the last decade, thanks to the spreading of the concept of Industry 4.0 and Smart Factory, more and more companies have started to investigate the robotic field looking for reliable solutions aiming at improving the efficiency of assembly lines.Promising technologies are connected to the speeding up of production stages like fast algorithms for object localization, as well as dexterous grippers for manipulation and assembly. Nowadays, most of the solutions for pick and place tasks involve the use of robotic grippers for grasping objects, while robotic manipulators are responsible for their accurate placements. Focusing on the grippers, although their simple structure can be appreciated, it greatly reduces their in-hand manipulation abilities, making unfeasible the twists of grasped objects and their release in a desired pose. As consequence, the efficiency of the pick and place operation is reduced since several adjustments of the robotic arm are required to accomplish the task.In this paper, a novel dexterous gripper coupled with a vision system algorithm for object localization and pose estimation are presented, and their performances in manipulating different objects are discussed. The designed gripper has a symmetrical structure with two finger modules, each one consisting in a couple of linear actuators arranged mutually orthogonal, so the translations in two axis, namely y and z directions, are allowed. As terminal part of each finger there is a revolute joint to whom is attached a fingertip modelled according to the shape of the target objects and easily replaceable. The embedded vision system algorithm adopted estimates position and orientation of the objects on a flat surface, and it coordinates the gripper placement to grasp them. The case study of the handling of a Spanish fan is presented and discussed in details.

Sub-OBB based object recognition and localization algorithm using range images

This paper presents a novel approach to recognize and estimate pose of the 3D objects in cluttered range images. The key technical breakthrough of the developed approach can enable robust object recognition and localization under undesirable condition such as environmental illumination variation as well as optical occlusion to viewing the object partially. First, the acquired point clouds are segmented into individual object point clouds based on the developed 3D object segmentation for randomly stacked objects. Second, an efficient shape-matching algorithm called Sub-OBB based object recognition by using the proposed oriented bounding box (OBB) regional area-based descriptor is performed to reliably recognize the object. Then, the 3D position and orientation of the object can be roughly estimated by aligning the OBB of segmented object point cloud with OBB of matched point cloud in a database generated from CAD model and 3D virtual camera. To detect accurate pose of the object, the iterative closest point (ICP) algorithm is used to match the object model with the segmented point clouds. From the feasibility test of several scenarios, the developed approach is verified to be feasible for object pose recognition and localization.

Moving Object Detection, Localization and Tracking Using Stereo Vison System

The aim of this study was to design an moving object detection, localization and tracking algorithm able to detect, localize and track especially humans and vehicles. We focused on triangulation techniques to calculate the position of the detected objects in a stereo vision rig coordinates frame. For objects detection and tracking the novel algorithm, based on statistical image processing methods, was proposed. Verification of a proper operation of the elaborated method was made by conducting series of experiments. Our results indicate that the algorithm localizes, detects and tracks objects accurately for the most tested conditions.