Location Of Target Object Research Articles

Automated Recognition of Submerged Body-like Objects in Sonar Images Using Convolutional Neural Networks

The Police Robot for Inspection and Mapping of Underwater Evidence (PRIME) is an uncrewed surface vehicle (USV) currently being developed for underwater search and recovery teams to assist in crime scene investigation. The USV maps underwater scenes using sidescan sonar (SSS). Test exercises use a clothed mannequin lying on the seafloor as a target object to evaluate system performance. A robust, automated method for detecting human body-shaped objects is required to maximise operational functionality. The use of a convolutional neural network (CNN) for automatic target recognition (ATR) is proposed. SSS image data acquired from four different locations during previous missions were used to build a dataset consisting of two classes, i.e., a binary classification problem. The target object class consisted of 166 196 × 196 pixel image snippets of the underwater mannequin, whereas the non-target class consisted of 13,054 examples. Due to the large class imbalance in the dataset, CNN models were trained with six different imbalance ratios. Two different pre-trained models (ResNet-50 and Xception) were compared, and trained via transfer learning. This paper presents results from the CNNs and details the training methods used. Larger datasets are shown to improve CNN performance despite class imbalance, achieving average F1 scores of 97% in image classification. Average F1 scores for target vs background classification with unseen data are only 47% but the end result is enhanced by combining multiple weak classification results in an ensemble average. The combined output, represented as a georeferenced heatmap, accurately indicates the target object location with a high detection confidence and one false positive of low confidence. The CNN approach shows improved object detection performance when compared to the currently used ATR method.

Grasp with push policy for multi-finger dexterity hand based on deep reinforcement learning

When interacting with the external environment, dexterous hands face various challenges, including interference in cluttered environments and difficulty in accurately locating target objects. We observe that the human hand is often at ease in the face of complex unstructured environments: pushing objects that can rearrange clutter, locating target objects, and creating space for fingers. In addition, the gripping action complements the push by achieving the precise movement of unrelated objects. Taking inspiration from this, to enable manipulators to perform accurate grasping tasks in complex environments, we propose a new data-driven grasping approach for robotic dexterous hands: the push-fit grasping approach. This method combines human grasping with model-free deep reinforcement learning to realize the robot's collaborative grasping ability. An end-to-end conditional converter grasping network is first trained, which takes the visual point cloud input to the action output. We use DQN algorithms for strategy training, further enhanced by fine-tuning methods for real-world learning. The experimental results show that the push strategy we learned significantly improves the grasping performance, and the success rate of cooperative grasping is increased by 8 %. It is worth noting that this fine-tuning method significantly reduces the actual training time, with the real world training taking up only one-fifth of the time in the simulated world. Even in demanding scenarios characterized by confusion and complexity, it facilitates rapid learning. This study provides new insights for effectively addressing complex challenges involving multi-fingered dexterous hands and human-computer interaction. Our code is available in the github repository.

Knowing where to go: Spatial memory guides eye and body movements in a naturalistic visual search task.

Most research on visual search has used simple tasks presented on a computer screen. However, in natural situations visual search almost always involves eye, head, and body movements in a three-dimensional (3D) environment. The different constraints imposed by these two types of search tasks might explain some of the discrepancies in our understanding concerning the use of memory resources and the role of contextual objects during search. To explore this issue, we analyzed a visual search task performed in an immersive virtual reality apartment. Participants searched for a series of geometric 3D objects while eye movements and head coordinates were recorded. Participants explored the apartment to locate target objects whose location and visibility were manipulated. For objects with reliable locations, we found that repeated searches led to a decrease in search time and number of fixations and to a reduction of errors. Searching for those objects that had been visible in previous trials but were only tested at the end of the experiment was also easier than finding objects for the first time, indicating incidental learning of context. More importantly, we found that body movements showed changes that reflected memory for target location: trajectories were shorter and movement velocities were higher, but only for those objects that had been searched for multiple times. We conclude that memory of 3D space and target location is a critical component of visual search and also modifies movement kinematics. In natural search, memory is used to optimize movement control and reduce energetic costs.

Clustering-inspired channel selection method for weakly supervised object localization

Weakly Supervised Object Localization (WSOL) aims to utilize the features learned by a classifier on the image-level labels to locate target objects. However, these existing channel selection methods for WSOL still cannot effectively select the important channels and remove the unimportant ones. To address this issue, we propose a Clustering-inspired Channel Selection method based on Class Activation Maps (CCS-CAM). Compared with the traditional methods, the advantage of CCS-CAM is that it is very simple yet effective for channel selection due to the K-means clustering based on Class Activation Maps. It can effectively ensure both object localization and classification accuracy. The effectiveness of the proposed CCS-CAM method has been demonstrated using multiple public datasets, with GT-Know Loc reaching 87.9% and 63.71% on the CUB200-2011 and ImageNet-1k respectively, which is superior to the other state-of-the-art methods.

End-to-end Visual Grounding Based on Query Text Guidance and Multi-stage Reasoning

<p>Visual grounding locates target objects or areas in the image based on natural language expression. Most current methods extract visual features and text embeddings independently, and then carry out complex fusion reasoning to locate target objects mentioned in the query text. However, such independently extracted visual features often contain many features that are irrelevant to the query text or misleading, thus affecting the subsequent multimodal fusion module, and deteriorating target localization. This study introduces a combined network model based on the transformer architecture, which realizes more accurate visual grounding by using query text to guide visual feature generation and multi-stage fusion reasoning. Specifically, the visual feature generation module reduces the interferences of irrelevant features and generates visual features related to query text through the guidance of query text features. The multi-stage fused reasoning module uses the relevant visual features obtained by the visual feature generation module and the query text embeddings for multi-stage interactive reasoning, further infers the correlation between the target image and the query text, so as to achieve the accurate localization of the object described by the query text. The effectiveness of the proposed model is experimentally verified on five public datasets and the model outperforms state-of-the-art methods. It achieves an improvement of 1.04%, 2.23%, 1.00% and +2.51% over the previous state-of-the-art methods in terms of the top-1 accuracy on TestA and TestB of the RefCOCO and RefCOCO+ datasets, respectively.</p> <p> </p>

Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots

Home service robots operating indoors, such as inside houses and offices, require the real-time and accurate identification and location of target objects to perform service tasks efficiently. However, images captured by visual sensors while in motion states usually contain varying degrees of blurriness, presenting a significant challenge for object detection. In particular, daily life scenes contain small objects like fruits and tableware, which are often occluded, further complicating object recognition and positioning. A dynamic and real-time object detection algorithm is proposed for home service robots. This is composed of an image deblurring algorithm and an object detection algorithm. To improve the clarity of motion-blurred images, the DA-Multi-DCGAN algorithm is proposed. It comprises an embedded dynamic adjustment mechanism and a multimodal multiscale fusion structure based on robot motion and surrounding environmental information, enabling the deblurring processing of images that are captured under different motion states. Compared with DeblurGAN, DA-Multi-DCGAN had a 5.07 improvement in Peak Signal-to-Noise Ratio (PSNR) and a 0.022 improvement in Structural Similarity (SSIM). An AT-LI-YOLO method is proposed for small and occluded object detection. Based on depthwise separable convolution, this method highlights key areas and integrates salient features by embedding the attention module in the AT-Resblock to improve the sensitivity and detection precision of small objects and partially occluded objects. It also employs a lightweight network unit Lightblock to reduce the network’s parameters and computational complexity, which improves its computational efficiency. Compared with YOLOv3, the mean average precision (mAP) of AT-LI-YOLO increased by 3.19%, and the detection precision of small objects, such as apples and oranges and partially occluded objects, increased by 19.12% and 29.52%, respectively. Moreover, the model inference efficiency had a 7 ms reduction in processing time. Based on the typical home activities of older people and children, the dataset Grasp-17 was established for the training and testing of the proposed method. Using the TensorRT neural network inference engine of the developed service robot prototype, the proposed dynamic and real-time object detection algorithm required 29 ms, which meets the real-time requirement of smooth vision.

Sliding Window, Hierarchical Classification, Regression, and Genetic Algorithm for RFID Indoor Positioning Systems

Several surveys in the Indoor Positioning System (IPS) use the Radio Signal Strength Indicator (RSSI) to locate target objects. However, RSSI suffers significantly from the multipath phenomenon and other environmental factors such as the absence of the field of view, reflective materials, excessive obstacles, and a high density of objects. To overcome such problems, we propose three new IPS approaches using Radio Frequency Identification (RFID) technology and the collaborative use of Sliding Window, Hierarchical Classification, Random Forest Regressor, and Genetic Algorithm. We tested the new approaches on three datasets obtained in different environments and compared them with two state-of-the-art IPS approaches (Landmarc and SVR-Landmarc). One of the main results got was a precision of 4.82 cm in an environment with 7,000 target tags, reducing by up to 92.91% the average error w.r.t. other IPS approaches presented in the literature, including more complex indoor scenarios.

A Design of Obstacle Avoidance and Grabbing Manipulator Based on Computer Vision for Warehouse Inspection Robot

The automatic manipulator has several potential applications in industrial systems. It is critical to research machine route planning and obstacle avoidance in order to fulfill the rising technical requirements. This work provides a technique for realizing the aforementioned tasks based on an enhanced obstacle avoidance algorithm that can automatically skip barriers and complete target sorting using visual identification and improved obstacle avoidance algorithms. The whole system consists of three parts: image collection, pose estimation and target grasping. Firstly, computer vision technology is used to recognize and locate target objects. Secondly, the robot's kinematics model is created, and combined with the visual perception information, the target grabbing and sorting are completed. Finally, the environment is built with Gazebo, and the robot is simulated. Simulation results verify the feasibility and effectiveness of the method.

An efficient single shot detector with weight-based feature fusion for small object detection

Object detection has been widely applied in various fields with the rapid development of deep learning in recent years. However, detecting small objects is still a challenging task because of the limited information in features and the complex background. To further enhance the detection accuracy of small objects, this paper proposes an efficient single-shot detector with weight-based feature fusion (WFFA-SSD). First, a weight-based feature fusion block is designed to adaptively fuse information from several multi-scale feature maps. The feature fusion block can exploit contextual information for feature maps with large resolutions. Then, a context attention block is applied to reinforce the local region in the feature maps. Moreover, a pyramids aggregation block is applied to combine the two feature pyramids to classify and locate target objects. The experimental results demonstrate that the proposed WFFA-SSD achieves higher mean Average Precision (mAP) under the premise of ensuring real-time performance. WFFA-SSD increases the mAP of the car by 4.12% on the test set of the CARPK.

Region Probability Map-Guided Fast Wide-Area Multiobject Detection

Detecting multiobjects in a wide-area scenario efficiently is a critical technology for industry and security applications. The detection performance has benefited enormously from the probability estimation of the unknown environment. Representative methods, such as particle filter (PF), construct the probability distribution model and iteratively locate target objects. However, randomly sampling and detecting image patches in the large covering field-of-view (FOV) make these methods inefficient and computation costly. To address these issues, we propose a region probability map (RPM)-guided fast wide-area detection system that can simultaneously detect multiobjects from a large FOV at 300 frames per second (fps) through a coarse-to-fine-grained detection paradigm. Specifically, a segmentation-based RPM generation module is introduced to assign probability measurements to different regions of the panoramic image, which models how likely the desired objects will occur in these regions. Then, based on the generated probability map of the whole scene, a novel RPM-guided PF framework is proposed to speed up the detection process by concentrating the detection on high-probability areas. Finally, a rapid and low-latency active detection system based on a wide-angle camera, a high-speed camera, and an ultrafast galvano mirror is implemented, which gains a 15.38% efficiency improvement while achieving more accurate detection compared with existing methods. Extensive experimental results verify the robustness and effectiveness of our proposed system.

Mobile Robot Navigation Using Deep Reinforcement Learning

Learning how to navigate autonomously in an unknown indoor environment without colliding with static and dynamic obstacles is important for mobile robots. The conventional mobile robot navigation system does not have the ability to learn autonomously. Unlike conventional approaches, this paper proposes an end-to-end approach that uses deep reinforcement learning for autonomous mobile robot navigation in an unknown environment. Two types of deep Q-learning agents, such as deep Q-network and double deep Q-network agents are proposed to enable the mobile robot to autonomously learn about collision avoidance and navigation capabilities in an unknown environment. For autonomous mobile robot navigation in an unknown environment, the process of detecting the target object is first carried out using a deep neural network model, and then the process of navigation to the target object is followed using the deep Q-network or double deep Q-network algorithm. The simulation results show that the mobile robot can autonomously navigate, recognize, and reach the target object location in an unknown environment without colliding with static and dynamic obstacles. Similar results are obtained in real-world experiments, but only with static obstacles. The DDQN agent outperforms the DQN agent in reaching the target object location in the test simulation by 5.06%.

Robust Object Positioning for Visual Robotics in Automatic Assembly Line under Data-Scarce Environments

Object positioning is a basic need for visual robotics in automatic assembly lines. An assembly line requires fast transfer to new object positioning tasks with few or no training data for deep learning algorithms, and the captured visual images usually suffer from partial missing and cropping and environmental lighting interference. These features call for efficient and robust arbitrary shape positioning algorithms under data-scarce and shape distortion cases. To this end, this paper proposes the Random Verify Generalised Hough Transform (RV-GHT). The RV-GHT builds a much more concise shape dictionary than traditional GHT methods with just a single training image. The location, orientation, and scaling of multiple target objects are given simultaneously during positioning. Experiments were carried out on a dataset in an automatic assembly line with real shape distortions, and the performance was improved greatly compared to the state-of-the art methods. Although the RV-GHT was initially designed for vision robotics in an automatic assembly line, it works for other object positioning mechatronics systems, which can be modelled as shape distortion on a standard reference object.

A development of visual-feedback automatic control for robotic manipulator

In this work, an automatic robotic manipulator control based on visual information for object tracking is presented. The tracking process employ Recurrent Neural Network and Kalman filter techniques to predict the location of target object which is used to calculate movement of the robot. A manipulator control process was developed to provide consistent motion and reduce vibration. We also propose an object analysis based on image processing technique for object size determination. We implemented the proposed system with an actual robotic manipulator with a gripper for experiments. The experiments shown that object size estimation method results the maximum accuracy at 1.13%. The proposed object tracking and manipulator control process achieved vibration reduction 3.05 times compared to simple control method and reduced the time of end-effector reaching the target object by 1.29sec.

Functional Specificity of the Affordance of Reaching.

The information that specifies whether an object is within reach is a complex pattern that depends on body-scaled parameters measured from an egocentric reference point. The pattern is a function of relevant body proportions (eye height, shoulder height [SH], arm length) with respect to the spatial location of the target object. In addition to not knowing how these factors map onto perception, it is also not known whether the egocentric viewpoint is centered at the eye or the shoulder. In three experiments, we systematically tested whether observers can perceive eye height and SH (Experiment 1), whether they can point accurately in the direction of a target object (Experiment 2), and whether they can point accurately to judge if the target object is within reach (Experiment 3). Experiment 1 demonstrated that participants are more accurate at judging their own eye height than SH. Experiment 2 revealed that participants can more accurately point to a target object's location when measured from the shoulder as a reference point than when measured from the eye. In Experiment 3, we showed that a higher-order variable that includes arm length, body height, and angle of declination to the target successfully predicted affordance judgments, regardless of a reference point. We consider this as evidence that the invariant is functionally specific, not tied to any one particular anatomical body part.

Mechanical Particle Filter-Based Active Vision System for Fast Wide-Area Multiobject Detection

Sliding window searching is a common paradigm employed for large Field-of-View (FOV) multi-object detection. However, low target resolution and irregular distribution of objects make the traditional method inefficient and hard to meet the demands of fast object detection in a wide area. To address the above issues, in this paper we design an ultrafast switching galvano-mirror based active vision system that can simultaneously search and detect multi-object at 300 frames per second (fps) in a large FOV, without the guidance of a high-resolution (HR) panoramic camera. Specifically, a novel mechanical particle filter (MPF) framework is proposed to improve the efficiency of searching potential objects by constructing the probability distribution model and then iteratively locating target objects, where each particle is represented by an image of a single small FOV that is controlled and captured by a switching galvano-mirror. The object detector is used to determine if the particle image contains target objects. Then, to reduce the total scanning cost for capturing all particles, we introduce a center partitioned scanning algorithm that significantly speeds up the scanning process by 10× or more. Moreover, we optimize the processing pipeline for low latency sensing by using a high-parallelism visual feedback scheme to improve efficiency. Finally, an efficient and fast active vision system is implemented, which is based on a 2-axis galvano-mirror and a high-speed camera without multi-camera fusion for active object searching. Abundant experimental results demonstrate the effectiveness of our proposed system in indoor environments and real-world scenarios. We also verify the performance in terms of scanning speed and detection accuracy.

Behavioral synergic relations between eye and postural movements in young adults searching to locate objects in room inside houses.

During precise gaze shifts, eye, head, and body movements exhibit synergic relations. In the present study, we tested the existence of behavioural synergic relations between eye and postural movements in a goal-directed, precise, visual search task (locate target objects in large images). More precisely, we tested if postural control could be adjusted specifically to facilitate precise gaze shifts. Participants also performed a free-viewing task (gaze images with no goal) and a fixation task. In both search and free-viewing tasks, young participants (n = 20; mean age = 22years) were free to move their eyes, head, and body segments as they pleased to self-explore the images with no external perturbation. We measured eye and postural kinematic movements. The results showed significant negative correlations between eye and postural (head and upper back) movements in the precise task, but not in the free-viewing task. The negative correlations were considered to be stabilizing and synergic. Indeed, the further the eyes moved, the more postural variables were adjusted to reduce postural sway. These results suggest that postural control was adjusted to succeed in subtle and active self-induced precise gaze shifts. Furthermore, partial correlations showed significant relations between (1) task performance to find target objects and (2) synergic relations between eye and postural movements. These later results tend to show that synergic eye-postural relations were performed to improve the task performance in the precise visual task.

Construction of multi-modal perception model of communicative robot in non-structural cyber physical system environment based on optimized BT-SVM model

With the rapid development of intelligent control technology, computer technology, bionics, artificial intelligence and other disciplines, more and more attention has been paid to the research of intelligent mobile robot technology, and autonomous positioning is the basis for mobile robots to conduct autonomous navigation and exploration research. Sensors assist each other to provide a wealth of perception information about the internal state of the robot and the external surrounding environment. This paper proposes a method for optimizing the Support Vector Machine (SVM) multi-classifier with a binary tree structure, which improves the accuracy of multi-modal tactile signal recognition. The improved particle swarm clustering algorithm is used to optimize the binary tree structure, reduce the error accumulation of the binary tree structure SVM multi-classifier, and further improve the accuracy of multi-modal tactile signal recognition. The effect of the method in this paper is verified by robot grasping experiments. The results show that the use of multi-modal information of two-dimensional images and three-dimensional point cloud images can effectively identify and locate target objects of different shapes. Compared with the processing method of two-dimensional or point cloud monomodal image information, the positioning error can be reduced by 54.8%, and the direction error can be reduced by 50.8%, which has better robustness and accuracy. The simulation results show that the improved PSOBT-SVM model has the best classification effect for artificial features, PCA features and spatio-temporal correlation features. The improved PSOBT-SVM optimizes the classification accuracy without changing the number of SVM classifiers, and proves its accuracy in classifying multimodal tactile signals.

Meaning and expected surfaces combine to guide attention during visual search in scenes.

How do spatial constraints and meaningful scene regions interact to control overt attention during visual search for objects in real-world scenes? To answer this question, we combined novel surface maps of the likely locations of target objects with maps of the spatial distribution of scene semantic content. The surface maps captured likely target surfaces as continuous probabilities. Meaning was represented by meaning maps highlighting the distribution of semantic content in local scene regions. Attention was indexed by eye movements during the search for target objects that varied in the likelihood they would appear on specific surfaces. The interaction between surface maps and meaning maps was analyzed to test whether fixations were directed to meaningful scene regions on target-related surfaces. Overall, meaningful scene regions were more likely to be fixated if they appeared on target-related surfaces than if they appeared on target-unrelated surfaces. These findings suggest that the visual system prioritizes meaningful scene regions on target-related surfaces during visual search in scenes.

Detection method of robot grasp based on lightweight network

When the robot arm uses the suction cup to grasp the task, it is faced with an unstructured scene, and it is difficult to accurately calculate the grasping posture of the robot due to the irregular placement of the object and its irregular shape. To solve this problem, a grasping detection method of manipulator based on lightweight convolutional neural network was proposed. Firstly, the Mobile Net-YOLOV4 algorithm based on lightweight convolutional neural network was used to detect the target object in the image, and the classification and location information of the target were obtained. Then according to the final detection results of the image threshold segmentation, the anchor point is corrected, and finally the corrected positioning result is obtained. The grasping experiment was carried out on the Probot anno manipulator platform. The experimental results show that, compared with other image processing methods, the proposed method can realize fast detection and location of irregular target objects, and has better robustness for the diversity of object morphology and environment.

WGI-Net: A weighted group integration network for RGB-D salient object detection

Salient object detection is used as a pre-process in many computer vision tasks (such as salient object segmentation, video salient object detection, etc.). When performing salient object detection, depth information can provide clues to the location of target objects, so effective fusion of RGB and depth feature information is important. In this paper, we propose a new feature information aggregation approach, weighted group integration (WGI), to effectively integrate RGB and depth feature information. We use a dual-branch structure to slice the input RGB image and depth map separately and then merge the results separately by concatenation. As grouped features may lose global information about the target object, we also make use of the idea of residual learning, taking the features captured by the original fusion method as supplementary information to ensure both accuracy and completeness of the fused information. Experiments on five datasets show that our model performs better than typical existing approaches for four evaluation metrics.