Localization Of Target Objects Research Articles

Transient Electromagnetic Monitoring of Permafrost: Mathematical Modeling Based on Sumudu Integral Transform and Artificial Neural Networks

Due to the ongoing global warming on the Earth, permafrost degradation has been extensively taking place, which poses a substantial threat to civil and industrial facilities and infrastructure elements, as well as to the utilization of natural resources in the Arctic and high-latitude regions. In order to prevent the negative consequences of permafrost thawing under the foundations of constructions, various geophysical techniques for monitoring permafrost have been proposed and applied so far: temperature, electrical, seismic and many others. We propose a cross-borehole exploration system for a high localization of target objects in the cryolithozone. A novel mathematical apparatus for three-dimensional modeling of transient electromagnetic signals by the vector finite element method has been developed. The original combination of the latter, the Sumudu integral transform and artificial neural networks makes it possible to examine spatially heterogeneous objects of the cryolithozone with a high contrast of geoelectric parameters, significantly reducing computational costs. We consider numerical simulation results of the transient electromagnetic monitoring of industrial facilities located on permafrost. The formation of a talik has been shown to significantly manifest itself in the measured electromagnetic responses, which enables timely prevention of industrial disasters and environmental catastrophes.

Vision- and Lidar-Based Autonomous Docking and Recharging of a Mobile Robot for Machine Tending in Autonomous Manufacturing Environments

Autonomous docking and recharging are among the critical tasks for autonomous mobile robots that work continuously in manufacturing environments. This requires robots to demonstrate the following abilities: (i) detecting the charging station, typically in an unstructured environment and (ii) autonomously docking to the charging station. However, the existing research, such as that on infrared range (IR) sensor-based, vision-based, and laser-based methods, identifies many difficulties and challenges, including lighting conditions, severe weather, and the need for time-consuming computation. With the development of deep learning techniques, real-time object detection methods have been widely applied in the manufacturing field for the recognition and localization of target objects. Nevertheless, those methods require a large amount of proper and high-quality data to achieve a good performance. In this study, a Hikvision camera was used to collect data from a charging station in a manufacturing environment; then, a dataset for the wireless charger was built. In addition, the authors of this paper propose an autonomous docking and recharging method based on the deep learning model and the Lidar sensor for a mobile robot operating in a manufacturing environment. In the proposed method, a YOLOv7-based object detection method was developed, trained, and evaluated to enable the robot to quickly and accurately recognize the charging station. Mobile robots can achieve autonomous docking to the charging station using the proposed Lidar-based approach. Compared to other methods, the proposed method has the potential to improve recognition accuracy and efficiency and reduce the computation costs for the mobile robot system in various manufacturing environments. The developed method was tested in real-world scenarios and achieved an average accuracy of 95% in recognizing the target charging station. This vision-based charger detection method, if fused with the proposed Lidar-based docking method, can improve the overall accuracy of the docking alignment process.

HA-FPN: Hierarchical Attention Feature Pyramid Network for Object Detection.

The goals of object detection are to accurately detect and locate objects of various sizes in digital images. Multi-scale processing technology can improve the detection accuracy of the detector. Feature pyramid networks (FPNs) have been proven to be effective in extracting multi-scaled features. However, most existing object detection methods recognize objects in isolation, without considering contextual information between objects. Moreover, for the sake of computational efficiency, a significant reduction in the channel dimension may lead to the loss of semantic information. This study explores the utilization of attention mechanisms to augment the representational power and efficiency of features, ultimately improving the accuracy and efficiency of object detection. The study proposed a novel hierarchical attention feature pyramid network (HA-FPN), which comprises two key components: transformer feature pyramid networks (TFPNs) and channel attention modules (CAMs). In TFPNs, multi-scaled convolutional features are embedded as tokens and self-attention is applied to across both the intra- and inter-scales to capture contextual information between the tokens. CAMs are employed to select the channels with rich channel information to alleviate massive channel information losses. By introducing contextual information and attention mechanisms, the HA-FPN significantly improves the accuracy of bounding box detection, leading to more precise identification and localization of target objects. Extensive experiments conducted on the challenging MS COCO dataset demonstrate that the proposed HA-FPN outperforms existing multi-object detection models, while incurring minimal computational overhead.

Deep Learning-based Mobile Robot Target Object Localization and Pose Estimation Research

Deep Learning-based Mobile Robot Target Object Localization and Pose Estimation Research

Multifunctional Robot Grasping System Based on Deep Learning and Image Processing

This paper firstly introduces the general architecture of the multifunctional harvesting robot grasping system; then, deep learning is used to investigate the object target recognition, and a set of target detection algorithm based on convolutional neural network is implemented; then, image processing technology is used to realize the function of target object localization, which can guide the multifunctional harvesting robot to complete the picking of the target multifunctional. The experimental results show that the multifunctional harvesting robot has a small calculation error of multifunctional coordinates and has a strong multifunctional recognition and positioning capability.

Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-Based Beam Search

To track the target in a video, current visual trackers usually adopt greedy search for target object localization in each frame, that is, the candidate region with the maximum response score will be selected as the tracking result of each frame. However, we found that this may be not an optimal choice, especially when encountering challenging tracking scenarios such as heavy occlusion and fast motion. In particular, if a tracker drifts, errors will be accumulated and would further make response scores estimated by the tracker unreliable in future frames. To address this issue, we propose to maintain multiple tracking trajectories and apply beam search strategy for visual tracking, so that the trajectory with fewer accumulated errors can be identified. Accordingly, this paper introduces a novel multi-agent reinforcement learning based beam search tracking strategy, termed BeamTracking. It is mainly inspired by the image captioning task, which takes an image as input and generates diverse descriptions using beam search algorithm. Accordingly, we formulate the tracking as a sample selection problem fulfilled by multiple parallel decision-making processes, each of which aims at picking out one sample as their tracking result in each frame. Each maintained trajectory is associated with an agent to perform the decision-making and determine what actions should be taken to update related information. More specifically, using the classification-based tracker as the baseline, we first adopt bi-GRU to encode the target feature, proposal feature, and its response score into a unified state representation. The state feature and greedy search result are then fed into the first agent for independent action selection. Afterwards, the output action and state features are fed into the subsequent agent for diverse results prediction. When all the frames are processed, we select the trajectory with the maximum accumulated score as the tracking result. Extensive experiments on seven popular tracking benchmark datasets validated the effectiveness of the proposed algorithm.

Adaptive Template and Transition Map for Real-Time Video Object Segmentation

Semi-supervised video object segmentation (semi-VOS) is required for many visual applications. This task is tracking class-agnostic objects from a given segmentation mask. Various approaches have been developed and achieved high accuracy in this field, but these previous models are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To significantly speed up inference while reducing performance gaps from those previous models, we introduce a fast segmentation model based on a template matching method and auxiliary loss with a transition map. Our template matching method consists of short-term and long-term matching. The short-term matching enhances target object localization by focusing on neighboring frames, while long-term matching improves fine details and handles object shape-changing by considering long-range frames. However, since both matching processes generate each template based on the previously estimated masks, this incurs error propagation for tracking objects in the next frames. To mitigate this problem, we add auxiliary loss with a newly proposed transition map for encouraging correction power to create accurate masks of the target object. Our model obtains $81.1\%~J\&F$ score at the speed of 78.3 FPS on the DAVIS16 benchmark and achieves $1.4\times $ faster speed and 11.3% higher accuracy than SiamMask, one of the fast semi-VOS models.

Real-time vision-based grasping randomly placed object by low-cost robotic arm using surf algorithm

Vision-Based Manipulation is popular and still have open issues in robotics. The camera is a very important part of this method to obtain desired data with image processing techniques. In this study, the Dynamic position-based look and move method was selected to control the 4 DOF robotic arm. For this method, the Kinect camera was used for image processing. Kinect is a special camera which consists of both RGB and infrared camera. SURF algorithm was selected to detect a target object from the target scene using Kinect RGB camera. 3-D target object localization was calculated using Kinect infrared camera with the point cloud. The obtained target object’s location is according to the camera and transformed according to the robotic arm base. Using inverse kinematics, desired joint angles were calculated according to the object position. Therefore, the robot is provided to make the desired motion and grasping the object by using gripper. Real-time implementation of the proposed method is carried out using Matlab-Simulink.

Identification of coronary arteries in CT images by Bayesian analysis of geometric relations among anatomical landmarks

We propose a robust method for the identification of coronary arteries in computed tomography angiography (CTA) images. Utilizing geometric relations among the target and reference objects, which are assumed to follow a Gaussian distribution, an anatomic and geometric model is designed by Bayesian inference, which provides robust geometric priors for the target object localization. As a prerequisite process for the identification of coronary arteries, partially broken coronary artery segments found in CTA images are grouped and reconnected by geometric analysis of higher order curves connecting the broken segments. The geometric properties such as curvature and torsion represent naturalness and consistency between the vessel segments. As a problem to identify coronary arteries from CTA images, we demonstrate the robustness and accuracy of the proposed method in comparison with existing methods including commercial workstations on a variety of CTA cases.

On large appearance change in visual tracking

This paper concerns on overcoming the challenges caused by drastic appearance change in visual tracking, especially the long-term appearance variation due to occlusion or large object deformation. We aim to build a long-term appearance model for robust tracking against large appearance change in two new respects: using historical and distinguishing cues to model target representation and extracting effective spatial objectness features from each frame to distinguish outliers. For the first purpose, an adaptive superpixel-based appearance model is formulated. Different from previous superpixel-based trackers, a complementary feature set is defined for the update model to preserve the features of those temporally disappeared object parts especially under occlusion and large deformation. For the second purpose, three new spatial objectness cues specially designed for tracking are defined, including surrounding comparison, edge density change and weighted superpixel straddling. With these spatial objectness cues, our method facilitates target object localization and ensures the target has similar edge distribution between adjacent frames. These cues greatly improve the ability of our method to distinguish the target from its surrounding background. The adaptive appearance model retains valuable features of historical results, and the spatial objectness cues are extracted from the current frame, and thus they are finally combined to complement with each other to solve large appearance changes. The extensive evaluations on the CVPR 2013 online object tracking benchmark and VOT 2014 datasets demonstrate the effectiveness of our method as compared with related trackers.

Using Syntax to Ground Referring Expressions in Natural Images

We introduce GroundNet, a neural network for referring expression recognition---the task of localizing (or grounding) in an image the object referred to by a natural language expression. Our approach to this task is the first to rely on a syntactic analysis of the input referring expression in order to inform the structure of the computation graph. Given a parse tree for an input expression, we explicitly map the syntactic constituents and relationships present in the tree to a composed graph of neural modules that defines our architecture for performing localization. This syntax-based approach aids localization of both the target object and auxiliary supporting objects mentioned in the expression. As a result, GroundNet is more interpretable than previous methods: we can (1) determine which phrase of the referring expression points to which object in the image and (2) track how the localization of the target object is determined by the network. We study this property empirically by introducing a new set of annotations on the GoogleRef dataset to evaluate localization of supporting objects. Our experiments show that GroundNet achieves state-of-the-art accuracy in identifying supporting objects, while maintaining comparable performance in the localization of target objects.

Training-less color object recognition for autonomous robotics

This paper looks at the challenge of object recognition from the perspective of achieving the final goals of practical real-world autonomous robot-vision applications; that of identifying a target of interest in the robot’s field of view and properly localizing its position in preparation for the higher-level goal of tracking and navigation. A unified framework is introduced that combines a multidimensional feature histogram approach with a multiscale pyramid approach for training-less color object recognition and localization, with direct application to autonomous robotic agents. This framework addresses the high computational cost associated with multidimensional processing by deriving the Multidimensional “Laplacian Feature Histogram Pyramid”, a novel approach to a unified multidimensional-multiscale histogram representation. Furthermore, a Taylor series formulation is employed to combine the multiscale levels of the multidimensional Laplacian feature histogram pyramid into one efficient multidimensional-multiscale “Laplacian-Taylor Feature Histogram” for rapid object recognition and localization. The paper describes the criteria for target detection and localization by autonomous robots and how this newly developed framework fits these needs. Comparative results demonstrate the robustness of this recognition framework to noise and localization of target objects in cluttered scenes.

Efficient Target Object Localization Model Based on Biologically Motivated Visual Selective Attention

This work proposes a novel biologically motivated visual selective attention model for efficient visual searching, which is implemented by integrating three attention mechanisms: bottom-up attention, top-down attention, and spatial attention. Bottom-up attention generates salient locations by reflecting top-down biases as well as three primitive visual features: intensity, edge and color. Prototype-based object perception is proposed for top-down attention, in which a 3-D color histogram is applied to generate a prototype of the target object. And experience based spatial attention can determine acceleration to localize a target object, which is modeled by using memorized spatial location information updated by object-localization experience. In order to verify the performance of the proposed visual selective attention model, we apply the proposed model to a real application in pedestrian traffic signal detection, to be utilized as part of a blind guide system. The proposed selective attention model shows plausible performance in terms of accuracy and computation time while efficiently localizing pedestrian traffic signals.

Top-Down Visual Saliency via Joint CRF and Dictionary Learning

Top-down visual saliency is an important module of visual attention. In this work, we propose a novel top-down saliency model that jointly learns a Conditional Random Field (CRF) and a visual dictionary. The proposed model incorporates a layered structure from top to bottom: CRF, sparse coding and image patches. With sparse coding as an intermediate layer, CRF is learned in a feature-adaptive manner; meanwhile with CRF as the output layer, the dictionary is learned under structured supervision. For efficient and effective joint learning, we develop a max-margin approach via a stochastic gradient descent algorithm. Experimental results on the Graz-02 and PASCAL VOC datasets show that our model performs favorably against state-of-the-art top-down saliency methods for target object localization. In addition, the dictionary update significantly improves the performance of our model. We demonstrate the merits of the proposed top-down saliency model by applying it to prioritizing object proposals for detection and predicting human fixations.

Multi-cue based tracking

Visual tracking is a central topic in computer vision. However, the accurate localization of target object in extreme conditions (such as occlusion, scaling, illumination change, and shape transformation) still remains a challenge. In this paper, we explore utilizing multi-cue information to ensure a robust tracking. Optical flow, color and depth clues are simultaneously incorporated in our framework. The optical flow can get a rough estimation of the target location. Then the part-based structure is adopted to establish the precise position, combining both color and depth statistics. In order to validate the robustness of the proposed method, we take four video sequences of different demanding situations and compare our method with five competitive ones representing state of the arts. Experiments prove the effectiveness of the proposed method.

Goal-oriented behavior sequence generation based on semantic commands using multiple timescales recurrent neural network with initial state correction

In this paper, to build an autonomous robot, we propose a novel scheme for a goal-oriented behavior sequence generation in tasks involving multiple objects. The scheme includes three major functions: (1) visual attention for target object localization; (2) automatic initial state correction based on experience using simple reinforcement learning, and (3) a suitable behavior sequence generation method based on multiple timescales recurrent neural networks (MTRNN). The proposed scheme systematically combines the three different major functions so that the autonomous bi-pad robot can automatically execute tasks involving multiple objects based on high level semantic commands given by human supervisor. The selective attention model continuously catches the visual environment to understand the current states of robot and perceive the relationship between current states of robot and the environment (depth perception and localization of a target object). If the current state is different from the initial state (depth perception and localization of a target object), the robot automatically adjust its current state to the initial state by integrating visual attention and simple reinforcement learning. After correcting the initial state of the robot, the behavior sequence generation functions can successfully generate suitable behavior timing signals, by integrating visual attention and MTRNN, based on the high level semantic commands given by human supervisor. Experimental results show that the proposed scheme can successfully generate suitable behavior timing, for a robot to autonomously achieve the tasks involving multiple objects, such as searching, approaching and hitting the target object using its arm.

Distance and Angle Measurement of Objects on an Oblique Plane Based on Pixel Number Variation of CCD Images

This paper presents an image-based system for measuring target objects on an oblique plane based on pixel number variation of charge-coupled device images for digital cameras by referencing to two arbitrarily designated points in the image frame. Based on an established relationship between the displacement of the camera movement along the photographing direction and the variation in pixel counts between the reference points in the images, photographic distance and incline angle for objects lying on an oblique plane can be calculated via the proposed method. As a real-case application of the proposed approach, 2-D localization of target objects in robot soccer competitions is also demonstrated to show the effectiveness of the proposed approach. To allow the use of widely available digital zoom cameras for ranging and localization by the proposed method, a parameter equivalent to the displacement due to the camera movement is also investigated and derived in this paper.

Induced Roelofs effect in reaching

Event Abstract Back to Event Induced Roelofs effect in reaching Bahareh Taghizadeh1 and Alexander Gail1, 2* 1 German Primate Center, Germany 2 Bernstein Center for Computational Neuroscience, Germany Goal-directed reaching requires accurate localization of the target object. But the localization of visual objects is prone to illusions induced by the visual context. In the induced Roelofs effect (IRE), the position of a task irrelevant visual object induces a shift in the localization of the visual target object. This is true when subjects have to indicate the position of the target object relative to an array of reference positions, e.g. by response keys or by pointing to it with delay. In contrast, when subjects in the same task indicate the position of the target object by pointing to it without instructed delay or by directly reaching towards it with or without delay, no IRE is induced [1]. This discrepancy was taken as evidence for separate visuospatial representations for direct sensorimotor processing compared to spatial cognitive processing [2]. Here we test if an IRE can also be induced for reaching movement towards the target stimulus, which would argue against such strictly separate processing. We asked human subjects to perform reaches towards visual target stimuli in the frontoparallel plane. Each trial started with a brief appearance of a reference array (RA) of five horizontally arranged boxes which indicated the potential target positions. Then the target and a surrounding task-irrelevant visual frame were flashed, followed by a decision array (DA) immediately after the cue/frame offset. The DA was identical to the RA, but could be placed at different positions on the screen. While keeping ocular fixation, subjects had to reach towards the DA box which corresponded to the RA box in which they had seen the flashed target. The results showed a reliable IRE for reaching. In our task the reach goal needed to be defined relative to a task-relevant object, i.e. in an object-based reference frame, while in previous experiments the reach goal was directly defined by the target objects. The results suggest that during object-based encoding of motor goal locations, the information of additional task-irrelevant objects can induce systematic mis-localizations of the reach goal. This finding argues against strictly separate visuospatial representations for direct sensorimotor processing compared to spatial cognitive processing.

Object localization in the presence of a strong heterogeneous background in fluorescent tomography

We propose a method for object localization in fluorescent tomography (FT) in the presence of a highly heterogeneous background. Existing approaches typically assume a homogeneous background distribution; thus, they are incapable of accurately accounting for the more general case of an unconstrained, possibly heterogeneous, background. The proposed method iteratively solves the inverse problem over a solution space partitioned into a background subspace and an object subspace to simultaneously estimate the background and localize the target fluorescent objects. Simulation results of this algorithm applied to continuous-wave FT demonstrate effective localization of target objects in the presence of highly heterogeneous background distributions.

The impact of temporal inaccuracies on 4DCT image quality

Accurate delineation of target volumes is one of the critical components contributing to the success of image-guided radiotherapy treatments and several imaging modalities are employed to increase the accuracy in target identification. Four-dimensional (4D) techniques are incorporated into existing radiation imaging techniques like computed tomography (CT) to account for the mobility of the target volumes. However, these methods in some cases introduce further inaccuracies in the target delineation when further quality assurance measures are not implemented. A source of commonly observed inaccuracy is the misidentification of the respiration cycles and resulting respiration phase assignments used in the construction of the 4D patient model. The aim of this work is to emphasize the importance of optimal respiration phase assignment during the 4DCT image acquisition process and to perform a quantitative assessment of the effect of inaccurate phase assignments on the overall image quality. The accuracy of the phase assignment was assessed by comparison with an independent calculation of the respiration phases. Misplaced phase assignments manifest themselves as deformations and artifacts in reconstructed images. These effects are quantified as volumetric discrepancies in the localization of target objects represented by spherical phantoms. Measurements are performed using a fully programmable motion phantom designed and built at Mayo Clinic (Rochester, MN). Implementation of a case based independent check and correction procedure is also demonstrated with emphasis on the use of this procedure in the clinical environment. Review of clinical 4D scans performed in this institution showed discrepancies in the phase assignments in about 40% of the cases when compared to our independent calculations. It is concluded that for improved image reconstruction, an independent check of the sorting procedure should be performed for each clinical 4DCT case.