Position Information Of Object Research Articles

Position Fusing and Refining for Clear Salient Object Detection.

Multilevel feature fusion plays a pivotal role in salient object detection (SOD). High-level features present rich semantic information but lack object position information, whereas low-level features contain object position information but are mixed with noises such as backgrounds. Appropriately addressing the gap between low-and high-level features is important in SOD. We first propose a global position embedding attention (GPEA) module to minimize the discrepancy between multilevel features in this article. We extract the position information by utilizing the semantic information at high-level features to resist noises at low-level features. Object refine attention (ORA) module is introduced to refine features used to predict saliency maps further without any additional supervision and heighten discriminative regions near the salient object, such as boundaries. Moreover, we find that the saliency maps generated by the previous methods contain some blurry regions, and we design a pixel value (PV) loss to help the model generate saliency maps with improved clarity. Experimental results on five commonly used SOD datasets demonstrated that the proposed method is effective and outperforms the state-of-the-art approaches on multiple metrics.

Position‐aware pushing and grasping synergy with deep reinforcement learning in clutter

AbstractThe positional information of objects is crucial to enable robots to perform grasping and pushing manipulations in clutter. To effectively perform grasping and pushing manipulations, robots need to perceive the position information of objects, including the coordinates and spatial relationship between objects (e.g., proximity, adjacency). The authors propose an end‐to‐end position‐aware deep Q‐learning framework to achieve efficient collaborative pushing and grasping in clutter. Specifically, a pair of conjugate pushing and grasping attention modules are proposed to capture the position information of objects and generate high‐quality affordance maps of operating positions with features of pushing and grasping operations. In addition, the authors propose an object isolation metric and clutter metric based on instance segmentation to measure the spatial relationships between objects in cluttered environments. To further enhance the perception capacity of position information of the objects, the authors associate the change in the object isolation metric and clutter metric in cluttered environment before and after performing the action with reward function. A series of experiments are carried out in simulation and real‐world which indicate that the method improves sample efficiency, task completion rate, grasping success rate and action efficiency compared to state‐of‐the‐art end‐to‐end methods. Noted that the authors’ system can be robustly applied to real‐world use and extended to novel objects. Supplementary material is available at https://youtu.be/NhG\_k5v3NnM}{https://youtu.be/NhG\_k5v3NnM.

The nature of neural object representations during dynamic occlusion

Objects disappearing briefly from sight due to occlusion is an inevitable occurrence in everyday life. Yet we generally have a strong experience that occluded objects continue to exist, despite the fact that they objectively disappear. This indicates that neural object representations must be maintained during dynamic occlusion. However, it is unclear what the nature of such representation is and in particular whether it is perception-like or more abstract, for example, reflecting limited features such as position or movement direction only. In this study, we address this question by examining how different object features such as object shape, luminance, and position are represented in the brain when a moving object is dynamically occluded. We apply multivariate decoding methods to Magnetoencephalography (MEG) data to track how object representations unfold over time. Our methods allow us to contrast the representations of multiple object features during occlusion and enable us to compare the neural responses evoked by visible and occluded objects. The results show that object position information is represented during occlusion to a limited extent while object identity features are not maintained through the period of occlusion. Together, this suggests that the nature of object representations during dynamic occlusion is different from visual representations during perception.

HSV Color-Space-Based Automated Object Localization for Robot Grasping without Prior Knowledge

We propose a simple and robust HSV color-space-based algorithm that can automatically extract object position information without human intervention or prior knowledge. In manufacturing sites with high variability, it is difficult to recognize products through robot machine vision, especially in terms of extracting object information accurately, owing to various environmental factors such as the noise around objects, shadows, light reflections, and illumination interferences. The proposed algorithm, which does not require users to reset the HSV color threshold value whenever a product is changed, uses ROI referencing method to solve this problem. The algorithm automatically identifies the object’s location by using the HSV color-space-based ROI random sampling, ROI similarity comparison, and ROI merging. The proposed system utilizes an IoT device with several modules for the detection, analysis, control, and management of object data. The experimental results show that the proposed algorithm is very useful for industrial automation applications under complex and highly variable manufacturing environments.

Attention-based object detection with saliency loss in remote sensing images

Geospatial object detection in remote sensing images is a challenging subject since objects in remote sensing images are dense, multioriented, and multiscale. We present an attention network for object detection in remote sensing images. Through channel attention and spatial attention, the framework pays more attention to important channels and emphasizes position information of objects. Meanwhile, saliency learning is proposed to enhance objects information. Furthermore, saliency loss is added to the loss function to guide network learning in the training stage. In addition, multiscale feature module is added into the network to capture scale variations. Experimental results on public remote sensing image datasets validate the effectiveness of the proposed method.

Retracted: Jointly network image processing: multi‐task image semantic segmentation of indoor scene based on CNN

Image semantic segmentation has always been a research hotspot in the field of robots. Its purpose is to assign different semantic category labels to objects by segmenting different objects. However, in practical applications, in addition to knowing the semantic category information of objects, robots also need to know the position information of objects to complete more complex visual tasks. Aiming at a complex indoor environment, this study designs an image semantic segmentation network framework of joint target detection. Using the parallel operation of adding semantic segmentation branches to the target detection network, it innovatively implements multi-vision task combining object classification, detection and semantic segmentation. By designing a new loss function, adjusting the training using the idea of transfer learning, and finally verifying it on the self-built indoor scene data set, the experiment proves that the method in this study is feasible and effective, and has good robustness.

Using vision-based object detection for link quality prediction in 5.6-GHz channel

Various smart connected devices are emerging like automated driving cars, autonomous robots, and remote-controlled construction vehicles. These devices have vision systems to conduct their operations without collision. Machine vision technology is becoming more accessible to perceive self-position and/or the surrounding environment thanks to the great advances in deep learning technologies. The accurate perception information of these smart connected devices makes it possible to predict wireless link quality (LQ). This paper proposes an LQ prediction scheme that applies machine learning to HD camera output to forecast the influence of surrounding mobile objects on LQ. The proposed scheme utilizes object detection based on deep learning and learns the relationship between the detected object position information and the LQ. Outdoor experiments show that LQ prediction proposal can well predict the throughput for around 1 s into the future in a 5.6-GHz wireless LAN channel.

Hybrid-Attention Network for RGB-D Salient Object Detection

Depth information has been widely used to improve RGB-D salient object detection by extracting attention maps to determine the position information of objects in an image. However, non-salient objects may be close to the depth sensor and present high pixel intensities in the depth maps. This situation in depth maps inevitably leads to erroneously emphasize non-salient areas and may have a negative impact on the saliency results. To mitigate this problem, we propose a hybrid attention neural network that fuses middle- and high-level RGB features with depth features to generate a hybrid attention map to remove background information. The proposed network extracts multilevel features from RGB images using the Res2Net architecture and then integrates high-level features from depth maps using the Inception-v4-ResNet2 architecture. The mixed high-level RGB features and depth features generate the hybrid attention map, which is then multiplied to the low-level RGB features. After decoding by several convolutions and upsampling, we obtain the final saliency prediction, achieving state-of-the-art performance on the NJUD and NLPR datasets. Moreover, the proposed network has good generalization ability compared with other methods. An ablation study demonstrates that the proposed network effectively performs saliency prediction even when non-salient objects interfere detection. In fact, after removing the branch with high-level RGB features, the RGB attention map that guides the network for saliency prediction is lost, and all the performance measures decline. The resulting prediction map from the ablation study shows the effect of non-salient objects close to the depth sensor. This effect is not present when using the complete hybrid attention network. Therefore, RGB information can correct and supplement depth information, and the corresponding hybrid attention map is more robust than using a conventional attention map constructed only with depth information.

Multi-Robot Avoidance Control Based on Omni-Directional Visual SLAM with a Fisheye Lens Camera

This paper proposes a noble avoidance control algorithm based on omni-directional visual simultaneous localization and mapping (OVSLAM) with a fisheye lens camera. In addition, a robot avoids colliding with an obstacle regardless of the obstacle’s state by analyzing the information of the object obtained from an OVSLAM approach. OVSLAM has many advantages for object detection and mapping because it can measure all information around a robot simultaneously. We therefore proposed an improved avoidance and formation control to configure a multi-robot system optimized for OVSLAM. This system creates a global map based on vector information and position information of objects obtained from a local map, and determines the avoidance method according to the type of object, which is classified by analyzing the odometry and vector and position information. We carried out a formation control experiment in an environment with static obstacles and a dynamic robot, and a formation control experiment in an environment with dynamic obstacles and a robot. The reliability of the proposed formation algorithm was verified through a comparison of maps based on the proposed algorithm and real maps while maintaining the formation by applying a real robot.

Small Object Detection with Multiscale Features

The existing object detection algorithm based on the deep convolution neural network needs to carry out multilevel convolution and pooling operations to the entire image in order to extract a deep semantic features of the image. The detection models can get better results for big object. However, those models fail to detect small objects that have low resolution and are greatly influenced by noise because the features after repeated convolution operations of existing models do not fully represent the essential characteristics of the small objects. In this paper, we can achieve good detection accuracy by extracting the features at different convolution levels of the object and using the multiscale features to detect small objects. For our detection model, we extract the features of the image from their third, fourth, and 5th convolutions, respectively, and then these three scales features are concatenated into a one-dimensional vector. The vector is used to classify objects by classifiers and locate position information of objects by regression of bounding box. Through testing, the detection accuracy of our model for small objects is 11% higher than the state-of-the-art models. In addition, we also used the model to detect aircraft in remote sensing images and achieved good results.

A Saliency Detection Based Unsupervised Commodity Object Retrieval Scheme

Commodity object retrieval is a key issue in the application of self-service shopping and so on. In this paper, we propose a saliency object detection-based unsupervised commodity object retrieval scheme. Since most commodity objects are conspicuous and not complicated in commodity images, saliency detection could predict a saliency box that indicates approximate position information of objects. The proposed scheme utilizes the saliency box to filter the proposals extracted by selective search. The reserved proposals have a big overlapping ratio with saliency box to a large extent. This paper composes both the saliency box and the reserved proposals as saliency proposals. Furthermore, we propose a channel weighting generalized mean pooling feature to represent saliency proposals. On one hand, the reduction of proposals’ number after filtering significantly improves the computational efficiency; on the other hand, the new feature more accurately represents the objects to be retrieved, which results in higher retrieval precision. In addition, we built and manually annotated a commodity data set named PRODUCT to evaluate the proposed method. Extensive experiments are also conducted on the databases INSTRE and Flick32. The results demonstrate the superior performance of our scheme compared with the other state-of-the-art methods.

A Mobile Robot Localization via Indoor Fixed Remote Surveillance Cameras.

Localization, which is a technique required by service robots to operate indoors, has been studied in various ways. Most localization techniques have the robot measure environmental information to obtain location information; however, this is a high-cost option because it uses extensive equipment and complicates robot development. If an external device is used to determine a robot’s location and transmit this information to the robot, the cost of internal equipment required for location recognition can be reduced. This will simplify robot development. Thus, this study presents an effective method to control robots by obtaining their location information using a map constructed by visual information from surveillance cameras installed indoors. With only a single image of an object, it is difficult to gauge its size due to occlusion. Therefore, we propose a localization method using several neighboring surveillance cameras. A two-dimensional map containing robot and object position information is constructed using images of the cameras. The concept of this technique is based on modeling the four edges of the projected image of the field of coverage of the camera and an image processing algorithm of the finding object’s center for enhancing the location estimation of objects of interest. We experimentally demonstrate the effectiveness of the proposed method by analyzing the resulting movement of a robot in response to the location information obtained from the two-dimensional map. The accuracy of the multi-camera setup was measured in advance.

An implicit spatial memory alignment effect.

The memory alignment effect is the advantage of reasoning from a perspective which is aligned with the frame of reference used to encode an environment in memory. It usually occurs when participants have to consciously take a perspective to perform a spatial memory task. The present experiment assesses whether the memory alignment effect can occur without requiring to consciously take a given perspective, when the misaligned perspective is only perceptively provided. In others words, does the memory alignment effect still arise when it is only implicitly prompted? Thirty participants learned a sequence of four objects' positions in a room from a north-as-up survey perspective. During the testing phase, they had to point to the direction of a target object from another object ('the reference') with a fixed north-up orientation. The background behind the reference object displayed either a uniform color (control condition) or a misaligned ground-level perspective. The latter displayed a reference object's position information which was either congruent with the studied environment (congruent misaligned condition) or incongruent (incongruent misaligned condition). Mean pointing errors were higher in the congruent misaligned condition than in the control condition, whereas the incongruent misaligned condition did not differ from the control one. The present study shows that the memory alignment effect can arise without requiring a conscious misaligned perspective taking. Moreover, the perceived misaligned perspective must share the same spatial content as the memorized spatial representation in order to induce an alignment effect.

A motion tracking method that applies a spread spectrum communication technique to tagging MR imaging.

We propose a new motion tracking method that encodes an object's position information using tagging magnetic resonance (MR) images as digital codes, and we assess the method's feasibility in stationary and moving phantoms. The encoding and decoding of tag patterns employ principles of spread spectrum communication. We used a segmented fast low angle shot cine sequence (FLASH) with spatial modulation of magnetization (SPAMM) preparation pulses to encode position information as 7-bit code words and used this spread code to decode the position information. To make 7-bit code words using tag images, we adjusted the flip angle and phase of the 4 composite radiofrequency (RF) pulses and the gradient magnetic field strength in the SPAMM pulse to generate 7 types of tag patterns. The proposed method required 7 acquisitions with 7 types of tag patterns. We compared results with images obtained by conventional tagging in stationary and moving phantom experiments. In a stationary phantom, the proposed method showed the same ability to identify pixel position as conventional tagging method using improved SNR images with the average of 7 acquisitions. In a moving phantom, pixel position was successfully decoded by the proposed method on a pixel-by-pixel basis. In this method, the motion of the phantom was detected by the simple calculation of the correlation coefficient of the code words. We introduced a spread spectrum communication technique to tagging MR imaging that regards tag patterns as digital codes, and we demonstrated the method's potential to detect pixel position in sub-pixel resolution.

Priorities for selection and representation in natural tasks

Selecting and remembering visual information is an active and competitive process. In natural environments, representations are tightly coupled to task. Objects that are task-relevant are remembered better due to a combination of increased selection for fixation and strategic control of encoding and/or retaining viewed information. However, it is not understood how physically manipulating objects when performing a natural task influences priorities for selection and memory. In this study, we compare priorities for selection and memory when actively engaged in a natural task with first-person observation of the same object manipulations. Results suggest that active manipulation of a task-relevant object results in a specific prioritization for object position information compared with other properties and compared with action observation of the same manipulations. Experiment 2 confirms that this spatial prioritization is likely to arise from manipulation rather than differences in spatial representation in real environments and the movies used for action observation. Thus, our findings imply that physical manipulation of task relevant objects results in a specific prioritization of spatial information about task-relevant objects, possibly coupled with strategic de-prioritization of colour memory for irrelevant objects.

Object detection and tracking method of AUV based on acoustic vision

This paper describes a new framework for object detection and tracking of AUV including underwater acoustic data interpolation, underwater acoustic images segmentation and underwater objects tracking. This framework is applied to the design of vision-based method for AUV based on the forward looking sonar sensor. First, the real-time data flow (underwater acoustic images) is pre-processed to form the whole underwater acoustic image, and the relevant position information of objects is extracted and determined. An improved method of double threshold segmentation is proposed to resolve the problem that the threshold cannot be adjusted adaptively in the traditional method. Second, a representation of region information is created in light of the Gaussian particle filter. The weighted integration strategy combining the area and invariant moment is proposed to perfect the weight of particles and to enhance the tracking robustness. Results obtained on the real acoustic vision platform of AUV during sea trials are displayed and discussed. They show that the proposed method can detect and track the moving objects underwater online, and it is effective and robust.

Statistical characteristics of landmark-based localization performance

Landmark-based localization is a prevalent method to obtain the position information of objects, such as autonomous robots, in industries. This study investigated the performance characteristics of landmark-based localization based on the popular multilateration algorithm from new perspectives. Two performance metrics were used, namely, accuracy and precision, which are defined as the mean and standard deviation of root mean squared error over the entire localization region. The general formulations for the two metrics were obtained, and the computation was accomplished by Monte Carlo simulation. The simulation was performed under a simplified localization configuration, and the following system factors were systematically studied, namely, landmark layout, landmark number, target location distribution, and ranging error type. The quality of landmark layout was, for the first time, quantitatively assessed. Optimal layout can be obtained by minimization of the localization accuracy and precision values with regard to the quality index. It was found that the index represented by the area of polygon enclosed by the landmarks is an excellent measure of layout quality. Also, the improvement of localization performance can be achieved by distributing landmarks more evenly, using more landmarks, placing the target closer to the center of localization region, or reducing the ranging errors. Furthermore, it was also demonstrated that the findings in simulation are generally confirmed by the experimental results using active radio-frequency identification (RFID) devices.