Problem Of Object Localization Research Articles

Dense center-direction regression for object counting and localization with point supervision

Object counting and localization problems are commonly addressed with point-supervised learning, which allows the use of less labor-intensive point annotations. However, learning based on point annotations poses challenges due to the high imbalance between the sets of annotated and unannotated pixels, which is often treated with Gaussian smoothing of point annotations and focal loss. However, these approaches still focus on the pixels in the immediate vicinity of the point annotations and exploit the rest of the data only indirectly. In this work, we propose a novel approach termed CeDiRNet for point-supervised learning that uses a dense regression of directions pointing towards the nearest object centers, i.e. center-directions. This provides greater support for each center point arising from many surrounding pixels pointing towards the object center. We propose a formulation of center-directions that allows the problem to be split into the domain-specific dense regression of center-directions and the final localization task based on a small, lightweight, and domain-agnostic localization network that can be trained with synthetic data completely independent of the target domain. We demonstrate the performance of the proposed method on six different datasets for object counting and localization and show that it outperforms the existing state-of-the-art methods. The code is accessible on GitHub at https://github.com/vicoslab/CeDiRNet.git.

MBAN: multi-branch attention network for small object detection.

Recent years small object detection has seen remarkable advancement. However, small objects are difficult to accurately detect in complex scenes due to their low resolution. The downsampling operation inevitably leads to the loss of information for small objects. In order to solve these issues, this article proposes a novel Multi-branch Attention Network (MBAN) to improve the detection performance of small objects. Firstly, an innovative Multi-branch Attention Module (MBAM) is proposed, which consists of two parts, i.e. Multi-branch structure consisting of convolution and maxpooling, and the parameter-free SimAM attention mechanism. By combining these two parts, the number of network parameters is reduced, the information loss of small objects is reduced, and the representation of small object features is enhanced. Furthermore, to systematically solve the problem of small object localization, a pre-processing method called Adaptive Clustering Relocation (ACR) is proposed. To validate our network, we conducted extensive experiments on two benchmark datasets, i.e. NWPU VHR-10 and PASCAL VOC. The findings from the experiment demonstrates the significant performance gains of MBAN over most existing algorithms, the mAP of MBAN achieved 96.55% and 84.96% on NWPU VHR-10 and PASCAL VOC datasets, respectively, which proves that MBAN has significant performance in small object detection.

QDETRv: Query-Guided DETR for One-Shot Object Localization in Videos

In this work, we study one-shot video object localization problem that aims to localize instances of unseen objects in the target video using a single query image of the object. Toward addressing this challenging problem, we extend a popular and successful object detection method, namely DETR (Detection Transformer), and introduce a novel approach –query-guided detection transformer for videos (QDETRv). A distinctive feature of QDETRv is its capacity to exploit information from the query image and spatio-temporal context of the target video, which significantly aids in precisely pinpointing the desired object in the video. We incorporate cross-attention mechanisms that capture temporal relationships across adjacent frames to handle the dynamic context in videos effectively. Further, to ensure strong initialization for QDETRv, we also introduce a novel unsupervised pretraining technique tailored to videos. This involves training our model on synthetic object trajectories with an analogous objective as the query-guided localization task. During this pretraining phase, we incorporate recurrent object queries and loss functions that encourage accurate patch feature reconstruction. These additions enable better temporal understanding and robust representation learning. Our experiments show that the proposed model significantly outperforms the competitive baselines on two public benchmarks, VidOR and ImageNet-VidVRD, extended for one-shot open-set localization tasks.

Learning Offset Probability Distribution for Accurate Object Detection

Object detection combines object classification and object localization problems. Current object detection methods heavily depend on regression networks to locate objects, which are optimized with various regression loss functions to predict offsets between candidate boxes and objects. However, these regression losses are difficult to assign the appropriate penalties for samples with large offset errors, resulting in suboptimal regression networks and inaccurate object offsets. In this article, we consider object location as offset bin classification problem, and propose a distance-aware offset bin classification network optimized with multiple binary cross entropy losses to learn various offset probability distribution, including single label distribution and distance-aware label distribution. On one hand, it provides gradient contributions for different samples based on the bounded probability instead of previous incalculable offset error. On the other hand, it explores the distance correlations between discrete offset bins to facilitate network learning. Specifically, we discretize the continuous offset into a number of bins, and predict the probability of each offset bin, in which the probability should be higher for the offset bin closer to the target offsets, and vice versa. Furthermore, we propose an expectation-based offset prediction and a hierarchical focusing method to improve the precision of prediction. We conduct extensive experiments to evaluate the effectiveness of our method. In addition, our method can be conveniently and flexibly inserted into existing object detection methods, which consistently achieves a large gain based on popular anchor-based and anchor-free methods on the PASCAL VOC, MS-COCO, KITTI, and CrowdHuman datasets. Code will be released at: https://github.com/QiuHeqian/DBC .

Multiparticle Kalman filter for object localization in symmetric environments

This study considers the object localization problem and proposes a novel multiparticle Kalman filter to solve it in complex and symmetric environments. Two well-known classes of filtering algorithms to solve the localization problem are Kalman filter-based methods and particle filter-based methods. We consider these classes, demonstrate their complementary properties, and propose a novel filtering algorithm that takes the best from two classes. We evaluate the multiparticle Kalman filter in symmetric and noisy environments. Such environments are especially challenging for both classes of classical methods. We compare the proposed approach with the particle filter since only this method is feasible if the initial state is unknown. In the considered challenging environments, our method outperforms the particle filter in terms of both localization error and runtime.

Real-Time Location System (RTLS) Based on the Bluetooth Technology for Internal Logistics

The problem of object localization in indoor environments is very important in order to make a company effective and to detect disruption in the logistics system in real-time. Present research investigates how the IoT (Internet of Things) location system based on Bluetooth can be implemented for this solution. The location based on the Bluetooth is hard to predict. Radio wave interference in this frequency is affected by other devices, steel, vessels containing water, and more. However, proper data processing and signal stabilization can increase the accuracy of the location. To be sure that the location system based on the BT (Bluetooth) can be implemented for real cases, an analysis of signal strength amplitude and disruption was made. The paper presents R&D (Research and Development) works with a practical test in real cases. The signal strength fluctuation for the receiver is between 7 and 10 dBm for ESP32 device and between 13 and 14 dBm for Raspberry. For commercial implementation the number of devices scanned in the time window is also important. For Raspberry, the optimal time window is 5 s; in this time six transmitters can be detected. ESP32 has a problem with detecting devices in a short time, as just two transmitters can be detected in 4–8 s time window. Localisation precision depends on the distance between transmitter and receiver, and the angle from the axis of the directional antenna. For the distance of 10 m the measurement error is 1.2–6.1 m, whilst for the distance of 40 m the measurement error is 4.9 to 24.6 m. Using a Kalman filter can reduce the localization error to 1.5 m.

DANIEL: A fast and robust consensus maximization method for point cloud registration with high outlier ratios

Correspondence-based point cloud registration is a cornerstone in computer vision, robotics, autonomous navigation and remote sensing, which seeks to estimate the best rigid transformation between two point clouds from the point correspondences established over 3D keypoints. Owing to limited robustness and accuracy, current 3D keypoint matching techniques are rather prone to yield outliers, probably in large numbers, thus making robust estimation (outlier rejection) for the registration problem very important. Unfortunately, previously proposed robust registration methods often suffer from high computation cost or insufficient robustness when encountering high (or even extreme) outlier ratios, hardly ideal enough for practical use. In this paper, we present a novel, time-efficient consensus maximization solver, named Double-layered sAmpliNg with consensus maximization based on stratIfied Element-wise compatibiLity (DANIEL), for robust point cloud registration. DANIEL is smartly designed with two layers of random sampling operation, in order to find the inlier subset with the lowest computational cost possible. Specifically, we (i) apply the rigidity constraint to prune raw outliers in the first layer of one-point sampling, (ii) introduce a series of stratified element-wise compatibility tests to conduct rapid compatibility checking between minimal models so as to realize more efficient consensus maximization in the second layer of two-point sampling, and (iii) employ probabilistic termination conditions to ensure the timely return of the final inlier set. By conducting a series of experiments on multiple real datasets, we demonstrate that the proposed solver DANIEL (i) is robust against over 99% of outliers, (ii) is also significantly faster than previous state-of-the-art robust algorithms, and (iii) is fairly practical for use in real-world applications such as object localization and scan matching problems.

Automatic Localization and Segmentation of Vertebrae for Cobb Estimation and Curvature Deformity

The long twisted fragile tube, termed as spinal cord, can be named as the second vital organ of Central Nervous System (CNS), after brain. In human anatomy, all crucial life activities are controlled by CNS. The spinal cord does not only control the flow of information from the brain to rest of the body, but also takes charge of our reflexes control and the mobility of body. It keeps the body upright and acts as the main support for the flesh and bones. Spine deformity can occur by birth, due to aging, injury or spine surgery. In this research article, we have proposed a new three step framework for analysis of spine deformity where we have introduced vertebrae segmentation as object localization problem. You Only Look Once (YOLO) is utilized for localization of vertebrae, which achieved the mAP of 97.5% for Mendeley dataset and 95.2% for Computational methods and clinical applications for Spine Imaging (CSI) 2016 dataset. In the second step, edge detection, is done by Holistic Edge Detection (HED) and for corner calculation, the Harris method is used. In the final step we calculated the Cobb angle for the deformity analysis. Mean Absolute Error (MAE) is calculated that was found to be less than 0.40° for Mendeley and 0.50° for CSI 2016 dataset. The classification of Lumbar Lordosis with corner point Cobb estimation method achieved an accuracy up to 98.04% for the Mendeley dataset and 81.25% for CSI 2016 dataset respectively. A comparative analysis is done for Cobb estimation and the results showed that the proposed framework has reduced mean error up to 2 degree.

Research on Adjustable Baseline Binocular Vision Measurement System

Binocular stereo vision belongs to machine vision. It uses two cameras to imitate human eyes to obtain two images of the object to be measured, and obtains the three-dimensional information of the object by calculating the positional deviation between the corresponding points of the images. The baseline length of the binocular camera determines the size of the observable distance and range. In today's increasingly complex work environment, the baseline of traditional binocular measurement equipment is fixed, and the usage scenarios are too limited. Therefore, this paper proposes a binocular vision measurement system with dynamically adjustable baselines. The system solves the problem of object localization at different depths by changing the baseline length.

Multistatic Localization With Unknown Transmitter Position and Signal Propagation Speed

In this letter, we address the multistatic object localization problem when the transmitter position and signal propagation speed are not known. By transforming the time delay measurement model, we first formulate a non-convex weighted least squares problem, which is then relaxed into a convex semidefinite programming (SDP) problem by applying semidefinite relaxation. It turns out that the relaxed SDP problem is too loose and fails to achieve the Cramer-Rao lower bound (CRLB) performance. To address this problem, we propose to tighten the relaxed SDP problem by adding a second-order cone constraint to reach better solutions. Simulation results show that the proposed method is able to achieve the CRLB.

Multi-person vision tracking approach based on human body localization features

This paper presents a multi-person vision tracking approach based on human body localization features to address the problem of interactive object localization and tracking in a home monitoring scenario. Firstly, the human body localization model is used to obtain the 3D position of the human body, which is then used to construct the human body motion model based on the Kalman filter method. At the same time, the human appearance model is constructed by fusing human color features and features of the histogram of oriented gradient to better characterize the human body. Secondly, the human body observation model is constructed based on the human body motion model and appearance model to measure the similarities between the human body state sequence in the historical frame and the human body observation result in the current frame, and the cost matrix is then obtained. Thirdly, the Hungarian maximum matching algorithm is employed to match each human body in the current and historical frames, and the exception detection mechanism is simultaneously constructed to further reduce the probability of human tracking and matching failure. Finally, a multi-person vision tracking verification platform was constructed, and the achieved average accuracy was 96.6% in the case of human body overlapping, occlusion, disappearance, and appearance; this verifies the feasibility and effectiveness of the proposed method.

Real-time plant phenomics under robotic farming setup: A vision-based platform for complex plant phenotyping tasks

Plant phenotyping in general refers to quantitative estimation of the plant’s anatomical, ontogenetical, physiological and biochemical properties. Analyzing big data is challenging, and non-trivial given the different complexities involved. Efficient processing and analysis pipelines are the need of the hour with the increasing popularity of phenotyping technologies and sensors. Through this work, we largely address the overlapping object segmentation & localization problem. Further, we dwell upon multi-plant pipelines that pose challenges as detection and multi-object tracking becomes critical for single frame/set of frames aimed towards uniform tagging & visual features extraction. A plant phenotyping tool named RTPP (Real-Time Plant Phenotyping) is presented that can aid in the detection of single/multi plant traits, modeling, and visualization for agricultural settings. We compare our system with the plantCV platform. The relationship of the digital estimations, and the measured plant traits are discussed that plays a vital roadmap towards precision farming and/or plant breeding.

Using Lora Technology To Solve The Problem Of Object Localization On IoT Systems

Internet of Things (IoT) is a leading place in the rapidly expanding Internet technologies. One of the uses of IoT technology is to track and control the location of objects. The purpose of this work is to investigate the feasibility of using LoRa technology to solve the object localization problem.The theoretical analysis of data technologies in existing IoT systems shows that the use of technologies such as GPS, Wi-Fi, Bluetooth is not always effective due to its high cost and low coverage. A better solution for building networks of interaction between IoT devices can be considered LoRa technology, which at the same time allows you to communicate over long distances, characterized by low power consumption and high resistance to interference and is economically attractive. LoRa is a wireless modulation scheme built on Chirp Spread Spectrum (CSS) modulation that makes it more interfering. It is based on LoRaWAN, which allows you to create a network for interaction between devices. Such a network can be used to solve the problem of geolocation.To determine the location of a device, you first need several reference points, the location of which is already known. There are three known methods for locating: triangulation, trilateral, and multilateration methods. Let us analyze the effectiveness of using these methods when applying LoRa.In triangulation, the location of the device can be estimated using the geometry of the triangle formed between the two reference points and the device. In our case, the transmitters or device will be at a great distance from each other, because this is LoRa specificity. Because of this, location error is increased.The trilateration method has already been used in localization studies for LoRa devices. The distance from the device can be estimated in two ways: by measuring the time of arrival (ToA), or within RSSI by determining the signal strength of the received signal to the receiver.In the ToA method, the master data for processing is the time at which the signal arrives from the device to the station. This method is characterized by high power consumption due to the need for synchronization between devices. In addition, the quality of the location is highly dependent on the environment.This method of localization, such as multilateration, does not require the determination of the distance from the device to each anchor point, more important is the difference in the distance between the anchor points themselves. TDoA is a popular method for localization because it does not require transmitter synchronization with receivers. Therefore, the most optimal method of localization can be considered the method of determining the difference in time of arrival (TDoA)The analysis shows that the object localization problem can be solved by using modern wireless localization algorithms using LoRa technology. The goal of further research is to create an optimal algorithm for solving the problem of localization of IoT devices.

K-Nearest Neighbor Classification for Pattern Recognition of a Reference Source Light for Machine Vision System

The design of machine vision applications allows automatic inspection, measuring systems, and robot guidance. Typical applications of industrial robots are based on no-contact sensors to give the robot information about the environment. Robot's machine vision requires photosensors or video cameras to make intelligent decisions about its localization. Video cameras used as image-capturing equipment are too costly in comparison with optical scanning systems (OSS). The OSS system provides spatial coordinates measurements that can be exploited to solve a wide variety of structural problems in real-time. Localization and guidance using machine learning (ML) techniques offer advantages due to signals captured can be transformed and be reduced for processing, storage, and displaying. The use of algorithms of ML enhances the performance of the optical system based on localization and guidance. Feature extraction represents an important part of ML techniques to transform the original raw data onto a low-dimensional subspace and holding relevant information. This work presents an improvement of an optical system based on <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -nearest neighbor ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -NN) technique to solve the object detection and localization problem. The utility of this improvement allows the optical system can discriminate between the reference source and the optical noise or interference. The OSS system presented in this article has been implemented in structural health monitoring to measure the angular position even under “lighting and weather conditions”. The feature extraction techniques used in this article were linear predictive coding (LPC), quartiles ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iquartile</sub> ), and autocorrelation coefficients (ACC). The results of using <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -NN and autocorrelation coefficients and quartiles predicted more than 98% of correct classification by using a reference source light as a class 1 and a light bulb as an optical noise and called class 2.

Unsupervised Anomaly Detection and Localization Based on Deep Spatiotemporal Translation Network

Anomaly detection is of great significance for intelligent surveillance videos. Current works typically struggle with object detection and localization problems due to crowded and complex scenes. Hence, we propose a Deep Spatiotemporal Translation Network (DSTN), novel unsupervised anomaly detection and localization method based on Generative Adversarial Network (GAN) and Edge Wrapping (EW). In training, we use only the frames of normal events in order to generate their corresponding dense optical flow as temporal features. During testing, since all the video sequences are input into the system, unknown events are considered as anomalous events due to the fact that the model knows only the normal patterns. To benefit from the information provided by both appearance and motion features, we introduce (i) a novel fusion of background removal and real optical flow frames with (ii) a concatenation of the original and background removal frames. We improve the performance of anomaly localization in the pixel-level evaluation by proposing (iii) the Edge Wrapping to reduce the noise and suppress non-related edges of abnormal objects. Our DSTN has been tested on publicly available anomaly datasets, including UCSD pedestrian, UMN, and CUHK Avenue. The results show that it outperforms other state-of-the-art algorithms with respect to the frame-level evaluation, the pixel-level evaluation, and the time complexity for abnormal object detection and localization tasks.

PML-LocNet: Improving Object Localization with Prior-induced Multi-view Learning Network.

This paper introduces a new model for Weakly Supervised Object Localization (WSOL) problems where only image-level supervision is provided. The key to solve such problems is to infer the object locations accurately. Previous methods usually model the missing object locations as latent variables, and alternate between updating their estimates and learning a detector accordingly. However, the performance of such alternative optimization is sensitive to the quality of the initial latent variables and the resulted localization model is prone to overfitting to improper localizations. To address these issues, we develop a Prior-induced Multi-view Learning Localization Network (PML-LocNet) which exploits both view diversity and sample diversity to improve object localization. In particular, the view diversity is imposed by a two-phase multi-view learning strategy, with which the complementarity among learned features from different views and the consensus among localized instances from each view are leveraged to benefit localization. The sample diversity is pursued by harnessing coarse-to-fine priors at both image and instance levels. With these priors, more emphasis would go to the reliable samples and the contributions of the unreliable ones would be decreased, such that the intrinsic characteristics of each sample can be exploited to make the model more robust during network learning. PML-LocNet can be easily combined with existing WSOL models to further improve the localization accuracy. Its effectiveness has been proved experimentally. Notably, it achieves 69.3% CorLoc and 50.4% mAP on PASCAL VOC 2007, surpassing the state-of-the-arts by a large margin.

Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system

AbstractIt is interesting to develop effective fish sampling techniques using underwater videos and image processing to automatically estimate and consequently monitor the fish biomass and assemblage in water bodies. Such approaches should be robust against substantial variations in scenes due to poor luminosity, orientation of fish, seabed structures, movement of aquatic plants in the background and image diversity in the shape and texture among fish of different species. Keeping this challenge in mind, we propose a unified approach to detect freely moving fish in unconstrained underwater environments using a Region-Based Convolutional Neural Network, a state-of-the-art machine learning technique used to solve generic object detection and localization problems. To train the neural network, we employ a novel approach to utilize motion information of fish in videos via background subtraction and optical flow, and subsequently combine the outcomes with the raw image to generate fish-dependent candidate regions. We use two benchmark datasets extracted from a large Fish4Knowledge underwater video repository, Complex Scenes dataset and the LifeCLEF 2015 fish dataset to validate the effectiveness of our hybrid approach. We achieve a detection accuracy (F-Score) of 87.44% and 80.02% respectively on these datasets, which advocate the utilization of our approach for fish detection task.

Exploring to learn visual saliency: The RL-IAC approach

The problem of object localization and recognition on autonomous mobile robots is still an active topic. In this context, we tackle the problem of learning a model of visual saliency directly on a robot. This model, learned and improved on-the-fly during the robot’s exploration provides an efficient tool for localizing relevant objects within their environment. The proposed approach includes two intertwined components. On the one hand, we describe a method for learning and incrementally updating a model of visual saliency from a depth-based object detector. This model of saliency can also be exploited to produce bounding box proposals around objects of interest. On the other hand, we investigate an autonomous exploration technique to efficiently learn such a saliency model. The proposed exploration, called Reinforcement Learning-Intelligent Adaptive Curiosity (RL-IAC) is able to drive the robot’s exploration so that samples selected by the robot are likely to improve the current model of saliency. We then demonstrate that such a saliency model learned directly on a robot outperforms several state-of-the-art saliency techniques, and that RL-IAC can drastically decrease the required time for learning a reliable saliency model.

Motion Planning and Reactive Control Algorithms for Object Manipulation in Uncertain Conditions

This work proposes the application of several smart strategies for object manipulation tasks. A real-time flexible motion planning method was developed to be adapted to typical in-store logistics scenarios. The solution combines and optimizes some state-of-the-art techniques to solve object recognition and localization problems with a new hybrid pipeline. The algorithm guarantees good robustness and accuracy for object detection through depth images. A standard planner plans collision-free trajectories throughout the whole task while a proposed reactive motion control is active. Distributed proximity sensors were adopted to locally modify the planned trajectory when unexpected or misplaced obstacles intervene in the scene. To implement a robust grasping phase, a novel slipping control algorithm was used. It dynamically computes the grasp force by adapting it to the actual object physical properties so as to prevent slipping. Experimental results carried out in a typical supermarket scenario demonstrate the effectiveness of the presented methods.

Iterative spectral clustering for unsupervised object localization

This paper addresses the problem of unsupervised object localization in an image. Unlike previous supervised and weakly supervised algorithms that require bounding box or image level annotations for training classifiers, we propose a simple yet effective technique for localization using iterative spectral clustering. This iterative spectral clustering approach along with appropriate cluster selection strategy in each iteration naturally helps in searching of object region in the image. In order to estimate the final localization window, we group the proposals obtained from the iterative spectral clustering step based on the perceptual similarity, and average the coordinates of the proposals from the top scoring groups. We benchmark our algorithm on challenging datasets like Object Discovery and PASCAL VOC 2007, achieving an average CorLoc percentage of 51% and 35% respectively which is comparable to various other weakly supervised algorithms despite being completely unsupervised.