Simultaneous Object Detection Research Articles

Simultaneous Object Detection and Distance Estimation for Indoor Autonomous Vehicles

Object detection is an essential and impactful technology in various fields due to its ability to automatically locate and identify objects in images or videos. In addition, object-distance estimation is a fundamental problem in 3D vision and scene perception. In this paper, we propose a simultaneous object-detection and distance-estimation algorithm based on YOLOv5 for obstacle detection in indoor autonomous vehicles. This method estimates the distances to the desired obstacles using a single monocular camera that does not require calibration. On the one hand, we train the algorithm with the KITTI dataset, which is an autonomous driving vision dataset that provides labels for object detection and distance prediction. On the other hand, we collect and label 100 images from a custom environment. Then, we apply data augmentation and transfer learning to generate a fast, accurate, and cost-effective model for the custom environment. The results show a performance of mAP0.5:0.95 of more than 75% for object detection and 0.71 m of mean absolute error in distance prediction, which are easily scalable with the labeling of a larger amount of data. Finally, we compare our method with other similar state-of-the-art approaches.

Multi-object 3D segmentation of brain structures using a geometric deformable model with a priori knowledge

ABSTRACT Brain structure segmentation in 3D Magnetic Resonance Images is crucial for understanding neurodegenerative disorders. Manual segmentation is error-prone, necessitating robust automated techniques. In this paper, we introduce a novel and robust approach for the simultaneous segmentation of multiple brain structures in MRI images. Our method involves the concurrent evolution of 3D surfaces toward predefined anatomical targets, employing an efficient multi-object generalized fast marching method (MOGFMM) for simultaneous object detection. Additionally, we propose an effective evolution function that integrates prior knowledge from anatomical and probabilistic atlases, as well as spatial relationships among the segmented structures. Each deformable surface corresponds to a specific structure. To validate our approach, we conducted experiments on a dataset of real brain images (IBSR) and compared the results with several state-of-the-art methods. The obtained results were promising, demonstrating the effectiveness and superiority of our developed method.

Simultaneous object detection and segmentation for patient-specific markerless lung tumor tracking in simulated radiographs with deep learning.

Real-time tumor tracking is one motion management method to address motion-induced uncertainty. To date, fiducial markers are often required to reliably track lung tumors with X-ray imaging, which carries risks of complications and leads to prolonged treatment time. A markerless tracking approach is thus desirable. Deep learning-based approaches have shown promise for markerless tracking, but systematic evaluation and procedures to investigate applicability in individual cases are missing. Moreover, few efforts have been made to provide bounding box prediction and mask segmentation simultaneously, which could allow either rigid or deformable multi-leaf collimatortracking. The purpose of this study was to implement a deep learning-based markerless lung tumor tracking model exploiting patient-specific training which outputs both a bounding box and a mask segmentation simultaneously. We also aimed to compare the two kinds of predictions and to implement a specific procedure to understand the feasibility of markerless tracking on individualcases. We first trained a Retina U-Net baseline model on digitally reconstructed radiographs (DRRs) generated from a public dataset containing 875 CT scans and corresponding lung nodule annotations. Afterwards, we used an independent cohort of 97 lung patients to develop a patient-specific refinement procedure. In order to determine the optimal hyperparameters for automatic patient-specific training, we selected 13 patients for validation where the baseline model predicted a bounding box on planning CT (PCT)-DRR with intersection over union (IoU) with the ground-truth higher than 0.7. The final test set contained the remaining 84 patients with varying PCT-DRR IoU. For each testing patient, the baseline model was refined on the PCT-DRR to generate a patient-specific model, which was then tested on a separate 10-phase 4DCT-DRR to mimic the intrafraction motion during treatment. A template matching algorithm served as benchmark model. The testing results were evaluated by four metrics: the center of mass (COM) error and the Dice similarity coefficient (DSC) for segmentation masks, and the center of box (COB) error and the DSC for bounding box detections. Performance was compared to the benchmark model including statistical testing forsignificance. A PCT-DRR IoU value of 0.2 was shown to be the threshold dividing inconsistent (68%) and consistent (100%) success (defined as mean bounding box DSC > 0.6) of PS models on 4DCT-DRRs. Thirty-seven out of the eighty-four testing cases had a PCT-DRR IoU above 0.2. For these 37 cases, the mean COM error was 2.6 mm, the mean segmentation DSC was 0.78, the mean COB error was 2.7 mm, and the mean box DSC was 0.83. Including the validation cases, the model was applicable to 50 out of 97 patients when using the PCT-DRR IoU threshold of 0.2. The inference time per frame was 170 ms. The model outperformed the benchmark model on all metrics, and the comparison was significant (p < 0.001) over the 37 PCT-DRR IoU > 0.2 cases, but not over the undifferentiated 84 testingcases. The implemented patient-specific refinement approach based on a pre-trained baseline model was shown to be applicable to markerless tumor tracking in simulated radiographs for lungcases.

A neural learning approach for simultaneous object detection and grasp detection in cluttered scenes.

Object detection and grasp detection are essential for unmanned systems working in cluttered real-world environments. Detecting grasp configurations for each object in the scene would enable reasoning manipulations. However, finding the relationships between objects and grasp configurations is still a challenging problem. To achieve this, we propose a novel neural learning approach, namely SOGD, to predict a best grasp configuration for each detected objects from an RGB-D image. The cluttered background is first filtered out via a 3D-plane-based approach. Then two separate branches are designed to detect objects and grasp candidates, respectively. The relationship between object proposals and grasp candidates are learned by an additional alignment module. A series of experiments are conducted on two public datasets (Cornell Grasp Dataset and Jacquard Dataset) and the results demonstrate the superior performance of our SOGD against SOTA methods in predicting reasonable grasp configurations "from a cluttered scene."

Simultaneous Segmentation and Classification of Pressure Injury Image Data Using Mask-R-CNN.

Pressure injuries (PIs) impose a substantial burden on patients, caregivers, and healthcare systems, affecting an estimated 3 million Americans and costing nearly $18 billion annually. Accurate pressure injury staging remains clinically challenging. Over the last decade, object detection and semantic segmentation have evolved quickly with new methods invented and new application areas emerging. Simultaneous object detection and segmentation paved the way to segment and classify anatomical structures. In this study, we utilize the Mask-R-CNN algorithm for segmentation and classification of stage 1-4 pressure injuries. Images from the eKare Inc. pressure injury wound data repository were segmented and classified manually by two study authors with medical training. The Mask-R-CNN model was implemented using the Keras deep learning and TensorFlow libraries with Python. We split 969 pressure injury images into training (87.5%) and validation (12.5%) subsets for Mask-R-CNN training. We included 121 random pressure injury images in our test set. The Mask-R-CNN model showed overall classification accuracy of 92.6%, and the segmentation demonstrated 93.0% accuracy. Our F1 scores for stages 1-4 were 0.842, 0.947, 0.907, and 0.944, respectively. Our Dice coefficients for stages 1-4 were 0.92, 0.85, 0.93, and 0.91, respectively. Our Mask-R-CNN model provides levels of accuracy considerably greater than the average healthcare professional who works with pressure injury patients. This tool can be easily incorporated into the clinician's workflow to aid in the hospital setting.

A comparison of deep learning algorithms on image data for detecting floodwater on roadways

Object detection and segmentation algorithms evolved significantly in the last decade. Simultaneous object detection and segmentation paved the way for real-time applications such as autonomous driving. Detection and segmentation of (partially) flooded roadways are essential inputs for vehicle routing and traffic management systems. This paper proposes an automatic floodwater detection and segmentation method utilizing the Mask Region-Based Convolutional Neural Networks (Mask-R-CNN) and Generative Adversarial Networks (GAN) algorithms. To train the model, manually labeled images with urban, suburban, and natural settings are used. The performances of the algorithms are assessed in accurately detecting the floodwater captured in images. The results show that the proposed Mask-R-CNN-based floodwater detection and segmentation outperform previous studies, whereas the GAN-based model has a straightforward implementation compared to other models.

A loss-balanced multi-task model for simultaneous detection and segmentation

Scene understanding comes in many flavors, two of the most popular being object detection and semantic segmentation, which act as two important aspects for scene understanding, and are applied to many areas, such as autonomous driving and intelligent surveillance. Although much progress has already been made, the two tasks of object detection and semantic segmentation are often investigated independently. In practice, scene understanding is complicated, and comprises many sub-tasks, so that research of learning multiple tasks simultaneously with a single model is feasible. With the interrelated goals of these two tasks, there is a strong motivation to improve the object detection accuracy with the help of semantic segmentation, and vice versa. In this paper, we propose a loss-balanced multi-task model for simultaneous object detection and semantic segmentation. We explore multi-task learning with sharing parameters based on deep learning to realize improved object detection and segmentation, and propose a single-stage deep architecture based on multi-task learning, jointly performing object detection and semantic segmentation to boost each other. With no more computation load in the inference compared with the baselines of SSD and FCN, we show that these two tasks, object detection and semantic segmentation, benefit from each other. Experimental results on Pascal VOC and COCO show that our method improves much in object detection and semantic segmentation compared with the corresponding baselines of both tasks.

Performance Evaluation of Edge Orientation Histograms Based System for Real-time Object Detection in Two Separate Platforms

Background: Real-time object detection has been attracting much attention recently due the increasing market need of such systems. Therefore, different detection algorithms and techniques have been evaluated to create a reliable detection system. The main challenge to implement a realtime reliable detection system relies on the algorithm training phase. During this phase, a large number of object image database needs to be prepared for each object to be detected. Objective: In this work, we implement a simultaneous object detection system based on local Edge Orientation Histograms (EOH) as feature extraction method with a smaller objects image database. Then, we evaluate the performance of this detection system in two separate platforms. Methods: We evaluated the performance of the detection of Ede Orientation Histograms against HAAR and Local Binary Patterns (LBP) algorithms using two different objects. After that, we discussed the evaluation of the detection systems on the standard platform in addition to the porting process into the embedded platform. Results: We achieved excellent results on both face and hands objects using less than 300 samples. This number is really negligible compared to the size of the image database used by state-of-the-art solutions. In terms of quality of detection, we have achieved more than 93% detection accuracy for the standard platform and 91.8% in the embedded platform for both face and hand objects. Conclusion: In this work, we demonstrated how Edge Orientation Histograms-based detection system gives better performance results than the algorithms evaluated against with less than 300 images database in two separate platforms.

Boosting Multi-Vehicle Tracking with a Joint Object Detection and Viewpoint Estimation Sensor.

In this work, we address the problem of multi-vehicle detection and tracking for traffic monitoring applications. We preset a novel intelligent visual sensor for tracking-by-detection with simultaneous pose estimation. Essentially, we adapt an Extended Kalman Filter (EKF) to work not only with the detections of the vehicles but also with their estimated coarse viewpoints, directly obtained with the vision sensor. We show that enhancing the tracking with observations of the vehicle pose, results in a better estimation of the vehicles trajectories. For the simultaneous object detection and viewpoint estimation task, we present and evaluate two independent solutions. One is based on a fast GPU implementation of a Histogram of Oriented Gradients (HOG) detector with Support Vector Machines (SVMs). For the second, we adequately modify and train the Faster R-CNN deep learning model, in order to recover from it not only the object localization but also an estimation of its pose. Finally, we publicly release a challenging dataset, the GRAM Road Traffic Monitoring (GRAM-RTM), which has been especially designed for evaluating multi-vehicle tracking approaches within the context of traffic monitoring applications. It comprises more than 700 unique vehicles annotated across more than 40.300 frames of three videos. We expect the GRAM-RTM becomes a benchmark in vehicle detection and tracking, providing the computer vision and intelligent transportation systems communities with a standard set of images, annotations and evaluation procedures for multi-vehicle tracking. We present a thorough experimental evaluation of our approaches with the GRAM-RTM, which will be useful for establishing further comparisons. The results obtained confirm that the simultaneous integration of vehicle localizations and pose estimations as observations in an EKF, improves the tracking results.

The challenge of simultaneous object detection and pose estimation: A comparative study

Detecting objects and estimating their pose remains as one of the major challenges of the computer vision research community. There exists a compromise between localizing the objects and estimating their viewpoints. The detector ideally needs to be view-invariant, while the pose estimation process should be able to generalize towards the category-level. This work is an exploration of using deep learning models for solving both problems simultaneously. For doing so, we propose three novel deep learning architectures, which are able to perform a joint detection and pose estimation, where we gradually decouple the two tasks. We also investigate whether the pose estimation problem should be solved as a classification or regression problem, being this still an open question in the computer vision community. We detail a comparative analysis of all our solutions and the methods that currently define the state of the art for this problem. We use PASCAL3D+ and ObjectNet3D datasets to present the thorough experimental evaluation and main results. With the proposed models we achieve the state-of-the-art performance in both datasets.

Learning 3D Object Templates by Quantizing Geometry and Appearance Spaces.

While 3D object-centered shape-based models are appealing in comparison with 2D viewer-centered appearance-based models for their lower model complexities and potentially better view generalizabilities, the learning and inference of 3D models has been much less studied in the recent literature due to two factors: i) the enormous complexities of 3D shapes in geometric space; and ii) the gap between 3D shapes and their appearances in images. This paper aims at tackling the two problems by studying an And-Or Tree (AoT) representation that consists of two parts: i) a geometry-AoT quantizing the geometry space, i.e. the possible compositions of 3D volumetric parts and 2D surfaces within the volumes; and ii) an appearance-AoT quantizing the appearance space, i.e. the appearance variations of those shapes in different views. In this AoT, an And-node decomposes an entity into constituent parts, and an Or-node represents alternative ways of decompositions. Thus it can express a combinatorial number of geometry and appearance configurations through small dictionaries of 3D shape primitives and 2D image primitives. In the quantized space, the problem of learning a 3D object template is transformed to a structure search problem which can be efficiently solved in a dynamic programming algorithm by maximizing the information gain. We focus on learning 3D car templates from the AoT and collect a new car dataset featuring more diverse views. The learned car templates integrate both the shape-based model and the appearance-based model to combine the benefits of both. In experiments, we show three aspects: 1) the AoT is more efficient than the frequently used octree method in space representation; 2) the learned 3D car template matches the state-of-the art performances on car detection and pose estimation in a public multi-view car dataset; and 3) in our new dataset, the learned 3D template solves the joint task of simultaneous object detection, pose/view estimation, and part localization. It can generalize over unseen views and performs better than the version5 of the DPM model in terms of object detection and semantic part localization.

Multi-robot system using low-cost infrared sensors

This paper presents a proposed set of the novel technique, methods, and algorithm for simultaneous path planning, area exploration, area retrieval, obstacle avoidance, object detection, and object retrieval autonomously by a multi-robot system. The proposed methods and algorithms are built considering the use of low cost infrared sensors with the ultimate function of efficiently exploring the given unknown area and simultaneously identifying desired objects by analyzing the physical characteristics of several of the objects that come across during exploration. In this paper, we have explained the scenario by building a coordinative multi-robot system consisting of two autonomously operated robots equipped with low-cost and low-range infrared sensors to perform the assigned task by analyzing some of the sudden changes in their environment. Along with identifying and retrieving the desired object, the proposed methodology also provide an inclusive analysis of the area being explored. The novelties presented in the paper may significantly provide a cost-effective solution to the problem of area exploration and finding a known object in an unknown environment by demonstrating an innovative approach of using the infrared sensors instead of high cost long range sensors and cameras. Additionally, the methodology provides a speedy and uncomplicated method of traversing a complicated arena while performing all the necessary and inter-related tasks of avoiding the obstacles, analyzing the area as well as objects, and reconstructing the area using all these information collected and interpreted for an unknown environment. The methods and algorithms proposed are simulated over a complex arena to depict the operations and manually tested over a physical environment which provided 78% correct results with respect to various complex parameters set randomly.

GEOMETRIC INVARIANTS CONSTRUCTION FOR SEMANTIC SCENE UNDERSTANDING FROM MULTIPLE VIEWS INSPIRED BY THE HUMAN VISUAL SYSTEM

Semantic scene understanding is one of the several significant goals of robotics. In this paper, we propose a framework that is able to construct geometric invariants for simultaneous object detection and segmentation using a simple pairwise interactive context term, for the sake of achieving a preliminary milestone of Semantic scene understanding. The context is incorporated as pairwise interactions between pixels, imposing a prior on the labeling. Our model formulates the multi-class image segmentation task as an energy minimization problem and finds a globally optimal solution using belief propagation and neural network. We experimentally evaluate the proposed method on three publicly available datasets: the MSRC-1, the CorelB datasets, and the PASCAL VOC database. Results show the applicability and efficacy of the proposed method to the multi-class segmentation problem.

Adaptive background generation for automatic detection of initial object region in multiple color-filter aperture camera-based surveillance system

In this paper, we present an adaptive background generation method for automatic selection of initial object regions, which realizes simultaneous object detection and depth estimation using multiple color-filter aperture (MCA) camera. Since the conventional background generation method does not fit the depth estimation using the MCA camera, we propose a novel color-based background generation method which can reduce interference in the object region for stable depth estimation. For efficient estimation of color shifting vectors in the extracted object region, a simplified elastic registration (ER) algorithm is used. The proposed simplified method is essential factor to realize realtime depth estimation and tracking, which is the primary condition for consumer applications. Finally, the object distance is determined by using the relationship between the pre-specified distance transformation function and the estimated shifting vectors of the corresponding object region. Although traditional depth estimation methods generally use dual cameras for stereo vision, the proposed method uses only a single camera for both object detection and depth estimation. Experimental results show that the proposed MCA camera-based object detection system can be used in a variety of consumer surveillance systems such as intelligent transport systems, 3D-based cameras and advanced safety vehicles.

Adaptive object detection and recognition based on a feedback strategy

Detecting and recognizing objects in environments with unpredictable illumination changes remains a challenging task. Existing algorithms employ a passive methodology to deal with these environments, where learning is performed from many samples taken under various lighting conditions or with some pre-designed color constancy models. In this paper, the challenges of unpredictable illumination changes are addressed through a feedback strategy. With the use of feedback, self-adaptation in object detection and recognition is possible in response to variable illumination. Self-adaptation is achieved through feedback from the recognition phase to the detection phase. A multilevel Markov random field (MRF) is adopted to model both the detection and recognition processes. The original MRF approach is extended to a model that encodes simultaneous object detection and recognition. Experimental results show the feasibility of the proposed framework.