Deformable Part-based Model Research Articles

Vision-Based Human Detection Techniques: A Descriptive Review

Cameras are being used everywhere for the safety and security of citizens in different countries. Using a machine to detect humans in a photo or a video frame is a very complicated and challenging task. Various techniques have been developed for this purpose, which mainly rely on Artificial Intelligence. This article aims to provide a comprehensive review and analysis of the literatures from a descriptive perspective, which is its main differentiator from the existing survey papers in this area. Firstly, the vision-based human detection techniques and classifiers are elucidated in conjunction with the variants of feature extraction techniques. Secondly, various pros and cons of such techniques are discussed. Then, an investigation has been conducted and reported based on the state-of-the-art human detection descriptors (e.g. Log-Average Miss Rate and accuracy). Although techniques such as Viola-Jones and Speeded-Up Robust Features can detect objects in real-time and overcome Scale-Invariant Feature Transform (SIFT) limitations, they are still sensitive to illuminated conditions. Other techniques such as SIFT, Bag of Words, Orthogonal Moments, and Histogram of oriented Gradients provide other interesting benefits which include insensitivity to occlusion and clutters, simplicity, low-order element construction and invariance to illuminated conditions; nevertheless, they are computationally expensive and sensitive to image rotation. A meticulous review along similar lines revealed that the Deformable Part-based Model performs relatively better due to its ability to deal with particular pose variations and multiple views, occlusion handling (partial) and is application-free while its counterparts focus on only a single aspect. This article highlights and provides a brief description of each available data-sets for human detection research. Various use-cases of human detection systems are also elaborated. Finally, various conclusions are derived based on the conducted review followed by recommendations for future directions and possibilities to further improve the speed and accuracy of human detection systems.

Efficient Scale-Adaptive License Plate Detection System

License plate detection is a common problem in traffic surveillance applications. Although some solutions have been proposed in the literature, their success is usually restricted to very specific scenarios, with their performance dropping in more demanding conditions. One of the main challenges to be addressed for this kind of systems is the varying scale of the license plates, which depends on the distance between the vehicles and the camera. Traditionally, systems have handled this issue by sequentially running single-scale detectors over a pyramid of images. This approach, although simplifies the training process, requires as many evaluations as considered scales, which leads to running times that grow linearly with the number of scales considered. In this paper, we propose a scale-adaptive deformable part-based model which, based on a well-known boosting algorithm, automatically models scale during the training phase by selecting the most prominent features at each scale and notably reduces the test detection time by avoiding the evaluation at different scales. In addition, our method incorporates an empirically constrained-deformation model that adapts to different levels of deformation shown by distinct local features within license plates. As shown in the experimental section, the proposed detector is robust and scale and perspective independent and can work in quite diverse scenarios. Experiments on two datasets show that the proposed method achieves a significantly better performance in comparison with other methods of the state of the art.

DSN: A New Deformable Subnetwork for Object Detection

Although deep convolutional neural networks have achieved great success in object detection, they depend heavily on considerable training data and do not have a specific mechanism to handle related challenging problems, such as object deformation. In this paper, we design a deformable subnetwork (DSN) to introduce the deformable part-based model (DPM) in the deep object detection framework. It is effective for handling object deformation and is composed of two significant parts: the deformation coefficient part and the deformation pooling part. The deformation coefficient part is responsible for generating the deformation coefficients for each position of the input. The deformation pooling part calculates the final score for each position which takes into account its displacement penalty relative to the root position. DSN is convenient for being embedded into the most prevalent object detection frameworks such as faster-RCNN or RFCN. More importantly, it does not impair the integrity of the original framework and only causes little time consumption. We show the effectiveness of DSN via experiments on the PASCAL VOC and COCO datasets, achieving state-of-the-art results, 82.7% for PASCAL VOC and 32.1% for COCO.

Detection of vehicle wheels from images using a pseudo‐wavelet filter for analysis of congested traffic

There is potential for significant savings if the safety of existing bridges can be more accurately assessed. For long-span bridges, congestion is the governing traffic load condition. The current methods of simulating congestion make assumptions about the axle-to-axle gaps maintained between vehicles. There is potential for improvement in congestion models if accurate data on axle-to-axle gaps can be obtained. In this study, the use of a camera to collect this information is put forward. A new image processing technique is proposed to detect wheels in variable light conditions. The method is based on a pseudo-wavelet filter that amplifies circles, in conjunction with an algorithm that weights features in the image according to their circularity. This new approach is compared with the Hough transform, template matching and the deformable part-based model (DPM) methods previously developed. In a sample set of 80 images, 96.9% of wheels are detected, considerably more than with the Hough transform and template matching methods. It also provides the same level of accuracy as DPM without requiring a training process.

Detection of Infantile Movement Disorders in Video Data Using Deformable Part-Based Model.

Movement analysis of infants’ body parts is momentous for the early detection of various movement disorders such as cerebral palsy. Most existing techniques are either marker-based or use wearable sensors to analyze the movement disorders. Such techniques work well for adults, however they are not effective for infants as wearing such sensors or markers may cause discomfort to them, affecting their natural movements. This paper presents a method to help the clinicians for the early detection of movement disorders in infants. The proposed method is marker-less and does not use any wearable sensors which makes it ideal for the analysis of body parts movement in infants. The algorithm is based on the deformable part-based model to detect the body parts and track them in the subsequent frames of the video to encode the motion information. The proposed algorithm learns a model using a set of part filters and spatial relations between the body parts. In particular, it forms a mixture of part-filters for each body part to determine its orientation which is used to detect the parts and analyze their movements by tracking them in the temporal direction. The model is represented using a tree-structured graph and the learning process is carried out using the structured support vector machine. The proposed framework will assist the clinicians and the general practitioners in the early detection of infantile movement disorders. The performance evaluation of the proposed method is carried out on a large dataset and the results compared with the existing techniques demonstrate its effectiveness.

Unified Partial Configuration Model Framework for Fast Partially Occluded Object Detection in High-Resolution Remote Sensing Images

Partially occluded object detection (POOD) has been an important task for both civil and military applications that use high-resolution remote sensing images (HR-RSIs). This topic is very challenging due to the limited object evidence for detection. Recent partial configuration model (PCM) based methods deal with occlusion yet suffer from the problems of massive manual annotation, separate parameter learning, and low training and detection efficiency. To tackle this, a unified PCM framework (UniPCM) is proposed in this paper. The proposed UniPCM adopts a part sharing mechanism which directly shares the root and part filters of a deformable part-based model (DPM) among different partial configurations. It largely reduces the convolution overhead during both training and detection. In UniPCM, a novel DPM deformation deviation method is proposed for spatial interrelationship estimation of PCM, and a unified weights learning method is presented to simultaneously obtain the weights of elements within each partial configuration and the weights between partial configurations. Experiments on three HR-RSI datasets show that the proposed UniPCM method achieves a much higher training and detection efficiency for POOD compared with state-of-the-art PCM-based methods, while maintaining a comparable detection accuracy. UniPCM obtains a training speedup of maximal 10× and 2.5× for airplane and ship, and a detection speedup of maximal 7.2×, 4.1× and 2.5× on three test sets, respectively.

Multiple Object Detection by a Deformable Part-Based Model and an R-CNN

Multiple object detection is a key challenge in object detection. Feature extraction and occlusion handling are two key elements in multiple object detection. However, existing methods do not perform well in these aspects during detecting multiple objects. A region-based convolutional network (R-CNN) has achieved a great success in region-based feature extraction, and the part filters in a deformable part-based model (DPM) are very suitable for detecting occluded objects. In this letter, we present a framework that integrates the R-CNN and the DPM for detecting multiple objects. In addition, we propose a new filter based on the dense subgraph discovery algorithm for refining the candidate proposals generated by the DPM. Through combining these two models, we can detect each single object with high accuracy among all objects in an image, especially in the situation that objects are stayed close together. Different from the traditional methods, our framework has the capability to detect multiple objects belonging to various classes rather than only one typical class such as a person or a car. Experimental results on the PASCAL VOC dataset show that our framework achieves better performance compared to using only the R-CNN or the DPM alone in multiple object detection.

A Semi-Supervised Image Classification Model Based on Improved Ensemble Projection Algorithm

Image classification has been an incredibly active research topic in recent years with widespread applications. Researchers have put forward many remarkable techniques and semi-supervised learning (SSL) is one among them. However, due to not taking the relationship of samples among different classes in consideration, previous approaches cannot often get a clear decision boundary. In this paper, we propose an improved classification model on the basis of SSL. First, we adopt a deformable part-based model to capture a stable global structure and salient objects, and then, we find a better decision boundary by our classification algorithm-based on an improved ensemble projection (IEP). Our IEP exploits the weighted average method. To evaluate the effectiveness of our approach, we do experiments not only with the LandUse-21 (L-21) data set, but also with an architecture style data set. Experimental results show that our approach is capable of achieving the state-of-the-art performance on the two data sets. For each class in L-21 data set, when 50 images are randomly chosen as training images, the multi-class average precision increases to 97.63%. Besides, for the architecture style data set, we achieve the best result with about 80% accuracy and have about a 10% improvement over the previous best work. Although there are a small number of labeled data used to train, we get the satisfactory performance.

Classify vehicles in traffic scene images with deformable part-based models

Vehicle classification is an important and challenging task in intelligent transportation systems, which has a wide range of applications. In this paper, we propose to integrate vehicle detection and vehicle classification into one single framework by using deformable part-based models. First of all, we use annotated vehicle images to train a deformable part-based model for each class of vehicles to be classified. Then, given a traffic scene image, we employ the obtained vehicle models to perform vehicle detection in it for vehicle extraction. After that, model alignment is performed on the extracted image crop, based on which features are extracted for creating a representation for the vehicle in the given image. We train a linear multi-class Support Vector Machine classifier based on representations of images in a validation set. Finally, we adopt the SVM classifier for vehicle classification. The proposed method is evaluated on the BIT-Vehicle Dataset, and can achieve an accuracy of $$91.08\%$$ , which is superior to methods used for comparison. Obtained results demonstrated the effectiveness of the proposed method.

Automatic and Fast PCM Generation for Occluded Object Detection in High-Resolution Remote Sensing Images

Partial configuration model (PCM) is an occluded object detection method in high-resolution remote sensing images (HR-RSIs) based on the deformable part-based model (DPM). However, it needs extra category predefinition, considerable part-level annotation, and repeated multimodel training. In this letter, an automatic and fast PCM generation method is proposed based on a novel part sharing mechanism. We propose to share parts from one trained DPM model (tDPM) among different models of partial configurations (PCs) to address the above problems. PCs are first designed according to part anchors of tDPM. The model is then generated through corresponding parts selection, root coverage cropping, and elements reweighing. This method avoids the need for manual category predefinition and part-level annotation, while largely reducing the computation of PCM training. Experimental results on three HR-RSI data sets show that the proposed method obtains a training speedup of $6.7\times $ and $2\times $ for each PC of airplane and ship categories, while achieving a comparable accuracy compared with PCM.

Training my car to see using virtual worlds

Computer vision technologies are at the core of different advanced driver assistance systems (ADAS) and will play a key role in oncoming autonomous vehicles too. One of the main challenges for such technologies is to perceive the driving environment, i.e. to detect and track relevant driving information in a reliable manner (e.g. pedestrians in the vehicle route, free space to drive through). Nowadays it is clear that machine learning techniques are essential for developing such a visual perception for driving. In particular, the standard working pipeline consists of collecting data (i.e. on-board images), manually annotating the data (e.g. drawing bounding boxes around pedestrians), learning a discriminative data representation taking advantage of such annotations (e.g. a deformable part-based model, a deep convolutional neural network), and then assessing the reliability of such representation with the acquired data. In the last two decades most of the research efforts focused on representation learning (first, designing descriptors and learning classifiers; later doing it end-to-end). Hence, collecting data and, especially, annotating it, is essential for learning good representations. While this has been the case from the very beginning, only after the disruptive appearance of deep convolutional neural networks that it became a serious issue due to their data hungry nature. In this context, the problem is that manual data annotation is a tiresome work prone to errors. Accordingly, in the late 00’s we initiated a research line consisting of training visual models using photo-realistic computer graphics, especially focusing on assisted and autonomous driving. In this paper, we summarize such a work and show how it has become a new tendency with increasing acceptance.

Occluded Object Detection in High-Resolution Remote Sensing Images Using Partial Configuration Object Model

Deformable-part-based model (DPM) has shown great success in object detection in recent years. However, its performance will degrade on partially occluded objects and is even worse on largely occluded objects in real remote sensing applications. To address this problem, a novel partial configuration object model (PCM) is developed in this paper. Compared to conventional single-layer DPMs, an extra partial configuration layer, which is composed of partial configurations defined according to possible occlusion patterns, is introduced in PCM to block the transmission of occlusion impact. During detection, each hypothesis from a partial configuration layer will infer the entire object based on spatial interrelationship and final detection results are obtained from the fusion of these possible entire objects using a weighted continuous clustering method. As PCM makes a better compromise between the deformation modeling flexibility of small parts and the discriminative shape-capturing capability of large DPM, its performance on occluded object detection will be improved. Moreover, occlusion states of detected objects can be inferred with the intermediate results of our model. Experimental results on multiple high-resolution remote sensing image datasets demonstrate the effectiveness of the proposed model.

Partial Occlusion Handling in Pedestrian Detection With a Deep Model

Part-based models have demonstrated their merit in object detection. However, there is a key issue to be solved on how to integrate the inaccurate scores of part detectors when there are occlusions, abnormal deformations, appearances, or illuminations. To handle the imperfection of part detectors, this paper presents a probabilistic pedestrian detection framework. In this framework, a deformable part-based model is used to obtain the scores of part detectors and the visibilities of parts are modeled as hidden variables. Once the occluded parts are identified, their effects are properly removed from the final detection score. Unlike previous occlusion handling approaches that assumed independence among the visibility probabilities of parts or manually defined rules for the visibility relationship, a deep model is proposed in this paper for learning the visibility relationship among overlapping parts at multiple layers. The proposed approach can be viewed as a general postprocessing of part-detection results and can take detection scores of existing part-based models as input. The experimental results on three public datasets (Caltech, ETH, and Daimler) and a new CUHK occlusion dataset ( http://www.ee.cuhk.edu.hk/~xgwang/CUHK_pedestrian.html ), which is specially designed for the evaluation of occlusion handling approaches, show the effectiveness of the proposed approach.

Hierarchical Adaptive Structural SVM for Domain Adaptation

A key topic in classification is the accuracy loss produced when the data distribution in the training (source) domain differs from that in the testing (target) domain. This is being recognized as a very relevant problem for many computer vision tasks such as image classification, object detection, and object category recognition. In this paper, we present a novel domain adaptation method that leverages multiple target domains (or sub-domains) in a hierarchical adaptation tree. The core idea is to exploit the commonalities and differences of the jointly considered target domains. Given the relevance of structural SVM (SSVM) classifiers, we apply our idea to the adaptive SSVM (A-SSVM), which only requires the target domain samples together with the existing source-domain classifier for performing the desired adaptation. Altogether, we term our proposal as hierarchical A-SSVM (HA-SSVM). As proof of concept we use HA-SSVM for pedestrian detection and object category recognition. In the former we apply HA-SSVM to the deformable part-based model (DPM) while in the latter HA-SSVM is applied to multi-category classifiers. In both cases, we show how HA-SSVM is effective in increasing the detection/recognition accuracy with respect to adaptation strategies that ignore the structure of the target data. Since, the sub-domains of the target data are not always known a priori, we shown how HA-SSVM can incorporate sub-domain structure discovery for object category recognition.

Weakly Supervised Large Scale Object Localization with Multiple Instance Learning and Bag Splitting.

Localizing objects of interest in images when provided with only image-level labels is a challenging visual recognition task. Previous efforts have required carefully designed features and have difficulty in handling images with cluttered backgrounds. Up-scaling to large datasets also poses a challenge to applying these methods to real applications. In this paper, we propose an efficient and effective learning framework called MILinear, which is able to learn an object localization model from large-scale data without using bounding box annotations. We integrate rich general prior knowledge into a learning model using a large pre-trained convolutional network. Moreover, to reduce ambiguity in positive images, we present a bag-splitting algorithm that iteratively generates new negative bags from positive ones. We evaluate the proposed approach on the challenging Pascal VOC 2007 dataset, and our method outperforms other state-of-the-art methods by a large margin; some results are even comparable to fully supervised models trained with bounding box annotations. To further demonstrate scalability, we also present detection results on the ILSVRC 2013 detection dataset, and our method outperforms supervised deformable part-based model without using box annotations.

DietCam: Multiview Food Recognition Using a Multikernel SVM.

Food recognition is a key component in evaluation of everyday food intakes, and its challenge is due to intraclass variation. In this paper, we present an automatic food classification method, DietCam, which specifically addresses the variation of food appearances. DietCam consists of two major components, ingredient detection and food classification. Food ingredients are detected through a combination of a deformable part-based model and a texture verification model. From the detected ingredients, food categories are classified using a multiview multikernel SVM. In the experiment, DietCam presents reliability and outperformance in recognition of food with complex ingredients on a database including 15,262 food images of 55 food types.

Domain Adaptation of Deformable Part-Based Models.

The accuracy of object classifiers can significantly drop when the training data (source domain) and the application scenario (target domain) have inherent differences. Therefore, adapting the classifiers to the scenario in which they must operate is of paramount importance. We present novel domain adaptation (DA) methods for object detection. As proof of concept, we focus on adapting the state-of-the-art deformable part-based model (DPM) for pedestrian detection. We introduce an adaptive structural SVM (A-SSVM) that adapts a pre-learned classifier between different domains. By taking into account the inherent structure in feature space (e.g., the parts in a DPM), we propose a structure-aware A-SSVM (SA-SSVM). Neither A-SSVM nor SA-SSVM needs to revisit the source-domain training data to perform the adaptation. Rather, a low number of target-domain training examples (e.g., pedestrians) are used. To address the scenario where there are no target-domain annotated samples, we propose a self-adaptive DPM based on a self-paced learning (SPL) strategy and a Gaussian Process Regression (GPR). Two types of adaptation tasks are assessed: from both synthetic pedestrians and general persons (PASCAL VOC) to pedestrians imaged from an on-board camera. Results show that our proposals avoid accuracy drops as high as 15 points when comparing adapted and non-adapted detectors.

Image Segmentation by Pre-Labeling Based on Part-Based Model

Interactive segmentation with graph cuts has become very popular and many priors have been introduced into graph cuts to improve the results. This paper proposed a method which uses the deformable part-based model to pre-label the seeds. First the deformable part-based model finds out the bounding box, then we can pre-label the seed point based on the assumption of compact shape. Our results show that our method can get more accurate result especially the appearance of the object and background are similar and the shape is compact.

Graph Cuts Based Image Segmentation with Part-Based Models

This study proposed an improved pre-labeling method based on deformable part models and HOG features for interactive segmentation with graph cuts. Because of the complex appearance of foreground and background, the result of segmentation is unsatisfactory. Many priors have been introduced into graph cuts to improve the segmentation results and our work is inspired by the shape prior. In this paper we use the deformable part-based model and HOG features to pre-label the seeds before the graph cuts algorithm. The user involvement is reduced and the performance of the graph cuts algorithm is improved at the first iteration. Our assumption is based on the compact shape. We assume that the area between the center of the part filter and root filter belongs to foreground. If the area covered by more filters, it will more probably be the foreground. Our results show that our method can get more accurate result especially the appearance of the object and background is similar and the shape of the object close to rectangle and eclipse.