Random Ferns Research Articles

Multiple pedestrian tracking in moving vehicle using online learning of random ferns and feature descriptor of pre-trained shallow Convolutional Neural Networks

In this paper, we introduce a multi-pedestrian tracking algorithm for tracking from a moving vehicle. The method is based on online learning of a random ferns (RF) tracker model using the output features of a convolutional neural network (CNN). For real-time application in vehicles, an online method is applied within the tracking-by-detection framework where data association between detections and trackers is conducted online. To predict the tracker's position, we perform particle filtering with tracker models inferred from a shallow CNN. In this study, You Only Look Once (YOLO), a real-time object detection system, was adopted as the pre-trained model. Although YOLO has an accurate network for object classification, it is not appropriate for real-time multi-pedestrian tracking. Therefore, we use modified YOLO to obtain a shallow version (S-YOLO) having fewer convolutional layers and fewer filters in these layers. To update the tracker in every frame, positive and negative samples are applied to the S-YOLO and retraining is performed. Then, we extract feature descriptors from the first fully connected layer of S-YOLO to train the RF tracker models. The proposed algorithm was successfully applied to various pedestrian video sequences and yielded a more accurate tracking performance than other existing method.

Online Tracker Optimization for Multi-Pedestrian Tracking Using a Moving Vehicle Camera

Multi-pedestrian tracking (MPT) on the road is closely related to a reduction in the possibility of pedestrian–vehicle collisions when using advanced driver assistance systems. Therefore, this paper focuses on MPT on real roads using a moving camera. Although convolutional neural network (CNN)-based single object tracking methods have recently been proposed, the online learning of CNN for MPT is a significant burden in terms of real-time processing. However, because the features extracted from CNN are a good representation and achieve a high generalization capability, to reduce the learning time, a shallow pre-trained CNN was applied in the present study as a feature extractor instead of an object tracker As an online tracker, this study uses random ferns (RF) with pre-trained CNN output feature, which can make the problem computationally tractable and robustness. However, a large RF requires significant amount of memory and computational complexity, and is still a burden in terms of online learning in cases of multi-pedestrian tracking. Therefore, we introduce a teacher–student model compression algorithm for selecting a few optimal ferns for each tracker, thereby, reducing the online learning time. The online learning of a student-RF tracker has an advantage in that it can adaptively update the tracker to achieve robustness against various changes, such as in pose and illumination, and partial occlusions within a limited period of time. The proposed algorithm was successfully applied to benchmark video sequences captured from a moving camera that include multiple pedestrians in various poses. Specifically, the proposed algorithm yields a more accurate tracking performance than the other state-of-the-art methods.

Exploiting Lightweight Statistical Learning for Event-Based Vision Processing.

This paper presents a lightweight statistical learning framework potentially suitable for low-cost event-based vision systems, where visual information is captured by a dynamic vision sensor (DVS) and represented as an asynchronous stream of pixel addresses (events) indicating a relative intensity change on those locations. A simple random ferns classifier based on randomly selected patch-based binary features is employed to categorize pixel event flows. Our experimental results demonstrate that compared to existing event-based processing algorithms, such as spiking convolutional neural networks (SCNNs) and the state-of-the-art bag-of-events (BoE)-based statistical algorithms, our framework excels in high processing speed (2× faster than the BoE statistical methods and >100× faster than previous SCNNs in training speed) with extremely simple online learning process, and achieves state-of-the-art classification accuracy on four popular address-event representation data sets: MNIST-DVS, Poker-DVS, Posture-DVS, and CIFAR10-DVS. Hardware estimation shows that our algorithm will be preferable for low-cost embedded system implementations.

Corrected Continuous Correlation Filter for Long-Term Tracking

Recently, a large number of visual tracking algorithms based on discriminative correlation filter have been proposed with demonstrated success. However, most of algorithms cannot well handle long-term videos in which the locating error may accumulate and lead to drifting or tracking failure. Hence, it is of great importance to design a robust long-term tracker which can effectively alleviate tracking drift and redetect the object in case of tracking failure. In this paper, a continuous correlation filter has been proposed to achieve subpixel object locations in continuous domain. For scale estimation, we present a novel multipyramid strategy and the optimal scale tracker is used to correct object locating error in return. Meanwhile, we learn an online random fern classifier to redetect the target in case of tracking failure. By analyzing the confidence of predicted location, we update the translation model conservatively by the reliable targets throughout the sequence. To evaluate the proposed algorithm, extensive experiments are conducted on a benchmark with 100 video sequences, which demonstrate that our tracking mechanism is well fit to tackle long-term sequences and outperforms the state-of-the-art methods.

基于移动平台热成像的行人检测随机蕨分类器

基于移动平台热成像的行人检测随机蕨分类器

Fast segmentation of kidney components using random forests and ferns.

This paper studies the feasibility of developing a fast and accurate automatic kidney component segmentation method. The proposed method segments the kidney into four components: renal cortex, renal column, renal medulla, and renal pelvis. In this article, we have proposed a highly efficient approach which strategically combines random forests and random ferns methods to segment the kidney into four components: renal cortex, renal column, renal medulla, and renal pelvis. The proposed method is designed following a coarse-to-fine strategy. The initial segmentation applies random forests and random ferns with a variety of features, and combines their results to obtain a coarse renal cortex region. Then the fine segmentation of four kidney components is achieved using the weighted forests-ferns approach with the well-designed potential energy features which are calculated based on the initial segmentation result. The proposed method was validated on a dataset with 37 contrast-enhanced CT images. Evaluation indices including Dice similarity coefficient (DSC), true positive volume fraction (TPVF), and false positive volume fraction (FPVF) are used to assess the segmentation accuracy. The proposed method was implemented and tested on a 64-bit system computer (Intel Core i7-3770 CPU, 3.4 GHz and 8 GB RAM). The experimental results demonstrated the high accuracy and efficiency for segmenting the kidney components: the mean Dice similarity coefficients were 89.85%, 80.60%, 86.63%, and 77.75% for renal cortex, column, medulla, and pelvis, respectively, for right and left kidneys. The computational time of segmenting the whole kidney into four components was about 3 s. The experimental results showed the feasibility and efficacy of the proposed automatic kidney component segmentation method. The proposed method applied an efficient weighted strategy to combine random forests and ferns, making full use of the advantages of both methods. The novel potential energy features help random forests effectively segment the kidney components and the background. The high accuracy and efficiency of our method make it practicable in clinical applications.

Body pose estimation in depth images for infant motion analysis.

Motion analysis of infants is used for early detection of movement disorders like cerebral palsy. For the development of automated methods, capturing the infant's pose accurately is crucial. Our system for predicting 3D joint positions is based on a recently introduced pixelwise body part classifier using random ferns, to which we propose multiple enhancements. We apply a feature selection step before training random ferns to avoid the inclusion of redundant features. We introduce a kinematic chain reweighting scheme to identify and to correct misclassified pixels, and we achieve rotation invariance by performing PCA on the input depth image. The proposed methods improve pose estimation accuracy by a large margin on multiple recordings of infants. We demonstrate the suitability of the approach for motion analysis by comparing predicted knee angles to ground truth angles.

Rotation-invariant object detection using Sector-ring HOG and boosted random ferns

The histogram of oriented gradients (HOG) is widely used for image description and has proven to be very effective. In some practical applications that lack an assumption of the object’s orientation, rotation-invariant detection is of vital significance. To address this problem, this paper presents a new visual feature, Sector-ring HOG (SRHOG), which is obtained by improving the gradient binning and spatial binning based on HOG. The new feature can convert planar image rotations into cyclic shifts of the final descriptor and thereby facilitate rotated object detection. After modifying boosted random ferns in SRHOG feature domain, we further propose two strategies for rotation-invariant object detection: one depends completely on the new feature’s characteristic, and the other introduces an orientation estimation step. The former is more suitable to ‘finding objects’ and the latter can provide the higher orientation estimation accuracy. Both the use of supervised learning and working in the gradient space make our approaches effective and robust. We show these properties by thorough testing on the public Freestyle Motocross dataset and our dataset for victim detection in post-disaster rescue efforts.

Real-Time Multi-View Facial Capture with Synthetic Training

We present a real-time multi-view facial capture system facilitated by synthetic training imagery. Our method is able to achieve high-quality markerless facial performance capture in real-time from multi-view helmet camera data, employing an actor specific regressor. The regressor training is tailored to specified actor appearance and we further condition it for the expected illumination conditions and the physical capture rig by generating the training data synthetically. In order to leverage the information present in live imagery, which is typically provided by multiple cameras, we propose a novel multi-view regression algorithm that uses multi-dimensional random ferns. We show that higher quality can be achieved by regressing on multiple video streams than previous approaches that were designed to operate on only a single view. Furthermore, we evaluate possible camera placements and propose a novel camera configuration that allows to mount cameras outside the field of view of the actor, which is very beneficial as the cameras are then less of a distraction for the actor and allow for an unobstructed line of sight to the director and other actors. Our new real-time facial capture approach has immediate application in on-set virtual production, in particular with the ever-growing demand for motion-captured facial animation in visual effects and video games.

Boosted Random Ferns for Object Detection.

In this paper we introduce the Boosted Random Ferns (BRFs) to rapidly build discriminative classifiers for learning and detecting object categories. At the core of our approach we use standard random ferns, but we introduce four main innovations that let us bring ferns from an instance to a category level, and still retain efficiency. First, we define binary features on the histogram of oriented gradients-domain (as opposed to intensity-), allowing for a better representation of intra-class variability. Second, both the positions where ferns are evaluated within the sliding window, and the location of the binary features for each fern are not chosen completely at random, but instead we use a boosting strategy to pick the most discriminative combination of them. This is further enhanced by our third contribution, that is to adapt the boosting strategy to enable sharing of binary features among different ferns, yielding high recognition rates at a low computational cost. And finally, we show that training can be performed online, for sequentially arriving images. Overall, the resulting classifier can be very efficiently trained, densely evaluated for all image locations in about 0.1 seconds, and provides detection rates similar to competing approaches that require expensive and significantly slower processing times. We demonstrate the effectiveness of our approach by thorough experimentation in publicly available datasets in which we compare against state-of-the-art, and for tasks of both 2D detection and 3D multi-view estimation.

Pedestrian Tracking Using Online Boosted Random Ferns Learning in Far-Infrared Imagery for Safe Driving at Night

Pedestrian-vehicle accidents that occur at night are a major social problem worldwide. Advanced driver assistance systems that are equipped with cameras have been designed to automatically prevent such accidents. Among the various types of cameras used in such systems, far-infrared (FIR) cameras are favorable because they are invariant to illumination changes. Therefore, this paper focuses on a pedestrian nighttime tracking system with an FIR camera that is able to discern thermal energy and is mounted on the forward roof part of a vehicle. Since the temperature difference between the pedestrian and background depends on the season and the weather, we therefore propose two models to detect pedestrians according to the season and the weather, which are determined using Weber–Fechner's law. For tracking pedestrians, we perform real-time online learning to track pedestrians using boosted random ferns and update the trackers at each frame. In particular, we link detection responses to trajectories based on similarities in position, size, and appearance. There is no standard data set for evaluating the tracking performance using an FIR camera; thus, we created the Keimyung University tracking data set (KMUTD) by combining the KMU sudden pedestrian crossing (SPC) data set [21] for summer nights with additional tracking data for winter nights. The KMUTD contains video sequences involving a moving camera, moving pedestrians, sudden shape deformations, unexpected motion changes, and partial or full occlusions between pedestrians at night. The proposed algorithm is successfully applied to various pedestrian video sequences of the KMUTD; specifically, the proposed algorithm yields more accurate tracking performance than other existing methods.

LBP-ferns-based feature extraction for robust facial recognition

Most facial recognition (FR) systems first extract discriminative features from a facial image and then perform classification. This paper proposes a method aimed at representing human facial traits and a low-dimensional feature extraction method using orthogonal linear discriminant analysis (OLDA). The proposed feature relies on a local binary pattern to represent texture information and random ferns to build a structural model. By concatenating its feature vectors, the proposed method achieves a high-dimensional descriptor of the input facial image. In general, the feature dimension is highly related to its discriminative ability. However, higher dimensionality is more complex to compute. Thus, dimensionality reduction is essential for practical FR applications. OLDA is employed to reduce the dimension of the extracted features and improve discriminative performance. With a representative FR database, the proposed method demonstrates a higher recognition rate and low computational complexity compared to existing FR methods. In addition, with a facial image database with disguises, the proposed algorithm demonstrates outstanding performance.

RGB-D 영상 포맷을 위한 결합형 무작위 Fern을 이용한 객체 검출

RGB-D 영상 포맷을 위한 결합형 무작위 Fern을 이용한 객체 검출

Semi self-training beard/moustache detection and segmentation simultaneously

This paper presents a robust, fully automatic and semi self-training system to detect and segment facial beard/moustache simultaneously in challenging facial images. Based on the observation that some certain facial areas, e.g. cheeks, do not typically contain any facial hair whereas the others, e.g. brows, often contain facial hair, a self-trained model is first built using a testing image itself. To overcome the limitation of that facial hairs in brows regions and beard/moustache regions are different in length, density, color, etc., a pre-trained model is also constructed using training data. The pre-trained model is only pursued when the self-trained model produces low confident classification results. In the proposed system, we employ the superpixel together a combination of two classifiers, i.e. Random Ferns (rFerns) and Support Vector Machines (SVM) to obtain good classification performance as well as improve time efficiency. A feature vector, consisting of Histogram of Gabor (HoG) and Histogram of Oriented Gradient of Gabor (HOGG) at different directions and frequencies, is generated from both the bounding box of the superpixel and the super pixel foreground. The segmentation result is then refined by our proposed aggregately searching strategy in order to deal with inaccurate landmarking points. Experimental results have demonstrated the robustness and effectiveness of the proposed system. It is evaluated in images drawn from three entire databases i.e. the Multiple Biometric Grand Challenge (MBGC) still face database, the NIST color Facial Recognition Technology FERET database and a large subset from Pinellas County database.

Human Detection Using Random Color Similarity Feature and Random Ferns Classifier.

We explore a novel approach for human detection based on random color similarity feature (RCS) and random ferns classifier which is also known as semi-naive Bayesian classifier. In contrast to other existing features employed by human detection, color-based features are rarely used in vision-based human detection because of large intra-class variations. In this paper, we propose a novel color-based feature, RCS feature, which is yielded by simple color similarity computation between image cells randomly picked in still images, and can effectively characterize human appearances. In addition, a histogram of oriented gradient based local binary feature (HOG-LBF) is also introduced to enrich the human descriptor set. Furthermore, random ferns classifier is used in the proposed approach because of its faster speed in training and testing than traditional classifiers such as Support Vector Machine (SVM) classifier, without a loss in performance. Finally, the proposed method is conducted in public datasets and achieves competitive detection results.

Automatic GUI test by using SIFT matching

In software development process, the last step is usually the Graphic User Interface(GUI) test, which is part of the final user experience (UE) test. Traditionally, there exist some GUI test tools in the market, such as Abbot Java GUI Test Framework and Pounder, in which testers pre-configure in the script all desired actions and instructions for the computer, nonetheless requiring too much of invariance of GUI environment; and they require reconfiguration in case of GUI changes, therefore still to be done mostly manually and hard for non-programmer testers to. Consequently, we proposed GUI tests by image recognition to automate the last process; we managed to innovate upon current algorithms such as SIFT and Random Fern, from which we develop the new algorithm scheme retrieving most efficient feature and dispelling inefficient part of each algorithm. Computers then apply the algorithm, to search for target patterns themselves and take subsequent actions such as manual mouse, keyboard and screen I/O automatically to test the GUI without any manual instructions. Test results showed that the proposed approach can accelerate GUI test largely compared to current benchmarks.

Face alignment under occlusion based on local and global feature regression

Shape alignment or estimation under occlusion is one of the most challenging tasks in computer vision field. Most previous works treat occlusion as noises or part models, which usually lead to low accuracy or inefficiencies. This paper proposes an efficient and accurate regression-based algorithm for face alignment. In this framework, local and global regressions are iteratively used to train a series of random forests in a cascaded manner. In training and testing process, each step consists of two layers. In the first layer, a set of highly discriminative local features are extracted from local regions according to locality principle. The regression forests are trained for each facial landmark independently using those local features. Then the leaf node of the regression tree is encoded by histogram statistic method and the final shape is estimated by a linear regression matrix. In the second layer, our proposed global features are generated. Then we use those features to train a random fern to keep the global shape constraints. Experiments show that our method has a high speed, but same or slightly lower accuracy than state of the art methods under occlusion condition. In order to gain a higher accuracy we use multi-random shape for initialization, which may slightly reduce the calculation efficiency as a trade-off.

Random clustering ferns for multimodal object recognition

We propose an efficient and robust method for the recognition of objects exhibiting multiple intra-class modes, where each one is associated with a particular object appearance. The proposed method, called random clustering ferns, combines synergically a single and real-time classifier, based on the boosted assembling of extremely randomized trees (ferns), with an unsupervised and probabilistic approach in order to recognize efficiently object instances in images and discover simultaneously the most prominent appearance modes of the object through tree-structured visual words. In particular, we use boosted random ferns and probabilistic latent semantic analysis to obtain a discriminative and multimodal classifier that automatically clusters the response of its randomized trees in function of the visual object appearance. The proposed method is validated extensively in synthetic and real experiments, showing that the method is capable of detecting objects with diverse and complex appearance distributions in real-time performance.

Interactive multiple object learning with scanty human supervision

We present a fast and online human-robot interaction approach that progressively learns multiple object classifiers using scanty human supervision. Given an input video stream recorded during the human-robot interaction, the user just needs to annotate a small fraction of frames to compute object specific classifiers based on random ferns which share the same features. The resulting methodology is fast (in a few seconds, complex object appearances can be learned), versatile (it can be applied to unconstrained scenarios), scalable (real experiments show we can model up to 30 different object classes), and minimizes the amount of human intervention by leveraging the uncertainty measures associated to each classifier.We thoroughly validate the approach on synthetic data and on real sequences acquired with a mobile platform in indoor and outdoor scenarios containing a multitude of different objects. We show that with little human assistance, we are able to build object classifiers robust to viewpoint changes, partial occlusions, varying lighting and cluttered backgrounds.

Invariant Hough random ferns for RGB-D-based object detection

This paper studies the challenging problem of object detection using rich image and depth features. An invariant Hough random ferns framework for RGB-D images is proposed here, which primarily consists of a rotation-invariant RGB-D local binary feature, random ferns classifier training, Hough mapping and voting, searches for the maxima, and back projection. In comparison with traditional three-dimensional local feature extraction techniques, this method is effective in reducing the amount of computation required for feature extraction and matching. Moreover, the detection results showed that the proposed method is robust against rotation and scale variations, changes in illumination, and part-occlusions. The authors believe that this method will facilitate the use of perception in fields such as robotics.