HMAX Model Research Articles

There Is a "U" in Clutter: Evidence for Robust Sparse Codes Underlying Clutter Tolerance in Human Vision.

The neural mechanisms underlying object recognition in cluttered scenes (i.e., containing more than one object) remain poorly understood. Studies have suggested that neural responses to multiple objects correspond to an average of the responses to the constituent objects. Yet, it is unclear how the identities of an unknown number of objects could be disentangled from a confounded average response. Here, we use a popular computational biological vision model to show that averaging-like responses can result from responses of clutter-tolerant neurons to suboptimal stimuli. The model also provides a novel prediction, that human detection ability should show a U-shaped dependency on target-clutter similarity, which is confirmed experimentally, supporting a simple, unifying account of how the brain performs object recognition in clutter.

Enhanced HMAX model with feedforward feature learning for multiclass categorization.

In recent years, the interdisciplinary research between neuroscience and computer vision has promoted the development in both fields. Many biologically inspired visual models are proposed, and among them, the Hierarchical Max-pooling model (HMAX) is a feedforward model mimicking the structures and functions of V1 to posterior inferotemporal (PIT) layer of the primate visual cortex, which could generate a series of position- and scale- invariant features. However, it could be improved with attention modulation and memory processing, which are two important properties of the primate visual cortex. Thus, in this paper, based on recent biological research on the primate visual cortex, we still mimic the first 100–150 ms of visual cognition to enhance the HMAX model, which mainly focuses on the unsupervised feedforward feature learning process. The main modifications are as follows: (1) To mimic the attention modulation mechanism of V1 layer, a bottom-up saliency map is computed in the S1 layer of the HMAX model, which can support the initial feature extraction for memory processing; (2) To mimic the learning, clustering and short-term memory to long-term memory conversion abilities of V2 and IT, an unsupervised iterative clustering method is used to learn clusters with multiscale middle level patches, which are taken as long-term memory; (3) Inspired by the multiple feature encoding mode of the primate visual cortex, information including color, orientation, and spatial position are encoded in different layers of the HMAX model progressively. By adding a softmax layer at the top of the model, multiclass categorization experiments can be conducted, and the results on Caltech101 show that the enhanced model with a smaller memory size exhibits higher accuracy than the original HMAX model, and could also achieve better accuracy than other unsupervised feature learning methods in multiclass categorization task.

Modeling object recognition in visual cortex using multiple firing k-means and non-negative sparse coding

The HMAX model developed by Serre et al. imitates the process of visual recognition in primates’ visual cortex. However, it has some limits in modeling the V2 neurons or higher level of visual cortex. We extend the model in some biologically plausible ways and construct a five-layer computational model, denoted as Sparse-HMAX model. First we use Gabor filters to describe the response properties of V1 neurons as in original HMAX model and describe C1 image patches with HOG descriptors. Then we integrate multiple firing k-means into the HMAX model to emulate the V2 neural responses and non-negative sparse coding to model V4 neurons. To investigate the efficacy of our proposed model, we perform experiments on three public databases: Caltech101, Caltech256 and GRAZ-01. Experimental results demonstrate that Sparse-HMAX model displays great improvements over the original HMAX model both in recognition accuracy and processing speed for object recognition. Our proposed method is also comparable to the prevalent approaches in recognition performance.

Combining two visual cortex models for robust face recognition

This paper describes a new robust face recognition system based on combining C2 features of HMAX model and visual attention model inspired from the behavior of neurons in ventral and dorsal stream of visual cortex. By visual attention model, visual attention points are selected which are discriminative areas of face images. C2 features are extracted from these visual attention points to create efficient C2 features and have robustness properties. These efficient C2 features are used to distinct multi-classes of faces. For experimental analysis, we use cropped Caltech faces database and do face recognition task by support vector machine (SVM) classifier on efficient C2 features of them. The results illustrate our proposed face recognition system using efficient C2 features are effective and can recognize faces in various illumination and expression conditions with high accuracy and speed in comparison with original C2 features.

Visual dictionaries as intermediate features in the human brain.

The human visual system is assumed to transform low level visual features to object and scene representations via features of intermediate complexity. How the brain computationally represents intermediate features is still unclear. To further elucidate this, we compared the biologically plausible HMAX model and Bag of Words (BoW) model from computer vision. Both these computational models use visual dictionaries, candidate features of intermediate complexity, to represent visual scenes, and the models have been proven effective in automatic object and scene recognition. These models however differ in the computation of visual dictionaries and pooling techniques. We investigated where in the brain and to what extent human fMRI responses to short video can be accounted for by multiple hierarchical levels of the HMAX and BoW models. Brain activity of 20 subjects obtained while viewing a short video clip was analyzed voxel-wise using a distance-based variation partitioning method. Results revealed that both HMAX and BoW explain a significant amount of brain activity in early visual regions V1, V2, and V3. However, BoW exhibits more consistency across subjects in accounting for brain activity compared to HMAX. Furthermore, visual dictionary representations by HMAX and BoW explain significantly some brain activity in higher areas which are believed to process intermediate features. Overall our results indicate that, although both HMAX and BoW account for activity in the human visual system, the BoW seems to more faithfully represent neural responses in low and intermediate level visual areas of the brain.

A hierarchical model of vision (HMAX) can also recognize speech

HMAX is a well-known computational model of visual recognition in cortex consisting of just two computational operations – a “template match” and non-linear pooling – alternating in a feedforward hierarchy in which receptive fields exhibit increasing specificity and invariance [1]. Interestingly, auditory recognition problems (such as speech recognition) share similar computational requirements, and recent work in auditory neuroscience suggests that auditory and visual cortex share similar anatomical and functional organization. Based on these similarities, we tested whether HMAX could support an auditory recognition task (specifically, word spotting). To test HMAX on word spotting, recorded speech samples from the TIMIT corpus [2] were first converted into time-frequency spectrograms using a computational model of the auditory periphery [3]. These spectrograms were then split into 750 ms frames and input to a standard HMAX model [4]. Based on observed similarities between the receptive fields in primary auditory cortex (spectro-temporal receptive fields, or STRFs) and primary visual cortex (typically modeled as oriented Gabor filters), we used S1 filters identical to those used in vision [4]. Similarly, S2 “patches” were randomly selected from C1 representations of speech sounds drawn from an independent speech corpus. One vs. all linear support vector machines (SVMs) were then trained to discriminate frames that contain a target word from those that did not. These SVMs were then tested on a novel set of test sentences using a sliding frame approach (750 ms frame size, 20 ms step size). For each frame in a sentence, the SVM produced a distance from the hyperplane, and a threshold value was applied to produce a binary classification whether or not the target word was present in the sentence. When tested on target words that appeared in a fixed context (i.e. SA sentences in TIMIT), performance was highly robust, with ROC areas consistently above 0.9. When tested on target words that appeared in variable contexts (i.e., SI sentences in TIMIT), performance was somewhat decreased with ROC areas around 0.8. This decrease in performance is likely due to the inclusion of “clutter” (i.e., target irrelevant features) within the frame, also commonly observed when HMAX is applied to visual object recognition tasks [1]. These results are novel in that they provide support for the hypothesis that the simple computational framework implemented in HMAX – consisting of a feedforward hierarchy of only two alternating computational operations – may generalize beyond vision to support auditory recognition as well. It is possible that such a representation could give rise to stable neural encodings that are invariant to behaviorally irrelevant characteristics as seen in higher order visual and auditory cortices [5,6]. While it is likely that this auditory version of the HMAX model would benefit from the use of more auditory-specific filters based on STRF models [7], the Gabor features used here are largely compatible with previous computational models based on STRFs up to the level of primary auditory cortex [8]. Additional benefit may also be gained by learning sparse representations from natural sounds, at both the S1 and S2 levels [9].

Classification and identification of vehicle type and make by cortex-like image descriptor HMAX

Identification of vehicle types from images is a challenging task. Though several methods have been proposed, most of the works has been done in strictly controlled conditions, where the feature information was calculated on ad hoc bases. In order to achieve better performance of make and model recognition, we emphasised the importance of feature description in a principled way from a biologically inspired vision modelling perspective. As a visual feature expression model in cortex, HMAX integrates general beliefs about the visual system in a quantitative framework. We applied HMAX for vehicle type recognition using a database that includes over 2,000 vehicle images of 26 classes recorded from surveillance cameras involving various complex photographic conditions. Experimental results using the HMAX model and multi-layer perceptron MLP offers a classification rate of 94% and averaged identification accuracy of 95%, which is higher than other commonly used classification algorithms such as kNN and SVM.

The emergence of orthographic word representations in the brain: evaluating a neural shape-based framework using fMRI and the HMAX model. Addendum to retraction notice: 'The emergence of orthographic word representations in the brain: evaluating a neural shape-based framework using fMRI and the HMAX model' by Braet, W., Kubilius, J., Wagemans, J., & Op de Beeck, H. P. doi:10.1093/cercor/bhs355 (published online

The emergence of orthographic word representations in the brain: evaluating a neural shape-based framework using fMRI and the HMAX model. Addendum to retraction notice: 'The emergence of orthographic word representations in the brain: evaluating a neural shape-based framework using fMRI and the HMAX model' by Braet, W., Kubilius, J., Wagemans, J., & Op de Beeck, H. P. doi:10.1093/cercor/bhs355 (published online

Introducing Memory and Association Mechanism Into a Biologically Inspired Visual Model

A famous biologically inspired hierarchical model (HMAX model), which was proposed recently and corresponds to V1 to V4 of the ventral pathway in primate visual cortex, has been successfully applied to multiple visual recognition tasks. The model is able to achieve a set of position- and scale-tolerant recognition, which is a central problem in pattern recognition. In this paper, based on some other biological experimental evidence, we introduce the memory and association mechanism into the HMAX model. The main contributions of the work are: 1) mimicking the active memory and association mechanism and adding the top down adjustment to the HMAX model, which is the first try to add the active adjustment to this famous model and 2) from the perspective of information, algorithms based on the new model can reduce the computation storage and have a good recognition performance. The new model is also applied to object recognition processes. The primary experimental results show that our method is efficient with a much lower memory requirement.

Modeling guidance and recognition in categorical search: bridging human and computer object detection.

Search is commonly described as a repeating cycle of guidance to target-like objects, followed by the recognition of these objects as targets or distractors. Are these indeed separate processes using different visual features? We addressed this question by comparing observer behavior to that of support vector machine (SVM) models trained on guidance and recognition tasks. Observers searched for a categorically defined teddy bear target in four-object arrays. Target-absent trials consisted of random category distractors rated in their visual similarity to teddy bears. Guidance, quantified as first-fixated objects during search, was strongest for targets, followed by target-similar, medium-similarity, and target-dissimilar distractors. False positive errors to first-fixated distractors also decreased with increasing dissimilarity to the target category. To model guidance, nine teddy bear detectors, using features ranging in biological plausibility, were trained on unblurred bears then tested on blurred versions of the same objects appearing in each search display. Guidance estimates were based on target probabilities obtained from these detectors. To model recognition, nine bear/nonbear classifiers, trained and tested on unblurred objects, were used to classify the object that would be fixated first (based on the detector estimates) as a teddy bear or a distractor. Patterns of categorical guidance and recognition accuracy were modeled almost perfectly by an HMAX model in combination with a color histogram feature. We conclude that guidance and recognition in the context of search are not separate processes mediated by different features, and that what the literature knows as guidance is really recognition performed on blurred objects viewed in the visual periphery.

Fast neuromimetic object recognition using FPGA outperforms GPU implementations.

Recognition of objects in still images has traditionally been regarded as a difficult computational problem. Although modern automated methods for visual object recognition have achieved steadily increasing recognition accuracy, even the most advanced computational vision approaches are unable to obtain performance equal to that of humans. This has led to the creation of many biologically inspired models of visual object recognition, among them the hierarchical model and X (HMAX) model. HMAX is traditionally known to achieve high accuracy in visual object recognition tasks at the expense of significant computational complexity. Increasing complexity, in turn, increases computation time, reducing the number of images that can be processed per unit time. In this paper we describe how the computationally intensive and biologically inspired HMAX model for visual object recognition can be modified for implementation on a commercial field-programmable aate Array, specifically the Xilinx Virtex 6 ML605 evaluation board with XC6VLX240T FPGA. We show that with minor modifications to the traditional HMAX model we can perform recognition on images of size 128 × 128 pixels at a rate of 190 images per second with a less than 1% loss in recognition accuracy in both binary and multiclass visual object recognition tasks.

C-HMAX: Artificial cognitive model inspired by the color vision mechanism of the human brain

Artificial cognitive models and computational neuroscience methods have garnered great interest from both neurologist and leading analysts in recent years. Among the cognitive models, HMAX has been widely used in computer vision systems for its robustness shape and texture features inspired by the ventral stream of the human brain. This work presents a Color-HMAX (C-HMAX) model based on the HMAX model which imitates the color vision mechanism of the human brain that the HMAX model does not include. C-HMAX is then applied to the German Traffic Sign Recognition Benchmark (GTSRB) which has 43 categories and 51 840 sample traffic signs with an accuracy of 98.41%, higher than most other models including linear discriminant analysis and multi-scale convolutional neural network.

Exploiting temporal continuity of views to learn visual object invariance.

Exploiting temporal continuity of views to learn visual object invariance.

Notice of retraction: 'The emergence of orthographic word representations in the brain: evaluating a neural shape-based framework using fMRI and the HMAX model' by Wouter Braet, Jonas Kubilius, Johan Wagemans and Hans P. Op de Beeck. doi:10.1093/Cercor/bhs355, published online November 16, 2012.

Notice of retraction: 'The emergence of orthographic word representations in the brain: evaluating a neural shape-based framework using fMRI and the HMAX model' by Wouter Braet, Jonas Kubilius, Johan Wagemans and Hans P. Op de Beeck. doi:10.1093/Cercor/bhs355, published online November 16, 2012.

Extended Coding and Pooling in the HMAX Model

This paper presents an extension of the HMAX model, a neural network model for image classification. The HMAX model can be described as a four-level architecture, with the first level consisting of multiscale and multiorientation local filters. We introduce two main contributions to this model. First, we improve the way the local filters at the first level are integrated into more complex filters at the last level, providing a flexible description of object regions and combining local information of multiple scales and orientations. These new filters are discriminative and yet invariant, two key aspects of visual classification. We evaluate their discriminative power and their level of invariance to geometrical transformations on a synthetic image set. Second, we introduce a multiresolution spatial pooling. This pooling encodes both local and global spatial information to produce discriminative image signatures. Classification results are reported on three image data sets: Caltech101, Caltech256, and fifteen scenes. We show significant improvements over previous architectures using a similar framework.

Evaluating a neural shape-based framework for the emergence of visual word form representations using fMRI and the HMAX model

Evaluating a neural shape-based framework for the emergence of visual word form representations using fMRI and the HMAX model

A stable biologically motivated learning mechanism for visual feature extraction to handle facial categorization.

The brain mechanism of extracting visual features for recognizing various objects has consistently been a controversial issue in computational models of object recognition. To extract visual features, we introduce a new, biologically motivated model for facial categorization, which is an extension of the Hubel and Wiesel simple-to-complex cell hierarchy. To address the synaptic stability versus plasticity dilemma, we apply the Adaptive Resonance Theory (ART) for extracting informative intermediate level visual features during the learning process, which also makes this model stable against the destruction of previously learned information while learning new information. Such a mechanism has been suggested to be embedded within known laminar microcircuits of the cerebral cortex. To reveal the strength of the proposed visual feature learning mechanism, we show that when we use this mechanism in the training process of a well-known biologically motivated object recognition model (the HMAX model), it performs better than the HMAX model in face/non-face classification tasks. Furthermore, we demonstrate that our proposed mechanism is capable of following similar trends in performance as humans in a psychophysical experiment using a face versus non-face rapid categorization task.

Texture Feature Extraction Inspired By Natural Vision System And Hmax Algorithm

In this paper, a new and effective method called HMAX is used for image texture. feature extraction. This method is inspired by the biological system of brain and human vision in order to create feature vectors for image recognition. A set of C2 features obtained from HMAX algorithm that are stable against changes in angle and size, are extracted from all image datasets firstly. Then using artificial neural networks and K-nearest neighbor classifiers, eight different types of natural texture images from VISTEX dataset are classified. In order to evaluate the HMAX feature extraction method, the classification results are compared with Gabor filter banks. Since HMAX model is consistent with natural vision system, it is expected to obtain a better accuracy compared to Gabor filter banks. Experimental results with artificial neural network and K-nearest neighbor classifier show that the accuracy of 90.12% and 84.50% respectively for HMAX features. They have significant improvements compared to Gabor filter banks which obtained 78.62% and 72% accuracy.

2P1-J04 スケールと位置に不変な物体識別を行う脳型視覚システム(ロボットビジョン(2))

We have developed a visual system achieving scale and position invariant object classification. The system includes a silicon retina, a field-programmable gate array (FPGA), and a personal computer (PC). Its algorithm is based on the HMAX model, which is a hierarchical model of object recognition in the visual cortex and offers scale and position invariant recognition. In order to perform a lot of spatial filtering required by the model effectively, we employed a resistive network. The output of the system was examined using visual stimuli displayed on a LCD monitor, and the system was successful in classifying several figures and alphabetical letters independently of scale and position of the target.

基于改进的HMAX算法的车型识别应用

本文实现了将改进的HMAX算法应用于车型识别领域。算法的主要创新点是采用ITTI显著性算法选取车辆图片显著点构造图库，通过采用对颜色、亮度和朝向三方面敏感度的综合，构造更具代表性的模板库，提高了算法的识别率。此外，通过计算同一模板与不同图片响应度值的方差，消除了冗余模板，降低了算法识别时间。分析和实验表明，改进的HMAX算法可以有效地实现车型识别，新提出的算法相对原先HMAX模型在识别率可以提高1%~2%，相对当前存在的其他车型识别算法在识别率方面可提升约5%~10%，在保证特征数量的前提下，识别率接近95%；此外，加入模版筛选方法的改进算法相对于原先HMAX模型在识别率基本保持不变的条件下，识别时间可以缩减到原先的1/4，最终根据效益值评估的最优组合中，识别率约为92%，识别时间为0.6 s/幅图片，相对于原先方法均达到了一定的提升。 The paper implements the improved HMAX model for vehicle type (make and model) recognition. The main improvement of this algorithm is using ITTI model with the characteristic of quick response to color, intensity or orien- tation discrimination, to select the saliency area. By this way the template can be more representative and more beneficial to the detection rate. In addition, the calculating of the variance of response degree between different images and the same template eliminates redundant templates, which make contribution to reducing the time of classification. Analyz- ing and experiments claims that the improved HMAX model can be effective and reliable by the detection rate at 95%, with enough features extracted, 5% - 10% higher than current vehicle type recognition methods and 1% - 2% higher than original HMAX model. In addition, with the template screening method added, the improved HMAX model can keep the detection rate and curtail the classification time by quarter time. According to the optimal setting by the final value estimation, the detection rate is about 92%, and the classification time is 0.6 s/image, with promotion to the past.