Low-level Feature Detection Research Articles

RBNet: An Ultrafast Rendering-Based Architecture for Railway Defect Segmentation

Inspection of railway defects is crucial for the safe and efficient operation of trains. Recent advancements in convolutional neural networks have led to the development of many effective detection and segmentation algorithms, however, these algorithms often struggle to balance efficiency and precision. In this paper, we present a rendering-based fully convolutional network that generates segmentation results through a coarse-to-fine approach. This allows our framework to make full use of low-level features while minimizing the number of parameters. Additionally, our network generates segmentation results from multiple scales of the feature map and uses residual connections to improve low-level feature detection. To improve training, we propose a novel method that augments the dataset by cutting and pasting images and corresponding ground truth labels horizontally. To better understand the patterns learned by our model, we also generate importance and uncertainty maps to make our model explainable. Our results show that the proposed method outperforms other state-of-the-art image segmentation methods with a higher frame rate and better performance.

Data-driven emergence of convolutional structure in neural networks

Exploiting data invariances is crucial for efficient learning in both artificial and biological neural circuits. Understanding how neural networks can discover appropriate representations capable of harnessing the underlying symmetries of their inputs is thus crucial in machine learning and neuroscience. Convolutional neural networks, for example, were designed to exploit translation symmetry, and their capabilities triggered the first wave of deep learning successes. However, learning convolutions directly from translation-invariant data with a fully connected network has so far proven elusive. Here we show how initially fully connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs, resulting in localized, space-tiling receptive fields. These receptive fields match the filters of a convolutional network trained on the same task. By carefully designing data models for the visual scene, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs, which has long been recognized as the hallmark of natural images. We provide an analytical and numerical characterization of the pattern formation mechanism responsible for this phenomenon in a simple model and find an unexpected link between receptive field formation and tensor decomposition of higher-order input correlations. These results provide a perspective on the development of low-level feature detectors in various sensory modalities and pave the way for studying the impact of higher-order statistics on learning in neural networks.

Reward Timing and Its Expression by Inhibitory Interneurons in the Mouse Primary Visual Cortex.

The primary sensory cortex has historically been studied as a low-level feature detector, but has more recently been implicated in many higher-level cognitive functions. For instance, after an animal learns that a light predicts water at a fixed delay, neurons in the primary visual cortex (V1) can produce "reward timing activity" (i.e., spike modulation of various forms that relate the interval between the visual stimulus and expected reward). Local manipulations to V1 implicate it as a site of learning reward timing activity (as opposed to simply reporting timing information from another region via feedback input). However, the manner by which V1 then produces these representations is unknown. Here, we combine behavior, in vivo electrophysiology, and optogenetics to investigate the characteristics of and circuit mechanisms underlying V1 reward timing in the head-fixed mouse. We find that reward timing activity is present in mouse V1, that inhibitory interneurons participate in reward timing, and that these representations are consistent with a theorized network architecture. Together, these results deepen our understanding of V1 reward timing and the manner by which it is produced.

An Extensive Approach to Features Detection and Description for 2-D Range Data Using Active B-splines

This letter presents three novel contributions for the two-dimensional laser scan data. First, a low-level point feature detector for stable interest point detection. Second, an approach for high-level B-splines features extraction, and finally, a local descriptor for the interest points based on the shape context representation of the uniformly distributed points on the B-splines features. The clues of stable interest points obtained from the detector are used in the B-splines features extraction from the scan data, which enables lightweight and accurate mapping of the environment using fewer control points instead of scan data points. This letter also proposes benchmark metrics to evaluate the performance of the extracted high-level features approach in comparison with the state-of-the-art methodology. A robust descriptor for the interest points offers an efficient association and correspondence matching. Consistent results compared with the state-of-the-art FALKO-BSC and FLIRT- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">β</i> -Grid features in experiments with standard datasets prove the stability of our proposed interest point detector and the proficiency of the descriptor. The results of efficient localization using the pair are also presented.

A Novel Real-time Automatic Angioectasia Detection Method in Wireless Capsule Endoscopy Video Feed.

The development of Wireless Capsule Endoscopy (WCE) revolutionized the examination of the small bowel for diseases. Upon swallowing a capsule (a microscopic camera that resembles an ordinary pill in both shape and size), images of the patient's gastrointestinal (GI) tract are wirelessly transmitted from it to an external recorder. The inspection of these images is, to this day, still manually performed by medical professionals - a lengthy, and especially prone to errors, process. One of the most common diagnoses is the presence of angioectasias, i.e. ectatic vessels on the GI tract that are predisposed to bleeding. In this paper, a novel method for automatic detection of these lesions is proposed, using a combination of low-level image processing, feature detection and machine learning, that can run in real-time without the need for specialized hardware or graphics cards, achieving 92.7% sensitivity and 99.5% specificity to angioectasias. This method can also be expanded to include more pathologies.

A Two-Stage Clustering Based 3D Visual Saliency Model for Dynamic Scenarios

Three-dimensional (3D) visual saliency is fundamental for vision-guided applications such as human–computer interaction in virtual reality, image quality assessment, object tracking, and event retrieval. Classical models for 3D visual saliency can draw an appropriate saliency map when the quality of the required depth maps or auxiliary cues is high enough. However, the depth map is usually impaired with artifacts (such as holes or noise) from faults in stereo matching or multipaths in range sensors. In these cases, challenges arise in those 3D visual saliency models because the core preliminary processes, such as the detection of low-level visual features, may fail. To solve this problem, we proposed a two-stage clustering-based 3D visual saliency model for human visual fixation prediction in dynamic scenarios. In this model, a two-stage clustering scheme is designed to handle the negative influence of impaired depth videos. With the help of this scheme, representative cues are selected for saliency modeling. After that, multimodal saliency maps are obtained from depth, color, and 3D motion cues. Finally, a cross-Bayesian model is designed for the pooling of multimodal saliency maps. The experimental results demonstrate that the proposed 3D saliency model based on two-stage clustering outperforms other state-of-the-art models on a variety of metrics. Furthermore, the consistency and robustness of our model are also verified.

Perceptual expertise impacts preattentive processing of visual simple feature: a visual mismatch negativity study.

Perceptual expertise can be defined as enhanced abilities of discriminating, learning, and recognizing object of special categories. It is unclear whether the perceptual expertise effects occur at the preattentive stage. In the present study, visual mismatch negativity (vMMN) elicited by deviant orientation (90°/270° vs. 0°) for faces, houses, and arrows, respectively, was investigated. Compared with standard stimuli (0° orientation), the orientation changes elicited posterior vMMNs for all deviant stimuli. vMMN amplitudes were similar between houses and arrows, except the significant delayed peak latency for houses. Importantly, compared with houses and arrows, vMMN elicited by the orientation change of faces was significantly decreased and delayed. These data indicated that the perceptual expertise for faces relevant to high-level configural processing impaired the change detection of low-level visual features at the preattentive stage of information processing.

Affordance Based Approach to Automatic Program Generation for Industrial Robots in Manufacturing

Due to the increasing demand for flexible and low-cost production, manufacturing solutions involving human robot interaction have become much sought-after. Robotic manufacturers meet the demand with a rising number of low cost robots specifically designed around safety and usability. However, the programming is still based on a tool centric teach-in. This paper discusses an affordance based approach for process programming in industrial manufacturing. Using low level feature detection and a consecutive evaluation, a fast programming method for industrial applications is presented. The paper presents the concept and a prototypic implementation for a welding process. Using the affordance detection, the system is able to identify relevant seams based on an image of the work space. The identified seams are then presented to the user for review with the means of augmented reality. Lastly, the system derives a welding program based on the detected seams. First experiments show promising results concerning programming speed and path accuracy for different work piece shapes and task definitions. Finally, based on the experience gained with the prototype, the outlook discusses the possibilities and further fields for future work.

Symmetric stability of low level feature detectors

We investigate the capability of low level feature detectors to consistently define feature keypoints in an image and its horizontally reflected (mirrored) image. It is our assertion that this consistency is a useful attribute of a feature detector and should be considered in assessing the robustness of a feature detector. We test ten of the most popular detectors using a popular dataset of 8677 images. We define a set of error measurements to help us to understand the invariance in keypoint position, size and angle of orientation, and we use SIFT descriptors extracted from the keypoints to measure the consistency of extracted feature descriptors. We conclude that the FAST and CenSurE detectors are perfectly invariant to bilateral symmetry, Good Features to Track and the Harris Corner detector produce consistent keypoints that can be matched using feature descriptors, and others vary in their invariance. SIFT is the least invariant of all the detectors that we test.

Darkroom

Specialized image signal processors (ISPs) exploit the structure of image processing pipelines to minimize memory bandwidth using the architectural pattern of line-buffering , where all intermediate data between each stage is stored in small on-chip buffers. This provides high energy efficiency, allowing long pipelines with tera-op/sec. image processing in battery-powered devices, but traditionally requires painstaking manual design in hardware. Based on this pattern, we present Darkroom, a language and compiler for image processing. The semantics of the Darkroom language allow it to compile programs directly into line-buffered pipelines, with all intermediate values in local line-buffer storage, eliminating unnecessary communication with off-chip DRAM. We formulate the problem of optimally scheduling line-buffered pipelines to minimize buffering as an integer linear program. Finally, given an optimally scheduled pipeline, Darkroom synthesizes hardware descriptions for ASIC or FPGA, or fast CPU code. We evaluate Darkroom implementations of a range of applications, including a camera pipeline, low-level feature detection algorithms, and deblurring. For many applications, we demonstrate gigapixel/sec. performance in under 0.5mm 2 of ASIC silicon at 250 mW (simulated on a 45nm foundry process), real-time 1080p/60 video processing using a fraction of the resources of a modern FPGA, and tens of megapixels/sec. of throughput on a quad-core x86 processor.

Advancing bag-of-visual-words representations for lesion classification in retinal images.

Diabetic Retinopathy (DR) is a complication of diabetes that can lead to blindness if not readily discovered. Automated screening algorithms have the potential to improve identification of patients who need further medical attention. However, the identification of lesions must be accurate to be useful for clinical application. The bag-of-visual-words (BoVW) algorithm employs a maximum-margin classifier in a flexible framework that is able to detect the most common DR-related lesions such as microaneurysms, cotton-wool spots and hard exudates. BoVW allows to bypass the need for pre- and post-processing of the retinographic images, as well as the need of specific ad hoc techniques for identification of each type of lesion. An extensive evaluation of the BoVW model, using three large retinograph datasets (DR1, DR2 and Messidor) with different resolution and collected by different healthcare personnel, was performed. The results demonstrate that the BoVW classification approach can identify different lesions within an image without having to utilize different algorithms for each lesion reducing processing time and providing a more flexible diagnostic system. Our BoVW scheme is based on sparse low-level feature detection with a Speeded-Up Robust Features (SURF) local descriptor, and mid-level features based on semi-soft coding with max pooling. The best BoVW representation for retinal image classification was an area under the receiver operating characteristic curve (AUC-ROC) of 97.8% (exudates) and 93.5% (red lesions), applying a cross-dataset validation protocol. To assess the accuracy for detecting cases that require referral within one year, the sparse extraction technique associated with semi-soft coding and max pooling obtained an AUC of 94.22.0%, outperforming current methods. Those results indicate that, for retinal image classification tasks in clinical practice, BoVW is equal and, in some instances, surpasses results obtained using dense detection (widely believed to be the best choice in many vision problems) for the low-level descriptors.

Semantic Control of Feature Extraction from Natural Scenes

In the early stages of image analysis, visual cortex represents scenes as spatially organized maps of locally defined features (e.g., edge orientation). As image reconstruction unfolds and features are assembled into larger constructs, cortex attempts to recover semantic content for object recognition. It is conceivable that higher level representations may feed back onto early processes and retune their properties to align with the semantic structure projected by the scene; however, there is no clear evidence to either support or discard the applicability of this notion to the human visual system. Obtaining such evidence is challenging because low and higher level processes must be probed simultaneously within the same experimental paradigm. We developed a methodology that targets both levels of analysis by embedding low-level probes within natural scenes. Human observers were required to discriminate probe orientation while semantic interpretation of the scene was selectively disrupted via stimulus inversion or reversed playback. We characterized the orientation tuning properties of the perceptual process supporting probe discrimination; tuning was substantially reshaped by semantic manipulation, demonstrating that low-level feature detectors operate under partial control from higher level modules. The manner in which such control was exerted may be interpreted as a top-down predictive strategy whereby global semantic content guides and refines local image reconstruction. We exploit the novel information gained from data to develop mechanistic accounts of unexplained phenomena such as the classic face inversion effect.

Prioritized Detection of Personally Familiar Faces

We investigated whether personally familiar faces are preferentially processed in conditions of reduced attentional resources and in the absence of conscious awareness. In the first experiment, we used Rapid Serial Visual Presentation (RSVP) to test the susceptibility of familiar faces and faces of strangers to the attentional blink. In the second experiment, we used continuous flash interocular suppression to render stimuli invisible and measured face detection time for personally familiar faces as compared to faces of strangers. In both experiments we found an advantage for detection of personally familiar faces as compared to faces of strangers. Our data suggest that the identity of faces is processed with reduced attentional resources and even in the absence of awareness. Our results show that this facilitated processing of familiar faces cannot be attributed to detection of low-level visual features and that a learned unique configuration of facial features can influence preconscious perceptual processing.

A Computational Learning Theory of Active Object Recognition Under Uncertainty

We present some theoretical results related to the problem of actively searching a 3D scene to determine the positions of one or more pre-specified objects. We investigate the effects that input noise, occlusion, and the VC-dimensions of the related representation classes have in terms of localizing all objects present in the search region, under finite computational resources and a search cost constraint. We present a number of bounds relating the noise-rate of low level feature detection to the VC-dimension of an object representable by an architecture satisfying the given computational constraints. We prove that under certain conditions, the corresponding classes of object localization and recognition problems are efficiently learnable in the presence of noise and under a purposive learning strategy, as there exists a polynomial upper bound on the minimum number of examples necessary to correctly localize the targets under the given models of uncertainty. We also use these arguments to show that passive approaches to the same problem do not necessarily guarantee that the problem is efficiently learnable. Under this formulation, we prove the existence of a number of emergent relations between the object detection noise-rate, the scene representation length, the object class complexity, and the representation class complexity, which demonstrate that selective attention is not only necessary due to computational complexity constraints, but it is also necessary as a noise-suppression mechanism and as a mechanism for efficient object class learning. These results concretely demonstrate the advantages of active, purposive and attentive approaches for solving complex vision problems.

Classification of Real and Imagined Sounds in Early Visual Cortex

Early visual cortex has been thought to be mainly involved in the detection of low-level visual features. Here we show that complex natural sounds can be decoded from early visual cortex activity, in the absence of visual stimulation and both when sounds are actually displayed and when they are merely imagined. Blindfolded subjects listened to three complex natural sounds (bird singing, people talking, traffic noise; Exp. 1) or received word cues (“forest”, “people”, “traffic”; Exp 2) to imagine the associated scene. fMRI BOLD activation patterns from retinotopically defined early visual areas were fed into a multivariate pattern classification algorithm (a linear support vector machine). Actual sounds were discriminated above chance in V2 and V3 and imagined sounds were decoded in V1. Also cross-classification, ie, training the classifier to real sounds and testing it to imagined sounds and vice versa, was successful. Two further experiments showed that an orthogonal working memory task does not interfer...

A detailed investigation into low-level feature detection in spectrogram images

Being the first stage of analysis within an image, low-level feature detection is a crucial step in the image analysis process and, as such, deserves suitable attention. This paper presents a systematic investigation into low-level feature detection in spectrogram images. The result of which is the identification of frequency tracks. Analysis of the literature identifies different strategies for accomplishing low-level feature detection. Nevertheless, the advantages and disadvantages of each are not explicitly investigated. Three model-based detection strategies are outlined, each extracting an increasing amount of information from the spectrogram, and, through ROC analysis, it is shown that at increasing levels of extraction the detection rates increase. Nevertheless, further investigation suggests that model-based detection has a limitation-it is not computationally feasible to fully evaluate the model of even a simple sinusoidal track. Therefore, alternative approaches, such as dimensionality reduction, are investigated to reduce the complex search space. It is shown that, if carefully selected, these techniques can approach the detection rates of model-based strategies that perform the same level of information extraction. The implementations used to derive the results presented within this paper are available online from http://stdetect.googlecode.com.

Electrophysiological evidence implicates automatic low-level feature detectors in perceptual asymmetry

Perceptual asymmetry has been demonstrated behaviorally using frequency modulated (FM) stimuli: a modulated tone is easier to detect among unmodulated distracters than the converse. We demonstrate perceptual asymmetry for FM tones in the mismatch negativity (MMN) component of the event-related potential, regardless of whether the participant attends to the tones. These results suggest that perceptual asymmetry reflects the automatic activation of low-level feature detectors in the auditory system.

Evaluation of Interest Point Detectors

Many different low-level feature detectors exist and it is widely agreed that the evaluation of detectors is important. In this paper we introduce two evaluation criteria for interest points' repeatability rate and information content. Repeatability rate evaluates the geometric stability under different transformations. Information content measures the distinctiveness of features. Different interest point detectors are compared using these two criteria. We determine which detector gives the best results and show that it satisfies the criteria well.

Performance enhancement for fuzzy adaptive resonancetheory (ART) neural networks

A modified fuzzy adaptive resonance theory neural network (ART) is used as a classifier for a texture recognition system. The system consists of a wavelet based low level feature detector and a high level ART classifier. The performance improvement is measured in terms of identification accuracy and computational burden.

A VLSI architecture for a half-edge-based corner detector

Corner detection is a low-level feature detection operator that is of great use in image processing applications, for example, optical flow and structure from motion by image correspondence. The detection of corners is a computationally intensive operation. Past implementations of corner detection techniques have been restricted to software. In this paper we propose an efficient very large-scale integration (VLSI) architecture for detection of corners in images. The corner detection technique is based on the half-edge concept and the first directional derivative of Gaussian. Apart from the location of the corner points, the algorithm also computes the corner orientation and the corner angle and outputs the edge map of the image. The symmetrical properties of the masks are utilized to reduce the number of convolutions effectively, from eight to two. Therefore, the number of multiplications required per pixel is reduced from 1800 to 392. Thus, the proposed architecture yields a speed-up factor of 4.6 over conventional convolution architectures. The architecture uses the principles of pipelining and parallelism and can be implemented in VLSI.