High Illumination Variations Research Articles

Deep learning-based hard spatial attention for driver in-vehicle action monitoring

Distracted driving is one of the main causes of deaths and injuries in the world. Monitoring driver behaviors through Driver Action Recognition (DAR) contributes significantly to building safer transportation systems. However, in naturalistic driving settings, this task is complex and challenging because of numerous difficulties, such as high illumination variation and cluttered and dynamic background. In this paper, we introduce a novel hard attention network that highlights the most pertinent driving-scene information while filtering out irrelevant data. Specifically, only local discriminative salient regions are exploited through a hard attention mechanism. The experimental results indicate that our approach significantly enhances DAR performance. We evaluated our network on three diverse state-of-the-art datasets recorded in real-world conditions: it achieves up to 95.83% in terms of safe driving recognition and up to 99.07% in terms of distraction detection. The proposed approach outperforms the soft attention-based DAR not only in detection and recognition performance but also in computation complexity by 38.71% less runtime. For reproducible research, the code is available at https://github.com/JEGHAMI/HSA-.

Kilometer-scale autonomous navigation in subarctic forests: challenges and lessons learned

Challenges inherent to autonomous wintertime navigation in forests include lack of a reliable Global Navigation Satellite System (GNSS) signal, low feature contrast, high illumination variations, and changing environment. This type of off-road environment is an extreme case of situations autonomous cars could encounter in northern regions. Thus, it is important to understand the impact of this harsh environment on autonomous navigation systems. To this end, we present a field report analyzing teach-and-repeat navigation in a subarctic forest while subject to fluctuating weather, including light and heavy snow, rain, and drizzle. First, we describe the system, which relies on point cloud registration to localize a mobile robot through a boreal forest, while simultaneously building a map. We experimentally evaluate this system in over 18.8km of autonomous navigation in the teach-and-repeat mode. Over 14 repeat runs, only four manual interventions were required, three of which were due to localization failure and another one caused by battery power outage. We show that dense vegetation perturbs the GNSS signal, rendering it unsuitable for navigation in forest trails. Furthermore, we highlight the increased uncertainty related to localizing using point cloud registration in forest trails. We demonstrate that it is not snow precipitation, but snow accumulation, that affects our system’s ability to localize within the environment. Finally, we expose some challenges and lessons learned from our field campaign to support better experimental work in winter conditions. Our dataset is available online.

UNSUPERVISED MULTI-CONSTRAINT DEEP NEURAL NETWORK FOR DENSE IMAGE MATCHING

Abstract. Dense image matching is essential to photogrammetry applications, including Digital Surface Model (DSM) generation, three dimensional (3D) reconstruction, and object detection and recognition. The development of an efficient and robust method for dense image matching has been one of the technical challenges due to high variations in illumination and ground features of aerial images of large areas. Nowadays, due to the development of deep learning technology, deep neural network-based algorithms outperform traditional methods on a variety of tasks such as object detection, semantic segmentation and stereo matching. The proposed network includes cost-volume computation, cost-volume aggregation, and disparity prediction. It starts with a pre-trained VGG-16 network as a backend and using the U-net architecture with nine layers for feature map extraction and a correlation layer for cost volume calculation, after that a guided filter based cost aggregation is adopted for cost volume filtering and finally the soft Argmax function is utilized for disparity prediction. The experimental conducted on a UAV dataset demonstrated that the proposed method achieved the RMSE (root mean square error) of the reprojection error better than 1 pixel in image coordinate and in-ground positioning accuracy within 2.5 ground sample distance. The comparison experiments on KITTI 2015 dataset shows the proposed unsupervised method even comparably with other supervised methods.

Dense Image-Matching via Optical Flow Field Estimation and Fast-Guided Filter Refinement

The development of an efficient and robust method for dense image-matching has been a technical challenge due to high variations in illumination and ground features of aerial images of large areas. In this paper, we propose a method for the dense matching of aerial images using an optical flow field and a fast-guided filter. The proposed method utilizes a coarse-to-fine matching strategy for a pixel-wise correspondence search across stereo image pairs. The pyramid Lucas–Kanade (L–K) method is first used to generate a sparse optical flow field within the stereo image pairs, and an adjusted control lattice is then used to derive the multi-level B-spline interpolating function for estimating the dense optical flow field. The dense correspondence is subsequently refined through a combination of a novel cross-region-based voting process and fast guided filtering. The performance of the proposed method was evaluated on three bases, namely, the matching accuracy, the matching success rate, and the matching efficiency. The evaluative experiments were performed using sets of unmanned aerial vehicle (UAV) images and aerial digital mapping camera (DMC) images. The results showed that the proposed method afforded the root mean square error (RMSE) of the reprojection errors better than ±0.5 pixels in image, and a height accuracy within ±2.5 GSD (ground sampling distance) from the ground. The method was further compared with the state-of-the-art commercial software SURE and confirmed to deliver more complete matches for images with poor-texture areas, the matching success rate of the proposed method is higher than 97% while SURE is 96%, and there is 47% higher matching efficiency. This demonstrates the superior applicability of the proposed method to aerial image-based dense matching with poor texture regions.

A new combination active contour model for segmenting texture image with low contrast and high illumination variations

This paper presents a novel combined energy functional based on edge and region information for active contour model, which can be applied to segment textured images containing low contrast and high illumination variations. In the proposed method, the edge features are calculated based on the phase-based approaches derived from the monogenic signal, which are robust to illumination variations in the image. These feature values are used in an edge energy functional to assist the active contour evolving towards the true object boundaries. To extract the region features, at first, we compute the normalized accumulated short-term autocorrelation (NASTA) values for the image, which suppress background clutter and enhance dissimilarities between objects and background. Next, a local cumulative distribution function (LCDF) of NASTA is calculated for every pixel in a local window around it and is used as a region feature for that pixel. Then, the obtained features are employed to define a new localized region-based energy functional that can correctly segment regions with same intensity mean and variance. The proposed edge and region energy terms are integrated and a regularization term is added to them to form our combined active contour. Experimental results indicate remarkable advantages of proposed method comparing to existing combination models.

A kernelized sparsity-based approach for best spectral bands selection for face recognition

We study face recognition in unconstrained illumination conditions. A twofold contribution is proposed: First, the robustness of four state-of-the-art algorithms, namely Multi-block Local Binary Pattern (MBLBP), Histogram of Gabor Phase Patterns (HGPP), Local Gabor Binary Pattern Histogram Sequence (LGBPHS) and Patterns of Oriented Edge Magnitudes (POEM-WPCA) against high illumination variation is studied. Second, we propose to enhance the performance of the four mentioned algorithms, which has been drastically decreased upon the day lighted face images provided by IRIS-M3 face database. For this purpose, we use visible narrow band subspectral images selected from the mentioned database. We formulate best spectral bands selection as a pursuit optimization problem wherein the vector of weights determining the importance of each visible spectral band is supposed to be sparse, and hence can be determined by minimizing its L1-norm. Several fusing approaches are then applied on selected best spectral bands using multi-scale and multi-orientation Gabor wavelets. The results highlight further the still challenging problem of face recognition in conditions with high illumination variation, as well as the effectiveness of our subspectral images based approach with its two components; bands selection and bands fusion, to increase the accuracy of the studied algorithms by at least 14 % upon the proposed database.

A fast color image enhancement algorithm based on Max Intensity Channel

In this paper, we extend image enhancement techniques based on the retinex theory imitating human visual perception of scenes containing high illumination variations. This extension achieves simultaneous dynamic range modification, color consistency, and lightness rendition without multi-scale Gaussian filtering which has a certain halo effect. The reflection component is analyzed based on the illumination and reflection imaging model. A new prior named Max Intensity Channel (MIC) is implemented assuming that the reflections of some points in the scene are very high in at least one color channel. Using this prior, the illumination of the scene is obtained directly by performing a gray-scale closing operation and a fast cross-bilateral filtering on the MIC of the input color image. Consequently, the reflection component of each RGB color channel can be determined from the illumination and reflection imaging model. The proposed algorithm estimates the illumination component which is relatively smooth and maintains the edge details in different regions. A satisfactory color rendition is achieved for a class of images that do not satisfy the gray-world assumption implicit to the theoretical foundation of the retinex. Experiments are carried out to compare the new method with several spatial and transform domain methods. Our results indicate that the new method is superior in enhancement applications, improves computation speed, and performs well for images with high illumination variations than other methods. Further comparisons of images from National Aeronautics and Space Administration and a wearable camera eButton have shown a high performance of the new method with better color restoration and preservation of image details.

Continuous variance estimation in video surveillance sequences with high illumination changes

Continuous estimation of signal statistics is an important issue in many video processing systems, such as motion detection in surveillance applications. In this paper we demonstrate how results of classical expressions for variance estimation decrease in accuracy when dealing with sequences containing high illumination variations. The paper also proposes a new estimation method, and shows how, under such conditions, the accuracy of the proposed method produces better results whilst maintaining performance in scenarios with smaller changes, thus improving the motion detection stage of a video surveillance system.