Poor Illumination Conditions Research Articles

SDL: Spectrum-Disentangled Representation Learning for Visible-Infrared Person Re-Identification

Visible-infrared person re-identification (RGB-IR ReID) is extremely important for the surveillance applications under poor illumination conditions. Since the difference in the feature representations not only lies in the person’ pose, viewpoint or illumination variations, but also comes from huge spectrum discrepancy, the task becomes practically very challenging. Existing RGB-IR ReID models focus on bridging the gap between RGB and IR images through shared feature embedding, subspace learning or via adversarial learning. However, these methods do not explicitly disregard the spectrum information which is otherwise irrelevant for ReID. Further, adversarial learning methods has less promising convergence. This motivates us to design a non-adversarial and fast disentanglement method to disentangle the spectrum information while learning the identity discriminative features. To extract these features, we propose a novel network with disentanglement loss which can distill identity features and dispel spectrum features. Our network has two branches, spectrum dispelling and spectrum distilling branch. On spectrum dispelling branch, we apply identification loss to learn the identity related and spectrum disentangled features. On spectrum distilling branch, we apply an identity-dispeller loss to fool the identity classifier so that it primarily learns spectrum related information. The entire network is trained in an end-to-end manner, which minimizes spectrum information and maximizes invariant identity relevant information at spectrum dispelling branch. Extensive experiments on existing datasets demonstrate the superior performance of our approach compared to the state-of-the-art.

3D MODELS OF THE QH31, QH32 AND QH33 TOMBS IN QUBBET EL HAWA (ASWAN, EGYPT)

Abstract. This paper describes the methodology employed to obtain 3D models of three funerary complexes (QH31, QH32 and QH33) of the Necropolis of Qubbet el Hawa (Aswan, Egypt) and the main results obtained. These rock-cut tombs are adjacent structures defined by complex geometries such as chambers, corridors and vertical shafts. The main goal of this study was to discover the spatial relationships between them and obtain a complete 3D model. In addition, some models with realistic textures of the burial chambers were demanded in order to analyse archaeological, architectural and geological aspects. The methodology was based on the use of Terrestrial Laser Scanning and Close Range Photogrammetry. In general, both techniques were developed in parallel for each tomb. Some elements presented difficulties because of their reduced dimensions, the presence of vertical falls, some objects stored in the tombs that generated occlusions of some walls, coincidence of other workers, poor illumination conditions, etc. The results included three complete 3D models obtained without texture and some parts of interest obtained with real textures. All models were merged into a global 3D model. The information extracted from this product has helped architects and archaeologists to contrast their premises about the spatial behaviour of the tombs. The results have also allowed the obtaining of the first 3D documentation of these tombs under the same reference system, allowing them to be studied completely. This information is very important for documentation purposes but also to understand the spatial behaviour of these structures and the excavation processes developed by ancient Egyptians 4000 years ago.

Texture collinearity foreground segmentation for night videos

One of the most difficult scenarios for unsupervised segmentation of moving objects is found in nighttime videos where the main challenges are the poor illumination conditions resulting in low-visibility of objects, very strong lights, surface-reflected light, a great variance of light intensity, sudden illumination changes, hard shadows, camouflaged objects, and noise. This paper proposes a novel method, coined COLBMOG (COLlinearity Boosted MOG), devised specifically for the foreground segmentation in nighttime videos, that shows the ability to overcome some of the limitations of state-of-the-art methods and still perform well in daytime scenarios. It is a texture-based classification method, using local texture modeling, complemented by a color-based classification method. The local texture at the pixel neighborhood is modeled as an N-dimensional vector. For a given pixel, the classification is based on the collinearity between this feature in the input frame and the reference background frame. For this purpose, a multimodal temporal model of the collinearity between texture vectors of background pixels is maintained. COLBMOG was objectively evaluated using the ChangeDetection.net (CDnet) 2014, Night Videos category, benchmark. COLBMOG ranks first among all the unsupervised methods. A detailed analysis of the results revealed the superior performance of the proposed method compared to the best performing state-of-the-art methods in this category, particularly evident in the presence of the most complex situations where all the algorithms tend to fail.

Enhanced situation awareness through CNN-based deep multimodal image fusion

Automated situation awareness (ASA) in a complex and dynamic setting is a challenging task. The accurate perception of environmental elements and events is critical for the successful completion of a mission. The key technology to implement ASA is target detection. However, in most situations, targets of interest that are at a distance are hard to identify due to the small size, complex background, and poor illumination conditions. Thus, multimodal (e.g., visible and thermal) imaging and fusion techniques are adopted to enhance the capability for situation awareness. A deep multimodal image fusion (DIF) framework is proposed to detect the target by fusing the complementary information from multimodal images with a deep convolutional neural network. The DIF is built and validated with the Military Sensing Information Analysis Center dataset. Extensive experiments were carried out to demonstrate the effectiveness and superiority of the proposed method in terms of both detection accuracy and computational efficiency.

Multiple Sound Source Position Estimation by Drone Audition Based on Data Association Between Sound Source Localization and Identification

Drone audition, or auditory processing for drones equipped with a microphone array, is expected to compensate for problems affecting drones’ visual processing, in particular occlusion and poor-illumination conditions. The current state of drone audition still assumes a single sound source. When a drone hears sounds originating from multiple sound sources, its sound-source localization function determines their directions. If two sources are very close to each other, the localization function cannot determine whether they are crossing or approaching-then-departing. This ambiguity in tracking multiple sound sources is resolved by data association. Typical methods of data association use each label of the separated sounds, but are prone to errors due to identification failures. Instead of labeling by classification, this study uses a set of classification measures determined by support vector machines (SVM) to avoid labeling failures and deal with unknown signals. The effectiveness of the proposed approach is validated through simulations and experiments conducted in the field.

An Experiment-Based Review of Low-Light Image Enhancement Methods

Images captured under poor illumination conditions often exhibit characteristics such as low brightness, low contrast, a narrow gray range, and color distortion, as well as considerable noise, which seriously affect the subjective visual effect on human eyes and greatly limit the performance of various machine vision systems. The role of low-light image enhancement is to improve the visual effect of such images for the benefit of subsequent processing. This paper reviews the main techniques of low-light image enhancement developed over the past decades. First, we present a new classification of these algorithms, dividing them into seven categories: gray transformation methods, histogram equalization methods, Retinex methods, frequency-domain methods, image fusion methods, defogging model methods and machine learning methods. Then, all the categories of methods, including subcategories, are introduced in accordance with their principles and characteristics. In addition, various quality evaluation methods for enhanced images are detailed, and comparisons of different algorithms are discussed. Finally, the current research progress is summarized, and future research directions are suggested.

Learning Modality-Consistency Feature Templates: A Robust RGB-Infrared Tracking System

With a large number of video surveillance systems installed for the requirement from industrial security, the task of object tracking, which aims to locate objects of interest in videos, is very important. Although numerous tracking algorithms for RGB videos have been developed in the decade, the tracking performance and robustness of these systems may be degraded dramatically when the information from RGB video is unreliable (e.g., poor illumination conditions or very low resolution). To address this issue, this paper presents a new tracking system, which aims to combine the information from RGB and infrared modalities for object tracking. The proposed tracking systems is based on our proposed machine learning model. Particularly, the learning model can alleviate the modality discrepancy issue under the proposed modality consistency constraint from both representation patterns and discriminability, and generate discriminative feature templates for collaborative representations and discrimination in heterogeneous modalities. Experiments on a variety of challenging RGB-infrared videos demonstrate the effectiveness of the proposed algorithm.

Improved Oct Image Enhancement using Clahe

Medical image processing is a challenging research field, since most captured images suffer from noise and poor contrast nature. The accuracy of details present in the medical image depends entirely on the captured image quality. The factor that affects the quality of the images includes poor illumination conditions, capturing devices and inexperienced technicians that may result in low contrast images. Hence, contrast enhancement techniques are necessary to improve the quality of OCT images for further processing. In this paper, the enhancement of OCT images is carried out using various enhancement techniques to identify the method that offers improvement in the enhancement quality of the image. It presents a comparative evaluation of enhancement techniques based on the performance indices calculated from the experimental results. The results of this research work suggest the better enhancement technique suitable for OCT images depending on the various performance metrics used prominently in medical imaging.

A Review of Enhancement and Segmentation Techniques for Digital Images

Image enhancement and segmentation are the two imperative steps while processing digital images. The goal of enhancement is to improve the quality of images so as to nullify the effect of poor illumination conditions during image acquisition. Afterwards, segmentation is performed to extract region of interest (ROI) from the background details of the image. There is a vast literature available for both the techniques. Therefore, this paper is intended to summarize the basic as well as advanced enhancement and segmentation techniques under a single heading; to provide an insight for future researches in the field of pattern recognition.

PsLSNet: Automated psoriasis skin lesion segmentation using modified U-Net-based fully convolutional network

The segmentation of psoriasis skin lesions from RGB color images is a challenging task in the computer vision, due to poor illumination conditions, the irregular shapes and sizes of psoriasis lesions, fuzzy boundaries between the lesions and the surrounding skin, and various artifacts such as skin hairs and camera reflections. The manual segmentation of lesions is very time-consuming and laborious for the dermatologist, and various automatic lesion segmentation approaches have therefore been presented by researchers in the recent past. However, these existing state-of-the-art approaches have various limitations, such as being highly dependent on feature engineering, showing poor performance in terms of accuracy and failing to consider challenging cases, as explained above. In view of this, we present an automated psoriasis lesion segmentation method based on a modified U-Net architecture, referred as PsLSNet. The architecture consists of a 29-layer deep fully convolutional network, for extracting spatial information automatically. In U-Net architecture there are two paths namely contracting and extracting, which are connected as U-shape. The proposed convolutional neural network also provides accelerated training by reducing the covariate shift through the implementation of batch normalization and is capable of segmenting the lesion even in challenging cases such as under poor acquisition conditions and in the presence of artifacts. In our experiment, we use 5241 images of psoriasis lesions collected from 1026 psoriasis patients by a dermatologist. The experimental results show effective performance metrics such as a Dice coefficient of 93.03% and an accuracy of 94.80%, with 89.60% sensitivity and 97.60% specificity, values that are significantly higher than for existing approaches.

Digital video microscopy enhanced by deep learning

Single particle tracking is essential in many branches of science and technology, from the measurement of biomolecular forces to the study of colloidal crystals. Standard current methods rely on algorithmic approaches: by fine-tuning several user-defined parameters, these methods can be highly successful at tracking a well-defined kind of particle under low-noise conditions with constant and homogenous illumination. Here, we introduce an alternative data-driven approach based on a convolutional neural network, which we name DeepTrack. We show that DeepTrack outperforms algorithmic approaches, especially in the presence of noise and under poor illumination conditions. We use DeepTrack to track an optically trapped particle under very noisy and unsteady illumination conditions, where standard algorithmic approaches fail. We then demonstrate how DeepTrack can also be used to track multiple particles and non-spherical objects such as bacteria, also at very low signal-to-noise ratios. In order to make DeepTrack readily available for other users, we provide a Python software package, which can be easily personalized and optimized for specific applications.

Visual Semantic Landmark-Based Robust Mapping and Localization for Autonomous Indoor Parking.

Autonomous parking in an indoor parking lot without human intervention is one of the most demanded and challenging tasks of autonomous driving systems. The key to this task is precise real-time indoor localization. However, state-of-the-art low-level visual feature-based simultaneous localization and mapping systems (VSLAM) suffer in monotonous or texture-less scenes and under poor illumination or dynamic conditions. Additionally, low-level feature-based mapping results are hard for human beings to use directly. In this paper, we propose a semantic landmark-based robust VSLAM for real-time localization of autonomous vehicles in indoor parking lots. The parking slots are extracted as meaningful landmarks and enriched with confidence levels. We then propose a robust optimization framework to solve the aliasing problem of semantic landmarks by dynamically eliminating suboptimal constraints in the pose graph and correcting erroneous parking slots associations. As a result, a semantic map of the parking lot, which can be used by both autonomous driving systems and human beings, is established automatically and robustly. We evaluated the real-time localization performance using multiple autonomous vehicles, and an repeatability of 0.3 m track tracing was achieved at a 10 kph of autonomous driving.

Heading Reference-Assisted Pose Estimation for Ground Vehicles

In this paper, heading reference-assisted pose estimation (HRPE) has been proposed to compensate inherent drift of visual odometry (VO) on ground vehicles, where an estimation error is prone to grow while the vehicle is making turns or in environments with poor features. By introducing a particular orientation as “heading reference,” a pose estimation framework has been presented to incorporate measurements from heading reference sensors into VO. A graph formulation is then proposed to represent the pose estimation problem under the commonly used graph optimization model. Simulations and experiments on KITTI data set and our self-collected sequences have been conducted to verify the accuracy and robustness of the proposed scheme. KITTI sequences and manually generated heading measurement with Gaussian noises are used in simulation, where rotational drift error is observed to be bounded. Compared with a pure VO, the proposed approach greatly reduces average translational localization error from 153.85 to 24.29 m and 23.80 m in self-collected stereo visual sequences with traveling distance over 4.5 km at the processing rates of 19.7 and 11.1 Hz, for the loosely coupled and tightly coupled models, respectively. Note to Practitioners —When the Global Positioning System is not available or reliable, visual odometry (VO) on ground vehicles is an efficient tool for estimating the pose, which involves translation and rotation. However, VO inherently suffers from drifting issue due to constant iterations. By adding a low-cost heading reference sensor, this paper first introduces graph optimization formulation of pose estimation and then presents a pose estimation framework which incorporates heading measurements to VO such that long-term translation and rotation estimation errors can be greatly reduced in real-time computation. As a supplementary to VO, performance may still deteriorate in environments with poor illumination conditions and high-frequency movements. The proposed approach may be further improved by fusing heading measurements from more sensors or being used to build a heading reference-assisted simultaneous localization and mapping (SLAM) system on any off-the-shelf SLAM framework.

Infrared and Visible Image Fusion Under Different Illumination Conditions Based on Illumination Effective Region Map

It is well known that the quality of visible images taken under different illumination conditions varies greatly, so the illumination factors will affect infrared and visible image fusion effects inevitably. This paper proposes an infrared and visible image fusion algorithm that satisfies poor illumination conditions. The algorithm is divided into three stages: image preprocessing, multiscale decomposition and image fusion. The purpose of image preprocessing is to improve the contrast of visible image and extract the visual salient regions of infrared image. In the multiscale analysis stage, infrared and visible images are decomposed into different scales by combining Gaussian transform and Rolling Guidance Filter, which effectively avoids the halo artifacts. In the image fusion stage, the fusion weights of base layer coefficients are determined based on the combination of the saliency map of infrared image and the Illumination Effective Region Map of visible image. The detail layer coefficients of images are fused by choose-max fusion rule based on the local variance of detail features of the original images. Experiments show that the fusion effects of the proposed algorithm are robust to illumination variations, and the fused images have good detail clarity and can preserve the effective information of the original images well in an unsatisfactory illumination condition.

Face-from-Depth for Head Pose Estimation on Depth Images.

Depth cameras allow to set up reliable solutions for people monitoring and behavior understanding, especially when unstable or poor illumination conditions make unusable common RGB sensors. Therefore, we propose a complete framework for the estimation of the head and shoulder pose based on depth images only. A head detection and localization module is also included, in order to develop a complete end-to-end system. The core element of the framework is a Convolutional Neural Network, called POSEidon +, that receives as input three types of images and provides the 3D angles of the pose as output. Moreover, a Face-from-Depth component based on a Deterministic Conditional GAN model is able to hallucinate a face from the corresponding depth image. We empirically demonstrate that this positively impacts the system performances. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Experimental results show that our method overcomes several recent state-of-art works based on both intensity and depth input data, running in real-time at more than 30 frames per second.

Dynamical stochastic resonance for non‐uniform illumination image enhancement

Images taken under poor illumination conditions have low contrast and dark tones. General dark image enhancement algorithms cannot effectively enhance these images without introducing over-enhancement, detail loss, and noise amplification. In this study, a simple and fast enhancement technique of non-uniform illumination images is proposed based on dynamical stochastic resonance (DSR). The low-contrast images are enhanced through the nonlinear iteration by solving monostable Langevin equation. Iteration parameters are dynamically adjusted according to the intensity distribution of the original images, which ensure the balance of visibility and naturalness in the entire areas. A threshold is defined to automatically confirm the optimal outputs. The enhanced image is obtained by fusing the DSR result, original component, and illumination compensation component. The computational time, no-reference perceptual quality assessment, and lightness order error are measured to evaluate the performance of experimental results. The subjective and objective comparison with state-of-the-art methods shows that our method performs well to enhance the non-uniform illumination images with a low-computational complexity.

Fuzzy granulation and constraint neighbourhood granulation structure for object classification in unevenly illuminated images

Classification of objects and background in a complex scene is a challenging early vision problem. Specifically, the problem is compounded under poor illumination conditions. In this paper, the problem of object and background classification has been addressed under unevenly illuminated conditions. The challenge is to extract the actual boundary under poorly illuminated portion. This has been addressed using the notion of rough sets and granular computing. In order to take care of differently illuminated portions over the image, adaptive window growing approach has been employed to partition the image into different windows and optimum threshold for classification over a given window has been determined by the four proposed granular computing based schemes. Over a particular window, illumination varies significantly posing a challenge for classification. In order to deal with these issues, we have proposed four schemes based on heterogeneous and non-homogeneous granulation. They are; (i) Heterogeneous Granulation based Window Growing (HGWG), (ii) Empirical Non-homogeneous Granulation based Window Growing (ENHWG), (iii) Fuzzy Gradient Non-homogeneous based Window Growing (FNHWG), (iv) Fuzzy Gradient Non-homogeneous Constrained Neighbourhood based Window Growing (FNHCNGWG). The proposed schemes have been tested with images from Berkeley image database, specifically with unevenly illuminated images having single and multiple objects. The performance of the proposed schemes has been evaluated based on four metrics. The performance of the FNHWG and FNHCNGWG schemes has been compared with Otsu, K-means, FCM, PCM, Pal’s method, HGWG, ENHWG and Fuzzy Non-homogeneous Neighbourhood based Window growing (FNHNGWG) schemes and found to be superior to the existing ones.

Adaptive Image Enhancement Based on Guide Image and Fraction-Power Transformation for Wireless Capsule Endoscopy.

Good image quality of the wireless capsule endoscopy (WCE) is the key for doctors to diagnose gastrointestinal (GI) tract diseases. However, the poor illumination, limited performance of the camera in WCE, and complex environment in the GI tract usually result in low-quality endoscopic images. Existing image enhancement methods only use the information of the image itself or multiple images of the same scene to accomplish the enhancement. In this paper, we propose an adaptive image enhancement method based on guide image and fraction-power transformation. First, intensities of endoscopic images are analyzed to assess the illumination conditions. Second, images captured under poor illumination conditions are enhanced by a brand-new image enhancement method called adaptive guide image based enhancement (AGIE). AGIE enhances low-quality images by using the information of a good quality image of the similar scene. Otherwise, images are enhanced by the proposed adaptive fraction-power transformation. Experimental results show that the proposed method improves the average intensity of endoscopic images by 64.20% and the average local entropy by 31.25%, which outperforms the state-of-art methods.

Vehicle Tracking at Nighttime by Kernelized Experts With Channel-Wise and Temporal Reliability Estimation

Despite the fact that in recent years, vision-based tracking approaches have made significant progress, the task of tracking vehicles at night still remains challenging. Visual information is strongly deteriorated or at least degraded due to poor illumination conditions. This reduces the perceptive ability of vision systems significantly and can even lead to target loss, resulting in false estimation and/or false prediction of object behavior. In this paper, we propose a novel online-learning method to track vehicles at night. Our method is based on the kernelized correlation filter and assembles different feature channels to kernelized experts. By estimating their reliabilities, we force the appearance model to focus on the most discriminative visual features to accomplish the classification. In addition, a temporal optimization step in conjunction with a memory model is used to remove outliers and keep the most reliable samples to train the tracker models. Experiments over various daytime and weather conditions show that our approach outperforms existing trackers at night and in case of bad weather while offering state-of-the-art performance in more favorable situations. As our tracker has only little computational cost, it is appropriate for use cases with real-time requirements like in automotive or industrial applications.

Hierarchical Discriminative Learning for Visible Thermal Person Re-Identification

Person re-identification is widely studied in visible spectrum, where all the person images are captured by visible cameras. However, visible cameras may not capture valid appearance information under poor illumination conditions, e.g, at night. In this case, thermal camera is superior since it is less dependent on the lighting by using infrared light to capture the human body. To this end, this paper investigates a cross-modal re-identification problem, namely visible-thermal person re-identification (VT-REID). Existing cross-modal matching methods mainly focus on modeling the cross-modality discrepancy, while VT-REID also suffers from cross-view variations caused by different camera views. Therefore, we propose a hierarchical cross-modality matching model by jointly optimizing the modality-specific and modality-shared metrics. The modality-specific metrics transform two heterogenous modalities into a consistent space that modality-shared metric can be subsequently learnt. Meanwhile, the modality-specific metric compacts features of the same person within each modality to handle the large intra-modality intra-person variations (e.g. viewpoints, pose). Additionally, an improved two-stream CNN network is presented to learn the multi-modality sharable feature representations. Identity loss and contrastive loss are integrated to enhance the discriminability and modality-invariance with partially shared layer parameters. Extensive experiments illustrate the effectiveness and robustness of the proposed method.