Complicated Illumination Research Articles

Precise Facial Landmark Detection by Reference Heatmap Transformer.

Most facial landmark detection methods predict landmarks by mapping the input facial appearance features to landmark heatmaps and have achieved promising results. However, when the face image is suffering from large poses, heavy occlusions and complicated illuminations, they cannot learn discriminative feature representations and effective facial shape constraints, nor can they accurately predict the value of each element in the landmark heatmap, limiting their detection accuracy. To address this problem, we propose a novel Reference Heatmap Transformer (RHT) by introducing reference heatmap information for more precise facial landmark detection. The proposed RHT consists of a Soft Transformation Module (STM) and a Hard Transformation Module (HTM), which can cooperate with each other to encourage the accurate transformation of the reference heatmap information and facial shape constraints. Then, a Multi-Scale Feature Fusion Module (MSFFM) is proposed to fuse the transformed heatmap features and the semantic features learned from the original face images to enhance feature representations for producing more accurate target heatmaps. To the best of our knowledge, this is the first study to explore how to enhance facial landmark detection by transforming the reference heatmap information. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.

Hierarchically adaptive image block matching under complicated illumination conditions

Image block matching is one of active research fields in image processing, which has been widely used in security monitoring and motion estimation. Due to great disparities in images of the same scene under various illumination, block matching has been a challenging task. To this end, we propose a hierarchical block matching method which is adaptive to the computational complexity and suited to complicated illumination environments. The approach is divided into two parts. First, in order to reduce searching time, the whole algorithm is established in the framework of pyramid structures. Second, the correlation coefficient and structural functions are adopted to evaluate the similarity between two images so as to get better matching results. Simulation results show that, compared with the classical three-step, four-step searching algorithms and OHBM algorithm, this algorithm can reduce the time complexity efficiently. Moreover, since the algorithm makes use of the structure details of images, it is robust to complicated scenes.

Fusing point cloud with image for object detection using convolutional neural networks

Addressing on the issues like varying object scale, complicated illumination conditions, and lack of reliable distance information in driverless applications, this paper proposes a multi-modal fusion method for object detection by using convolutional neural networks. The depth map is generated by mapping LiDAR point cloud onto the image plane and taken as input data together with the RGB image. The input data is also processed by the sliding window to reduce information loss. Two feature extracting networks are used to extract features of the image and the depth map respectively. The generated feature maps are fused through a connection layer. The objects are detected by processing the fused feature map through position regression and object classification. Non-maximal suppression is used to optimize the detection results. The experimental results on the KITTI dataset show that the proposed method is robust in various illumination conditions and especially effective on detecting small objects. Compared with other methods, the proposed method exhibits integrated advantages in terms of detection accuracy and speed.

Robust Face Alignment by Multi-Order High-Precision Hourglass Network.

Heatmap regression (HR) has become one of the mainstream approaches for face alignment and has obtained promising results under constrained environments. However, when a face image suffers from large pose variations, heavy occlusions and complicated illuminations, the performances of HR methods degrade greatly due to the low resolutions of the generated landmark heatmaps and the exclusion of important high-order information that can be used to learn more discriminative features. To address the alignment problem for faces with extremely large poses and heavy occlusions, this paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model, which are seamlessly integrated into a novel multi-order high-precision hourglass network (MHHN). The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT). At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information. To the best of our knowledge, this is the first study to explore heatmap subpixel regression for robust and high-precision face alignment. The experimental results from challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

Vision reconstruction based on planar laser with nonrestrictive installation position and posture relative to 2D reference.

The active-vision-based reconstruction is an advanced approach to characterize the spatial object. In the active vision, the planar laser is a kind of the moderate structured light for the robustness and adaptivity in the complicated illumination environment. However, as the planar laser is determined by 3 unknown degrees of freedom, the laser sensor should be calibrated in the immovable coordinate frame of reference (CFR), or be pre-calibrated in the coordinate frame of the camera (CFC) and moved with CFC. In the study, as the main improvement, the laser plane is attached to the simple 2D reference without strict constraints of the relative installation position and posture. The laser plane is desired to be flexibly moved and registered in CFC. The calibration is modeled by the closed loop of the camera, the cylindrical object, the planar laser and the 2D reference, in which the camera is twice considered as the bridge between the cylindrical object and the 2D reference. The calibration contributes the planar laser that is determined as a constant vector in CFR. Furthermore, the reconstruction is performed by the calibrated laser plane and refined by the optimization method. The method errors of 1.263, 1.517, 1.633, and 1.803 mm are evaluated by the experiments, which indicate the flexibility of the method and applicable prospects in the spatial reconstruction field.

Deep Feature Ranking for Person Re-Identification

Person re-identification plays a critical part in many surveillance applications. Due to complicated illumination environments and various viewpoints, it is still a challenging problem to extract robust features. To solve this issue, we propose a novel deep feature ranking scheme. Our main contribution is to rank achieved deep features, which are obtained by classic deep learning model, and set the sort order number as our feature vector, named as ordinal deep features (ODFs). Person re-identification results are acquired by ranking person candidates by measuring distance based on ODFs. Since applying for rank orders rather than original feature values, our method achieves robust results, especially under the situation of viewpoints shift. Comprehensive experiments are carried out to demonstrate the significance of the proposed feature. Meanwhile, comparative experiments are applied over the publicly available dataset, our method achieves promising performance and outperforms the state of the art methods. Moreover, we applied the proposed feature in the scenario of image classification and discussed the effectiveness.

An RGB-D-Based Cross-Field of View Pose Estimation System for a Free Flight Target in a Wind Tunnel

Estimating the real-time pose of a free flight aircraft in a complex wind tunnel environment is extremely difficult. Due to the high dynamic testing environment, complicated illumination condition, and the unpredictable motion of target, most general pose estimating methods will fail. In this paper, we introduce a cross-field of view (FOV) real-time pose estimation system, which provides high precision pose estimation of the free flight aircraft in the wind tunnel environment. Multiview live RGB-D streams are used in the system as input to ensure the measurement area can be fully covered. First, a multimodal initialization method is developed to measure the spatial relationship between the RGB-D camera and the aircraft. Based on all the input multimodal information, a so-called cross-FOV model is proposed to recognize the dominating sensor and accurately extract the foreground region in an automatic manner. Second, we develop an RGB-D-based pose estimation method for a single target, by which the 3D sparse points and the pose of the target can be simultaneously obtained in real time. Many experiments have been conducted, and an RGB-D image simulation based on 3D modeling is implemented to verify the effectiveness of our algorithm. Both the real scene’s and simulation scene’s experimental results demonstrate the effectiveness of our method.

Camera calibration method with checkerboard pattern under complicated illumination

Camera calibration is a key step in three-dimensional (3-D) reconstruction; however, the calibration accuracy and stability often suffer from complicated illumination in real applications. The camera calibration method using the checkerboard pattern is improved for high-precision calibration under complicated illumination. First, an improved Harris corner detector based on color constancy, as well as a subpixel optimization method based on prior knowledge of the checkerboard, are proposed to overcome the influence of bad lighting. Second, a checkerboard pattern with a central circle point is designed for better reference point location for uneven illumination. The checkerboard identification by Delaunay triangulation is also improved for the situation of discontinuity caused by over-saturation. Finally, the camera parameter optimization process is improved to reduce the calibration error produced by complicated illumination. Experimental results show that the improvements achieve more accurate and stable calibration results in complicated lighting conditions compared with traditional methods.

Robust lane detection and tracking using multiple visual cues under stochastic lane shape conditions

As one of the essential components of environment perception techniques for an intelligent vehicle, lane detection is confronted with challenges including robustness against the complicated disturbance and illumination, also adaptability to stochastic lane shapes. To overcome these issues, we proposed a robust lane detection method named classification-generation-growth-based (CGG) operator to the detected lines, whereby the linear lane markings are identified by synergizing multiple visual cues with the a priori knowledge and spatial–temporal information. According to the quality of linear lane fitting, the linear and linear-parabolic models are dynamically switched to describe the actual lane. The Kalman filter with adaptive noise covariance and the region of interests (ROI) tracking are applied to improve the robustness and efficiency. Experiments were conducted with images covering various challenging scenarios. The experimental results evaluate the effectiveness of the presented method for complicated disturbances, illumination, and stochastic lane shapes.

Adaptive enhancement for nonuniform illumination images via nonlinear mapping

Nonuniform illumination images suffer from degenerated details because of underexposure, overexposure, or a combination of both. To improve the visual quality of color images, underexposure regions should be lightened, whereas overexposure areas need to be dimmed properly. However, discriminating between underexposure and overexposure is troublesome. Compared with traditional methods that produce a fixed demarcation value throughout an image, the proposed demarcation changes as local luminance varies, thus is suitable for manipulating complicated illumination. Based on this locally adaptive demarcation, a nonlinear modification is applied to image luminance. Further, with the modified luminance, we propose a nonlinear process to reconstruct a luminance-enhanced color image. For every pixel, this nonlinear process takes the luminance change and the original chromaticity into account, thus trying to avoid exaggerated colors at dark areas and depressed colors at highly bright regions. Finally, to improve image contrast, a local and image-dependent exponential technique is designed and applied to the RGB channels of the obtained color image. Experimental results demonstrate that our method produces good contrast and vivid color for both nonuniform illumination images and images with normal illumination.

Aphid Identification and Counting Based on Smartphone and Machine Vision

Exact enumeration of aphids before the aphids outbreak can provide basis for precision spray. This paper designs counting software that can be run on smartphones for real-time enumeration of aphids. As a first step of the method used in this paper, images of the yellow sticky board that is aiming to catch insects are segmented from complex background by using GrabCut method; then the images will be normalized by perspective transformation method. The second step is the pretreatment on the images; images of aphids will be segmented by using OSTU threshold method after the effect of random illumination is eliminated by single image difference method. The last step of the method is aphids’ recognition and counting according to area feature of aphids after extracting contours of aphids by contour detection method. At last, the result of the experiment proves that the effect of random illumination can be effectively eliminated by using single image difference method. The counting accuracy in greenhouse is above 95%, while it can reach 92.5% outside. Thus, it can be seen that the counting software designed in this paper can realize exact enumeration of aphids under complicated illumination which can be used widely. The design method proposed in this paper can provide basis for precision spray according to its effective detection insects.

A dynamical systems novel approach for accurate human eyes location

To accurately locate human eye features, such as eye contours, iris boundaries and the coordinates of pupil centers and canthi, a hierarchical approach was proposed. First, an AdaBoost based eye detector was trained for extracting the sub-images containing eyes areas, and the sub-images were enhanced by single-scale Retinex algorithm. Second, the pupil centers and iris boundaries were located by gradient Hough circle transform in the sub-images. Third, the eyes areas were segmented by FCM clustering for constructing the initial signed distance function which was the Level Set function of C-V Model. Finally, the eye contours and canthi were located using C-V Model. With the combination of FCM clustering and C-V Model, the convergent speed of C-V Model was increased by 64.1% on the Purdue AR face test set, while the locating accuracy was increased by 8.3%. This approach is immune to complicated background and illumination changes, showing its high robustness. Results prove the efficiency of this approach.

Adaptive and accurate color edge extraction method for one-shot shape acquisition

This paper presents an approach to extract accurate color edge information using encoded patterns in hue, saturation, and intensity (HSI) color space. This method is applied to one-shot shape acquisition. Theoretical analysis shows that the hue transition between primary and secondary colors in a color edge is based on light interference and diffraction. We set up a color transition model to illustrate the hue transition on an edge and then define the segmenting position of two stripes. By setting up an adaptive HSI color space, the colors of the stripes and subpixel edges are obtained precisely without a dark laboratory environment, in a low-cost processing algorithm. Since this method does not have any constraints for colors of neighboring stripes, the encoding is an easy procedure. The experimental results show that the edges of dense modulation patterns can be obtained under a complicated environment illumination, and the precision can ensure that the three-dimensional shape of the object is obtained reliably with only one image.

Particle Filter Object Tracking Algorithm Based on Color and Gradient

With the robustness of a single color which is not high in standard particle filter tracking, a fusion of color and gradient particle filter algorithm is proposed. By the advantages of color described the target ’global and gradients described the shape of structure, they are weighted fusion to form a new integrated histogram and applied to the particle filter framework. The experimental results show that compared with the traditional particle filter algorithm, the text of the algorithm can achieve relatively reliable target tracking under complicated background and illumination changes, with better robustness and reliability.

License Plate Location Based on Texture Analysis and Projection Approach

Since it is difficult to locate license plate quickly and precisely under complicated background and different illumination conditions, therefore, this paper proposes a high-speed and precise approach to solve such kind of problem on the basis of texture analysis and projection method. The approach utilizes both rich textural feature and projective statistical feature of characters in LP region to carry out coarse and fine license plate location. The experiment shows that, under different illumination, car models and background conditions, such approach is proved to be fast, precise and practical.

Robust intelligent PCA‐based face recognition framework using GNP‐fuzzy data mining

AbstractTraditional principal component analysis (PCA) based face recognition algorithms have a low recognition accuracy due to the influence of noise and illumination changes. This paper proposes a robust, intelligent PCA‐based face recognition framework in the complicated illumination database when using multiple training images per person (MTIP‐CID). There are mainly two improvements in the proposed method. One is that a face‐recognition‐oriented genetic‐based clustering algorithm is introduced to reduce the influence of a large number of classes on the classification accuracy in the MTIP‐CID. The other is that a classifier based on fuzzy class association rules (FCARs) is applied to mine the inherent relationships between eigenfaces and to improve the robustness of PCA‐based face recognition in noisy environments. Experimental results on the extended Yale‐B database demonstrate that the proposed framework performs better and is more robust against noise compared with other traditional face recognition algorithms, i.e. linear discriminant analysis (LDA) and local binary patterns (LBPs). © 2013 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Photobleaching imprinting microscopy: seeing clearer and deeper

We present a generic sub-diffraction-limited imaging method - photobleaching imprinting microscopy (PIM) - for biological fluorescence imaging. A lateral resolution of 110 nm was measured, more than a twofold improvement over the optical diffraction limit. Unlike other super-resolution imaging techniques, PIM does not require complicated illumination modules or specific fluorescent dyes. PIM is expected to facilitate the conversion of super-resolution imaging into a routine lab tool, making it accessible to a much broader biological research community. Moreover, we show that PIM can increase the image contrast of biological tissue, effectively extending the fundamental depth limit of multi-photon fluorescence microscopy.

Combined feedback method for designing a free-form optical system with complicated illumination patterns for an extended LED source

A combined feedback design method based on variable separation mapping is proposed in this paper to design free-form optical systems for an extended LED source with complicated illumination patterns. In this method, macro energy division and micro illuminance distribution feedback modifications are carried out according to the deviation between the simulated illumination results and the target requirements. The free-form optical system is then regenerated, and the deviation could be minimized through multiple iterations. Results indicate that free-form optical system designed by this method could achieve precise energy distribution, high regional illuminance uniformity (89.7%), and high light output efficiency (94.9%) simultaneously.

Face skin color detection in complicated illumination conditions

Face skin color detection is sensitive to illumination variations.This paper presented a novel method to detect the face skin color regions in complicated illumination conditions.Firstly,a skin color model in complicated illumination conditions was created using YCbCr color space.Secondly,the skin color model was applied to detect the face skin color regions in the input image.Finally,the 4-connected regions were calculated in the detection result and be used to eliminate the non-face regions and recover the real face regions.The experimental results demonstrate that the proposed method can detect the face skin color regions correctly in complicated illumination conditions.

Uniformly sampling multi-resolution analysis for image-based relighting

Image-based relighting allows us to efficiently light a scene under complicated illumination conditions. However, the traditional cubemap based multi-resolution analysis unevenly samples the spherical surface, with a higher sampling rate near the face corners and a lower one near the face centers. The non-uniformity penalizes the efficiency of data representation. This paper presents a uniformly sampling multi-resolution analysis approach, namely the icosahedron spherical wavelets (ISW), for image-based relighting under time-varying distant environment. Since the proposed ISW approach provides a highly uniform sampling distribution over the spherical domain, we thus can efficiently handle high frequency variations locally in the illumination changes as well as reduce the number of wavelet coefficients needed in the renderings. Furthermore, visual artifacts are demonstrated to be better suppressed in the proposed ISW approach. Compared with the traditional cubemap based multi-resolution analysis approach, we show that our approach can effectively produce higher quality image sequences that are closer to the ground truth in terms of percentage square errors.