Poor Illumination Research Articles

A novel fusion method for infrared and visible images under poor illumination conditions

Most infrared and visible image fusion methods are designed based on the fact that visible images have rich scene information and more details like edges and textures than infrared images, while infrared images have prominent thermal target information. However, under poor illumination conditions, most areas of visible images are dark, may contain much noise, and lack the corresponding detail information compared with infrared images. As a result, fused images of those methods suffer from information loss, low contrast, and non-obvious targets. To solve this problem, we propose a novel fusion method. Firstly, an improved rolling guidance filter, named RF-RGF, is proposed to decompose source images into small-scale detail, large-scale detail, and base layers. Secondly, for the fusion of small-scale detail layers, a new nonlinear function-based rule is proposed to transfer more texture information from source images under poor illumination conditions to the fused images. For the fusion of large-scale detail layers, a novel fusion rule based on the weighted sum of support values (WSSV) is constructed to retain details effectively. Then, for the fusion of base layers, the rule based on the visual saliency map (VSM) is adopted to ensure high contrast and a well overall look of the fused image. Moreover, BIMEF and morphological bright and dark details (MBD) are used to further enhance the fused image’s contrast and details, making targets more obvious. Specifically, BIMEF is adopted to enhance the visible image before decomposition. The MBD obtained by two selective rules based on morphological top- and bottom-transformations (MTB) is used to enhance the base layer. Experimental results show that the fusion performance of the proposed method is better than other methods (including some state-of-the-art methods), especially in artifact suppression, information retention, contrast improvement, and target enhancement.

Supervised Attention Network for Arbitrary-Shaped Text Detection in Edge-Fainted Noisy Scene Images

Text mining in noisy situations, like poor contrast and fainted edges, is one of the challenging areas of research in the domain of social networks and computer vision. Scene text detection is a complicated task as text regionhaving varying span in term of size, orientation, aspect ratio, color, font, and script. Furthermore, the contrast of a scene image varies drastically in noisy situations due to poor illumination and image filtering. This faints the text edges and make the task of detection more challenging. In this article, we bring forward a semantic edge supervised spatial-channel attention network, known as SESANet, for detecting arbitrary-shaped text instances in noisy scene images with faint text edges. Our network learns multiscale (MS) supervised edge semantic, pixel-wise spatial structure information, and interchannel dependencies for precisely localizing the text masks in scene images with poor contrast and illumination. Our network is efficient, precise, and fast in nature. SESANet captures rich, dense, discriminative, and MS semantic information. The experimental results show the success of the proposed network. It shows a superior performance with regard to recall on the publicly available benchmark datasets. A new dataset scene images, named as Edge-fainted Noisy Arbitrary-shaped Scene Text (EFNAST) dataset, having varying noise density, poor contrast, low illumination, and faint edges is created.

CNN-based Image Processing algorithm for autonomous optical navigation of Hera mission to the binary asteroid Didymos

Hera mission is the European Space Agency’s contribution to the international collaboration with NASA for the planetary defence, i.e. Asteroid Impact Deflection Assessment aiming to deflect the trajectory of its target binary asteroid system (65803) Didymos. The Early Characterization Phase and the Detailed Characterization Phase of Hera mission are two phases of the proximity operations with the objective to physically and dynamically characterize Didymos. During these phases, an Image Processing algorithm is required to estimate the position of the centroid of the primary to enable Line of Sight navigation. However, the performance of standard Image Processing algorithms is affected by the disturbances of the image, such as poor illumination conditions, the presence of external bodies and the irregular shape of the target. This research addresses this challenge by developing a robust Convolutional Neural Networks-based Image Processing algorithm to estimate the position of the centroids of Didymos and its moon Dimorphos, the pseudorange from the primary and the Sun phase angle. The training, validation and testing datasets are generated with the software Planet and Asteroid Natural scene Generation Utility using the Early Characterization Phase and the Detailed Characterization Phase trajectories as case scenario. The position in the image of the centroids of Didymos and Dimorphos is estimated using their respective position vectors. To estimate the pseudorange, the developed algorithm regresses a set of keypoints on the visible border of Didymos and evaluates its apparent radius. For the Sun phase angle, the pixel position of the subsolar point of the primary is leveraged. The High-Resolution Network is the Convolutional Neural Network architecture applied to detect keypoints with superior spatial precision. Even with the considered disturbances, the analysis shows that the proposed algorithm is able to provide an accurate estimation of the mentioned outputs for all the Early Characterization Phase trajectory and for 77.33% of the Detailed Characterization Phase trajectory, improving the robustness and autonomy of the mission navigation.

Seeing the unseen: Wifi-based 2D human pose estimation via an evolving attentive spatial-Frequency network

Camera-based human pose estimation has become popular due to its wide applications and easy implementation. However, it is not applicable under several circumstances such as poor illumination, occlusion, and private protection. In this work, we utilize Wifi signals to estimate 2D human poses, which is challenging because Wifi signals are abstract and contain limited information. To address these challenges, we develop an evolving attentive spatial-frequency network to discover the relationship between signal variation and body movement for Wifi-based 2D human pose estimation. By first taking dilated CSI sequences as inputs, a spatial-frequency encoder is then introduced to effectively integrate static spatial information and dynamic frequency information from CSI signals. Finally, we design an evolving attention module to enable our model to attend to certain channels of features. Due to a lack of benchmarks, we propose two Wifi-based human pose estimation datasets, the General Pose Estimation dataset (GPE) and Specific Pose Estimation dataset (SPE), which have been released as a public download at project page. Extensive experiments on the proposed datasets show that our model outperforms the state-of-the-art method by at least 16% in terms of PCK@20 (percentage of correct keypoints).

Improving Pedestrian Safety Using Ultra-Wideband Sensors: A Study of Time-to-Collision Estimation.

Pedestrian safety has been evaluated based on the mean number of pedestrian-involved collisions. Traffic conflicts have been used as a data source to supplement collision data because of their higher frequency and lower damage. Currently, the main source of traffic conflict observation is through video cameras that can efficiently gather rich data but can be limited by weather and lighting conditions. The utilization of wireless sensors to gather traffic conflict data can augment video sensors because of their robustness to adverse weather conditions and poor illumination. This study presents a prototype of a safety assessment system that utilizes ultra-wideband wireless sensors to detect traffic conflicts. A customized variant of time-to-collision is used to detect conflicts at different severity thresholds. Field trials are conducted using vehicle-mounted beacons and a phone to simulate sensors on vehicles and smart devices on pedestrians. Proximity measures are calculated in real-time to alert smartphones and prevent collisions, even in adverse weather conditions. Validation is conducted to assess the accuracy of time-to-collision measurements at various distances from the phone. Several limitations are identified and discussed, along with recommendations for improvement and lessons learned for future research and development.

Echo-Enhanced Embodied Visual Navigation.

Visual navigation involves a movable robotic agent striving to reach a point goal (target location) using vision sensory input. While navigation with ideal visibility has seen plenty of success, it becomes challenging in suboptimal visual conditions like poor illumination, where traditional approaches suffer from severe performance degradation. We propose E3VN (echo-enhanced embodied visual navigation) to effectively perceive the surroundings even under poor visibility to mitigate this problem. This is made possible by adopting an echoer that actively perceives the environment via auditory signals. E3VN models the robot agent as playing a cooperative Markov game with that echoer. The action policies of robot and echoer are jointly optimized to maximize the reward in a two-stream actor-critic architecture. During optimization, the reward is also adaptively decomposed into the robot and echoer parts. Our experiments and ablation studies show that E3VN is consistently effective and robust in point goal navigation tasks, especially under nonideal visibility.

Thermal image-based hand gesture recognition for worker-robot collaboration in the construction industry: A feasible study

Worker-robot collaboration (WRC) is a promising solution for complex construction tasks, which can integrate the robots’ advantages in strength and accuracy with human ability in intuitive decision-making and adaptability. A new imperative objective for real-world WRC is to design a user-friendly interface to support safe and efficient worker-robot interactions. Vision-based hand gesture is a simple but effective solution. However, existing methods mainly depend on 3-channel RGB images captured by visible cameras, which are prone to be affected by on-site environmental disturbances, such as poor illumination, fog, and dust. Moreover, previous networks strive for high accuracy, neglecting computational efficiency (e.g., model size and latency) when implementing the network on resource-constrained devices like mobile construction robots. Against this backdrop, this research presented a feasibility study to investigate whether hand signals can be detected from thermal images and designed a lightweight network that has fewer parameters and obtains lower latency without compromising accuracy. Experimental results indicated that thermal images were robust to different lighting conditions, and the proposed model achieved a high classification accuracy (97.54 %) with 1.8 M parameters. The comparative study demonstrated the superiority of our model to other advanced lightweight models, illustrating the feasibility of the developed method in supporting safe WRC applications by recognizing workers’ hand signals.

Road safety benefits and challenges associated with pedestrian footbridge patronage along the Madina-Adenta highway

ABSTRACT Pedestrian fatalities constitute a fourth of all road deaths in Ghana. Footbridges are recent countermeasures deployed to improve pedestrians’ safety in Ghana. The main objectives of the research were to: (i) identify the lapses on the footbridges and their effects on pedestrian crossing behavior in the vicinity of the edifices, (ii) explore the reasons why pedestrians do not use the footbridges and, (iii) evaluate the road safety impact of the footbridges on pedestrian safety. Naturalistic observations and peak-hour pedestrian counts on the footbridges and within the vicinities of the facilities were carried out. Focus group discussions were also conducted to understand pedestrian crossing and walking behavior at the footbridges. Lastly, pedestrian casualties were analyzed for 3 years before and after the bridge construction. There were 165 pedestrian casualties along the highway section. Of these, 29% were fatal while 40% of the victims were hospitalized for at least 24-hours. Many pedestrians still cross the multilane-highway at grade. The main reasons for non-use of the bridges include heights, lengths, security, poor illumination and hawking. Pedestrian injuries have significantly declined during the after period compared with the before period. Findings have implications for road safety education and siting of future footbridges.

OPTIMIZATION OF LIGHTING INTENSITY IN BATAM PUBLIC LIBRARY READING ROOM

Lighting is one of the most critical components that support human life and other living creature. Poor illumination can impede users’ ability to use a building or space effectively. Library is one of the places that require good light spread so that users feel comfortable when doing activities in it. The Batam City public library is a library located on the fourth floor of the Batam City joint office. This library only has four small apertures in the form of windows for a 13.2 meter x 9 meter area, therefore sunlight cannot distribute evenly. Therefore, one solution to ensure that the library room has a suitable distribution of light is to employ artificial lighting. According to a questionnaire distributed to visitors and staff of Batam City public libraries, 82.4% of respondents believed that lighting in Batam City public libraries was still inadequate. This study used data from direct field observations, interviews with staff and library visitors, questionnaire distribution, and literature review. This research uses quantitative methods by calculating light intensity through the simulation of dialux software. Simulations were carried out in the catalogue room and the library reading room. Changing the colour of the wall surface, the furniture arrangement and the lights' placement are some experiments done to provide a good distribution of light. This study aims to evaluate and obtain lighting simulation results for the public library in Batam City that adheres to the SNI standard (300 LUX).

A fundus image enhancer based on illumination-guided attention and optic disc perception GAN

High-quality fundus images are essential for ophthalmologists in clinical diagnosis of eye diseases. For reasons such as the acquisition process and the retinal disease, most fundus images may suffer poor illumination, blur, and low contrast. In order to obtain high-quality fundus images, a novel network model for fundus image enhancement is constructed in this paper. The model introduces a generated adversarial network (GAN) with illumination-guided attention and optic disc perception. A U-Net with illumination-guided attention is used as the generator, and a global discriminator and a local discriminator with optic disc segmentation are used in the learning process. The model can improve the exposure level of overexposed and underexposed fundus images and also stretch the contrast. Experiments are performed on multiple fundus image datasets taken by the handheld device and high-end device to verify the robustness of the proposed illumination-guided attention and optic disc perception GAN (IGAODPGAN). The retinal structure and pathological characteristics are highlighted significantly after enhancement. Our method is superior to other state-of-the-art algorithms in terms of both subjective and objective assessment, which can provide an efficient solution for fundus image enhancement, especially for the non-uniform illuminated fundus images.

Enhance Mobile Phone Digital Camera Images Using Band Reject Filter and Discrete Fourier Transform

These days, We know that mobile phones with camera are very common to used by most of the people because of their ease of use and carry . So I figured I need for mobile phone images processing. These images are often subjected to a variety of types of distortion, such as noise, poor illumination, and blur due to transmission , image acquisition and mobile movement during capturing scene. This Degradation leads to produce distorted images are not suitable for viewing or application. To overcome this degradation, this paper presents image improvement in mobile phone by distortion removal using Band Reject Filter and DFT to obtain on the best clear image. Periodic noise is sinusoidal of several of certain frequencies and periodic in nature. It looks identical to the bars that the image is covered. The proposed method is subjected to eliminate this noise from a set of images using Band Reject Filter using discrete Fourier transform that attenuate and replace a number of degradation bands. Then, computes the peak signal to noise ratio metric for each band. After that, select the largest value of each band for the purpose the making one clear image. Finally, the bands that carry value large are merged to produce a one image. The final image is converted to the spatial domain using an inverse Fourier transform. The experimental outcomes indicate that the proposed method improves the degraded image by a greater percentage without using the better value for metric and makes image more clear and informative compare with the input image.

Multi-modal pedestrian detection with misalignment based on modal-wise regression and multi-modal IoU

Multi-modal pedestrian detection, which integrates visible and thermal sensors, has been developed to overcome many limitations of visible-modal pedestrian detection, such as poor illumination, cluttered background, and occlusion. By adopting the combination of multiple modalities, we can efficiently detect pedestrians even with poor visibility. Nevertheless, the critical assumption of multi-modal pedestrian detection is that multi-modal images are perfectly aligned. In general, however, this assumption often becomes invalid in real-world situations. Viewpoints of the different modal sensors are usually different. Then, the positions of pedestrians on the different modal images have disparities. We proposed a multi-modal faster-RCNN specifically designed to handle misalignment between two modalities. The faster-RCNN consists of a region proposal network (RPN) and a detector. We introduce position regressors for both modalities in the RPN and the detector. Intersection over union (IoU) is one of the useful metrics for object detection but is defined only for a single-modal image. We extend it into multi-modal IoU to evaluate the preciseness of both modalities. Our experimental results with the proposed evaluation metrics demonstrate that the proposed method has comparable performance with state-of-the-art methods and outperforms them for data with significant misalignment.

FACE RECOGNITION SYSTEM FOR AUTOMATIC DOOR ACCESS CONTROL

A face recognition system for automatic door access control has been developed in this work with a view to providing a relatively more robust and foolproof access control which can provide better security and reduce human errors inherent in other conventional methods. The system was designed with machine learning and artificial intelligence to capture faces, train faces with machine mode, and run trained faces to grant access to the user. The system uses the RaspberryPi module, camera module, servo motor and the GSM module which were all incorporated into the fabricated building to make up the prototype developed to provide access control by means of facial biometrics. In order to grant access to registered users, various photos of the users were taken in different positions and expressions with proper illumination. The user’s face is been captured by the camera module and saved in the database with the help of Raspberry Pi Module. Good lighting condition and other favorable conditions helps the camera module to recognize faces and sends signal to the Raspberry Pi which processes these images and opens the door with the help of the servo motor. The developed prototype was used to train fifty (50) users. It granted access to all fifty (50) users when there was proper illumination and pose but five (5) and nine (9) users respectively were denied access due to challenges of poor illumination and pose variation.

A two-stage framework for optical coherence tomography angiography image quality improvement.

Optical Coherence Tomography Angiography (OCTA) is a new non-invasive imaging modality that gains increasing popularity for the observation of the microvasculatures in the retina and the conjunctiva, assisting clinical diagnosis and treatment planning. However, poor imaging quality, such as stripe artifacts and low contrast, is common in the acquired OCTA and in particular Anterior Segment OCTA (AS-OCTA) due to eye microtremor and poor illumination conditions. These issues lead to incomplete vasculature maps that in turn makes it hard to make accurate interpretation and subsequent diagnosis. In this work, we propose a two-stage framework that comprises a de-striping stage and a re-enhancing stage, with aims to remove stripe noise and to enhance blood vessel structure from the background. We introduce a new de-striping objective function in a Stripe Removal Net (SR-Net) to suppress the stripe noise in the original image. The vasculatures in acquired AS-OCTA images usually exhibit poor contrast, so we use a Perceptual Structure Generative Adversarial Network (PS-GAN) to enhance the de-striped AS-OCTA image in the re-enhancing stage, which combined cyclic perceptual loss with structure loss to achieve further image quality improvement. To evaluate the effectiveness of the proposed method, we apply the proposed framework to two synthetic OCTA datasets and a real AS-OCTA dataset. Our results show that the proposed framework yields a promising enhancement performance, which enables both conventional and deep learning-based vessel segmentation methods to produce improved results after enhancement of both retina and AS-OCTA modalities.

Automatic Detection of Corrosion in Large-Scale Industrial Buildings Based on Artificial Intelligence and Unmanned Aerial Vehicles

In recent years, Artificial Intelligence (AI) provided essential tools to enhance the productivity of activities related to civil engineering, particularly in design, construction, and maintenance. In this framework, the present work proposes a novel AI computer vision methodology for automatically identifying the corrosion phenomenon on roofing systems of large-scale industrial buildings. The proposed method can be incorporated into computational packages for easier integration by the industry to enhance the inspection activities’ performance. For this purpose, a dedicated image database with more than 8k high-resolution aerial images was developed for supervised training. An Unmanned Aerial Vehicle (UAV) was used to acquire remote georeferenced images safely and efficiently. The corrosion anomalies were manually annotated using a segmentation strategy summing up 18,381 instances. These anomalies were identified through instance segmentation using the Mask based Region-Convolution Neural Network (Mask R-CNN) framework adjusted to the created dataset. Some adjustments were performed to enhance the performance of the classification model, particularly defining an adequate input image size, data augmentation strategy, Intersection over a Union (IoU) threshold during training, and type of backbone network. The inferences show promising results, with correct detections even under complex backgrounds, poor illumination conditions, and instances of significantly reduced dimensions. Furthermore, in scenarios without a roofing system, the model proved reliable, not producing any false positive occurrences. The best model achieved metrics’ values equal to 65.1% for the bounding box detection Average Precision (AP) and 59.2% for the mask AP, considering an IoU of 50%. Regarding classification metrics, the precision and recall were equal to 85.8% and 84.0%, respectively. The developed methodology proved to be extremely valuable for guiding infrastructure managers in taking physically informed decisions based on the real assets condition.

A depth information aided real-time instance segmentation method for space task scenarios under CPU platform

Visual instance segmentation ability is one of the effective means to promote the autonomy and intelligence of space agents. However, due to the limited airborne computing capacity, current methods are difficult to deploy on space agents, because these methods are developed based on GPU. To this end, this paper proposes a novel framework of instance segmentation by introducing depth information and combining traditional computer vision techniques with an object detection method. This framework provides a new idea for the implementation of instance segmentation. The experiment results show that the proposed method achieves a real-time performance under a common laptop CPU platform. In addition, thanks to the introduction of depth information, the proposed method can obtain better segmentation results compared to Mask R–CNN and SOLOv2 in complex scenes (poor illumination and occlusion). Finally, because the semantic information is obtained by the object detection method in this paper, the model training adopts a weakly supervised manner from bounding-box annotations, which can reduce various costs of data labeling to a certain extent.

Ayur-PlantNet: An unbiased light weight deep convolutional neural network for Indian Ayurvedic plant species classification

Plant species classification using photo-based computer vision is challenging due to some crucial factors such as poor illumination, overlapping leaves, occlusions from cross-domain scene elements, inter-class similarities, intra-class variations, and blurred scene elements. The current state of art methods concerning photo-based plant species classification fails to deal with the challenges described above. In the proposed method, an unbiased lightweight deep convolutional neural network named Ayur-PlantNet is proposed to classify forty Ayurvedic plant species. The model built from scratch is trained and tested on 6000 samples with segmented plant regions. From a comparative study with pre-trained models; Resnet34, Resnet50, VGG16, MobileNetV3_Large, EfficientNetwork_B4, and Densenet121. It is noticed that the Ayur-PlantNet can produce an accuracy of 92.27% with reduced trainable parameters and computational complexity than pre-trained models. The experimental results prove that Ayur-PlantNet architecture is dominant compared to other deep learning models.

SFAF-MA: Spatial Feature Aggregation and Fusion With Modality Adaptation for RGB-Thermal Semantic Segmentation

The fusion of RGB and thermal images has profound implications for the semantic segmentation of challenging urban scenes, such as those with poor illumination. Nevertheless, existing RGB-T fusion networks pay less attention to modality differences; i.e., RGB and thermal images are commonly fused with fixed weights. In addition, spatial context details are lost during regular extraction operations, inevitably leading to imprecise object segmentation. To improve the segmentation accuracy, a novel network named spatial feature aggregation and fusion with modality adaptation (SFAF-MA) is proposed in this paper. The modality difference adaptive fusion (MDAF) module is introduced to adaptively fuse RGB and thermal images with corresponding weights generated from an attention mechanism. In addition, the spatial semantic fusion (SSF) module is designed to tap into more information by capturing multiscale perceptive fields with dilated convolutions of different rates, and aggregate shallower-level features with rich visual information and deeper-level features with strong semantics. Compared with existing methods on the public MFNet dataset and PST900 dataset, the proposed network significantly improves the segmentation effectiveness. The code is available at https://github.com/hexunjie/SFAF-MA.

A One-Stage Domain Adaptation Network With Image Alignment for Unsupervised Nighttime Semantic Segmentation.

In this paper, we tackle the problem of semantic segmentation for nighttime images that plays an equally important role as that for daytime images in autonomous driving, but is also much more challenging due to very poor illuminations and scarce annotated datasets. It can be treated as an unsupervised domain adaptation (UDA) problem, i.e., applying other labeled dataset taken in the daytime to guide the network training meanwhile reducing the domain shift, so that the trained model can generalize well to the desired domain of nighttime images. However, current general-purpose UDA approaches are insufficient to address the significant appearance difference between the day and night domains. To overcome such a large domain gap, we propose a novel domain adaptation network "DANIA" for nighttime semantic image segmentation by leveraging a labeled daytime dataset (the source domain) and an unlabeled dataset that contains coarsely aligned day-night image pairs (the target daytime and nighttime domains). These three domains are used to perform a multi-target adaptation via adversarial training in the network. Specifically, for the unlabeled day-night image pairs, we use the pixel-level predictions of static object categories on a daytime image as a pseudo supervision to segment its counterpart nighttime image. We also include a step of image alignment to relieve the inaccuracy caused by the misalignment between day-night image pairs by estimating a flow to refine the pseudo supervision produced by daytime images. Finally, a re-weighting strategy is applied to further improve the predictions, especially boosting the prediction accuracy of small objects. The proposed DANIA is a one-stage adaptation framework for nighttime semantic segmentation, which does not train additional day-night image transfer models as a separate pre-processing stage. Extensive experiments on Dark Zurich and Nighttime Driving datasets show that our DANIA achieves state-of-the-art performance for nighttime semantic segmentation.

Brightness and Contrast Enhancement Method for Color Images Via Pairing Adaptive Gamma Correction and Histogram Equalization

For enhanced adaptability to poor light enhancement whilst achieving high image contrast, a new method for color image correction based on the advantages of non-linear function in grey transformation and histogram equalization techniques is proposed in this work. Firstly, the original red, green and blue (RGB) image is converted into the HSV color space, and the V channel is used for enhancement. An adaptive gamma generator is proposed to adaptively calculate gamma parameters in accordance with dark, medium, or bright image conditions. The computed gamma parameters are used to propose a cumulative distribution function that produces an optimized curve for illumination values. Next, a second modified equalization is performed to evenly correct the offset of the illumination curve values on the basis of the equal probability of the available values only. Finally, the processed V channel replaces the original V channel, and the new HSV model returns to the RGB color space. Experiments show that the proposed method can significantly improve the low contrast and poor illumination of the color image whilst preserving the color and details of the original image. Results from benchmark data sets and measurements indicate that the proposed method outperforms other state-of-the-art methods.