Illumination Changes Research Articles

Enhancing vehicle re-identification via synthetic training datasets and re-ranking based on video-clips information

Vehicle re-identification (ReID) aims to find a specific vehicle identity across multiple non-overlapping cameras. The main challenge of this task is the large intra-class and small inter-class variability of vehicles appearance, sometimes related with large viewpoint variations, illumination changes or different camera resolutions. To tackle these problems, we proposed a vehicle ReID system based on ensembling deep learning features and adding different post-processing techniques. In this paper, we improve that proposal by: incorporating large-scale synthetic datasets in the training step; performing an exhaustive ablation study showing and analyzing the influence of synthetic content in ReID datasets, in particular CityFlow-ReID and VeRi-776; and extending post-processing by including different approaches to the use of gallery video-clips of the target vehicles in the re-ranking step. Additionally, we present an evaluation framework in order to evaluate CityFlow-ReID: as this dataset has not public ground truth annotations, AI City Challenge provided an on-line evaluation service which is no more available; our evaluation framework allows researchers to keep on evaluating the performance of their systems in the CityFlow-ReID dataset.

Bioconvection pattern of Euglena under periodical illumination.

Microorganisms respond to environmental conditions and often spontaneously form highly ordered convection patterns. This mechanism has been well-studied from the viewpoint of self-organization. However, environmental conditions in nature are usually dynamic. Naturally, biological systems respond to temporal changes in environmental condition. To elucidate the response mechanisms in such a dynamic environment, we observed the bioconvection pattern of Euglena under periodical changes in illumination. It is known that Euglena shows localized bioconvection patterns under constant homogeneous illumination from the bottom. Periodical changes in light intensity induced two different types of spatiotemporal patterns: alternation of formation and decomposition over a long period and complicated transition of pattern over a short period. Our observations suggest that pattern formation in a periodically changing environment is of fundamental importance to the behavior of biological systems.

Object tracking in surveillance video based on Siamese network considering geographical spatiotemporal constraints

AbstractIdentifying and tracking objects in surveillance videos is an important means of mining information during surveillance. Currently, most object‐tracking methods rely only on image features, which cannot accurately express the motion of the tracked object in real geographical scenes and are easily influenced by occlusion and surrounding objects having similar features. However, tracked objects, such as pedestrians and vehicles, usually move in geographical space with fixed patterns of motion, and the motion in a short time is constrained by geographical space and time, the motion trajectory is predictable, and the range of motion is limited. Therefore, based on the SiamFC object tracking framework, this study introduces geographical spatiotemporal constraints into the tracking framework and proposes the GeoSiamFC method. The objective of this is to: (1) construct the mapping relationship between geographical space and image space to solve the problem that the pixel movement within the image after perspective imaging cannot accurately express the motion of the tracked object in a real geographical scene; (2) add candidate search areas according to the predicted trajectory location to correct the tracking errors caused by the occlusion of the object; and (3) focus on the search for the range of motion of the mapped pixel within the image space according to the limited geographical range of motion of the tracked objects in a certain time to reduce the interference of similar objects within the search area. In this study, separate experiments were conducted on a common test dataset using multiple methods to deal with challenges such as occlusion and illumination changes. In addition, a robust test dataset with noise addition and luminance adjustment based on the common test dataset was used. The results show that GeoSiamFC outperforms other object‐tracking methods in the common test dataset with a precision score of 0.995 and a success score of 0.756 compared with most other object‐tracking algorithms under the base condition of using only shallow networks. Moreover, GeoSiamFC maintained the highest precision score (0.970) and high success score (0.734) in the more challenging robust test dataset as well. The tracking speed of 59 frames per second far exceeds the real‐time requirement of 25 FPS. Geographical spatiotemporal constraints were considered to improve tracker performance while providing real‐time feedback on the motion trajectory of the target in geographical space. Thus, the proposed method is suitable for real‐time tracking of the motion trajectory of a target in real geographical scenes in various surveillance videos.

Condition-invariant and compact visual place description by convolutional autoencoder

AbstractVisual place recognition (VPR) in condition-varying environments is still an open problem. Popular solutions are convolutional neural network (CNN)-based image descriptors, which have been shown to outperform traditional image descriptors based on hand-crafted visual features. However, there are two drawbacks of current CNN-based descriptors: (a) their high dimension and (b) lack of generalization, leading to low efficiency and poor performance in real robotic applications. In this paper, we propose to use a convolutional autoencoder (CAE) to tackle this problem. We employ a high-level layer of a pre-trained CNN to generate features and train a CAE to map the features to a low-dimensional space to improve the condition invariance property of the descriptor and reduce its dimension at the same time. We verify our method in four challenging real-world datasets involving significant illumination changes, and our method is shown to be superior to the state-of-the-art. The code of our work is publicly available at https://github.com/MedlarTea/CAE-VPR.

An LDOP approach for face identification under unconstrained scenarios

ABSTRACT In unconstrained environments, it encounters a number of challenges when considering handcrafted features for face recognition, including changes in pose, illumination and facial expression and plastic surgery variations, look-alike faces and selfie images. As majority of the published works on local descriptors are based on the relationship between the centre pixel and neighbourhood pixels at different radial widths, by including the relationship between different neighbourhood pixels of a target pixel with it can facilitate the decoupling of the discriminatory features further. To exploit such relationships and to mitigate the challenges, the proposed study reports a novel local descriptor called Local Directional Octa Pattern (LDOP) that makes use of derivative operation and three well-known metrics, namely, histogram intersection, K-nearest neighbour and Euclidean distance for face identification. Rigorous experiments have been conducted on six benchmark face databases, namely, Extended Yale Face B, Labeled Faces in the Wild, Plastic Surgery, Look-alike, UMDAA-02-FD and ORL, confirming the superiority of the proposed local descriptor over state-of-the-art descriptors with accuracies of 97.37%, 49.09%, 75.47%, 31.2% and 93.31% at Rank–10 and 100% at Rank-5 determined on the databases, respectively.

Color constancy for daylight illumination changes in anomalous trichromats and dichromats.

Color constancy is the perceptual stability of surface colors under temporal changes in the illumination spectrum. The illumination discrimination task (IDT) reveals worse discrimination for "bluer" illumination changes in normal-trichromatic observers (changes towards cooler color temperatures on the daylight chromaticity locus), indicating greater stability of scene colors or better color constancy, compared with illumination changes in other chromatic directions. Here, we compare the performance of individuals with X-linked color-vision deficiencies (CVDs) to normal trichromats on the IDT performed in an immersive setting with a real scene illuminated by spectrally tunable LED lamps. We determine discrimination thresholds for illumination changes relative to a reference illumination (D65) in four chromatic directions, roughly parallel and orthogonal to the daylight locus. We find, using both a standard CIELUV metric and a cone-contrast metric tailored to distinct CVD types, that discrimination thresholds for daylight changes do not differ between normal trichromats and CVD types, including dichromats and anomalous trichromats, but thresholds for atypical illuminations do differ. This result extends a previous report of illumination discrimination ability in dichromats for simulated daylight changes in images. In addition, using the cone-contrast metric to compare thresholds for bluer and yellower daylight changes with those for unnatural redder and greener changes, we suggest that reduced sensitivity to daylight changes is weakly preserved in X-linked CVDs.

Implementation of 1D convolutional neural network for improvement remote photoplethysmography measurement

<span lang="EN-US">Remote photoplethysmography (rPPG) for non-contact heart rate measurement has been widely developed and shows good development. However, motion artifact due to changes in illumination and subject movement is still the main problem. Especially when measurements are taken in real conditions. In these conditions, it will be vulnerable to rPPG signal readings with poor signal quality. So, in this paper, it is proposed to classify the signal quality using one dimensional convolutional neural network (1D CNN). The classification is carried out based on the extraction of the temporal features of the rPPG signal that has been obtained from the plane orthogonal to skin algorithm and the magnitude of the subject's movement when measured. The classification results are entered into a compensated network if the signal obtained shows moderate quality. The compensated network will provide a more accurate estimate of hr value. The test was carried out using a dataset of 10 subjects, each measured with 3 different types of illumination. In the experiments conducted, the system's performance showed an improvement compared to the POS algorithm alone. The experiment found that the mean absolute error measurement was 2.78, and the mean error was relative at 3.67%.</span>

Siamese PointNet: 3D Head Pose Estimation with Local Feature Descriptor

Head pose estimation is an important part of the field of face analysis technology. It can be applied to driver attention monitoring, passenger monitoring, effective information screening, etc. However, illumination changes and partial occlusion interfere with the task, and due to the non-stationary characteristic of the head pose change process, normal regression networks are unable to achieve very accurate results on large-scale synthetic training data. To address the above problems, a Siamese network based on 3D point clouds was proposed, which adopts a share weight network with similar pose samples to constrain the regression process of the pose’s angles; meanwhile, a local feature descriptor was introduced to describe the local geometric features of the objects. In order to verify the performance of our method, we conducted experiments on two public datasets: the Biwi Kinect Head Pose dataset and Pandora. The results show that compared with the latest methods, our standard deviation was reduced by 0.4, and the mean error was reduced by 0.1; meanwhile, our network also maintained a good real-time performance.

IMAGE SEGMENTATION WITH A CONVOLUTIONAL NEURAL NETWORK WITHOUT POOLING LAYERS IN DERMATOLOGICAL DISEASE DIAGNOSTICS SYSTEMS

Context. The problem of automating of the segmentation of spectral-statistical texture images is considered. The object of research is image processing in dermatological disease diagnostic systems.
 Objective. The aim of the research is to improve the segmentation performance of color images of psoriasis lesions by elaboration of a deep learning convolutional neural network without pooling layers.
 Method. The convolutional neural network is proposed to process a three-channel psoriasis image with a specified size. The initial color images were scaled to the specified size and then inputed on the neural network. The architecture of the proposed neural network consists of four convolutional layers with batch normalization layers and ReLU activation function. Feature maps from the output of these layers were inputted to the 1*1 convolutional layer with the Softmax activation function. The resulting feature maps were inputted to the image pixel classification layer. When segmenting images, convolutional and pooling layers extract the features of image fragments, and fully connected layers classify the resulting feature vectors, forming a partition of the image into homogeneous segments. The segmentation features are evaluated as a result of network training using ground-truth images which segmented by an expert. Such features are robust to noise and distortion in images. The combination of segmentation results at different scales is determined by the network architecture. Pooling layers were not included in the architecture of the proposed convolutional neural network since they reduce the size of feature maps compared to the size of the original image and can decrease the segmentation performance of small psoriasis lesions and psoriasis lesions of complex shape.
 Results. The proposed convolutional neural network has been implemented in software and researched for solving the problem of psoriasis images segmentation.
 Conclusions. The use of the proposed convolutional neural network made it possible to enhance the segmentation performance of plaque and guttate psoriasis images, especially at the edges of the lesions. Prospects for further research are to study the performance of the proposed CNN then abrupt changes in color and illumination, blurring, as well as the complex background areas are present on dermatological images, for example, containing clothes or fragments of the interior. It is advisable to use the proposed CNN in other problems of color image processing to segment statistical or spectral-statistical texture regions on a uniform or textured background.

The effect of surface gloss on categorical color constancy in real scenes.

Categorical color constancy has been widely investigated and found to be very robust. As one of object material properties, the surface gloss was found to barely contribute to color constancy in a natural viewing condition. In this study, the effect of surface gloss on categorical color constancy was investigated by asking eight observers to categorize 208 Munsell matte surfaces and 260 Munsell glossy surfaces under D65, F, and TL84 illuminants in a viewing chamber with a uniform gray background. A color constancy index based on the centroid shift of the color category was used to evaluate color constancy degree of each color category across illumination changes from D65 to F or TL84 illuminant. The result showed that both matte and glossy surfaces showed almost perfect color constancy on all color categories under F and TL84 illuminants, and there was no significant difference between them. This result suggests that surface gloss has little effect on categorical color constancy in a uniform gray background where the local surround cue was present, which is consistent with the previous findings.

Invariant categorical color regions across illuminant change coincide with focal colors.

Are there regions in a color space where color categories are invariant across illuminant changes? If so, what characteristics make them more stable than other regions? To address these questions, we asked observers to give a color name to 424 colored surfaces, presented one at a time, under various chromatic illuminants. Results showed a high degree of categorical color constancy, especially under illuminants that occur in the natural environment. It was also shown that surfaces selected as a focal color (the best example of a color category) are more resistant to illuminant change than nonfocal color samples. We additionally ran an asymmetric color matching experiment to quantify the shift of color appearance induced by illuminant changes using surfaces that were all named gray, thereby disentangling the appearance-based color constancy from the categorical color constancy (which are often confounded). Results suggested that the appearance of color samples largely shifted owing to illuminant changes, even though all samples were named gray; showing that the constancy of a color category is substantially more robust than the constancy of color appearance.

Vision-based real-time structural vibration measurement through deep-learning-based detection and tracking methods

Structural vibration measurement is crucial in structural health monitoring and structural laboratory tests. Traditional contact type sensors are usually required to be attached to the test specimens, which may be difficult to install, and may affect the structural properties and response. Non-contact type wireless sensors are usually expensive and require specialized workers to install and operate. In recent years, vision-based tracking methods for structural vibration measurement have gained increasing interests due to their high accuracy, non-contact feature and low cost. However, traditional vision-based tracking algorithms are susceptible to external environmental conditions such as illumination and background noise. In this paper, two real-time methods, YOLOv3-tiny and YOLOv3-tiny-KLT, are proposed to track structural motions. In the first method, YOLOv3-tiny is established based on the YOLOv3 architecture to localize customized markers where structural displacements are directly determined from the bounding boxes generated. The second method, YOLOv3-tiny-KLT, is a more advanced method which combines the YOLOv3-tiny detector and the traditional KLT tracking algorithm. The pretrained YOLOv3-tiny is deployed to localize the targets automatically, which will then be tracked by Kanade‐Lucas‐Tomasi algorithm. YOLOv3-tiny is intended to provide baseline vibration measurement when the KLT tracking gets lost. The proposed methods were implemented for the videos of shake table tests on a two-storey steel structure. Parametric studies were conducted for the YOLOv3-tiny-KLT method to examine its sensitivity to the tracking parameters. The results show that the proposed method is capable of achieving real-time speed and high accuracy, when compared with the traditional displacement sensors including linear variable differential transducer (LVDT) and String Pots. It is also found that the combined YOLOv3-tiny-KLT approach achieves higher accuracy than YOLOv3-tiny only method, and higher robustness than KLT only method against illumination changes and background noise.

Development of an automated system for the soldering of USB cables

Large demand for soldering multi-wire cables (e.g., USB cables, earphone cables, etc.) is reflected in the rapidly expanding 3C industry (i.e., Computer, Communication and Consumer electronics). Currently, several pre-processing steps need to be first carried out: separating → sorting → pressing . These steps are challenging for the robot, mainly because of the deformable nature, small size, and couplings between multiple wires. As such, existing solutions are entirely manual. In this paper, a new automated system for the soldering of multi-wire cable is proposed, in which the whole procedure, including all the pre-processing steps, is fully automated. The key novelty is the development of a series of new mechanical modules that are able to disentangle multiple wires then manipulate them into the structured configuration, laying the foundation for the subsequent soldering operation. The transition between multiple modules is automatically achieved using a deep vision network to guarantee high accuracy (i.e., < 1 mm ) under different levels of illumination and/or cluttered backgrounds. Moreover, an optimization scheme has been proposed for wire sorting to shorten the operation time and improve mass production efficiency. Experimental results show that the speed of manufacturing is around 112 pics/hr and the success rate for each module is above 93%, thereby proving the feasibility of the developed robot. • The development of a fully-automatic system for the soldering of multi-wire cable. • The construction of a deep vision network for accurate detection of wires in the presence of changes in illumination and cluttered background. • The design of an optimization scheme capable of shortening the time of wire sorting and improving mass production efficiency.

Pedestrian Re-Identification Algorithm Based on Multivariate Manifold Metric-Anti-Noise Manifold Space Learning

Deep learning is widely used in pedestrian re-identification, but pedestrian images are susceptible to changes in illumination, environmental noise, and other factors. These factors have affected the in-depth promotion of this type of method. First, the existing models cannot effectively extract the structural information of low-light pedestrian samples. Second, environmental noise information will cause the model to fail to extract stable and reliable feature information. In this paper, to solve the problems as mentioned above, we propose a multi-group manifold metric learning model. Our method can obtain manifold conditions under different lighting conditions. It can retain the inherent structure of low-dimensional spatial data in high-dimensional space, and realize the extraction of features of pedestrians in low-light conditions. In addition, we proposed an anti-noise manifold space learning method and designed a manifold module network structure. The manifold modules are stacked layer by layer to construct a manifold convolutional neural network further. Our method can extract robust pedestrian image features and reduce noise disturbance to the model. We use noise and low-light images as the main experimental objects. The experimental results show that our proposed method can achieve the most advanced performance in multiple ReID data sets.

Cluster-based segmentation for tobacco plant detection and classification

Tobacco is one of the major economical crops in the agriculture sector. It is essential to detect tobacco plants using unmanned aerial vehicle (UAV) images for improved crop yield and plays an important role in the early treatment of tobacco plants. The proposed research work is carried out in three phases: In the first phase, we collect images from UAV’s and apply the French Commision Internationale de l'eclairage (CIE) L*a*b colour space model as pre-processing operations and segmentation. And then two prominent motion descriptors namely histogram of flow (HOF) and motion boundary histogram (MBH) are combined with the optimal histogram of oriented gradients (HOG) descriptor for exploring optimal motion trajectory and spatial measurements. And finally, the spatial variations with respect to the scale and illumination changes are incorporated using the optimal HOG descriptor. Here both dense motion patterns and HOG are refined using hierarchical feature selection using principal component analysis (PCA). The proposed model is trained and evaluated on different tobacco UAV image datasets and done a comparative analysis of different machine learning (ML) algorithms. The proposed model achieves good performance with 95% accuracy and 92% of sensitivity.

Indoor Visual Re-Localization for Long-Term Autonomous Robots Based on Object-Level Features and Semantic Relationships

Visual re-localization has become one of the key technologies for long-term autonomous robots. Existing methods, mostly focusing on addressing day-night, weather, and seasonal changes, are not applicable in indoor scenarios. At the same time, the layouts of objects in indoor scenes are highly dynamic over time due to human interactions with the environment, which makes indoor re-localization challenging. This letter presents a novel indoor visual re-localization method for long-term autonomous robots. First, a scene graph model is proposed, incorporating object-level features and semantic relationships, which overcomes the influence of dynamic objects by understanding the interactions among objects. Then, a visual re-localization method is developed based on the proposed scene graph model. It adopts graph matching technologies to incorporate pairwise object interactions as important features for re-localization, and designs a feature reweighting strategy to further reduce the impact of outliers in dynamic scenes. The proposed re-localization method has been verified in both photorealistic simulation environments and real-world scenarios. The results show that our approach exhibits higher robustness to diverse object changes and performs comparably to the state-of-the-art methods when illumination changes occur.

Body Correlation Network for Person Re-Identifications

&lt;p&gt;Visual information accounts for approximately 80 to 85 percent of the information available daily in modern cities. As such, person re-identification, an instance-level image retrieval task, has become an important research topic in computer vision, machine learning, and other fields in recent years. Traditional person re-identification methods based on convolutional neural networks only extract the global feature information of people. Thus, when external factors such as changes in occlusion and illumination disturb people, the recognition performance of these methods substantially decreases. We therefore develop body correlation network (BC-Net), which takes full advantage of images of body parts and the correlations between them. Specifically, BC-Net uses body part feature information and correlation feature information as nodes and edges, respectively, and then uses graph convolutions to learn the overall topology of people. To improve use of crucial feature information, we also design a unique method of propagation between nodes and edges. We conduct extensive comparative experiments on the Market-1501 and DukeMTMC-reID datasets, and the results demonstrate that BC-Net outperforms other state-of-the-art techniques.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

Biologically meaningful moonlight measures and their application in ecological research

Light availability is one of the key drivers of animal activity, and moonlight is the brightest source of natural light at night. Moon phase is commonly used but, while convenient, it can be a poor proxy for lunar illumination on the ground. While the moon phase remains effectively constant within a night, actual moonlight intensity is affected by multiple factors such as disc brightness, position of the moon, distance to the moon, angle of incidence, and cloud cover. A moonlight illumination model is presented for any given time and location, which is significantly better at predicting lunar illumination than moon phase. The model explains up to 92.2% of the variation in illumination levels with a residual standard error of 1.4%, compared to 60% explained by moon phase with a residual standard error of 22.6%. Importantly, the model not only predicts changes in mean illumination between nights but also within each night, providing greater temporal resolution of illumination estimates. An R package moonlit facilitating moonlight illumination modelling is also presented. Using a case study, it is shown that modelled moonlight intensity can be a better predictor of animal activity than moon phase. More importantly, complex patterns of activity are shown where animals focus their activity around certain illumination levels. This relationship could not be identified using moon phase alone. The model can be universally applied to a wide range of ecological and behavioural research, including existing datasets, allowing a better understanding of lunar illumination as an ecological resource.Significance statementMoon phase is often used to represent lunar illumination as an environmental niche, but it is a poor proxy for actual moonlight intensity on the ground. A model is therefore proposed to estimate lunar illumination for any given place and time. The model is shown to provide a significantly better prediction of empirically measured lunar illumination than moon phase. Importantly, it also has much higher temporal resolutions, allowing to not only detect selectiveness for light levels between nights but also within each night, which is not achievable with moon phase alone. This offers unprecedented opportunities to study complex activity patterns of nocturnal species using any time-stamped data (GPS trackers, camera traps, song meters, etc.). It can also be applied to historical datasets, as well as facilitate future research planning in a wide range of ecological and behavioural studies.

Global Perception-Based Robust Parking Space Detection Using a Low-Cost Camera

In the city, parking space detection can help car owners save time in finding parking spaces, and it can also provide help for autonomous parking. However, existing parking space detection methods based on embedded geomagnetic sensors are usually costly and complex to deploy in a large area. To reduce costs while obtaining sufficient performance, it is desirable to detecting parking spaces with cameras. We adopt a low-cost visual method to solve the above problems, using a convolutional neural network to achieve the parking space classification. We propose a general module called Global Perceptual Feature Extractor (GPFE) based on transformer to achieve global attention, which can be easily combined with other classification networks to improve the accuracy and robustness of the entire system, and alleviate the impact of illumination changing and other issues. We verify the proposed method with other methods on three public parking space detection datasets (CNREXT, PKLot and ACPDS). The results show that the accuracy of the network combined with the GPFE module has been generally improved, especially on scenes with low and high illumination, which verifies the robustness and performance of the module.

Io’s Optical Aurorae in Jupiter’s Shadow

Decline and recovery timescales surrounding eclipse are indicative of the controlling physical processes in Io’s atmosphere. Recent studies have established that the majority of Io’s molecular atmosphere, SO2 and SO, condenses during its passage through Jupiter’s shadow. The eclipse response of Io’s atomic atmosphere is less certain, having been characterized solely by ultraviolet aurorae. Here we explore the response of optical aurorae for the first time. We find oxygen to be indifferent to the changing illumination, with [O i] brightness merely tracking the plasma density at Io’s position in the torus. In shadow, line ratios confirm sparse SO2 coverage relative to O, since their collisions would otherwise quench the emission. Io’s sodium aurora mostly disappears in eclipse and e-folding timescales, for decline and recovery differ sharply: ∼10 minutes at ingress and nearly 2 hr at egress. Only ion chemistry can produce such a disparity; Io’s molecular ionosphere is weaker at egress due to rapid recombination. Interruption of a NaCl+ photochemical pathway best explains Na behavior surrounding eclipse, implying that the role of electron impact ionization is minor relative to photons. Auroral emission is also evident from potassium, confirming K as the major source of far red emissions seen with spacecraft imaging at Jupiter. In all cases, direct electron impact on atomic gas is sufficient to explain the brightness without invoking significant dissociative excitation of molecules. Surprisingly, the nonresponse of O and rapid depletion of Na is opposite the temporal behavior of their SO2 and NaCl parent molecules during Io’s eclipse phase.