Natural Illumination Conditions Research Articles

Flexible retinomorphic vision sensors with scotopic and photopic adaptation for a fully flexible neuromorphic machine vision system

AbstractBioinspired neuromorphic machine vision system (NMVS) that integrates retinomorphic sensing and neuromorphic computing into one monolithic system is regarded as the most promising architecture for visual perception. However, the large intensity range of natural lights and complex illumination conditions in actual scenarios always require the NMVS to dynamically adjust its sensitivity according to the environmental conditions, just like the visual adaptation function of the human retina. Although some opto‐sensors with scotopic or photopic adaption have been developed, NMVSs, especially fully flexible NMVSs, with both scotopic and photopic adaptation functions are rarely reported. Here we propose an ion‐modulation strategy to dynamically adjust the photosensitivity and time‐varying activation/inhibition characteristics depending on the illumination conditions, and develop a flexible ion‐modulated phototransistor array based on MoS2/graphdiyne heterostructure, which can execute both retinomorphic sensing and neuromorphic computing. By controlling the intercalated Li+ ions in graphdiyne, both scotopic and photopic adaptation functions are demonstrated successfully. A fully flexible NMVS consisting of front‐end retinomorphic vision sensors and a back‐end convolutional neural network is constructed based on the as‐fabricated 28 × 28 device array, demonstrating quite high recognition accuracies for both dim and bright images and robust flexibility. This effort for fully flexible and monolithic NMVS paves the way for its applications in wearable scenarios.

Algal-bacterial shortcut nitrogen removal model with seasonal light variations.

The algal-bacterial shortcut nitrogen removal (ABSNR) process can be used to treat high ammonia strength wastewaters without external aeration. However, prior algal-bacterial SNR studies have been conducted under fixed light/dark periods that were not representative of natural light conditions. In this study, laboratory-scale photo-sequencing batch reactors (PSBRs) were used to treat anaerobic digester sidestream under varying light intensities that mimicked summer and winter conditions in Tampa, FL, USA. A dynamic mathematical model was developed for the ABSNR process, which was calibrated and validated using data sets from the laboratory PSBRs. The model elucidated the dynamics of algal and bacterial biomass growth under natural illumination conditions as well as transformation processes for nitrogen species, oxygen, organic and inorganic carbon. A full-scale PSBR with a 1.2 m depth, a 6-day hydraulic retention time (HRT) and a 10-day solids retention time (SRT) was simulated for treatment of anaerobic digester sidestream. The full-scale PSBR could achieve >90% ammonia removal, significantly reducing the nitrogen load to the mainstream wastewater treatment plant (WWTP). The dynamic simulation showed that ABSNR process can help wastewater treatment facilities meet stringent nitrogen removal standards with low energy inputs.

Deep Shape and SVBRDF Estimation using Smartphone Multi‐lens Imaging

AbstractWe present a deep neural network‐based method that acquires high‐quality shape and spatially varying reflectance of 3D objects using smartphone multi‐lens imaging. Our method acquires two images simultaneously using a zoom lens and a wide angle lens of a smartphone under either natural illumination or phone flash conditions, effectively functioning like a single‐shot method. Unlike traditional multi‐view stereo methods which require sufficient differences in viewpoint and only estimate depth at a certain coarse scale, our method estimates fine‐scale depth by utilising an optical‐flow field extracted from subtle baseline and perspective due to different optics in the two images captured simultaneously. We further guide the SVBRDF estimation using the estimated depth, resulting in superior results compared to existing single‐shot methods.

Changes in the Chemical and Biological Properties of Arable Soils with Different Duration of the Freezing Period

The results of a 2-year vegetation experiment to study the effect of the duration of the frost-free period on the chemical properties and biological activity of agrokashtan soils and agrochernozem are presented. The task of the experiment was to assess changes in soil properties that may occur with further warming of the climate and a reduction in the time of stay of soils in the frozen state. The first variant of the experiment provided for incubation of soils in a frozen state in winter for 160 days, the second – 56 days, in the third variant, the soil was incubated throughout the winter period at a temperature of 0°C. Before winter incubation, the soil was moistened to 60% of FMC, mineral fertilizers and straw were applied. In the spring, spring wheat of the Zlata variety of the selection of the Moscow NIISH “Nemchinovka” was sown in the soil. Throughout the growing season, soil samples were kept in the vegetation pavilion in conditions of natural moisture and illumination. Sampling and measurements of biological properties were carried out in the spring period 10 days after the samples were transferred from the refrigerator to the vegetation pavilion. It was found that, depending on the duration of the freezing period of the studied soils, the most sensitive indicators for temperature changes were microbial biomass (C-LED), basal respiration rate (V-BASRR) of the microbial community and the number of microorganisms (NMO) growing on soil agar and on a rich medium. The soils that were incubated without freezing had the lowest values of C-LED and V-BASRR. The increase in the freezing period of the soil affected the acidity of the soils. In the variants with prolonged freezing in the 2nd year of the study, the pH value increased. For agrochernozem, a decrease in the content of P2O5 was noted as the freezing period increased. For agrokashtan soils, this pattern was less pronounced. In these soils, a tendency to decrease the C : N ratio was revealed with an increase in the time spent in the frozen state. The content of Corg, as well as macro- and microelements in all variants practically did not change during the 2 years of the experiment.

Seeing through Wavy Water–Air Interface: A Restoration Model for Instantaneous Images Distorted by Surface Waves

Imaging through a wavy water–air interface is challenging since light rays are bent by unknown amounts, leading to complex geometric distortions. Considering the restoration of instantaneous distorted images, this paper proposes an image recovery model via structured light projection. The algorithm is composed of two separate parts. In the first part, an algorithm for the determination of the instantaneous shape of the water surface via structured light projection is developed. Then, we synchronously recover the distorted airborne scene image through reverse ray tracing in the second part. The experimental results show that, compared with the state-of-the-art methods, the proposed method not only can overcome the influence of changes in natural illumination conditions for WAI reconstruction, but also can significantly reduce the distortion and achieve better performance.

Image Segmentation of UAV Fruit Tree Canopy in a Natural Illumination Environment

Obtaining canopy area, crown width, position, and other information from UAV aerial images and adjusting spray parameters in real-time according to this information is an important way to achieve precise pesticide application in orchards. However, the natural illumination environment in the orchard makes extracting the fruit tree canopy difficult. Hereto, an effective unsupervised image segmentation method is developed in this paper for fast fruit tree canopy acquisition from UAV images under natural illumination conditions. Firstly, the image is preprocessed using the shadow region luminance compensation method (SRLCM) that is proposed in this paper to reduce the interference of shadow areas. Then, use Naive Bayes to obtain multiple high-quality color features from 10 color models was combined with ensemble clustering to complete image segmentation. The segmentation experiments were performed on the collected apple tree images. The results show that the proposed method’s average precision rate, recall rate, and F1-score are 95.30%, 84.45%, and 89.53%, respectively, and the segmentation quality is significantly better than ordinary K-means and GMM algorithms.

Feasibility of detecting apple scab infections using low-cost sensors and interpreting radiation interactions with scab lesions

ABSTRACT Apple scab is a disease caused by the fungus Venturia inaequalis (Cke.) Wint. which can spread rapidly throughout orchards diminishing tree productivity and causing huge losses in marketable fruit. Efficient orchard reconnaissance and early detection of infections can inform fungicide applications for effective disease control and a range of new low-cost sensors offer a means of imaging orchards as the basis of scab detection. This study evaluates the potential contribution of three imaging devices: a multispectral (VIS-NIR) camera, thermal camera and a 3D sensor, for the detection of scab on young apple plants. In a controlled experiment, apple seedings were infected with scab and disease progression was imaged daily under natural illumination conditions in a glasshouse with minimal image processing. Whilst the thermal and 3D sensors images were deemed unsuitable for scab detection, the high-resolution multispectral imagery was exceptionally effective, with the NIR band (800–1000 nm) permitting the earliest scab detection due to the substantially lower reflectance of the fungal structures of V. inaequalis relative to healthy leaf tissue. We offer a model of near-infrared radiation interactions with the fungus and leaf interactions to explain reflectance characteristics of scab-infected leaves throughout the growth cycle of the pathogen. The simple, low-cost remote-sensing approach developed here holds considerable promise for providing timely information on tree infection to improve the efficiency of apple scab disease management routines.

A NOVEL INTRINSIC IMAGE DECOMPOSITION METHOD TO RECOVER ALBEDO FOR AERIAL IMAGES IN PHOTOGRAMMETRY PROCESSING

Abstract. Recovering surface albedos from photogrammetric images for realistic rendering and synthetic environments can greatly facilitate its downstream applications in VR/AR/MR and digital twins. The textured 3D models from standard photogrammetric pipelines are suboptimal to these applications because these textures are directly derived from images, which intrinsically embedded the spatially and temporally variant environmental lighting information, such as the sun illumination, direction, causing different looks of the surface, making such models less realistic when used in 3D rendering under synthetic lightings. On the other hand, since albedo images are less variable by environmental lighting, it can, in turn, benefit basic photogrammetric processing. In this paper, we attack the problem of albedo recovery for aerial images for the photogrammetric process and demonstrate the benefit of albedo recovery for photogrammetry data processing through enhanced feature matching and dense matching. To this end, we proposed an image formation model with respect to outdoor aerial imagery under natural illumination conditions; we then, derived the inverse model to estimate the albedo by utilizing the typical photogrammetric products as an initial approximation of the geometry. The estimated albedo images are tested in intrinsic image decomposition, relighting, feature matching, and dense matching/point cloud generation results. Both synthetic and real-world experiments have demonstrated that our method outperforms existing methods and can enhance photogrammetric processing.

Detecting ripe fruits under natural occlusion and illumination conditions

For an accurate detection of ripe fruits under uneven illumination and ubiquitous occlusion conditions, this paper proposes a method to detect and locate ripe fruits based on machine vision. There are four key steps in this method including image graying and background removal, binary image optimization, true contour fragments extraction, and fruit fitting. For testing the proposed method, field experiments were conducted with tomato and citrus, and the ripe fruits in complex environments were successfully detected and located. From the detection experiments, it showed that the recognition rate for ripe fruits in the near zone of the proposed method was higher than 97.44%, the average time consumption was 0.2966 s, and the positioning error was less than 4.41%. In addition, it can be concluded from the comparative experiment that the proposed method is superior to conventional Hough transform, random Hough transform, and other methods based on deep learning in terms of detection rate, time performance and positioning accuracy. Therefore, it can be applied to picking robots for real-time detecting and locating ripe fruits.

Deep relightable appearance models for animatable faces

We present a method for building high-fidelity animatable 3D face models that can be posed and rendered with novel lighting environments in real-time. Our main insight is that relightable models trained to produce an image lit from a single light direction can generalize to natural illumination conditions but are computationally expensive to render. On the other hand, efficient, high-fidelity face models trained with point-light data do not generalize to novel lighting conditions. We leverage the strengths of each of these two approaches. We first train an expensive but generalizable model on point-light illuminations, and use it to generate a training set of high-quality synthetic face images under natural illumination conditions. We then train an efficient model on this augmented dataset, reducing the generalization ability requirements. As the efficacy of this approach hinges on the quality of the synthetic data we can generate, we present a study of lighting pattern combinations for dynamic captures and evaluate their suitability for learning generalizable relightable models. Towards achieving the best possible quality, we present a novel approach for generating dynamic relightable faces that exceeds state-of-the-art performance. Our method is capable of capturing subtle lighting effects and can even generate compelling near-field relighting despite being trained exclusively with far-field lighting data. Finally, we motivate the utility of our model by animating it with images captured from VR-headset mounted cameras, demonstrating the first system for face-driven interactions in VR that uses a photorealistic relightable face model.

Generalized Adiabatic Theorems: Quantum Systems Driven by Modulated Time-Varying Fields

We present generalized adiabatic theorems for closed and open quantum systems that can be applied to slow modulations of rapidly varying fields, such as oscillatory fields that occur in optical experiments and light induced processes. The generalized adiabatic theorems show that a sufficiently slow modulation conserves the dynamical modes of time dependent reference Hamiltonians. In the limiting case of modulations of static fields, the standard adiabatic theorems are recovered. Applying these results to periodic fields shows that they remain in Floquet states rather than in energy eigenstates. More generally, these adiabatic theorems can be applied to transformations of arbitrary time-dependent fields, by accounting for the rapidly varying part of the field through the dynamical normal modes, and treating the slow modulation adiabatically. As examples, we apply the generalized theorem to (a) predict the dynamics of a two level system driven by a frequency modulated resonant oscillation, a pathological situation beyond the applicability of earlier results, and (b) to show that open quantum systems driven by slowly turned-on incoherent light, such as biomolecules under natural illumination conditions, can only display coherences that survive in the steady state.

Seasonal variation in UVA light drives hormonal and behavioural changes in a marine annelid via a ciliary opsin.

The right timing of animal physiology and behavior ensures the stability of populations and ecosystems. In order to predict anthropogenic impacts on these timings, more insight is needed into the interplay between environment and molecular timing mechanisms. This is particularly true in marine environments.Using high-resolution, long-term daylight measurements from a habitat of the marine annelid Platynereis dumerilii, we find that temporal changes in UVA/deep violet intensities, more than longer wavelengths, can provide annual time information, which differs from annual changes in photoperiod. We developed experimental setups that resemble natural daylight illumination conditions, and automated, quantifiable behavioral tracking. Experimental reduction of UVA/deep violet light (app. 370-430nm) under long photoperiod (LD16:8) significantly decreases locomotor activities, comparable to the decrease caused by short photoperiod (8:16). In contrast, altering UVA/deep violet light intensities does not cause differences in locomotor levels under short photoperiod. This modulation of locomotion by UVA/deep violet light under long photoperiod requires c-opsin1, an UVA/deep violet-sensor employing Gi-signalling. C-opsin1 also regulates the levels of rate-limiting enzymes for monogenic amine synthesis and of several neurohormones, including PDF, Vasotocin (Vasopressin/Oxytocin) and NPY-1.Our analyses indicate a complex inteplay between UVA intensities and photoperiod as indicators of annual time.

Dissolved Black Carbon Facilitates Photoreduction of Hg(II) to Hg(0) and Reduces Mercury Uptake by Lettuce (Lactuca sativa L.).

Here, we investigated the photoreduction of Hg(II) (Hg(NO3)2) mediated by dissolved black carbon (DBC, <0.45 μm size fraction) collected from water extracts of biochar derived by pyrolyzing crop residues (rice, soybean, and peanut). Under simulated sunlight conditions, the presence of 5 mg C/L DBC significantly facilitated the production of Hg(0) from Hg(II) (initially at 10 nmol/L) with a reduction ratio of 73 ± 4% in 5.3 h. Inhibition of photolysis-induced reactive oxygen species by a quencher or removal of dissolved oxygen indicated that Hg(II) was mainly reduced by superoxide anion (O2•-). Reduction by electrons transferred from photoexcited DBC components or by organic free radicals produced from photo-Fenton-like reactions was also proposed to play a role. Contrary to dissolved humic substances, the DBC-mediated photoreduction of Hg(II) led to unique positive mass-independent isotopic fractionation (MIF) of Hg(0) (Δ199Hg = 1.8 ± 0.3‰), which was attributed to the dominance of secondary Hg(II) reduction by O2•-. The leachate from soil amended with rice biochar at 1-5% mass ratios exhibited significantly higher photocatalytic efficiency than that from unamended soil (wherein the reduced Hg(0) increased from 27 ± 1 to 63 ± 2% in maximum), and the efficiency positively correlated with the percentage of amended biochar. Under natural illumination conditions, the total mercury and/or methylmercury uptake by roots, shoots, and leaves of lettuce (Lactuca sativa L.) grown in water extracts of rice biochar-amended soil was consistently lower (up to 70 ± 20%) than that without the biochar amendment. The findings highlight that DBC might play an important and previously unrecognized role in the biogeochemical cycle and the environmental impact of mercury.

Electrochemical assessment of pigments-binding medium interactions in oil paint deterioration: a case study on indigo and Prussian blue

The degradation of laboratory oil paint film specimens containing indigo and Prussian blue pigments and pictorial samples from the Sant Francesc de Paula painting exhibited in the Tomàs Balvey Arxiu Museum (Cardedeu (Catalonia), Spain) has been studied by voltammetry of immobilized particles. This technique, combined with light microscopy, scanning electron microscopy-energy dispersive X-ray analysis, nanoindentation-atomic force microscopy, attenuated total reflectance-Fourier-transform infrared spectroscopy and gas chromatography–mass spectroscopy techniques permits the proposal of a dual scheme for the degradation of the pigments when naturally aged and submitted to accelerated UVA aging. Under conditions of moderate temperature, humidity and natural illumination, and low gradients of these parameters, Prussian blue acts as a radical scavenger moderating the production of reactive oxygen species produced in the oil binding medium by the action of ultraviolet radiation, resulting in the formation, in the solid state, of the solid-solution, {KFeIII[FeII(CN)6]}x{FeIII[FeIII(CN)6]}1–x, known as Berlin green, which then promotes the formation of indigo adducts with radicals. In several localized areas of the Sant Francesc de Paula paint showing strong degradation, Prussian blue acts as a promoter of the indigo oxidation to isatin, thus resulting in a considerable chromatic shift.

Computer-aided classification of suspicious pigmented lesions using wide-field images

Background and objectiveEarly identification of melanoma is conducted through whole-body visual examinations to detect suspicious pigmented lesions, a situation that fluctuates in accuracy depending on the experience and time of the examiner. Computer-aided diagnosis tools for skin lesions are typically trained using pre-selected single-lesion images, taken under controlled conditions, which limits their use in wide-field scenes. Here, we propose a computer-aided classifier system with such input conditions to aid in the rapid identification of suspicious pigmented lesions at the primary care level. Methods133 patients with a multitude of skin lesions were recruited for this study. All lesions were examined by a board-certified dermatologist and classified into “suspicious” and “non-suspicious”. A new clinical database was acquired and created by taking Wide-Field images of all major body parts with a consumer-grade camera under natural illumination condition and with a consistent source of image variability. 3–8 images were acquired per patient on different sites of the body, and a total of 1759 pigmented lesions were extracted. A machine learning classifier was optimized and build into a computer aided classification system to binary classify each lesion using a suspiciousness score. ResultsIn a testing set, our computer-aided classification system achieved a sensitivity of 100% for suspicious pigmented lesions that were later confirmed by dermoscopy examination (“SPL_A”) and 83.2% for suspicious pigmented lesions that were not confirmed after examination (“SPL_B”). Sensitivity for non-suspicious lesions was 72.1%, and accuracy was 75.9%. With these results we defined a suspiciousness score that is aligned with common macro-screening (naked eye) practices. ConclusionsThis work demonstrates that wide-field photography combined with computer-aided classification systems can distinguish suspicious from non-suspicious pigmented lesions, and might be effective to assess the severity of a suspicious pigmented lesions. We believe this approach could be useful to support skin screenings at a population-level.

Visual Measurement of Water Level under Complex Illumination Conditions.

Image-based water level measurement is a visual-sensing technique which automatically inspects the reading of the water line via image processing instead of the human eye. It can be realized easily on an existing video surveillance system and has advantages like low cost, non-contact, as well as results that are verifiable. It has the potential to be widely used in flood and waterlogging monitoring, while facing the challenge that water-line detection under complex natural or artificial illumination conditions is quite difficult in field applications. To handle this problem, a method is proposed assuming that the water line is generally located on the row with the largest local change of gray or edge features in the image of the water gauge. The water line is determined by coarse-to-fine detection of the position of the maximum mean difference (MMD) of the horizontal projections of gray and edge images. Image-based flow-level measurement systems were developed at two measurement sites. In situ comparative experiments were conducted with the float-type stage gauge and other image-based methods. The results show that the fusion of gray and edge features can overcome the shortcomings of single feature methods under complex illumination conditions such as dim light, glares, shadows and artificial night lighting. A coarse-to-fine strategy utilizes the periodicity of the surface pattern distribution of the standard bicolor water gauge, which improves the reliability of water-line detection. The resolution and accuracy of water-level measurement are 1 mm and 1 cm, respectively. In particular, the MMD value is efficient at identifying extremely unfavorable conditions and reducing gross errors.

MORPHOLOGICAL AND MORPHOMETRICAL FEATURES IN DUNALIELLA SALINA (CHLAMYDOMONADALES, DUNALIELLACEAE) DURING THE TWO-PHASE CULTIVATION MODE

The paper presents studies of morphological and morphometrical characteristics of green halophilic carotenogenic microalga Dunaliella salina (Dunal) Teodoresco, 1905, from the south-west region of Crimean Peninsula. D. salina was cultivated in two-phase mode under conditions of natural illumination at the premises of A.O. Kovalevsky Institute of Biology of the Southern Seas of RAS (IBSS), Sevastopol, Russia. The maximum D. salina cell density was 1.69·106 cell/ml in the “green” phase and 0.84·106 cell/ml in the “red” growth phase. The maximum productivity by cell number reached 0.15 ∙ 106 cell/(ml·day)in the “green” phase while it was lower by 73% in the “red” phase (0.04∙106 cell/(ml·day). Along with the maximum productivity, linear growth stage in the first phase was characterized by a maximum fraction of small (up to 500 μm3 in volume) cells (about 15-29%) and a decrease in cell volume by 40-45% as compared with initial value. The mean of D. salina cell volume in the “red” phase was 30% higher than in the “green” phase. At the same time, the large cell fraction in the “red” phase was consistently high (15-35%). The patterns of change in morphological and morphometrical cell parameters were in accordance to stage and conditions of growth. Thus, cell elongation was noted in the stage of linear growth, while under unfavorable conditions at growth-declining stage cells became more round-shaped, with orange and tile-red coloration and granulation of cell content. It was shown that morphological and morphometric cell parameters can serve as additional criteria for assessment of physiological condition in D. salina culture. The experiment demonstrated the prospects for two-stage D. salina cultivation in Crimea.

On Stereophotogrammetry of a Perturbed Sea Surface

We consider models of the formation of a spatio-angular brightness distribution of the light scattered from profiled rough surfaces and mirror surfaces under natural illumination conditions. The developed analytical model of the sea surface stereophotogrammetry is based on the analysis of relationships for space and angle derivatives of the brightness of the recorded light field of the sky, which is reflected by the surface.

Object Detection under Natural Illumination Conditions using Superpixels and Local Binary Pattern Feature

This paper proposes a novel object detection method under natural illumination conditions in which a set of features, texture features and Local Binary Pattern (LBP) features, are extracted from the images acquired by a colour camera. The goal of this study is to develop a robust and fast algorithm to detect immature green citrus fruit in individual trees from colour images acquired under natural outdoor conditions. Colour and shape features are used to remove the complex background as much as possible. Since leaves share much more similarities with green citrus fruit in colour and to some extent in the shape, texture features are used for citrus detection. Statistical features, Tamura features and LBP features are used to build the KNN classifier. Experimental results show that the proposed approach provides fairly good object detection performance and confirms an efficient way for outdoor green citrus detection.

Snapshot channeled imaging spectrometer using geometric phase holograms.

In this paper, we present the design and experimental demonstration of a snapshot imaging spectrometer based on channeled imaging spectrometry (CIS) and channeled imaging polarimetry (CIP). Using a geometric phase microlens array (GPMLA) with multiple focal lengths, the proposed spectrometer selects wavelength components within its designed operating waveband of 450-700 nm. Compared to other snapshot spectral imagers, its key components are especially suitable for roll-to-roll (R2R) rapid fabrication, which gives the spectrometer potential for low-cost mass production. The principles and proof-of-concept experimental system of the sensor are described in detail, followed by lab validation and outdoor measurement results which demonstrate the sensor's ability to resolve spectral and spatial contents under both experimental and natural illumination conditions.