Ground-based Camera Research Articles

Improving Real-Scene 3D Model Quality of Unmanned Aerial Vehicle Oblique-Photogrammetry with a Ground Camera

In UAV (unmanned aerial vehicle) oblique photogrammetry, the occlusion of ground objects, particularly at their base, often results in low-quality real-scene 3D models. To address this issue, we propose a method to enhance model quality by integrating ground-based camera images. This innovative image acquisition method allows the rephotographing of areas in the 3D model that exhibit poor quality. Three critical parameters for reshooting are the reshooting distance and the front- and side-overlap ratios of reshooting images. The proposed method for improving 3D model quality focuses on point accuracy, dimensional accuracy, texture details, and the triangular mesh structure. Validated by a case study involving a complex building, this method demonstrates that integrating ground camera photos significantly improves the overall quality of the 3D model. The findings show that optimal settings for reshooting include a distance (in meter units) of 1.5–1.6 times the camera’s focal length (in millimeter units), a front overlap ratio of 30%, and a side overlap ratio of 20%. Furthermore, we conclude that an overlap rate of 20–30% in reshooting is a usable value.

Fixed-Wing UAV Pose Estimation Using a Self-Organizing Map and Deep Learning

In many Unmanned Aerial Vehicle (UAV) operations, accurately estimating the UAV’s position and orientation over time is crucial for controlling its trajectory. This is especially important when considering the landing maneuver, where a ground-based camera system can estimate the UAV’s 3D position and orientation. A Red, Green, and Blue (RGB) ground-based monocular approach can be used for this purpose, allowing for more complex algorithms and higher processing power. The proposed method uses a hybrid Artificial Neural Network (ANN) model, incorporating a Kohonen Neural Network (KNN) or Self-Organizing Map (SOM) to identify feature points representing a cluster obtained from a binary image containing the UAV. A Deep Neural Network (DNN) architecture is then used to estimate the actual UAV pose based on a single frame, including translation and orientation. Utilizing the UAV Computer-Aided Design (CAD) model, the network structure can be easily trained using a synthetic dataset, and then fine-tuning can be done to perform transfer learning to deal with real data. The experimental results demonstrate that the system achieves high accuracy, characterized by low errors in UAV pose estimation. This implementation paves the way for automating operational tasks like autonomous landing, which is especially hazardous and prone to failure.

Stage and discharge prediction from documentary time-lapse imagery

Imagery from fixed, ground-based cameras is rich in qualitative and quantitative information that can improve stream discharge monitoring. For instance, time-lapse imagery may be valuable for filling data gaps when sensors fail and/or during lapses in funding for monitoring programs. In this study, we used a large image archive (>40,000 images from 2012 to 2019) from a fixed, ground-based camera that is part of a documentary watershed imaging project (https://plattebasintimelapse.com/). Scalar image features were extracted from daylight images taken at one-hour intervals. The image features were fused with United States Geological Survey stage and discharge data as response variables from the site. Predictions of stage and discharge for simulated year-long data gaps (2015, 2016, and 2017 water years) were generated from Multi-layer Perceptron, Random Forest Regression, and Support Vector Regression models. A Kalman filter was applied to the predictions to remove noise. Error metrics were calculated, including Nash-Sutcliffe Efficiency (NSE) and an alternative threshold-based performance metric that accounted for seasonal runoff. NSE for the year-long gap predictions ranged from 0.63 to 0.90 for discharge and 0.47 to 0.90 for stage, with greater errors in 2016 when stream discharge during the gap period greatly exceeded discharge during the training periods. Importantly, and in contrast to gap-filling methods that do not use imagery, the high discharge conditions in 2016 could be visually (qualitatively) verified from the image data. Half-year test sets were created for 2016 to include higher discharges in the training sets, thus improving model performance. While additional machine learning algorithms and tuning parameters for selected models should be tested further, this study demonstrates the potential value of ground-based time-lapse images for filling large gaps in hydrologic time series data. Cameras dedicated for hydrologic sensing, including nighttime imagery, could further improve results.

Post-midnight purple arc and patches appeared on the high latitude part of the auroral oval: Dawnside counterpart of STEVE?

The phenomenon known as strong thermal emission velocity enhancement (STEVE) is a purple/mauve arc-shaped atmospheric glow observed at lower latitudes of the auroral oval on the duskside. Simultaneous observations using a ground-based camera and a low-altitude satellite have shown that STEVE is accompanied by rapid westward ion flows. Such fast ion flows are termed the subauroral ion drift (SAID) or subauroral polarization stream (SAPS). Similarly, an eastward fast ion flow known as the dawnside auroral polarization stream (DAPS) is observed within the Region 1 current on the dawnside. If the optical phenomenon triggered by SAID/SAPS corresponds to STEVE, a comparable optical phenomenon should be driven by DAPS. Thus far, however, such a phenomenon has not been reported. This study discovers, for the first time, a purple-colored optical phenomenon characterized by the fast eastward ion flows, a possible signature of DAPS, occurring poleward of the bright green arc in the post-midnight sector. We present color all-sky images obtained by a ground-based commercial digital camera, along with wide-coverage optical measurements and in-situ data from low-altitude satellites. The results imply that this glow requires not only a high-speed ion flow but also its sharp latitudinal gradient at the boundary between the Region 1 and 2.Graphical

Intrahour Direct Normal Irradiance Forecasting Based on Sky Image Processing and Time-Series Analysis

The present paper exhibits a hybrid model for intrahour forecasting of direct normal irradiance (DNI). It combines a knowledge-based model, which is used for clear-sky DNI forecasting from DNI measurements, with a machine-learning-based model, that evaluates the impact of atmospheric disturbances on the solar resource, through the processing of high dynamic range sky images provided by a ground-based camera. The performance of the hybrid model is compared with that of two machine learning models based on past DNI observations only. The results highlight the pertinence of combining knowledge-based models with data-driven models, and of integrating sky-imaging data in the DNI forecasting process. Parts of this paper were published as journal articleKarout, Y.; Thil, S.; Eynard, J.; Guillot, E.; Grieu, S. Hybrid intrahour DNI forecast model based on DNI measurements and sky-imaging data. Solar Energy. 2023, 249, 541-558. https://doi.org/10.1016/j.solener.2022.11.032

Aerosol Optical Depth Measurements from a Simulated Low-Cost Multi-Wavelength Ground-Based Camera: A Clear Case over a Peri-Urban Area

This paper highlights the advantages of an affordable multi-wavelength ground-based camera, called WaltRCam, for monitoring Aerosol Optical Depth (AOD) in a clear case over a peri-urban area. To simulate the performance of this low-cost camera, for which data are not yet available, we use data from an expensive hyperspectral camera (HSI) to mimic its characteristics. Our methodology is based on the construction of look-up tables using the DART (Discrete Anisotropic Radiative Transfer) 3D radiative transfer model. DART simulates the different spectra observed by the WaltRCam camera, which then provides the AODs for all image pixels in near-real-time. Moreover, DART is coupled to a 3D scale-model of the city of Toulouse (dating from 2014) to model complex urban geometries and to associate specific optical properties to the various objects that make up the environment. Moreover, we use a neural-network-based method to recognize the various objects in the image in order to take into account only pixels common between the observations. In this way, we take account of changes to the peri-urban area, such as vegetation growth, construction, demolition of buildings, etc. The results of this study show that the WaltRCam camera, by capturing eight wavelengths, can deliver convincing results compared with ground and satellite reference data, with a correlation coefficient of 0.9 and an average RMSE of less than 0.02.

A RGB-Thermal based adaptive modality learning network for day–night wildfire identification

Wildfires have long been a danger to the atmosphere and ecological environment. With the advancement of deep learning technology and sensor equipment, wildfire identification methods based on optical (RGB) or thermal infrared (TIR) images have made significant progress. However, single-modal identification technologies often experience serious performance degradation in challenging scenarios. Therefore, a multi-modal framework that combines both RGB and TIR is necessary for accurate wildfire identification. Despite this, current RGB-T based identification methods focus solely on developing fusion strategies to learn shared representations of the two modalities, while ignoring the effect of modality-specific features. To address this issue, we propose a novel wildfire identification framework that can adaptively learn modality-specific and shared features. Specifically, our model utilizes two parallel encoders to extract multi-scale RGB and TIR features, and integrates these modality-specific features into the fusion feature layer. Each modality-specific decoder and shared decoder corresponds to three independent label supervisions that oversee feature adaptive learning. To validate the effectiveness of our approach, we propose a new RGB-T paired wildfire semantic segmentation dataset that includes flames of different scales captured under varying lighting conditions from multiple UAV-based and ground-based cameras. Finally, we evaluate our proposed method alongside benchmarking single-modal methods and state-of-the-art multi-modal methods on two tasks: a common test setting under various lighting conditions, and a generalization test across three subsets. The experimental results demonstrate the superiority of our proposed method, with the average ＋6.41% (IoU) and ＋3.39% (F1-score) improvement over the second-best RGB-T semantic segmentation method.

HARNESSING PHOTOGRAMMETRY FOR WHITEWATER SLALOM COURSES IN NATURAL RIVERBEDS: AN INNOVATIVE APPROACH FOR COURSE DESIGN AND ANALYSIS

Abstract. Whitewater slalom, a thrilling and challenging water sport, requires carefully designed courses that provide a dynamic and engaging experience for athletes. Traditional methods of designing slalom courses in natural riverbeds have relied on manual measurements and estimations, which can be time-consuming, labour-intensive, and subject to human error. However, with the advent of photogrammetry, a cutting-edge technology that involves the use of aerial or ground-based cameras to capture and process 3D data from images, there is a new frontier in designing and analysing white water slalom courses.This article explores the usage of photogrammetry in the context of white-water slalom course design in natural riverbeds. It highlights how photogrammetry can revolutionize the course design process by providing accurate and detailed 3D models of the riverbed terrain, which can be used for virtual simulations, rapid prototyping, and precise analysis of different design options. The advantages of using photogrammetry include improved accuracy, efficiency, and cost-effectiveness in comparison to traditional methods.The article concludes by discussing the challenges and limitations of using photogrammetry for whitewater slalom course design, including issues related to data acquisition, processing, and accuracy. Despite these challenges, the potential of photogrammetry in revolutionizing the design and analysis of white-water slalom courses in natural riverbeds is immense, and it opens up exciting opportunities for further research, innovation, and advancement in the field of water sports.

Precipitable water vapor retrievals using a ground-based infrared sky camera in subtropical South America

Abstract. Atmospheric precipitable water vapor (PWV) is a critical quantity in fast-changing weather processes. Current retrieval techniques lack the spatial and/or temporal resolution necessary for a full PWV characterization. Here we investigate a retrieval method using an all-sky ground-based camera comprising a 14-bit 644 × 512-pixel microbolometer sensor array. The radiometrically calibrated infrared downwelling spectral radiance, Lλ, was acquired at rates of up to 3 min−1. For the studied site (23.56∘ S, 46.74∘W; 786 m a.s.l.) and spectral interval, Lλ is sensitive to the PWV; the vertical distribution of humidity; and their temporal, spatial, or seasonal variations. By comparing measured and simulated Lλ, we show that the PWV can be retrieved from prior knowledge of the local humidity profile. This information can originate from radiosonde data or statistical analysis of past vertical humidity distributions. Comparison with sun photometer PWV retrievals, for stable atmospheric conditions, showed an agreement of the average PWV within 2.8 % and a precision of subsequent retrievals of 1.9 %. The PWV was also retrieved as a bi-dimensional array, allowing for the investigation of spatial inhomogeneities of humidity distribution. The method can be used for daytime or nighttime retrievals, under partly cloudy sky conditions. Potential applications include studies on convection initiation processes.

Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis

Abstract Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ∼ 85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Furthermore, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.

Fast Pasture Classification Method using Ground-based Camera and the Modified Green Red Vegetation Index (MGRVI)

Fast Pasture Classification Method using Ground-based Camera and the Modified Green Red Vegetation Index (MGRVI)

Hybrid intrahour DNI forecast model based on DNI measurements and sky-imaging data

Short-term forecasting of direct normal irradiance (DNI), which refers to photons that did not interact with the atmosphere on their way to the observer, is of great interest in the advanced management of concentrated solar power systems. So, this paper exhibits a hybrid model dedicated to the intrahour forecasting of DNI (horizons are 5, 10, and 15 min). This hybrid model combines a knowledge-based model with a machine learning model: the knowledge-based model is used for clear-sky DNI forecasting from DNI measurements; the machine learning model evaluates the impact of the atmospheric disturbances on the solar resource, through the processing of high dynamic range sky images provided by a ground-based camera. The hybrid model is evaluated by comparing its performance, for different sky conditions, with that of two machine learning models based on past DNI observations only. The results highlight the pertinence of combining knowledge-based models with data-driven models, and of integrating sky-imaging data in the DNI forecasting process: the proposed hybrid model successfully manages clear-sky, overcast, and mixed situations. Regardless of the forecast horizon, the hybrid model outperforms the reference models both in normalized root mean squared error and in DNI ramp detection.

Correction and calibration of atmospheric impact observations in GOES GLM data.

The Earth's atmosphere is impacted daily by both meteoroids and artificial objects. Calibrated observations of the emitted light at sufficiently high sampling rates can enable or improve the estimation of impactor attributes such as size, cohesion, trajectory, and composition, but are difficult to obtain owing to the unpredictability, brevity, and high dynamic (brightness) range of impacts. Ground-based camera systems have successfully monitored small regions of the atmosphere at video frame rates and with limited radiometric capabilities, but most impacts occur over the 70% of the Earth's surface covered by water and are therefore missed by these networks. The Geostationary Lightning Mapper (GLM) instruments aboard Geostationary Operational Environmental Satellites 16 and 17 provide near-hemispherical coverage at 500 frames per second. These data have been shown to contain the signatures of many independently confirmed impacts, often from both viewing angles simultaneously, and constitute an observational resource that is currently unparalleled in the public domain. NASA's Asteroid Threat Assessment Project has implemented an automated impact detection pipeline that processes data from GLM daily. Given a detected impact, the GLM data contain a wealth of information for use in quantitative follow-up analyses. However, impact events differ from lightning in ways that violate key assumptions built into GLM's design. The result is that GLM's onboard processing introduces errors into pixel observations of impact events and the calibrated energies near the periphery of the detector may be substantially overestimated. We present methods for mitigating these and other issues to produce a data product more suitable for impact analyses than the existing GLM lightning product.

Simulation, image generation, and tomographic reconstruction of idealized volcanic plumes

ABSTRACT This paper presents a volcanic plume simulation and image generation framework, alongside a method for the tomographic reconstruction of volcanic ash plumes. The simulation framework facilitates the generation and processing of imagery analogous to that produced by real-world multi-spectral infrared observations of volcanic emissions. With this required imagery simulated, methods for the tomographic reconstruction of volcanic plumes can be tested. This simulation approach was undertaken due to the lack of suitable real-world multi-spectral and multi-angle imagery, and the difficulties and dangers of arranging multiple ground-based cameras or operating UASs in proximity to an active volcano. The efficacy of the simulation framework is demonstrated through a series of sensitivity analyses, assessing the change in reconstruction accuracy when modifying simulation variables such as the number and distribution of images, spatial resolution, and camera pointing inaccuracy. It is proposed that the results of these analyses can be used to inform the design and optimization of real-world observation campaigns.

Comparison of Deep Learning Models for the Classification of Noctilucent Cloud Images

Optically thin layers of tiny ice particles near the summer mesopause, known as noctilucent clouds, are of significant interest within the aeronomy and climate science communities. Ground-based optical cameras mounted at various locations in the arctic regions collect the dataset during favorable summer times. In this paper, first, we compare the performances of various deep learning-based image classifiers against a baseline machine learning model trained with support vector machine (SVM) algorithm to identify an effective and lightweight model for the classification of noctilucent clouds. The SVM classifier is trained with histogram of oriented gradient (HOG) features, and deep learning models such as SqueezeNet, ShuffleNet, MobileNet, and Resnet are fine-tuned based on the dataset. The dataset includes images observed from different locations in northern Europe with varied weather conditions. Second, we investigate the most informative pixels for the classification decision on test images. The pixel-level attributions calculated using the guide back-propagation algorithm are visualized as saliency maps. Our results indicate that the SqueezeNet model achieves an F1 score of 0.95. In addition, SqueezeNet is the lightest model used in our experiments, and the saliency maps obtained for a set of test images correspond better with relevant regions associated with noctilucent clouds.

Assessment of Nighttime Cloud Cover Products from MODIS and Himawari-8 Data with Ground-Based Camera Observations

Comparing cloud cover (CC) products from different satellites with the same ground-based CC dataset provides information on the similarities or differences of values among satellite products. For this reason, 42-month CC products from Moderate Resolution Imaging Spectrometer’s (MODIS) Collection 6.1 daily cloud cover products (MOD06_L2, MYD06_L2, MOD08_D3, and MYD08_D3) and Himawari-8 are compared with the ground-based camera datasets. The comparison shows that CC from MODIS differs from ground measurement CC by as much as 57% over Chiba, Japan, when low CC is observed by the camera. This indicates MODIS’s ability to capture high-level clouds that are not effectively seen from the ground. When the camera detects high CC, an indication of the presence of low-level clouds, CC from MODIS is relatively higher than the CC from the camera. In the case of Himawari-8 data, when the camera observes low CC, this difference is around 0.7%. This result indicates that high-level clouds are not effectively observed, but the Himawari-8 data correlates well with camera observations. When the camera observes high CC, Himawari-8-derived CC is lower by around 10% than CC from the camera. These results show the potential of continuous observations of nighttime clouds using the camera to provide a dataset that can be used for intercomparison among nighttime satellite CC products.

Study on Changes of NDVI by Growth Stages of Winter Forage Crop Using a Ground-based Camera System

Study on Changes of NDVI by Growth Stages of Winter Forage Crop Using a Ground-based Camera System

Tracking the Endogenous Dynamics of the Solfatara Volcano (Campi Flegrei, Italy) through the Analysis of Ground Thermal Image Temperatures

In the last decades, thermal infrared ground-based cameras have become effective tools to detect significant spatio-temporal anomalies in the hydrothermal/volcanic environment, possibly linked to impending eruptions. In this paper, we analyzed the temperature time-series recorded by the ground-based Thermal Infrared Radiometer permanent network of INGV-OV, installed inside the Solfatara-Pisciarelli area, the most active fluid emission zones of the Campi Flegrei caldera (Italy). We investigated the temperatures’ behavior in the interval 25 June 2016–29 May 2020, with the aim of tracking possible endogenous hydrothermal/volcanic sources. We performed the Independent Component Analysis, the time evolution estimation of the spectral power, the cross-correlation and the Changing Points’ detection. We compared the obtained patterns with the behavior of atmospheric temperature and pressure, of the time-series recorded by the thermal camera of Mt. Vesuvius, of the local seismicity moment rate and of the CO2 emission flux. We found an overall influence of exogenous, large scale atmospheric effect, which dominated in 2016–2017. Starting from 2018, a clear endogenous forcing overcame the atmospheric factor, and dominated strongly soil temperature variations until the end of the observations. This paper highlights the importance of monitoring and investigating the soil temperature in volcanic environments, as well as the atmospheric parameters.

The Impacts of Spatial Resolution, Viewing Angle, and Spectral Vegetation Indices on the Quantification of Woody Mediterranean Species Seasonality Using Remote Sensing

Discriminating between woody plant species using a single image is not straightforward due to similarity in their spectral signatures, and limitations in the spatial resolution of many sensors. Seasonal changes in vegetation indices can potentially improve vegetation mapping; however, for mapping at the individual species level, very high spatial resolution is needed. In this study we examined the ability of the Israel/French satellite of VENμS and other sensors with higher spatial resolutions, for identifying woody Mediterranean species, based on the seasonal patterns of vegetation indices (VIs). For the study area, we chose a site with natural and highly heterogeneous vegetation in the Judean Mountains (Israel), which well represents the Mediterranean maquis vegetation of the region. We used three sensors from which the indices were derived: a consumer-grade ground-based camera (weekly images at VIS-NIR; six VIs; 547 individual plants), UAV imagery (11 images, five bands, seven VIs) resampled to 14, 30, 125, and 500 cm to simulate the spatial resolutions available from some satellites, and VENμS Level 1 product (with a nominal spatial resolution of 5.3 m at nadir; seven VIs; 1551 individual plants). The various sensors described seasonal changes in the species’ VIs at different levels of success. Strong correlations between the near-surface sensors for a given VI and species mostly persisted for all spatial resolutions ≤125 cm. The UAV ExG index presented high correlations with the ground camera data in most species (pixel size ≤125 cm; 9 of 12 species with R ≥ 0.85; p < 0.001), and high classification accuracies (pixel size ≤30 cm; 8 species with >70%), demonstrating the possibility for detailed species mapping from space. The seasonal dynamics of the species obtained from VENμS demonstrated the dominant role of ephemeral herbaceous vegetation on the signal recorded by the sensor. The low variance between the species as observed from VENμS may be explained by its coarse spatial resolution (effective ground spatial resolution of 7.5) and its non-nadir viewing angle (29.7°) over the study area. However, considering the challenging characteristics of the research site, it may be that using a VENμS type sensor (with a spatial resolution of ~1 m) from a nadir point of view and in more homogeneous and dense areas would allow for detailed mapping of Mediterranean species based on their seasonality.

Birds not in flight: using camera traps to observe ground use of birds at a wind-energy facility

Abstract Context Camera trapping is increasingly used to collect information on wildlife occurrence and behaviour remotely. Not only does the technique provide insights into habitat use by species of interest, it also gathers information on non-target species. Aims We implemented ground-based camera trapping to investigate the behaviours of ground-dwelling birds, a technique that has largely been unutilised for studying birds, especially in wind-energy facilities. Methods We used camera traps to monitor activities of Agassiz’s desert tortoises (Gopherus agassizii) at their self-constructed burrows in a wind-energy facility near Palm Springs, California, USA. While doing so, we collected data on numerous burrow commensals, including birds. Key results Monitoring from late spring to mid-autumn in one year showed regular use of tortoise burrows and the immediate area by 12 species of birds, especially passerines. The most abundant species, as indicated by the number of photographs, but not necessarily individuals, was the rock wren (Salpinctes obsoletus), with a total of 1499 events. Birds appeared to use the interior or proximate vicinity of burrows for gathering nesting material, displaying, feeding, dust bathing and other activities. Of the bird species observed, 10 are known to be occasional casualties of turbine-blade strikes. The minimum known-age of a burrow had a positive relationship with bird counts. Conclusions Using camera traps focused at ground level can be a useful tool in avian conservation efforts because it is an effective technique for measuring bird presence, activity and behaviour in altered habitats such as wind farms, especially for those species that are low flyers or ground dwellers. Implications Acquiring data over the long term by using ground-based monitoring with camera traps could add to our understanding of avian behaviour and habitat use in relation to wind-energy infrastructure and operations, and help determine the vulnerability of avifauna that utilise the area.