All-sky Imager Research Articles

The MDW Hα Sky Survey: Data Release 0

The Mittelman–di Cicco–Walker (MDW) Hα Sky Survey is an autonomously operated and ongoing all-sky imaging survey in the narrowband Hα wavelength. The survey was founded by amateur astronomers and is presented here in its first stage of refinement for rigorous scientific use. Each field is exposed through an Hα filter with a 3 nm bandwidth for a total of 4 hr, with a pixel scale of 3.″2. Here, we introduce the first Data Release of the MDW Hα Survey (Data Release 0, or DR0), spanning 238 fields in the region of Orion (∼3100 deg2). DR0 includes: calibrated mean fields, star-removed mean fields, a point-source catalog matched to Data Release 1 of the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS1) and the Isaac Newton Telescope Galactic Plane Survey (IGAPS), and mosaics. 3 3 DR0 components are available at http://mdw.astro.columbia.edu; the DR0 catalog can also be found on the AAS Journals Zenodo repository: doi:10.5281/zenodo.12747455.

Short-term forecast of solar irradiance components using an alternative mathematical approach for the identification of cloud features

Solar energy technologies require precise solar forecasting to reduce power generation losses and protect equipment from irradiance fluctuations. This study introduces an alternative methodology for short-term forecasting of direct normal irradiance (DNI) and global horizontal irradiance (GHI) utilizing ground-based sky images captured by a single device. A low-cost all-sky imager (ASI) was developed, which implements an angular transformation and an optical flow technique to extract cloud features such as shape and velocity. A mathematical model calculates cloud transmittance based on pixel intensity, eliminating complex training steps. Results from a 30-day experimental campaign, incorporating diverse meteorological conditions, were compared against a secondary standard solarimetric station, a smart persistence model, and state-of-the-art approaches. The DNI forecast achieved an RMSE (relative error) of 46.79 W/m2 (11.99%) for 1-min intervals and 90.21 W/m2 (17.54%) for 10-min intervals, while GHI ranged from 31.73 W/m2 (4.68%) to 75.02 W/m2 (13.63%). Pearson correlation coefficients exceeded 0.9 overall, reaching 0.98 and 0.99 for the 1-min DNI and GHI forecasts, and 0.91 and 0.96 for the 10-min DNI and GHI forecasts, respectively, underscoring the system’s accuracy and robustness in complex meteorological scenarios.

The cloud cover and meteorological parameters at the Lenghu site on the Tibetan Plateau

Abstract The cloud cover and meteorological parameters serve as fundamental criteria for an astronomical observatory working in optical and infrared wavelengths. In this paper, we present a systematic assessment of key meteorological parameters at the Lenghu astronomical observing site on the Tibetan Plateau. The datasets adopted includes the meteorological parameters collected at the local weather stations at the site and in the Lenghu Town, the sky brightness acquired by the Sky Quality Meters and all-sky images from a digital camera, the ERA5 reanalysis database and global climate monitoring data. From 2019 to 2023, the fractional observable time of photometric condition is 69.70%, 74.97%, 70.26%, 74.27% and 65.12%, respectively, which is influenced by a variety of meteorological parameters. Large-scale air-sea interactions affect the climate at Lenghu site, which in fact delivers a clue to understand the irregularity of 2023. Specifically, precipitable water vapor at Lenghu site is correlated to both the westerly wind index and the summer North Atlantic Oscillation index, the yearly average temperature of Lenghu site is observed to increase significantly during the occurrence of a strong El Niño event, and the relative humidity anomaly at Lenghu site is correlated to the Pacific Decadal Oscillation index. The decrease of fractional observing time in 2023 was due to the ongoing strong El Niño event and relevant global climate change. We underscore the substantial role of global climate change in regulating astronomical observing conditions and the necessity for long-term continuous monitoring of the astronomical meteorological parameters at Lenghu site.

Estimating the drift velocity of plasma bubbles in airglow images using the scale invariant feature transform and the speeded up robust feature algorithms

Equatorial plasma bubbles (EPBs) are ionospheric irregularities that can significantly impact radio communication, causing signal scintillation that leads to signal loss and error in position calculations. In this study, we introduced a new technique to estimate the eastward and northward drift velocities of the EPB from ASI data by tracking the Scale Invariant Feature Transform (SIFT) and the Speeded Up Robust Features (SURF) points. The technique has been tested by studying multiple EPBs events detected over the Brazilian sector during the autumn equinox, spring equinox and summer solstice. The chosen events occurred during quiet nights (Kp index ≤ 3) and were observable through two All-Sky imager (ASI) stations. Moreover, the study shows the performance of the technique under different circumstances. It achieves optimal performance on clear-sky nights; however, cloud presence within the ASI field of view increases the error within the estimated values. The estimated velocities were found to be within the range reported by previous studies in the Brazilian sector.

Dataset for Machine Learning: Explicit All-Sky Image Features to Enhance Solar Irradiance Prediction

Prediction of solar irradiance is crucial for photovoltaic energy generation, as it helps mitigate intermittencies caused by atmospheric fluctuations such as clouds, wind, and temperature. Numerous studies have applied machine learning and deep learning techniques from artificial intelligence to address this challenge. Based on the recently proposed Hybrid Prediction Method (HPM), this paper presents an original and comprehensive dataset with nine attributes extracted from all-sky images developed using image processing techniques. This dataset and analysis of its attributes offer new avenues for research into solar irradiance forecasting. To ensure reproducibility, the data processing workflow and the standardized dataset have been meticulously detailed and made available to the scientific community to promote further research into prediction methods for photovoltaic energy generation.

Cloud Segmentation and Matching Using Deep Learning in All-Sky Images

Cloud Segmentation and Matching Using Deep Learning in All-Sky Images

A machine learning approach for estimating the drift velocities of equatorial plasma bubbles based on All-Sky Imager and GNSS observations

Equatorial Plasma Bubbles (EPBs) are zones characterized by fluctuations in plasma densities which form in the low-latitude ionosphere primarily during the post-sunset. They subject radio signals to amplitude and phase variabilities, affecting the functioning of technological systems that utilize the Global Navigation Satellite Systems (GNSS) signals for navigation. Thus, understanding EPB occurrence patterns and morphological features is vital for mitigating their effects. In this work, we employed two GNSS receivers and an All-Sky Imager (ASI) to conduct simultaneous observations on the morphology of EPBs over Brazil. The main objectives of the study were (1) to develop a Random Forest (RF) machine-learning model to estimate and predict the zonal drift velocities of EPBs, and (2) to compare the model predictions with actual EPB drifts inferred from the two instruments, as well as zonal neutral wind speeds obtained from the Horizontal Wind Model (HWM-14). In the model development, we utilized reliable EPB drift measurements made during geomagnetically quiet days between 2013 and 2017 in Brazil. The model predicted the velocities based on parameters including the day of the year, universal time, critical frequency of the F2 layer (foF2), solar and interplanetary indices. The correlation coefficients of 0.98 and 0.96 and RMSE values of 10.61 m/s and 10.06 m/s were obtained upon training and validation correspondingly. We evaluated the accuracy of the model in predicting EPB drifts on two geomagnetically quiet nights where an average correlation coefficient of 0.89 and an RMSE of 15.74 m/s were obtained. The predicted drifts, the zonal neutral wind velocities, and the GNSS and ASI velocity measurements were put into context for validation purposes. Overall, the velocities were comparable and ranged between ∼100 m/s and ∼30 m/s from the hours of 00 UT to 05 UT. The results confirmed the accuracy and applicability of the model, revealing the ionosphere-thermosphere coupling influence on the nocturnal propagation of EPBs under the full activation of the F region dynamo.

Investigation on Chasing and Interaction of Traveling Ionospheric Disturbances Based on Multi‐Instrument

AbstractIn this study, we use multi‐instrument observations (all‐sky imager (ASI), global navigation satellite system (GPS) receivers, digisonde) to study the interaction of nighttime medium‐scale traveling ionospheric disturbances (MSTIDs) on 13 November 2018. The most attractive aspect of this event is that the interaction appeared between two dark bands both propagated southwestward. The airglow observations show that the latter band moved faster and caught up with the former, and these two bands merged into a new one. The propagating characteristics and morphology of the MSTIDs changed during the interaction process. The simulations from the Thermosphere‐Ionosphere‐Electrodynamics General Circulation Model (TIEGCM) suggested that the ionospheric background zonal winds and electron density distributions could play essential roles in the interaction of the MSTIDs. Moreover, the merging process might be associated with the electrostatic reconnection.

Exploring the relationship between STEVE and SAID during three events observed by SuperDARN

The phenomenon known as strong thermal emission velocity enhancement (STEVE) is a narrow optical structure that may extend longitudinally for thousands of kilometers. Initially observed by amateur photographers, it has recently garnered researchers’ attention. STEVE has been associated with a rapid westward flow of ions in the ionosphere, known as subauroral ion drift (SAID). In this work, we investigate three occurrences of STEVE, using data from one of the Time History of Events and Macroscale Interactions during Substorms (THEMIS) ground-based all-sky imagers (ASIs) located at Pinawa, Manitoba, and from the Super Dual Auroral Radar Network (SuperDARN). This approach allows us to verify the correlation between STEVE and SAID, as well as analyze the temporal variation of SAID observed during STEVE events. Our results suggest that the SAID activity starts before the STEVE, and the magnitude of the westward flow decreases as the STEVE progresses toward the end of its optical manifestation.

Interactions between MSTIDs and Ionospheric Irregularities in the Equatorial Region Observed on 13–14 May 2013

We investigate the interactions between medium-scale traveling ionospheric disturbances (MSTIDs) and the equatorial ionization anomaly (EIA) as well as between MSTIDs and equatorial plasma bubbles (EPBs) on the night of 13–14 May 2013, based on observations from multiple instruments (an all-sky imager, digisonde, and global positioning system (GPS)). Two dark bands (the low plasma density region) for the MSTIDs were observed moving toward each other, encountering and interacting with the EIA, and subsequently interacting again with the EIA before eventually dissipating. Then, a new dark band of MSTIDs moved in the southwest direction, drifted into the all-sky imager’s field of view (FOV), and interacted with the EIA. Following this interaction, a new dark band split off from the original dark band, slowly moved in the northeast direction, and eventually faded away in a short time. Subsequently, the original southwestward-propagating dark band of the MSTIDs encountered eastward-moving EPBs, leading to an interaction between the MSTIDs and the EPBs. Then, the dark band of the MSTIDs faded away, while the EPBs grew larger with a pronounced westward tilt. The results from various observational instruments indicate the pivotal role played by the high-density region of the EIA in the occurrence of various interaction processes. In addition, this study also revealed that MSTIDs propagating into the equatorial region can significantly impact the morphology and evolution characteristics of EPBs.

A Review and Evaluation of the State of Art in Image-Based Solar Energy Forecasting: The Methodology and Technology Used

The increasing penetration of solar energy into the grid has led to management difficulties that require high accuracy forecasting systems. New techniques and approaches are emerging worldwide every year to improve the accuracy of solar power forecasting models and reduce uncertainty in predictions. This article aims to evaluate and compare various solar power forecasting methods based on their characteristics and performance using imagery. To achieve this goal, this article presents an updated analysis of diverse research, which is classified in terms of the technologies and methodologies applied. This analysis distinguishes studies that use ground-based sensor measurements, satellite data processing, or all-sky camera images, as well as statistical regression approaches, artificial intelligence, numerical models, image processing, or a combination of these technologies and methods. Key findings include the superior accuracy of hybrid models that integrate multiple data sources and methodologies, and the promising potential of all-sky camera systems for very short-term forecasting due to their ability to capture rapid changes in cloud cover. Additionally, the evaluation of different error metrics highlights the importance of selecting appropriate benchmarks, such as the smart persistence model, to enhance forecast reliability. This review underscores the need for continued innovation and integration of advanced technologies to meet the challenges of solar energy forecasting.

Innovative cloud quantification: deep learning classification and finite-sector clustering for ground-based all-sky imaging

Abstract. Accurate cloud quantification is essential in climate change research. In this work, we construct an automated computer vision framework by synergistically incorporating deep neural networks and finite-sector clustering to achieve robust whole-sky image-based cloud classification, adaptive segmentation and recognition under intricate illumination dynamics. A bespoke YOLOv8 (You Only Look Once 8) architecture attains over 95 % categorical precision across four archetypal cloud varieties curated from extensive annual observations (2020) at a Tibetan highland station. Tailor-made segmentation strategies adapted to distinct cloud configurations, allied with illumination-invariant image enhancement algorithms, effectively eliminate solar interference and substantially boost quantitative performance even in illumination-adverse analysis scenarios. Compared with the traditional threshold analysis method, the cloud quantification accuracy calculated within the framework of this paper is significantly improved. Collectively, the methodological innovations provide an advanced solution to markedly escalate cloud quantification precision levels imperative for climate change research while offering a paradigm for cloud analytics transferable to various meteorological stations.

Studies on the propagation dynamics and source mechanism of quasi-monochromatic gravity waves observed over São Martinho da Serra (29° S, 53° W), Brazil

Abstract. A total of 209 events of quasi-monochromatic atmospheric gravity waves (QMGWs) were acquired over 5 years of gravity waves (GWs) observation in southern Brazil. The observations were made by measuring the OH (hydroxyl radical) emission using an all-sky imager hosted by the Southern Space Observatory (SSO) coordinated by the National Institute for Space Research at São Martinho da Serra (RS) (29.44° S, 53.82° W). Using a two-dimensional fast-Fourier-Transform-based spectral analysis, it has been shown that the QMGWs have horizontal wavelengths of 10–55 km, periods of 5–74 min, and phase speeds up to 100 m s−1. The waves exhibited clear seasonal dependence on the propagation direction with anisotropic behavior, propagating mainly toward the southeast during the summer and autumn seasons and mainly toward the northwest during the winter. On the other hand, the propagation directions in the spring season exhibited a wide range from northwest to south. A complementary backward ray-tracing result revealed that the significant factors contributing to the propagation direction of the QMGWs are their source locations and the dynamics of the background winds per season. Three case studies in winter were selected to investigate further the propagation dynamics of the waves and determine their possible source location. We found that the jet stream associated with the cold front and their interaction generated these three GW events.

Simultaneous OI 630 nm imaging observations of thermospheric gravity waves and associated revival of fossil depletions around midnight near the equatorial ionization anomaly (EIA) crest

Abstract. We report F-region airglow imaging of fossil plasma depletions around midnight that revived afresh under persisting thermospheric gravity wave (GW) activity. An all-sky imager recorded these events in OI 630 nm imaging over Ranchi (23.3° N, 85.3° E; mlat. ∼19° N), India, on 16 April 2012. Northward-propagating and east–west-aligned GWs (λ∼210 km, v∼64 m s−1, and τ∼0.91 h) were seen around midnight. Persisting for ∼2 h, this GW activity revived two co-existing and eastward-drifting fossil depletions, DP1 and DP2. GW-driven revival was prominently seen in depletion DP1, wherein its apex height grew from ∼600 to >800 km, and the level of intensity depletion increased from ∼17 % to 50 %. The present study is novel in the sense that simultaneous observations of thermospheric GW activity and the associated evolution of depletion in OI 630 nm airglow imaging, as well as that around local midnight, have not been reported earlier. The current understanding is that GW phase fronts aligned parallel to the geomagnetic field lines and eastward-propagating are more effective in seeding Rayleigh–Taylor (RT) instability. Here, GW fronts were east–west-aligned (i.e., perpendicular to the geomagnetic field lines) and propagated northward, yet they revived fossil depletions.

Multidimensional Feature Extraction Based Minutely Solar Irradiance Forecasting Method Using All-Sky Images

Multidimensional Feature Extraction Based Minutely Solar Irradiance Forecasting Method Using All-Sky Images

Auroral breakup detection in all-sky images by unsupervised learning

Abstract. Due to a large number of automatic auroral camera systems on the ground, image data analysis requires more efficiency than what human expert visual inspection can provide. Furthermore, there is no solid consensus on how many different types or shapes exist in auroral displays. We report the first attempt to classify auroral morphological forms by an unsupervised learning method on an image set that contains both nightside and dayside aurora. We used 6 months of full-colour auroral all-sky images captured at a high-Arctic observatory on Svalbard, Norway, in 2019–2020. The selection of images containing aurora was performed manually. These images were then input into a convolutional neural network called SimCLR for feature extraction. The clustered and fused features resulted in 37 auroral morphological clusters. In the clustering of auroral image data with two different time resolutions, we found that the occurrence of 8 clusters strongly increased when the image cadence was high (24 s), while the occurrence of 14 clusters experienced little or no change with changes in input image cadence. We therefore investigated the temporal evolution of a group of eight “active aurora” clusters. Time periods for which this active aurora persisted for longer than two consecutive images with a maximum cadence of 6 min coincided with ground-magnetic deflections, and their occurrence was found to maximize around magnetic midnight. The active aurora onsets typically included vortical auroral structures and equivalent current patterns typical for substorms. Our findings therefore suggest that our unsupervised image clustering method can be used to detect auroral breakups in ground-based image datasets with a temporal accuracy determined by the image cadence.

Automatic Classification of All-Sky Nighttime Cloud Images Based on Machine Learning

Cloud-induced atmospheric extinction and occlusion significantly affect the effectiveness and quality of telescope observations. Real-time cloud-cover distribution and long-term statistical data are essential for astronomical siting and telescope operations. Visual inspection is currently the primary approach for analyzing cloud distribution at ground-based astronomical sites. However, the main disadvantages of manual observation methods are human subjectivity, heavy workloads, and poor real-time performance. Therefore, a real-time automatic cloud image classification method is desperately needed. This paper presents a novel cloud identification method named the PSO+XGBoost model, which combines eXtreme Gradient Boosting (XGBoost) with particle-swarm optimization (PSO). The entire cloud image is divided into 37 sub-regions to identify the distribution of the clouds more precisely. Nineteen features, including the sky background, star density, lighting conditions, and subregion grayscale values, are extracted. The experimental results have shown that the overall classification accuracy is 96.91%, and our model can outperform several state-of-the-art baseline methods. Our approach achieves high accuracy in comparison with the manual observation methods. Moreover, this method meets telescope real-time scheduling requirements.

Effects of Equatorial Plasma Bubbles on Multi-GNSS Signals: A Case Study over South China

Equatorial plasma bubbles (EPBs) occur frequently in low-latitude areas and have a non-negligible impact on navigation satellite signals. To systematically analyze the effects of a single EPB event on multi-frequency signals of GPS, Galileo, GLONASS, and BDS, all-sky airglow images over South China are jointly used to visually determine the EPB structure and depletion degree. The results reveal that scintillations, or GNSS signal fluctuations, are directly linked to EPBs and that the intensity of scintillation is positively correlated with the airglow depletion intensity. The center of the airglow depletion often corresponds to stronger GNSS scintillation, while the edge of the bubble, which is considered to have the largest density gradient, corresponds to relatively smaller scintillation instead. This work also systematically analyzes the responses of multi-constellation and multi-frequency signals to EPBs. The results show that the L2 and L5 frequencies are more susceptible than the L1 frequency is. For different constellations, Galileo’s signal has the best tracking stability during an EPB event compared with GPS, GLONASS, and BDS. The results provide a reference for dual-frequency signal selection in precise positioning or TEC calculation, that is, L1C and L2L for GPS, L1C and L5Q for Galileo, L1P and L2C for GLONASS, and L1P and L5P for BDS. Notably, BDS-2 is significantly weaker than BDS-3. And inclined geosynchronous orbit (IGSO) satellites have abnormal data error rates, which should be related to the special signal path trajectory of the IGSO satellite.

Post-midnight purple arc and patches appeared on the high latitude part of the auroral oval: Dawnside counterpart of STEVE?

The phenomenon known as strong thermal emission velocity enhancement (STEVE) is a purple/mauve arc-shaped atmospheric glow observed at lower latitudes of the auroral oval on the duskside. Simultaneous observations using a ground-based camera and a low-altitude satellite have shown that STEVE is accompanied by rapid westward ion flows. Such fast ion flows are termed the subauroral ion drift (SAID) or subauroral polarization stream (SAPS). Similarly, an eastward fast ion flow known as the dawnside auroral polarization stream (DAPS) is observed within the Region 1 current on the dawnside. If the optical phenomenon triggered by SAID/SAPS corresponds to STEVE, a comparable optical phenomenon should be driven by DAPS. Thus far, however, such a phenomenon has not been reported. This study discovers, for the first time, a purple-colored optical phenomenon characterized by the fast eastward ion flows, a possible signature of DAPS, occurring poleward of the bright green arc in the post-midnight sector. We present color all-sky images obtained by a ground-based commercial digital camera, along with wide-coverage optical measurements and in-situ data from low-altitude satellites. The results imply that this glow requires not only a high-speed ion flow but also its sharp latitudinal gradient at the boundary between the Region 1 and 2.Graphical

Satellite Visibility During the 2024 April Total Eclipse

On 2024 April 8, tens of millions of people across North America will be able to view a total solar eclipse. Such astronomical events have been important throughout history, but with nearly 10,000 satellites in orbit, we question whether total eclipses will now reveal a sky full of satellites, fundamentally changing this experience for humanity. Using the current population of Starlink satellites, we find that the brightest satellites would be naked-eye visible in dark skies, but the high sky brightness during totality will make them undetectable to the unaided eye. Our model does not take into account the effects of chance reflections from large, flat surfaces like solar panels, which we expect will cause glints and flares that could be visible from large satellites and abandoned rocket bodies. Time-lapse all-sky imaging might reveal satellites during the eclipse.