Space-weather HMI Active Region Patches Research Articles

Multiclass Classification of Solar Flares in Imbalanced Data Using Ensemble Learning and Sampling Methods

Solar flares are intense bursts of radiation across the electromagnetic spectrum on the surface of the Sun. They are categorized into four classes: B, C, M, and X, depending on their intensity, with X-class flares being the strongest. Being able to predict a flare’s class before its occurrence is critical for anticipating the severity of its impact on Earth. We used the Space-weather HMI Active Region Patches (SHARP) parameters available from Stanford’s Joint Science Operations Center (JSOC) to train machine learning models to classify these flares. However, predicting the flare class is a challenging task, as it is a multiclass classification probleminvolving imbalanced data due to the small number of X-class flares in a solar cycle. We propose a new method that uses a combination of random undersampling and the synthetic minority oversampling technique (SMOTE) to combat the imbalanced data problem. Furthermore, we develop an ensemble algorithm that uses nine classifiers as base learners and logistic regression as meta-learner. Experimental results show that the proposed method is effective in predicting solar flares, especially the most intense X-class flares, within the next 24 hours.

Forecasting solar flares with a transformer network

Space weather phenomena, including solar flares and coronal mass ejections, have significant influence on Earth. These events can cause satellite orbital decay due to heat-induced atmospheric expansion, disruption of GPS navigation and telecommunications systems, damage to satellites, and widespread power blackouts. The potential of flares and associated events to damage technology and disrupt human activities motivates prediction development. We use Transformer networks to predict whether an active region (AR) will release a flare of a specific class within the next 24 h. Two cases are considered: ≥C-class and ≥M-class. For each prediction case, separate models are developed. We train the Transformer to use time-series data to classify 24- or 48-h sequences of data. The sequences consist of 18 physical parameters that characterize an AR from the Space-weather HMI Active Region Patches data product. Flare event information is obtained from the Geostationary Operational Environmental Satellite flare catalog. Our model outperforms a prior study that similarly used only 24 h of data for the ≥C-class case and performs slightly worse for the ≥M-class case. When compared to studies that used a larger time window or additional data such as flare history, results are comparable. Using less data is conducive to platforms with limited storage, on which we plan to eventually deploy this algorithm.

Computation of Winding-Based Magnetic Helicity and Magnetic Winding Density for SHARP Magnetograms in Spherical Coordinates

Magnetic helicity has been used widely in the analysis and modelling of solar active regions. However, it is difficult to evaluate and interpret helicity in spherical geometry since coronal magnetic fields are rooted in the photosphere and helicity is susceptible to gauge choices. Recent work extended a geometrical definition of helicity from Cartesian to spherical domains, by interpreting helicity as the average, flux-weighted pairwise winding of magnetic-field lines. In this paper, by adopting the winding-based definition of helicity, we compute helicity and winding in spherical coordinates for SHARP (Spaceweather HMI Active Region Patches) magnetograms. This is compared with results obtained in Cartesian coordinates to quantitatively investigate the effect of spherical geometry. We find that the Cartesian approximations remain mostly valid, but for active regions with large spatial extents or strong field strengths (usually leading to flares and coronal mass ejections) there are significant deviations due to surface curvature that must be accounted for.

Two-stage Hierarchical Framework for Solar Flare Prediction

Solar flares, often accompanied by coronal mass ejections and other solar phenomena, are one of the most important sources affecting space weather. It is important to investigate the forecast approach of solar flares to mitigate their destructive effect on the Earth. Statistical analysis, associated with data from 2010 to 2017 in Space-weather HMI Active Region Patches (SHARPs) collected by the Solar Dynamics Observatory's Helioseismic and Magnetic Imager, reveals that there is a distribution divergence between the two types of active regions (ARs) of solar flares. A two-stage hierarchical prediction framework is formulated to better utilize this intrinsic distribution information. Specially, we pick up the ARs where at least one solar flare event occurs within the next 48 hr as flaring ARs through balanced random forest and naive Bayesian methods and then predict the events from flaring ARs by a cascade module of learning models. The empirical evaluation of SHARPs data from 2016 to 2019 verifies the promising performance of our framework, e.g., 0.727 for the true skill statistic.

Operational prediction of solar flares using a transformer-based framework

Solar flares are explosions on the Sun. They happen when energy stored in magnetic fields around solar active regions (ARs) is suddenly released. Solar flares and accompanied coronal mass ejections are sources of space weather, which negatively affects a variety of technologies at or near Earth, ranging from blocking high-frequency radio waves used for radio communication to degrading power grid operations. Monitoring and providing early and accurate prediction of solar flares is therefore crucial for preparedness and disaster risk management. In this article, we present a transformer-based framework, named SolarFlareNet, for predicting whether an AR would produce a gamma-class flare within the next 24 to 72 h. We consider three gamma classes, namely the geM5.0 class, the geM class and the geC class, and build three transformers separately, each corresponding to a gamma class. Each transformer is used to make predictions of its corresponding gamma-class flares. The crux of our approach is to model data samples in an AR as time series and to use transformers to capture the temporal dynamics of the data samples. Each data sample consists of magnetic parameters taken from Space-weather HMI Active Region Patches (SHARP) and related data products. We survey flare events that occurred from May 2010 to December 2022 using the Geostationary Operational Environmental Satellite X-ray flare catalogs provided by the National Centers for Environmental Information (NCEI), and build a database of flares with identified ARs in the NCEI flare catalogs. This flare database is used to construct labels of the data samples suitable for machine learning. We further extend the deterministic approach to a calibration-based probabilistic forecasting method. The SolarFlareNet system is fully operational and is capable of making near real-time predictions of solar flares on the Web.

A Large-Scale Dataset of Three-Dimensional Solar Magnetic Fields Extrapolated by Nonlinear Force-Free Method

It has been widely accepted that solar magnetic field manipulates all solar activities, especially violent solar bursts in solar corona. Thus, it is extremely important to reconstruct three-dimentional (3D) magnetic field of solar corona from really observed photospheric magnetogram. In this paper, a large-scale dataset of 3D solar magnetic fields of active regions is built by using the nonlinear force-free magnetic field (NLFFF) extrapolation from vector magnetograms of Helioseismic and Magnetic Imager (HMI) on Solar Dynamics Observatory (SDO). In this dataset, all space-weather HMI active region patches (SHARPs) with the corresponding serial numbers of national oceanic and atmospheric administration (NOAA) are included. They are downloaded from the SHARP 720 s series of JSOC every 96 minutes. In addition, each sample is labelled with a finer grained label for solar flare forecast. This paper is with the purpose of open availability of data resource and source code to the peers for refraining from repeated labor of data preparation. Meanwhile, with such a large-scale, high spatio-temporal resolution and high quality scientific data, we anticipate a wide attention and interest from artificial intelligence (AI) and computer vision communities, for exploring AI for astronomy over such a large-scale dataset.

Association Between Magnetic Pressure Difference and the Movement of Solar Pores

Solar pores are closely related to the concentration, dissipation, and transportation of solar magnetic flux. Their observable characteristics can provide constraints on models and simulations of magnetic flux emergence and formation. The specific property investigated in this study is their horizontal movement. The aim is to investigate whether the movement is correlated with any observable quantities. Our statistical analysis of 61 compact pores identified from the Spaceweather HMI Active Region Patches (SHARP) from 2011 to 2018 indicates that the direction of movement is often either parallel or antiparallel to the direction of maximum magnetic pressure difference (dP mag) at the opposite sides of the edges of the pores. The correlation coefficients for both the parallel and antiparallel cases are higher than 0.74. Despite the high correlation, our analysis using the transfer entropy indicates no significant causal relationship between the direction of motion and the direction of maximum dP mag.

Solar Flare Index Prediction Using SDO/HMI Vector Magnetic Data Products with Statistical and Machine-learning Methods

Solar flares, especially the M- and X-class flares, are often associated with coronal mass ejections. They are the most important sources of space weather effects, which can severely impact the near-Earth environment. Thus it is essential to forecast flares (especially the M- and X-class ones) to mitigate their destructive and hazardous consequences. Here, we introduce several statistical and machine-learning approaches to the prediction of an active region’s (AR) flare index (FI) that quantifies the flare productivity of an AR by taking into account the number of different class flares within a certain time interval. Specifically, our sample includes 563 ARs that appeared on the solar disk from 2010 May to 2017 December. The 25 magnetic parameters, provided by the Space-weather HMI Active Region Patches (SHARP) from the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory, characterize coronal magnetic energy stored in ARs by proxy and are used as the predictors. We investigate the relationship between these SHARP parameters and the FI of ARs with a machine-learning algorithm (spline regression) and the resampling method (Synthetic Minority Oversampling Technique for Regression with Gaussian Noise). Based on the established relationship, we are able to predict the value of FIs for a given AR within the next 1 day period. Compared with other four popular machine-learning algorithms, our methods improve the accuracy of FI prediction, especially for a large FI. In addition, we sort the importance of SHARP parameters by the Borda count method calculated from the ranks that are rendered by nine different machine-learning methods.

Solar flare catalog based on SDO/AIA EUV images: Composition and correlation with GOES/XRS X-ray flare magnitudes

Supervised Machine Learning (ML) models for solar flare prediction rely on accurate labels for a given input data set, commonly obtained from the GOES/XRS X-ray flare catalog. With increasing interest in utilizing ultraviolet (UV) and extreme ultraviolet (EUV) image data as input to these models, we seek to understand if flaring activity can be defined and quantified using EUV data alone. This would allow us to move away from the GOES single pixel measurement definition of flares and use the same data we use for flare prediction for label creation. In this work, we present a Solar Dynamics Observatory (SDO) Atmospheric Imaging Assembly (AIA)-based flare catalog covering flare of GOES X-ray magnitudes C, M and X from 2010 to 2017. We use active region (AR) cutouts of full disk AIA images to match the corresponding SDO/Helioseismic and Magnetic Imager (HMI) SHARPS (Space weather HMI Active Region Patches) that have been extensively used in ML flare prediction studies, thus allowing for labeling of AR number as well as flare magnitude and timing. Flare start, peak, and end times are defined using a peak-finding algorithm on AIA time series data obtained by summing the intensity across the AIA cutouts. An extremely randomized trees (ERT) regression model is used to map SDO/AIA flare magnitudes to GOES X-ray magnitude, achieving a low-variance regression. We find an accurate overlap on 85% of M/X flares between our resulting AIA catalog and the GOES flare catalog. However, we also discover a number of large flares unrecorded or mislabeled in the GOES catalog.

Knowledge‐Informed Deep Neural Networks for Solar Flare Forecasting

AbstractRecently, although various deep learning techniques have been applied to building space weather prediction models, a large amount of relevant prior knowledge of solar eruptions and magnetic properties is ignored during the model development. By integrating prior knowledge in flare production into the convolutional neural network (CNN) structures, we have developed a knowledge‐informed deep neural network model aiming at forecasting solar flares. The line‐of‐sight magnetograms of Space‐weather HMI Active Region Patches (SHARP) from May 2010 to December 2018 are selected. We have surveyed the relationships between solar flares and both the active region (AR) area and magnetic type. When integrating prior knowledge into the CNNs, three aspects are considered: (a) keeping the magnetic structure unchanged (data preprocessing) while filling SHARP magnetograms into squares, (b) grouping the data samples into two subsets according to different flare productivities (sample grouping), and (c) adding AR area as an extra input parameter to the CNN (extra input parameter implementation). Pure CNN model, Fusion model 1, and Fusion model 2 are built to forecast M‐class or above flares in the next 48 hr, which involve data preprocessing, data preprocessing and sample grouping, and all the three aspects, respectively. Fusion model 2 that augments the most prior knowledge has the best performance. Our results imply that prior knowledge can play an important role in building deep learning flare forecasting models. In the future, adopting knowledge‐informed deep neural networks will be an effective way to further improve the forecasting performance for other space weather events.

Studying the conditions for magnetic reconnection in solar flares with and without precursor flares

Forecasting of solar flares remains a challenge due to the limited understanding of the underlying physical processes of these events, in particular magnetic reconnection. Studies have indicated that changes to the photospheric magnetic fields associated with magnetic reconnection – particularly in relation to the field helicity – occur during solar flare events. This study utilized data from the Solar Dynamics Observatory (SDO) Helioseismic and Magnetic Imager (HMI) and SpaceWeather HMI Active Region Patches (SHARPs) to analyze full vector-field component data of the photospheric magnetic field during solar flare events within a near decade long HMI dataset. Analysis of the data was used to identify and compare the trends of differing flare classes for varying time intervals leading up to an event, as well as the trends of flares that occur with and without a precursor flare, in order to discern signatures of the physical mechanisms involved. The data suggests that active regions that produce flares with precursors are continuing to evolve and appreciably more complex than those that produce a flare without a precursor. Additionally, precursor flares were found to enhance the shear across an active region, helping set up the conditions necessary for a larger solar flare to occur. Ultimately none of the SHARP parameters showed a distinct signature of magnetic reconnection.

SMARPs and SHARPs: Two Solar Cycles of Active Region Data

Abstract We present a new data product, called Space-Weather MDI Active Region Patches (SMARPs), derived from maps of the solar surface magnetic field taken by the Michelson Doppler Imager on board the Solar and Heliospheric Observatory. Together with the Space-Weather HMI Active Region Patches (SHARPs), derived from similar maps taken by the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory, these data provide a continuous and seamless set of maps and keywords that describe every active region observed over the last two solar cycles, from 1996 to the present day. In this paper, we describe the SMARP data and compare it to the SHARP data.

Case Studies of Photospheric Magnetic Field Properties of Active Regions Associated with X-class Solar Flares

Solar flares are a transient phenomenon occurred in the active region (AR) on the Sun’s surface, producing intense emissions in EUV and soft X-ray that can wreak havoc in the near-Earth space mission and satellite as well as radio-based communication and navigation. The ARs are accompanied with strong magnetic fields and manifested as dark spots on the photosphere. To understand the photospheric magnetic field properties of the ARs that produce intense flares, two ARs associated with X-class flares, namely AR 12192 and AR 12297, occurred respectively on 25 October 2014 and 11 March 2015, are studied in terms of magnetic classification and various physical magnetic parameters. Solar images from the Langkawi National Observatory (LNO) and physical magnetic parameters from the Space-weather HMI Active Region Patches (SHARP) are used in this study. A total of seven SHARP magnetic parameters are examined which are calculated as sums of various magnetic quantities and have been identified as useful predictors for flare forecast. These two ARs are classified as βγδ sunspots whereas their formation and size are quite different from each other. Our results showed that the intensity of a flare has little relationship with the area of an AR and the magnetic free energy; and the temporal variation of individual magnetic parameter has no obvious and consistent pre-flare feature. It is concluded that the temporal variation of individual magnetic parameter may not be useful for predicting the onset time of a flare.

Detection of reconnection signatures in solar flares

Solar flare forecasting is limited by the current understanding of mechanisms that govern magnetic reconnection, the main physical phenomenon associated with these events. As a result, forecasting relies mainly on climatological correlations to historical events rather than the underlying physics principles. Solar physics models place the neutral point of the reconnection event in the solar corona. Correspondingly, studies of photospheric magnetic fields indicate changes during solar flares—particularly in relation to the field helicity—on the solar surface as a result of the associated magnetic reconnection. This study utilizes data from the Solar Dynamics Observatory (SDO) Helioseismic and Magnetic Imager (HMI) and SpaceWeather HMI Active Region Patches (SHARPs) to analyze full vector-field component data of the photospheric magnetic field during solar flares within a large HMI dataset (May 2010 through September 2019). This analysis is then used to identify and compare trends in the different categories of flare strengths and determine indications of the physical phenomena taking place.

Predicting Solar Flares Using a Novel Deep Convolutional Neural Network

Abstract Space weather forecasting is very important, and the prediction of space weather, especially for solar flares, has increasingly attracted research interests with the numerous recent breakthroughs in machine learning. In this study, we propose a novel convolutional neural network (CNN) model to make binary class prediction for both ≥C-class and ≥M-class flares within 24 hr. We collect magnetogram samples of solar active regions (ARs) provided by the Space-weather Helioseismic and Magnetic Imager Active Region Patches (SHARP) data from 2010 May to 2018 September. These samples are used to construct 10 separate data sets. Then, after training, validating, and testing our model, we compare the results of our model with previous studies in several metrics, with a focus on the true skill statistic (TSS). The major results are summarized as follows. (1) We propose a method of shuffle and split cross-validation (CV) based on AR segregation, which is the first attempt to verify the validity and stability of the model in flare prediction. (2) The proposed CNN model achieves a relatively high score of TSS = 0.749 ± 0.079 for ≥M-class prediction, and TSS = 0.679 ± 0.045 for ≥C-class prediction, which is greatly improved compared with previous studies. (3) The model trained on 10 CV data sets is considerably robust and stable in making flare prediction for both ≥C class and ≥M class. Our experimental results indicate that our proposed CNN model is a highly effective method for flare forecasting, with quite excellent prediction performance.

Predicting Solar Flares Using a Long Short-term Memory Network

Abstract We present a long short-term memory (LSTM) network for predicting whether an active region (AR) would produce a ϒ-class flare within the next 24 hr. We consider three ϒ classes, namely ≥M5.0 class, ≥M class, and ≥C class, and build three LSTM models separately, each corresponding to a ϒ class. Each LSTM model is used to make predictions of its corresponding ϒ-class flares. The essence of our approach is to model data samples in an AR as time series and use LSTMs to capture temporal information of the data samples. Each data sample has 40 features including 25 magnetic parameters obtained from the Space-weather HMI Active Region Patches and related data products as well as 15 flare history parameters. We survey the flare events that occurred from 2010 May to 2018 May, using the Geostationary Operational Environmental Satellite X-ray flare catalogs provided by the National Centers for Environmental Information (NCEI), and select flares with identified ARs in the NCEI flare catalogs. These flare events are used to build the labels (positive versus negative) of the data samples. Experimental results show that (i) using only 14–22 most important features including both flare history and magnetic parameters can achieve better performance than using all 40 features together; (ii) our LSTM network outperforms related machine-learning methods in predicting the labels of the data samples. To our knowledge, this is the first time that LSTMs have been used for solar-flare prediction.

A statistical study of photospheric magnetic field properties of active regions associated with M- and X-class flares using SDO/HMI vector magnetic field data

With the aim of understanding the photospheric magnetic field properties of active regions (ARs) that produce M-class or greater flares, physical magnetic parameters of ARs provided by the Space-weather HMI Active Region Patches (SHARP) are examined. A total of 223 flares are collected in 83 ARs, including 31 X- and 192 M-class flares. In this study, twelve SHARP parameters are analyzed, which are the sums of various physical quantities (e.g., magnetic flux, magnetic free energy density, vertical current, current helicity, shear angle, and Lorentz force). For X- and M-class flares, good correlation coefficients (≳ 0.90) are obtained among the seven magnetic parameters, namely, total unsigned quantities of (1) magnetic flux, (2) vertical current, (3) current helicity, and (4) flux near polarity inversion line, as well as (5) total photospheric magnetic free energy density, (6) total magnitude of Lorentz force (TOTBSQ), and (7) area of AR. A threshold for low bound is estimated for each of the seven magnetic parameters based on X-class flares. Only ∼5% of M-flares occur below the estimated thresholds. A time difference (TSF) is calculated between the time when all the magnetic parameter thresholds but TOTBSQ are met (T1) and the time when the flare starts. The TOTBSQ is excluded in the T1 calculation because its value is often missing in the interval of interest. Results show that (1) ∼77% of X-flares occur within 7 days after T1, with an occurrence peak at TSF=2 (∼20% probability), (2) ∼78% of M-flares also occur within 7 days after T1, with an occurrence peak at TSF=4 (∼20% probability), and (3) no major flare occurs after TSF=11. It is found that the GOES soft X-ray peak flux has a very weak relationship with the magnetic free energy density. This weak relationship can be interpreted in terms of the energy partition in a flare. In conclusion, the obtained seven thresholds will be useful for monitoring an AR that has a high potential to produce an M- or X-class flare.

Forecasting Solar Flares Using Magnetogram-based Predictors and Machine Learning

We propose a forecasting approach for solar flares based on data from Solar Cycle 24, taken by the Helioseismic and Magnetic Imager (HMI) on board the Solar Dynamics Observatory (SDO) mission. In particular, we use the Space-weather HMI Active Region Patches (SHARP) product that facilitates cut-out magnetograms of solar active regions (AR) in the Sun in near-real-time (NRT), taken over a five-year interval (2012 - 2016). Our approach utilizes a set of thirteen predictors, which are not included in the SHARP metadata, extracted from line-of-sight and vector photospheric magnetograms. We exploit several Machine Learning (ML) and Conventional Statistics techniques to predict flares of peak magnitude >M1 and >C1, within a 24 h forecast window. The ML methods used are multi-layer perceptrons (MLP), support vector machines (SVM) and random forests (RF). We conclude that random forests could be the prediction technique of choice for our sample, with the second best method being multi-layer perceptrons, subject to an entropy objective function. A Monte Carlo simulation showed that the best performing method gives accuracy ACC=0.93(0.00), true skill statistic TSS=0.74(0.02) and Heidke skill score HSS=0.49(0.01) for >M1 flare prediction with probability threshold 15% and ACC=0.84(0.00), TSS=0.60(0.01) and HSS=0.59(0.01) for >C1 flare prediction with probability threshold 35%.

Non-neutralized Electric Currents in Solar Active Regions and Flare Productivity

We explore the association of non-neutralized currents with solar flare occurrence in a sizable sample of observations, aiming to show the potential of such currents in solar flare prediction. We use the regularly produced high quality vector magnetograms by the Helioseismic Magnetic Imager and more specifically the Space weather HMI Active Region Patches (SHARP). Through a newly established method, that incorporates detailed error analysis, we calculate the non-neutralized currents contained in active regions (AR). Two predictors are produced, namely the total and the maximum unsigned non-neutralized current. Both are tested in AR time-series and a representative sample of point-in-time observations during the interval 2012-2016. The average values of non-neutralized currents in flaring active regions are by more than an order of magnitude higher than in non-flaring regions and correlate very well with the corresponding flare index. The temporal evolution of these parameters appears to be connected to physical processes, such as flux emergence and/or magnetic polarity inversion line formation that are associated with increased solar flare activity. Using Bayesian inference of flaring probabilities, it is shown that the total unsigned non-neutralized current outperforms significantly the total unsigned magnetic flux and other well established current-related predictors. Thus, it shows good prospects for inclusion in an operational flare forecasting service. We plan to use the new predictor in the framework of the FLARECAST project along with other highly performing predictors.

Detailed analysis of dynamic evolution of three Active Regions at the photospheric level before flare and CME occurrence

We present a combined analysis of the applications of the weighted horizontal magnetic gradient (denoted as WGM in Korsós et al. (2015)) method and the magnetic helicity tool (Berger and Field, 1984) employed for three active regions (ARs), namely NOAA AR 11261, AR 11283 and AR 11429. We analysed the time series of photospheric data from the Solar Dynamics Observatory taken between August 2011 and March 2012. During this period the three ARs produced a series of flares (eight M- and six X-class) and coronal mass ejections (CMEs). AR 11261 had four M-class flares and one of them was accompanied by a fast CME. AR 11283 had similar activities with two M- and two X-class flares, but only with a slow CME. Finally, AR 11429 was the most powerful of the three ARs as it hosted five compact and large solar flare and CME eruptions. For applying the WGM method we employed the Debrecen sunspot data catalogue, and, for estimating the magnetic helicity at photospheric level we used the Space-weather HMI Active Region Patches (SHARP’s) vector magnetograms from SDO/HMI (Solar Dynamics Observatory/Helioseismic and Magnetic Imager). We followed the evolution of the components of the WGM and the magnetic helicity before the flare and CME occurrences. We found a unique and mutually shared behaviour, called the U-shaped pattern, of the weighted distance component of WGM and of the shearing component of the helicity flux before the flare and CME eruptions. This common pattern is associated with the decreasing-receding phases yet reported only known to be a necessary feature prior to solar flare eruption(s) but found now at the same time in the evolution of the shearing helicity flux. This result leads to the conclusions that (i) the shearing motion of photospheric magnetic field may be a key driver for solar eruption in addition to the flux emerging process, and that (ii) the found decreasing-approaching pattern in the evolution of shearing helicity flux may be another precursor indicator for improving the forecasting of solar eruptions.