Gridded Data Research Articles

Crop sequence boundaries using USDA National Agricultural Statistics Service historic cropland data layers

Gridded landcover datasets like the NASS Cropland Data Layer (CDL) provide a useful resource for analyses of cropland management. However, many farm operation decisions are made at the field level, not the pixel level. To capture relationships between land cover and field characteristics – size, contiguity, etc. – some method is needed to aggregate gridded data into crop fields. To provide a uniform and consistent approach for aggregation of gridded data at the field level over a series of years, this research project developed a set of Crop Sequence Boundaries (CSBs), which are polygons that delineate areas of homogeneous cropping sequences for the contiguous US. The CSBs are open-sourced algorithm-based, geospatial polygons derived using historic CDLs together with road and rail networks to capture areas with common cropping sequences. The CSB approach used geospatial functions in Google Earth Engine (GEE) and in the ArcGIS Pro application. These geospatial functions are run in parallel by sub-dividing the contiguous US into smaller regions based on road and rail boundaries to prevent overlaps or gaps in the data. As a new set of algorithmically delineated field polygons, the CSBs enhance applications requiring large-scale crop mapping with vector-based data.

A Prediction‐Traversal Approach for Compressing Scientific Data on Unstructured Meshes with Bounded Error

AbstractWe explore an error‐bounded lossy compression approach for reducing scientific data associated with 2D/3D unstructured meshes. While existing lossy compressors offer a high compression ratio with bounded error for regular grid data, methodologies tailored for unstructured mesh data are lacking; for example, one can compress nodal data as 1D arrays, neglecting the spatial coherency of the mesh nodes. Inspired by the SZ compressor, which predicts and quantizes values in a multidimensional array, we dynamically reorganize nodal data into sequences. Each sequence starts with a seed cell; based on a predefined traversal order, the next cell is added to the sequence if the current cell can predict and quantize the nodal data in the next cell with the given error bound. As a result, one can efficiently compress the quantized nodal data in each sequence until all mesh nodes are traversed. This paper also introduces a suite of novel error metrics, namely continuous mean squared error (CMSE) and continuous peak signal‐to‐noise ratio (CPSNR), to assess compression results for unstructured mesh data. The continuous error metrics are defined by integrating the error function on all cells, providing objective statistics across nonuniformly distributed nodes/cells in the mesh. We evaluate our methods with several scientific simulations ranging from ocean‐climate models and computational fluid dynamics simulations with both traditional and continuous error metrics. We demonstrated superior compression ratios and quality than existing lossy compressors.

Data assimilation with machine learning for constructing gridded rainfall time series data to assess long-term rainfall changes in the northeastern regions in India

ABSTRACT Data scarcity and unavailability of observed rainfalls in the northeastern states of India limit prediction of extreme hydro-climatological changes. To fill this gap, a data assimilation approach has been applied to re-construct accurate high-resolution gridded (5 km2) daily rainfall data (2001–2020), which include seasonality assessment, statistical evaluation, and bias correction. Random forest (RF) and support vector regression were used to predict rainfall time series, and a comparison between machine learning and data assimilation-based gridded rainfall data was performed. Five gridded rainfall datasets, namely, Indian Monsoon Data Assimilation and Analysis (IMDAA) (12 km2), APHRODITE (25 km2), India Meteorological Department (25 km2), PRINCETON (25 km2), and CHIRPS (25 and 5 km2), have been utilized. For re-constructed rainfall datasets (5 km2), the comparative seasonality and change assessment have been performed with respect to other rainfall datasets. CHIRPS and APHRODITE datasets have shown better similarities with IMDAA. The RF and assimilated rainfall (AR) have superiority based on bias and extremity, and AR data were recognized as the best accurate data (>0.8). Precipitation change analysis (2021–2100) performed utilizing the bias corrected and downscaled CMIP6 datasets showed that the dry spells will be enhanced. Considering the CMIP6 moderate emission scenario, i.e., SSP245, the wet spell will be enhanced in future; however, when considering SSP585 (representing the extreme worst case), the wet spells will be decreased.

Method for Delineating the Formula Limit of the Continental Shelf under the Maximum Area Principle Constraint

In current practices of determining continental shelf area, the measured sediment thickness data do not effectively reflect the distribution of sediments across the area due to its dispersed nature. This issue raises potential limitations in unknown optimal survey line layout strategies for maximizing the posterior determination area. This paper adopts the binary search algorithm, relies on existing sediment thickness grid data, and uses geodetic formulas to build an ellipsoidal surface grid distance calculation model. This model quickly screens potential areas for the 1% sediment thickness line candidate points set. By constraining the azimuth parameter values during the construction process of the ellipsoidal point feature buffer zones, efficiently select the candidate points set for the 1% sediment thickness line. Furthermore, by elucidating the essential meanings of points on the formula limit and combining the candidate points set of the foot of the continental slope (FOS)+60 n mile line, the polygon minimal convex hull construction technique and a baseline points optimization algorithm with a length threshold are used to efficiently select points on formula limit. Experimental results demonstrate that this method can effectively assist coastal states in optimizing the determination of continental shelf area to the fullest extent under the length threshold requirements of the United Nations Convention on the Law of the Sea. Experiments have proven that compared to the traditional intersection method, the method presented in this paper can help coastal countries delineate a larger continental shelf area. In typical application scenarios, the gain in area can reach 77,278,427 m2 accounting for 0.51% of the total area.

Long-term effects of early life rainfall shocks on foundational cognitive skills: Evidence from Peru

Global warming is changing precipitation patterns, particularly harming communities in low-and-middle income countries (LMICs). Whilst the long-term effects of being exposed to rainfall shocks early in life on school-achievement tests are well-established, there is little population-based evidence from LMICs on the mechanisms through which these shocks operate. Executive functions (EFs) are key for children’s learning abilities. This paper analyses the effects of early exposure to rainfall shocks on four foundational cognitive skills (FCSs), including EFs that have been found to be key predictors of educational success. These skills were measured via a series of tablet-based tasks administered in Peru as part of the Young Lives longitudinal study (YLS). We combine the YLS data with gridded data on monthly precipitation to generate monthly, community-level rainfall shock estimates. The key identification strategy relies on temporary climatic shocks being uncorrelated with other latent determinants of FCSs development. Our results show significant negative effects of early life exposure to rainfall shocks on EFs—especially, on working memory—measured in later childhood. We also find evidence of rainfall shocks decreasing households’ abilities to invest in human capital, which may affect both FCSs and domain-specific test scores. Finally, there is suggestive, but not conclusive, evidence that a conditional-cash-transfer program providing poor households with additional financial resources might partially offset the effects of the rainfall shocks.

Evasive attacks against autoencoder-based cyberattack detection systems in power systems

The digital transformation process of power systems towards smart grids is resulting in improved reliability, efficiency and situational awareness at the expense of increased cybersecurity vulnerabilities. Given the availability of large volumes of smart grid data, machine learning-based methods are considered an effective way to improve cybersecurity posture. Despite the unquestionable merits of machine learning approaches for cybersecurity enhancement, they represent a component of the cyberattack surface that is vulnerable, in particular, to adversarial attacks. In this paper, we examine the robustness of autoencoder-based cyberattack detection systems in smart grids to adversarial attacks. A novel iterative-based method is first proposed to craft adversarial attack samples. Then, it is demonstrated that an attacker with white-box access to the autoencoder-based cyberattack detection systems can successfully craft evasive samples using the proposed method. The results indicate that naive initial adversarial seeds cannot be employed to craft successful adversarial attacks shedding insight on the complexity of designing adversarial attacks against autoencoder-based cyberattack detection systems in smart grids.

Development of Alternate Climate Divisions for Colorado Based on Gridded Data

The official climate divisions for the contiguous United States are used for a wide range of purposes, including ongoing climate monitoring, and through NOAA’s long-standing nClimDiv dataset. In Colorado, the climate divisions are based around the basins of the large rivers that flow out of the state. However, considering the complex topography and climate of the state, these divisions do not always represent key climate variations and changes. This study builds upon an approach first developed by Wolter and Allured to establish alternate climate divisions that more closely reflect observed climate variability across Colorado. Hierarchical cluster analysis is applied to gridded temperature and precipitation data (NOAA’s nClimGrid) from 1950-2021 to identify areas with similar climate variability, then manual inspection is used to establish 11 divisions. These resulting divisions are being used in an updated state-level climate change assessment. The method is flexible and uses open-source tools that could be extended to other regions or datasets.

Research on the Analysis Method of Measured Pressure Data during the Transient Process of Load Rejection in Hydropower Stations

Abstract This paper proposes a method to analyze the measured pressure data in the transition process of hydropower station. The method first separates the measured pressure data to the mean pressure and pressure fluctuation data, then divides the pressure fluctuation data into different time domain grids, and performs the confidence probability processing on the pressure fluctuation data in each time grid, and finally obtains the credible measured pressure data of transient process. In this paper, the specific implementation steps of the method are introduced by taking the results of a load rejection test of a pumped storage power station as an example.

Evaluation of Grid-Based Aridity Indices in Classifying Aridity Zones in Iraq

In this study, the aridity index (AI) based on gridded climate data was validated for defining aridity and classifying aridity zones in Iraq through comparison with the results obtained by the station-based aridity index. Gauge-based gridded climate data taken from Climatic Research Unit Timeseries (CRU TS) were used to determine the annual value of four aridity indices (Lang, De Martonne, Ernic and UNEP AI) over the period 1998-2011. The results showed that the aridity distribution maps derived using grid-based aridity indices were reasonably close to those found using station-based ones. The four aridity indices properly identified similar aridity (dryness) classifications in both the station-based and grid-based aridity maps. The area percentage of each aridity class predicted by grid-based AIs was also compared with that obtained by the station-based AIs. The results showed that the variances between the area percentages predicted by grid-based AIs and those estimated using station-based AIs are fairly slight. The Lang AI exhibited the least variance (0.4%) while the De Martonne AI had the biggest variance (-4.8%). Despite these minor variances, it is however possible to conclude that the grid-based aridity index classified the aridity zones of Iraq as properly as the station-based aridity index did.

Demand side management with wireless channel impact in IoT-enabled smart grid system

Demand Side Management (DSM) is a vital issue in smart grids, given the time-varying user demand for electricity and power generation cost over a day. On the other hand, wireless communications with ubiquitous connectivity and low latency have emerged as a suitable option for smart grid. The design of any DSM system using a wireless network must consider the wireless link impairments, which is missing in existing literature. In this paper, we propose a DSM system using a Real-Time Pricing (RTP) mechanism and a wireless Neighborhood Area Network (NAN) with data transfer uncertainty. A Zigbee-based Internet of Things (IoT) model is considered for the communication infrastructure of the NAN. A sample NAN employing XBee and Raspberry Pi modules is also implemented in real-world settings to evaluate its reliability in transferring smart grid data over a wireless link. The proposed DSM system determines the optimal price corresponding to the optimum system welfare based on the two-way wireless communications among users, decision-makers, and energy providers. A novel cost function is adopted to reduce the impact of changes in user numbers on electricity prices. Simulation results indicate that the proposed system benefits users and energy providers. Furthermore, experimental results demonstrate that the success rate of data transfer significantly varies over the implemented wireless NAN, which can substantially impact the performance of the proposed DSM system. Further simulations are then carried out to quantify and analyze the impact of wireless communications on the electricity price, user welfare, and provider welfare.

Real Estate Website

Abstract: The motivation to develop Real Estate Web Application comes from my urge to learn Visual Studio .NET 2005 for building the business logic of the application, SQL server 2005 for database designing and using new web technologies like AJAX, Java Script for website designing. The most influential factor for selecting this application is to add some innovative features to the search engine of a Real Estate Website which can make the task of a property buyer easy to search for property listings. The most significant feature in these websites is the interactive search criterion which lets the buyer specify their requirements to get the correct set of records from the database. The search tool should be strong enough to include all the required features which a buyer may desire. Also, each search requires a post back call to the database to retrieve some set of records but making the website AJAX enabled prevents the annoying post backs. Thus, each call refreshes just the data grid to display the listings rather the whole web page. Some other new features like drag and drop tool to save the listings, retrieving related results for each search, etc make this application more feature rich.

Urban pollution: A global perspective

We use worldwide gridded satellite data to analyse how population size and density affect urban PM2.5 pollution. We find that more populated and denser grid cells are more exposed to pollution. However, across urban areas, exposure increases with cities’ population size but decreases with density. Moreover, the population effect is driven mostly by population commuting to core cities rather than the core city population itself. We analyse heterogeneity by geography and income levels. A counterfactual simulation shows that PM2.5 exposure could fall by up to 40% if population size were equalized across all cities within countries, but the relocation of population from large to small cities that maximizes welfare would be small.

Trends in intraseasonal temperature variability in Europe: Comparison of station data with gridded data and reanalyses

AbstractTrends in temperature variability are often referred to have higher effect on temperature extremes than trends in the mean. We investigate trends in three complementary measures of intraseasonal temperature variability: (a) standard deviation of mean daily temperature (SD), (b) mean absolute value of day‐to‐day temperature change (DTD) and (c) 1‐day lagged temporal autocorrelation of temperature (LAG). It is a well‐established fact that different types of data (station, gridded, reanalyses) provide different temperature characteristics and particularly their trends. Moreover, we have uncovered that trends in measures of variability are considerably sensitive to data inhomogeneities. Therefore, we use five different datasets, one station based (ECA&D), one gridded (EOBS) and three reanalyses (JRA‐55, NCEP/NCAR, 20CR), and compare them. The period from 1961 to 2014 where all datasets overlap is examined, and the linear regression method is utilized to calculate trends of investigated measures in summer and winter. Intraseasonal temperature variability tends to decrease in winter, especially in eastern and northern Europe, where trends below −7%·decade−1 are detected for all measures. Decreases in DTD and LAG (indicating increase in persistence) prevail also in summer while summer SD tends to increase. The increase in the width of temperature distribution and the simultaneous increase in persistence indicate a tendency towards the rise in the frequency of extended extreme events in summer. Unlike previous studies, our results imply that reanalyses are not the least accurate in determining trends. JRA‐55 appears to be the least diverging from other datasets, while the largest discrepancies are detected for DTD at station data.

A unified dataset for pre-processed climate indicators weighted by gridded economic activity

Although high-resolution gridded climate variables are provided by multiple sources, the need for country and region-specific climate data weighted by indicators of economic activity is becoming increasingly common in environmental and economic research. We process available information from different climate data sources to provide spatially aggregated data with global coverage for both countries (GADM0 resolution) and regions (GADM1 resolution) and for a variety of climate indicators (total precipitations, average temperatures, average SPEI). We weigh gridded climate data by population density, night-time light intensity, cropland, and concurrent population count – all proxies of economic activity – before aggregation. Climate variables are measured daily, monthly, and annually, covering (depending on the data source) a time window from 1900 (at the earliest) to 2023. We pipeline all the preprocessing procedures in a unified framework, and we validate our data through a systematic comparison with those employed in leading climate impact studies.

Enhanced Wind Field Spatial Downscaling Method Using UNET Architecture and Dual Cross-Attention Mechanism

Before 2008, China lacked high-coverage regional surface observation data, making it difficult for the China Meteorological Administration Land Data Assimilation System (CLDAS) to directly backtrack high-resolution, high-quality land assimilation products. To address this issue, this paper proposes a deep learning model named UNET_DCA, based on the UNET architecture, which incorporates a Dual Cross-Attention module (DCA) for multiscale feature fusion by introducing Channel Cross-Attention (CCA) and Spatial Cross-Attention (SCA) mechanisms. This model focuses on the near-surface 10-m wind field and achieves spatial downscaling from 6.25 km to 1 km. We conducted training and validation using data from 2020–2021, tested with data from 2019, and performed ablation experiments to validate the effectiveness of each module. We compared the results with traditional bilinear interpolation methods and the SNCA-CLDASSD model. The experimental results show that the UNET-based model outperforms SNCA-CLDASSD, indicating that the UNET-based model captures richer information in wind field downscaling compared to SNCA-CLDASSD, which relies on sequentially stacked CNN convolution modules. UNET_CCA and UNET_SCA, incorporating cross-attention mechanisms, outperform UNET without attention mechanisms. Furthermore, UNET_DCA, incorporating both Channel Cross-Attention and Spatial Cross-Attention mechanisms, outperforms UNET_CCA and UNET_SCA, which only incorporate one attention mechanism. UNET_DCA performs best on the RMSE, MAE, and COR metrics (0.40 m/s, 0.28 m/s, 0.93), while UNET_DCA_ars, incorporating more auxiliary information, performs best on the PSNR and SSIM metrics (29.006, 0.880). Evaluation across different methods indicates that the optimal model performs best in valleys, followed by mountains, and worst in plains; it performs worse during the day and better at night; and as wind speed levels increase, accuracy decreases. Overall, among various downscaling methods, UNET_DCA and UNET_DCA_ars effectively reconstruct the spatial details of wind fields, providing a deeper exploration for the inversion of high-resolution historical meteorological grid data.

Strategic Transportation Planning: Soft Mobility Network Planning Technical Report

This technical report presents a comprehensive approach to soft mobility network planning, focusing on assessing the potential for growth in bicycle infrastructure. Utilizing GIS-based analysis and multi-criteria decision-making, the study aims to identify and prioritize areas for new bike infrastructure development. The methodology involved defining influential factors, selecting variables from available datasets, assigning value levels, weighting parameters, and generating cycling infrastructure potential maps. Factors such as connectivity, safety, and attractiveness are considered, with each variable assessed and scored based on current conditions and ideal benchmarks. The results highlight high-potential areas for infrastructure improvement and propose desired lines to enhance the existing network. Limitations of the study, including data currency and grid dimensions, are acknowledged, suggesting avenues for future research and improvement. Overall, the study underscores the importance of integrated planning approaches and the utilization of spatial analysis tools in promoting cycling as a sustainable and accessible mode of transportation

The role of data-driven methods in power system security assessment from aggregated grid data

Data-driven techniques have been considered as an enabling technology for reducing the computational burden of both static and dynamic power system security analysis. Anyway, the studies reported in the literature mainly focused on inferring from historical data the mapping between the bus variables before and after a certain contingencies set, while, to the best of the Author's knowledge, limited contributions have been devoted to try and classify the power system security state by processing aggregated grid data. This is a relevant issue to address for a Transmission System Operator since it could allow a sensible decrease in the computational burden and, considering that aggregated grid data can be reliably predicted from several hours to one day ahead, it may enable the evolution of security assessment to security forecasting. In trying and filling this research gap, this paper explores the role of machine learning and feature selection algorithms. A realistic case study involving 2 years of synthetic grid data simulated on the Italian power system model against future potential operational scenarios characterized by a high share of renewables is presented and discussed to identify the most promising computing paradigms, analyzing the criticality of tuning the feature selection and classifier algorithms.

An enhanced semi-supervised learning method with self-supervised and adaptive threshold for fault detection and classification in urban power grids

With the rapid development of urban power grids and the large-scale integration of renewable energy, traditional power grid fault diagnosis techniques struggle to address the complexities of diagnosing faults in intricate power grid systems. Although artificial intelligence technologies offer new solutions for power grid fault diagnosis, the difficulty in acquiring labeled grid data limits the development of AI technologies in this area. In response to these challenges, this study proposes a semi-supervised learning framework with self-supervised and adaptive threshold (SAT-SSL) for fault detection and classification in power grids. Compared to other methods, our method reduces the dependence on labeling data while maintaining high recognition accuracy. First, we utilize frequency domain analysis on power grid data to filter abnormal events, then classify and label these events based on visual features, to creating a power grid dataset. Subsequently, we employ the Yule–Walker algorithm extract features from the power grid data. Then we construct a semi-supervised learning framework, incorporating self-supervised loss and dynamic threshold to enhance information extraction capabilities and adaptability across different scenarios of the model. Finally, the power grid dataset along with two benchmark datasets are used to validate the model’s functionality. The results indicate that our model achieves a low error rate across various scenarios and different amounts of labels. In power grid dataset, When retaining just 5% of the labels, the error rate is only 6.15%, which proves that this method can achieve accurate grid fault detection and classification with a limited amount of labeled data.

Trends in temperature and precipitation at high and low elevations in the main mountain ranges of the Iberian Peninsula (1894–2020): The Sierra Nevada and the Pyrenees

AbstractThis study describes temperature and precipitation trends in the two National Parks located in the two highest mountain ranges on the Iberian Peninsula: the Sierra Nevada (Sierra Nevada National Park, SN) and the Pyrenees (Aigüestortes i Sant Maurici National Park, ASM). Special focus is placed on analysing disparities between the lowlands and the highlands, as well as the agreement between observational data and grid data (IBERIA01 and E‐OBS). For this purpose, a quality‐controlled and homogeneity‐adjusted database of the daily maximum temperature, minimum temperature and precipitation (SMADS database) has been generated. Regional trends in mean temperature indicate that warming in ASM (0.17°C·decade−1) was greater than in the Sierra Nevada (SN) (0.13°C·decade−1) in the longest joint period, 1930–2020. For annual precipitation, the trends over the past nine decades were negative, although not significantly. Only the summer in SN showed a significant negative trend, which has intensified in recent decades to −13.4%·decade−1 for 1975–2020. A parallel evolution was observed in the annual mean temperature of the highlands (>1500 m) and lowlands (<1500 m) of ASM, with a common trend of 0.17°C·decade−1, while in SN negative elevation‐dependent warming was detected. Differences between lowlands and highlands were also noted in precipitation trends in both mountain ranges: a positive trend in precipitation was found in the lowlands while in the highlands practically null trends (ASM) or decreasing precipitation trends (SN) were detected. The comparison of the Spanish Mountain Adjusted Daily Series (SMADS) results with the IBERIA01 and E‐OBS grid series yielded differences no greater than ±0.2°C·decade−1. No notable differences were detected between the regional trends calculated with observational series or with grid series. These results worsened when the differences in the trends detected in the individual observed temperature series were compared against the corresponding grid series.

Post-stack 3D merging to fast-track regional interpretation – offshore Otway Basin case study

Ideally when combining legacy 3D seismic surveys, differences in acquisition parameters warrant full pre-stack reprocessing from field data. However, there are occasions where this is not possible due to time, financial or data access constraints; a valuable alternative is post-stack merging and enhancement of existing migrations. This case was to produce a regularised and seamless 3D dataset of the highest possible quality, for the offshore Otway Basin, within 2 months. The input migrated volumes varied by data extent, migration methodology, angle range and grid orientation. Fourteen input volumes totalling 8092 km2 were post-stack merged and processed to produce a continuous and consistent volume, enabling more efficient and effective interpretation of the region. The surveys were regularised onto a common grid, optimised for structural trends, prior to survey matching. A mis-tie analysis algorithm, applied over a time window optimised for interpretation of key events, was used to derive corrections for timing, phase and amplitude, using a reference. This was followed by time-variant spectral and amplitude matching to improve continuity between volumes. Additional enhancements including noise removal and lateral amplitude scaling were also applied. The final merged volume offers significant uplift over the inputs, providing better imaging of structure and events and dramatically improving the efficiency and quality of interpretation. This enables rapid reconnaissance of the area by explorers.