Spatial Regression Test Research Articles

Research on Quality Control Method of Surface Temperature Observations for Complex Physical Geography

Abstract This article aims to improve the quality control (QC) of surface daily temperature observations over complex physical geography. A new QC method based on multiverse optimization algorithm, variational mode decomposition, and kernel extreme learning machine (MVO–VMD–KELM) was eomployed to identify potential outliers. For the selected six regions with complex physical geography, the inverse distance weighting (IDW), the spatial regression test (SRT), KELM, and the empirical mode decomposition improved KELM (EMD–KELM) methods were employed to test the proposed method. The results indicate that the MVO–VMD–KELM method outperformed other methods in all the cases. The MVO–VMD–KELM method yielded better mean absolute error (MAE), root-mean-square error (RMSE), index of agreement (IOA), and Nash–Sutcliffe model efficiency coefficient (NSC) values than others via the analysis of evaluation metrics for different cases. The comparison results led to the recommendation that the proposed method is an effective quality control method in identifying the seeded errors for the surface daily temperature observations.

Research on quality control methods for surface temperature observations via spatial correlation analysis

AbstractThis article proposed a new quality control method (CS‐MSF) to identify potential outliers in the surface temperature observations. The CS‐MSF method employed cosine similarity and moving surface fitting to obtain the estimated value of the target station. For the regions with complex terrain and low weather station density, another quality control method (CE‐GBDT) was employed to compensate for the shortcomings of CS‐MSF. Compared to the spatial regression test method (SRT) and inverse distance weighting method (IDW), the results indicated that CS‐MSF outperformed SRT and IDW in all the cases. And CE‐GBDT was superior to the other methods for the regions with complex terrain and low weather station density. The comparison results led to the recommendation that the two proposed methods are effective quality control methods in identifying the seeded errors for the surface temperature observations.

Does the Proximity of the Area Affect in Incidence of Stunting? : Study on Densely Populated Provinces in Indonesia

The prevalence of stunting in Indonesia is still above WHO standards, so it needs to be addressed immediately,given the adverse effects on individuals and countries. This study aims to analyze the incidence of stunting inEast Java by paying attention to the proximity of the area as an effort to solved stunting problems. The studywas conducted using secondary data from 2017 from official government agency reports. Administrative area(regency/city) analysis unit. Analysis using spatial regression test. The results show that Moran’s I test showsa spatial dependency or location autocorrelation (p <0.20). The Lagrange Multiplier SAR spatial regression testcan explain that spatial factors can increase 5% greater (75%) of the 5 selected factors that cause toddler stunting,compared to using the OLS classic regression test. It could be concluded that spatial factors, namely geographicareas, can increase the percentage of clarity in the regression modeling for the incidence of stunting in an area. Inthis study, spatial factors are known to be associated with regional proximity and ethnic similarity, namely theMadurese ethnicity.

The Analysis Of Characteristics, Habitat Spatial Distribution, and The Impacts On Anopheles Sp. Larvae Dencities at Perimeter and Buffer Area in The Mota’ain Cross-Border Station Of Port Health Office Kupang 2021

East Nusa Tenggara (NTT) is still the third largest contributor to Malaria cases in Indonesia, with a prevalence of 12%, Papua (18.4%) and West Papua (26.1%). Directly border with The Democratic Republic of Timor Leste (RDTL) with Mota'ain as the main crossing point, increases the risk of infectious diseases transmission. Mota’ain is located in coastal areas, swamps, river basins, former fish ponds, and rice fields potentially to be Anopheles sp. habitats. The purpose of this research is to examine the characteristics and spatial distribution of potential breeding habitats for Anopheles sp. in Mota'ain. This is a descriptive analytic study with a cross-sectional design. The Anopheles sp mosquito bionomic area, which included swamps, rice fields, estuaries around the perimeter, and the buffer area of the Mota'ain border, served as the research sample. The spatial regression test was used to analyze the data, and the results revealed three points of Anopheles sp larvae habitats, namely fish ponds, swamps, and river basin. In the buffer area, 24 (80%) of the larvae habitats were found within a 200-meter radius, and 24 (80%) of the larvae habitats were found within a 400-meter radius 6 (20%). There was no habitat found around the perimeter. With a habitat index of 50%, the density of Anopheles sp larvae per catch ranged from 0.04 to 12.03. Anopheles sp larvae discovered included Anopheles Barbirostris, Anopheles Subpictus, and Anopheles Vagus. Swamps have the most habitat (3,004.7 m2), while river basins have the least (3,004.7 m2) (2.93 m2). The Spatial Regression Test revealed that habitat area (p = 0.014), water pH (p = 0.015), and salinity (p = 0.00012) were significant. This means that habitat area, water pH, and salinity all have an effect on Anopheles sp. density, whereas temperature has no effect (p = 0.778). Cleaning aquatic plants and employing larval predators to prevent larval breeding.

ANALISIS SPASIAL KETERJANGKAUAN RETAILER ROKOK TERHADAP PERILAKU MEROKOK PADA SISWA SEKOLAH MENENGAH ATAS (SMA) DI KEC. WANGI-WANGI DAN WANGIWANGI SELATAN KABUPATEN WAKATOBI

Background: Indonesia is still the third country with the most active smokers in the world (61.4 million smokers), after China and India. Data on cases of smoking violations at school in 2017 were 64 cases of smoking violations in all high schools in Wangi-Wangi District and 45 cases in high schools in Wangi-Wangi Selatan District and until August 2018 there were 68 reported cases of smoking violations in high school in these 2 subdistricts and is predicted to continue to increase.
 Objective: To analyze the spatial relationship between the afford ability of cigarette retailers by high school students on smoking behavior spatially in Wangi-wangi District and South Wangi-wangi District, Wakatobi Regency.
 Methods: Type of research is an analytic survey, using a cross sectional study design with consecutive sampling technique. Where the sample is high school students with a sample of 94 people.
 Results: The results showed that there was a significant correlation between the affordability of retailers (p-Value = 0.048) with the smoking behavior of high school students PR 2.09 (95% CI 0.015-4.123), and the smoking behavior of friends (p-Value = 0.010) with the smoking behavior of high school students PR 1.82 (95% CI 1,096-13,662), GeoDa spatial regression test shows that retailer’s affordability (p = 0.00922), friend’s smoking behavior (p = 0.00204) and parental smoking behavior (p = 0.03181) shows spatial relation ship with smoking behavior, where the parental smoking behavior variable has weak spatial autocorrelation because its Morran’s index is close to zero.
 Conclusion: There is a spatial relationship between the affordability of retailers, friend smoking behavior and smoking behavior of parents with smoking habits of high school students in Wangi-Wangi District and WangiWangi South District, Wakatobi Regency
 

A quality control method for surface hourly temperature observations via gene‐expression programming

ABSTRACTThis article aims to find the best‐fit regression function for the quality control (QC) for the surface hourly temperature. A new QC method based on gene‐expression programming (GEP) was employed to identify potential outliers in the surface hourly temperature observations. Compared to the spatial regression test (SRT) method and inverse distance weighting (IDW) method, the results indicate that both GEP and SRT outperform IDW in all the cases. The GEP method and the SRT method display equivalent effects in most of the cases. But GEP was found to be superior to the other methods when the weather station density was low. For all cases, the GEP method could yield a smaller estimated error than the other methods, respectively. The results of the comparison led to the recommendation that the GEP method is an effective QC method in identifying the seeded errors for the surface hourly temperature.

Spatial regression test for ensuring temperature data quality in southern Spain

Quality assurance of meteorological data is crucial for ensuring the reliability of applications and models that use such data as input variables, especially in the field of environmental sciences. Spatial validation of meteorological data is based on the application of quality control procedures using data from neighbouring stations to assess the validity of data from a candidate station (the station of interest). These kinds of tests, which are referred to in the literature as spatial consistency tests, take data from neighbouring stations in order to estimate the corresponding measurement at the candidate station. These estimations can be made by weighting values according to the distance between the stations or to the coefficient of correlation, among other methods. The test applied in this study relies on statistical decision-making and uses a weighting based on the standard error of the estimate. This paper summarizes the results of the application of this test to maximum, minimum and mean temperature data from the Agroclimatic Information Network of Andalusia (southern Spain). This quality control procedure includes a decision based on a factor f, the fraction of potential outliers for each station across the region. Using GIS techniques, the geographic distribution of the errors detected has been also analysed. Finally, the performance of the test was assessed by evaluating its effectiveness in detecting known errors.

Quality Control Program for Real-Time Hourly Temperature Observation in Taiwan

AbstractThis paper introduces a quality control (QC) program for the real-time hourly land surface temperature observation developed by the Central Weather Bureau in Taiwan. There are three strategies involved. The first strategy is a range check scheme that inspects whether the observation falls inside the climatological limits of the station to screen out the obvious outliers. Limits are adjusted according to the station’s elevation. The second strategy is a spatial check scheme that scrutinizes whether the observation falls inside the derived confidence interval, according to the data from the reference stations and the correlations among the stations, to judge the reliability of the data. The scheme is specialized, as it employs the theorems of unbiased and minimum error estimators to determine the weights. The performance evaluation results show that the new method is in theory superior to the spatial regression test (You et al.). The third strategy is a temporal check scheme that examines whether the temperature difference of two successive observations exceeds the temperature variation threshold for judging the rationality of the data. Different thresholds are applied for the data observed in different times under different rainfall conditions. Procedurally, the observation must pass the range check first and then go through the spatial or the temporal check. The temporal check is applied only when the spatial check is unavailable. Post-examinations of the data from 2014 show that the QC program is able to filter out most of the significant errors.

Optimisation in the Design of Environmental Sensor Networks with Robustness Consideration.

This work proposes the design of Environmental Sensor Networks (ESN) through balancing robustness and redundancy. An Evolutionary Algorithm (EA) is employed to find the optimal placement of sensor nodes in the Region of Interest (RoI). Data quality issues are introduced to simulate their impact on the performance of the ESN. Spatial Regression Test (SRT) is also utilised to promote robustness in data quality of the designed ESN. The proposed method provides high network representativeness (fit for purpose) with minimum sensor redundancy (cost), and ensures robustness by enabling the network to continue to achieve its objectives when some sensors fail.

Restoration of missing dry-bulb temperature data with long-term gaps (up to 60 days) for use in building performance monitoring and analysis—Part II

The lack of standard procedures for filling climatic data has the potential to undermine design, monitoring, and control efforts aimed at climate-responsive building design, performance monitoring, and energy efficiency. This article addresses the challenge of long-term missing gaps in dry-bulb temperature data by examining three spatial methods, namely the inverse distance weighting (IDW) method, the spatial regression test (SRT) method, and the substitution with best match data (SSBM) method, as well as two temporal methods, namely the temporal regression test (TRT) method and the temporal substitution with best match data (TSBM) method. Using these methods, missing dry-bulb temperature data with long-term gaps, ranging from 1 to 60 days, are restored for use in building performance monitoring and analysis. Three one-year, hourly datasets were used to evaluate the performance of these approaches. Each method was applied to deal with artificial gaps which were generated randomly and represented different seasons of a year. In terms of the difference between estimated values and measured values, three evaluation indices, namely mean absolute error (MAE), root mean square error (RMSE), and standard error of bias (BIASSTD), were utilized. The comparison results show that spatial methods are better than temporal methods. The confidence level of the SRT method was further investigated by applying this method to existing data and missing data, and examining its performance. The results indicate that the uncertainty of the SRT method can be predicted and at least two neighboring stations are recommended when using it. This is the second part of the research results obtained through the ASHRAE 1413 research project (in press) with a focus on introducing gap-filling methods for long-term gaps in dry-bulb temperature.

Estimation of Uncertainty in Temperature Observations Made at Meteorological Stations Using a Probabilistic Spatiotemporal Approach

AbstractA probabilistic spatiotemporal approach based on a spatial regression test (SRT-PS) is proposed for the quality control of climate data. It provides a quantitative probability that represents the uncertainty in each temperature observation. The assumption of SRT-PS is that there might be large uncertainty in the station record if there is a large residual difference between the record estimated in the spatial regression test and the true station record. The result of SRT-PS is expressed as a confidence probability ranging from 0 to 1, where a value closer to 1 indicates less uncertainty. The potential of SRT-PS to estimate quantitatively the uncertainty in temperature observations was demonstrated using an annual temperature dataset for China for the period 1971–2000 with seeded errors. SRT-PS was also applied to assess a real dataset, and was compared with two traditional quality control approaches: biweight mean and biweight standard deviation and SRT. The study provides a new approach to assess quantitatively the uncertainty in temperature observations at meteorological stations.

Interpolation of Missing Temperature Data at Meteorological Stations Using P-BSHADE*

Abstract Some climate datasets are incomplete at certain places and times. A novel technique called the point estimation model of Biased Sentinel Hospitals-based Area Disease Estimation (P-BSHADE) is introduced to interpolate missing data in temperature datasets. Effectiveness of the technique was empirically evaluated in terms of an annual temperature dataset from 1950 to 2000 in China. The P-BSHADE technique uses a weighted summation of observed stations to derive unbiased and minimum error variance estimates of missing data. Both the ratio and covariance between stations were used in calculation of these weights. In this way, interpolation of missing data in the temperature dataset was improved, and best linear unbiased estimates (BLUE) were obtained. Using the same dataset, performance of P-BSHADE was compared against three estimators: kriging, inverse distance weighting (IDW), and spatial regression test (SRT). Kriging and IDW assume a homogeneous stochastic field, which may not be the case. SRT employs spatiotemporal data and has the potential to consider temperature nonhomogeneity caused by topographic differences, but has no objective function for the BLUE. Instead, P-BSHADE takes into account geographic spatial autocorrelation and nonhomogeneity, and maximizes an objective function for the BLUE of the target station. In addition to the theoretical advantages of P-BSHADE over the three other methods, case studies for an annual Chinese temperature dataset demonstrate its empirical superiority, except for the SRT from 1950 to 1970.

Quality Control of Soil Water Data in Applied Climate Information System—Case Study in Nebraska

Soil moisture is a key state variable from both climate and hydrologic cycle assessment perspectives. Recently, automated measurements of soil moisture with sensors deployed at sites in a real-time monitoring network have provided valuable new data to monitor the soil water resource. However, to assure the quality of the data, quality control (QC) tools are needed. Earlier studies left little literature on the QC of soil water data as measurements were generally not part of a network that routinely collected measurements. This paper presents a systematic QC analysis and methodology to evaluate the performance of candidate QC techniques using a spatially-extensive soil water data set. The six tests included are based on the general behavior of soil moisture, the statistical characteristics of the measurements, the soil properties, and the precipitation measurements. The threshold, step change, and spatial regression test proved most effective in identifying data problems. The results demonstrate that these methods will lead to early identification of potential instrument failures and other disturbances to the soil water measurements.

Comparison of methods for spatially estimating station temperatures in a quality control system

AbstractThe inverse distance weighting (IDW) and spatial regression test (SRT) methods provide data estimates for a station of interest based on the measurements at neighbouring stations. This paper evaluates the performance of the two approaches across the USA in estimating maximum and minimum daily temperature where the estimates are compared to actual measured data. The performance of these approaches was assessed using the coefficient of efficiency, explained variance, root mean square error, systematic and non‐systematic errors. The t‐test and variance test were also used to compare the performances of the two methods. In addition, two other versions of the IDW were tested. The first IDW modification was intended to determine the importance of adding a lapse rate correction to the surrounding stations. The second IDW modification used the intermediate estimates from the SRT method and therefore, by comparison to SRT, showed the relative importance of using SRT weights. The spatial regression approach was found to be superior to all versions of the IDW method especially in the coastal and mountainous regions. The spatial regression approach successfully resolves the systematic differences caused by temperature lapse rate with elevation, which is not accounted for in the inverse distance weighting method. Both the SRT and the IDW methods are found to perform relatively poorly when the weather station density is low. Copyright © 2007 Royal Meteorological Society

Relationship of flagging frequency to confidence intervals in the statistical regression approach for automated quality control of Tmax and Tmin

AbstractWith the widespread use of electronic interfaces in data collection, many networks have increased, or will increase, the sampling rate and add more sensors. The associated increase in data volume will naturally lead to an increased reliance on automatic quality assurance (QA) procedures. The number of data entries flagged for further manual validation can be affected by the choice of confidence intervals in statistically based QA procedures, which in turn affects the number of bad entries classified as good measurements. At any given station, a number of confidence intervals for the Spatial Regression Test (SRT) were specified and tested in this study, using historical data for both the daily minimum (Tmin) and maximum (Tmax), to determine how the frequency of flagging is related to the choice of confidence interval. An assessment of the general relationship of the number of data flagged to the specified confidence interval over a set of widely dispersed stations in the High Plains was undertaken to determine whether a single confidence factor would suffice, at all stations, to identify a moderate number of flags. This study suggests that using a confidence factor ‘f’ larger than 2.5 to specify the confidence interval will flag a reasonable number of measurements (<1%) for further manual validation and a single confidence factor can be applied for a state. This paper initially compares two formulations of the SRT method. This comparison is followed by an analysis of the percentage of observations flagged as a function of confidence interval. Copyright © 2007 Royal Meteorological Society

An Improved QC Process for Temperature in the Daily Cooperative Weather Observations

Abstract TempVal is a spatial component of data quality assurance algorithms applied by the National Climatic Data Center (NCDC), and it has been used operationally for about 4 yr. A spatial regression test (SRT) approach was developed at the regional climate centers for climate data quality assurance and was found to be superior to currently used quality control (QC) procedures for the daily maximum and minimum air temperature. The performance of the spatial quality assessment procedures has been evaluated by assessing the rate with which seeded errors are identified. A complete dataset with seeded errors for the year 2003 for the contiguous United States was examined for both the maximum and minimum air temperature. The spatial regression quality assessment component (SRT), originating in the Automated Climate Information System (ACIS), and TempVal, originating in the NCDC database, were applied separately and evaluated through the ratio of identified seeded errors to the total number of seeds. The spatial regression test applied in the ACIS system was found to perform better in identifying the seeded errors. For all months, the relative frequency of correct identification of wrong data is 0.72 and 0.83 for TempVal and SRT, respectively. The goal of the comparison was to evaluate quality assurance techniques that could improve data quality assessment at the NCDC, and the results of the comparison led to the recommendation that the SRT be included in the NCDC quality assessment methodology.

Quality Control of Weather Data during Extreme Events

Abstract Quality assurance (QA) procedures have been automated to reduce the time and labor necessary to discover outliers in weather data. Measurements from neighboring stations are used in this study in a spatial regression test to provide preliminary estimates of the measured data points. The new method does not assign the largest weight to the nearest estimate but, instead, assigns the weights according to the standard error of estimate. In this paper, the spatial test was employed to study patterns in flagged data in the following extreme events: the 1993 Midwest floods, the 2002 drought, Hurricane Andrew (1992), and a series of cold fronts during October 1990. The location of flagged records and the influence zones for such events relative to QA were compared. The behavior of the spatial test in these events provides important information on the probability of making a type I error in the assignment of the quality control flag. Simple pattern recognition tools that identify zones wherein frequent flagging occurs are illustrated. These tools serve as a means of resetting QA flags to minimize the number of type I errors as demonstrated for the extreme events included here.

Sensitivity Analysis of Quality Assurance Using the Spatial Regression Approach—A Case Study of the Maximum/Minimum Air Temperature

Abstract Both the spatial regression test (SRT) and inverse distance weighting (IDW) methods have been applied to provide estimates for the maximum air temperature (Tmax) and the minimum air temperature (Tmin) in the Applied Climate Information System (ACIS). This is critical to the processes of estimating missing data and identifying suspect data and is undertaken here to ensure quality data in ACIS. The SRT method was previously found to be superior to the IDW method; however, the sensitivity of the performance of both methods to input parameters has not been evaluated. A set of analyses is presented for both methods whereby the sensitivity to the radius of inclusion, the regression time window, the regression time offset, and the number of stations used to make the estimates are examined. Comparisons were also conducted between the SRT and the IDW methods. The performance of the SRT method stabilized when 10 or more stations were applied in the estimates. The optimal number of stations for the IDW method varies from only a few to 30. The results indicate that the best estimates obtained using the IDW method are still inferior to the worst estimates obtained using the SRT method.

Performance of Quality Assurance Procedures for an Applied Climate Information System

Abstract Valid data are required to make climate assessments and to make climate-related decisions. The objective of this paper is threefold: to introduce an explicit treatment of Type I and Type II errors in evaluating the performance of quality assurance procedures, to illustrate a quality control approach that allows tailoring to regions and subregions, and to introduce a new spatial regression test. Threshold testing, step change, persistence, and spatial regression were included in a test of three decades of temperature and precipitation data at six weather stations representing different climate regimes. The magnitude of thresholds was addressed in terms of the climatic variability, and multiple thresholds were tested to determine the number of Type I errors generated. In a separate test, random errors were seeded into the data and the performance of the tests was such that most Type II errors were made in the range of ±1°C for temperature, not too different from the sensor field accuracy. The study underscores the fact that precipitation is more difficult to quality control than temperature. The new spatial regression test presented in this document outperformed all the other tests, which together identified only a few errors beyond those identified by the spatial regression test.