Real-time weather-adaptive water flow and leakage forecasting using an explainable unified deep neural network

Background climate change increases cardiovascular risks from humidity, but effects of specific metrics-including relative humidity (RH), mixing ratio (MR), specific humidity (SH), absolute vapor pressure (AVP), vapor pressure deficit (VPD), and dew point temperature (DPT)-on aging are unclear. Methods we analyzed China Health and Aging Longitudinal Study data (CHARLS, 2011-2015; n=25,614, ≥45y) with high-resolution humidity measurements. Multivariate logistic regression and weighted quantile sum (WQS) models assessed associations between these six humidity indicators and heart disease, adjusted for confounders. Results MR showed the strongest association with heart disease (aOR=2.05, 95%CI:1.89-2.22 per IQR), followed by SH (aOR=2.04) and VPD (aOR=1.57). WQS identified VPD as most influential (45% weight), but MR correlated perfectly with SH (r=1), supporting its use as a primary indicator. RH, AVP and DPT showed weaker associations. Conclusion: MR and VPD are key humidity-related predictors of heart disease in middle-aged and older adults. Early-warning systems and targeted interventions should focus on MR, with VPD as a complementary metric, especially in vulnerable populations.

The weather research and forecasting model has been applied to derive information on meteorological variables for the period with high concentrations of PM 10 (1–30 December 2009) in SW Poland. Three one-way nested domains have been used and the results for the innermost domain have been compared with surface and radiosonde meteorological measurements for pressure (PRES), air temperature (TMP), specific humidity (SPFH), wind speed (WIND) and direction (WDIR). The model results are in good agreement with the surface measurements for TMP, PRES and SPFH, with the index of agreement (IOA) above 0.9. The model underestimate the observed PRES, TMP and SPFH except for the mountainous site Śniezka. The WIND is biased high, the overall IOA is 0.62, and range from 0.41 to 0.73 for all stations. The IOA is above 0.73 for TMP and SPFH for radiosonde measurements and the errors decrease with height.

Reducing pipe leakage is one of the top priorities for water companies, with many investing in higher quality sensor coverage to improve flow forecasting and detection of leaks. Most research on this topic is focused on leakage detection through the analysis of sensor data from district metered areas (DMAs), aiming to identify bursts after their occurrence. This study is a step towards the development of ‘self-healing’ water infrastructure systems. In particular, machine learning and deep learning-based algorithms are applied to forecasting the anomalous water flow experienced during bursts (new leakage) in DMAs at various temporal scales, thereby aiding in the health monitoring of water distribution systems. This study uses a dataset of over 2,000 DMAs in North Yorkshire, UK, containing flow time series recorded at 15-minute intervals for a period of one year. Firstly, the method of isolation forests is used to identify anomalies in the dataset, which are cross referenced with entries in the water mains repair log, indicating the occurrence of bursts. Going beyond leakage detection, this research proposes a hybrid deep learning framework named FLUIDS (Forecasting Leakage and Usual flow Intelligently in water Distribution Systems). A recurrent neural network (RNN) is used for mean flow forecasting, which is then combined with forecasted residuals obtained through real-time Kalman filtering. While providing expected day-to-day flow demands, this framework also aims to issue sufficient early warning for any upcoming anomalous flow or possible leakages. For a given forecast period, the FLUIDS framework can be used to compute the probability of flow exceeding a pre-defined threshold, thus allowing decision-making for any necessary interventions. This can inform targeted repair strategies that best utilize resources to minimize leakages and disruptions. The FLUIDS framework is statistically assessed and compared against the state-of-practice minimum night flow (MNF) methodology. Based on the statistical analyses, it is concluded that the proposed framework performs well on the unobserved test dataset for both regular and leakage water flows.

Chapter 9 - Leak Detection Performance, Testing, and Tuning

Irrigated cotton (Gossypium hirsutum L.) is produced mainly in Northwest China, where groundwater is heavily used. To alleviate water scarcity and increase regional economic benefits, a four-year (2016–2019) field experiment was conducted in Qira Oasis, Xingjiang Province, to evaluate irrigation water use efficiency (IWUE) in cotton production using the Root Zone Water Quality Model (RZWQM2), that was calibrated and validated using volumetric soil water content (θ), soil temperature (Tsoil°) and plant transpiration (T), along with cotton growth and yield data collected from full and deficit irrigation experimental plots managed with a newly developed Decision Support System for Irrigation Scheduling (DSSIS). In the validation phase, RZWQM2 adequately simulated (S) topsoil θ and Tsoil°, as well as cotton growth (average index of agreement (IOA) > 0.76). Relative root mean squared error (RRMSE) and percent bias (PBIAS) of cotton seed yield were 8% and 2.5%, respectively, during calibration, and 20% and −10.3% during validation. The cotton crop’s (M) T was well S (−18% < PBIAS < 14% and IOA > 0.95) for both full and deficit irrigation fields. The validated RZWQM2 model was subsequently run with seven irrigation scenarios with 850 to 350 mm water (Irr850, Irr750, Irr700, Irr650, Irr550, Irr450, and Irr350) and long-term (1990–2019) weather data to determine the best IWUE. Simulation results showed that the Irr650 treatment generated the greatest cotton seed yield (4.09 Mg ha−1) and net income (US $3165 ha−1), while the Irr550 treatment achieved the greatest IWUE (6.53 kg ha−1 mm−1) and net water production (0.94 $ m−3). These results provided farmers guidelines to adopt deficit irrigation strategies.

The multiport hybrid molecular pump is a new type of molecular pump that was specifically designed for use in helium mass spectrometer leak detectors. This pump enables the instrument to detect both downstream and counter-current leaks. Access to different extraction ports can fulfill various leak detection requirements, which broadens the application range of these detectors and improves their detection efficiency. This paper establishes a model for the calculation of the parameters of a helium mass spectrometer leak detector under fine inspection conditions, studies the influence of different turbine blade parameters and fine inspection port positions on pumping performance and leak detection, discusses a method to determine turbine blade parameters and fine inspection port positions under fine inspection conditions, and provides a theoretical basis for the development of helium mass spectrometer leak detection technology and the optimization of multiport hybrid molecular pump structures. The results show that reducing the blade inclination and increasing the number and height of blades can improve the detection sensitivity. Provided that the hybrid molecular pump has good pumping capacity, the helium mass spectrometer leak detector can obtain high detection sensitivity when the single blade inclination angle is 25°, the number of turbine blades is 25 to 30, the blade height is 3 mm, and the blade thickness is 0.6 mm to 0.8 mm. To determine the position of the inspection port, it is necessary to consider the actual requirements of the leak detector, such as the working pressure of the inspected parts, the leakage rate and the minimum detectable leakage rate. When the inspection port is between the 7th- and 8th-stage blade rows, optimum working pressure of the mass spectrometer chamber for better leak detection performance and large pressure detection range are guaranteed.

Recent advances in SCADA and leak detection system technologies lead to higher scan rates and faster model speeds. As these model speeds increase and the inherent mathematical uncertainties in implicit method solutions are reduced, errors and uncertainties in measurement of the physical properties of the fluids transported by pipeline come to dominate the confidence calculations for computer generated leak alerts in the control center. The ability to collect more data must be supported by the need for better model data in order to achieve optimal leak detection system performance. This is particularly true when the products transported are non-homogeneous and have strong viscosity-vs-temperature relationships. These are characteristics of crude oils in California’s San Joaquin Valley where significant heating is required to pump these oils in an efficient manner. Proper characterization and correct mathematical expression of these physical properties in leak models has become critical. This paper presents these new developments in the context of an implementation of this new technology for the Pacific Pipeline System (PPS). PPS is a recently constructed and commissioned 209 km (130-mile), 50.8 cm (20″) diameter, insulated, hot crude oil pipeline between the southern portion of California’s San Joaquin Valley and refineries in the Los Angeles basin. Operational temperatures in this line vary from ambient to 82.2°C (180°F) with pressures ranging from 345 kPa (50 psi) to 11,720 kPa (1700 psi). Due to the unique geometry of the line, facilities along the route include pumping stations, metering stations and numerous “throttle-type” pressure reduction facilities. On PPS, a high-speed leak detection model is supported by a fiber optic (OC-1) communication backbone with data rate capacities in excess of 50 Megabits Per Second (MPS). Total scan times for the distributed communication system have been reduced to 1/4 second — each facility reports data to the SCADA host four times each second. A corresponding 1/4 second leak detection model cycle leads to selection of Methods of Characteristics segments on the order of 260 meters (850 feet). This resolution, in conjunction with the advanced instrumentation package of PPS, makes detection of very small leaks realizable. This paper starts with an overview of the system and combines a mix of the theoretical requirements imposed by the mathematical solutions with a practical description of the laboratory procedures and propagated experimental errors. The paper reviews temperature-related errors and uncertainties and their influence on leak detection performance.

We report on the development and validation of a neural network (NN) model of PM_(10) concentrations in terms of photochemical measurements of NO, NO_2 and O_3 and temporal parameters that include the day of the week and the day of the year with its sinusoidal variation. A long-term record (≈10 yr) from 2001-2012 (inclusive) assembled from measurements taken at 10 station nodes in the air quality monitoring network of the Greater Athens Area in Greece has been used. Eight synoptic categorizations of the circulation at 850 hPa were used to partition the data record, and to train individual NNs with Bayesian regularization using 90% of available data for different atmospheric conditions. The time series of PM_(10) estimates was then reconstructed from the partitioned output. As a control, a NN without synoptic clustering was trained on the same data. The remaining 10% of the data was used for testing the simulation performance. NN models with synoptic clustering achieved an average root mean square error (RMSE) ≈ 16 μg m^(-3) across the station nodes with an average index of agreement (IA) of 0.71 (somewhat better than the control network whose performance statistics were RMSE ≈ 17 μg m^(-3) and IA = 0.61, respectively). For routine measurements below the EU Air Quality Directive limit value of 50 μg m^(-3), the average error is as low as RMSE ≈ 11 μg m^(-3) across the station nodes. NN models were found to strongly outperform analogous MLR models over all station nodes.

In semantic segmentation datasets, classes of high importance are oftentimes underrepresented, e.g., humans in street scenes. Neural networks are usually trained to reduce the overall number of errors, attaching identical loss to errors of all kinds. However, this is not necessarily aligned with intuition. For instance, an overlooked pedestrian seems more severe than an incorrectly detected one. One possible remedy is to deploy different decision rules by introducing class priors that assign more weight to underrepresented classes. While reducing the false negatives of the underrepresented class, at the same time this leads to a considerable increase of false positive indications. In this work, we combine decision rules with methods for false positive detection. Therefore, we fuse false negative detection with uncertainty based false positive meta classification. We present the efficiency of our method for the semantic segmentation of street scenes on the Cityscapes dataset based on predicted instances of the human class. In the latter we employ an advanced false positive detection method using uncertainty measures aggregated over instances. We, thereby, achieve improved trade-offs between false negative and false positive samples of the underrepresented classes.

In the agricultural sector, tomato leaf diseases signify a lot because they result in a lower crop yield and quality. Timely detection and classification of diseases help to ensure early interventions and effective treatment solutions. Nonetheless, the existing methods are confined by the dataset imbalance which affects class distribution negatively and thus results in poor models, especially for rare diseases. The research is designed to improve the capability of tomato leaf disease identification by investing a new deep-learning method beyond the challenge of imbalanced class distribution. By balancing the dataset, we aim to improve classification accuracy as we pay more attention to the under-represented classes. The proposed GAN-based method that combines the Weighted Loss Function to produce tomato leaf disease synthetic images is underrepresented. They improve the quality of the entire dataset, and the images from every class are now in a more balanced proportion. A CNN, which is the convolutional neural network, is trained for the classifier, with the weighted loss function as a part of the model. We used Genetic Algorithm (GA) for hyperparameter optimization of the CNN. It helps in emphasizing the learning process from the under-represented class. The suggested one will not only decrease the accuracy of tomato leaf disease detection but also increase it. Therefore, the synthetic images created by GAN enhance the dataset since the class distribution is brought to equilibrium. The incorporation of the weighted loss function into the model’s training process makes it very effective in handling with the class instability problem and consequently, the model can identify both common and rare diseases. From the outcomes of this study, it can be concluded that it is feasible to employ GAN and one loser weights function to solve the problem of class imbalance in tomato leaf disease recognition. A suggested approach that increases the model’s accuracy and reliability could be a good move to enhancing a reliable method of disease detection in the agricultural sector.

Abstract. The Vertical City Weather Generator (VCWG) is a computationally efficient urban microclimate model developed to predict temporal and vertical variation of potential temperature, wind speed, specific humidity, and turbulent kinetic energy. It is composed of various sub-models: a rural model, an urban vertical diffusion model, a radiation model, and a building energy model. Forced with weather data from a nearby rural site, the rural model is used to solve for the vertical profiles of potential temperature, specific humidity, and friction velocity at 10 m a.g.l. The rural model also calculates a horizontal pressure gradient. The rural model outputs are applied to a vertical diffusion urban microclimate model that solves vertical transport equations for potential temperature, momentum, specific humidity, and turbulent kinetic energy. The urban vertical diffusion model is also coupled to the radiation and building energy models using two-way interaction. The aerodynamic and thermal effects of urban elements, surface vegetation, and trees are considered. The predictions of the VCWG model are compared to observations of the Basel UrBan Boundary Layer Experiment (BUBBLE) microclimate field campaign for 8 months from December 2001 to July 2002. The model evaluation indicates that the VCWG predicts vertical profiles of meteorological variables in reasonable agreement with the field measurements. The average bias, root mean square error (RMSE), and R2 for potential temperature are 0.25 K, 1.41 K, and 0.82, respectively. The average bias, RMSE, and R2 for wind speed are 0.67 m s−1, 1.06 m s−1, and 0.41, respectively. The average bias, RMSE, and R2 for specific humidity are 0.00057 kg kg−1, 0.0010 kg kg−1, and 0.85, respectively. In addition, the average bias, RMSE, and R2 for the urban heat island (UHI) are 0.36 K, 1.2 K, and 0.35, respectively. Based on the evaluation, the model performance is comparable to the performance of similar models. The performance of the model is further explored to investigate the effects of urban configurations such as plan and frontal area densities, varying levels of vegetation, building energy configuration, radiation configuration, seasonal variations, and different climate zones on the model predictions. The results obtained from the explorations are reasonably consistent with previous studies in the literature, justifying the reliability and computational efficiency of VCWG for operational urban development projects.

The purpose of the present study is to examine the agreement of various existing definitions of metabolic syndrome for Japanese. One hundred thirty-two apparently healthy men and 147 apparently healthy women underwent testing and diagnosis for metabolic syndrome using 5 different definitions of metabolic syndrome for Japanese, including a newly proposed definition: a modified National Cholesterol Education Program definition replacing abdominal obesity with C-reactive protein. The agreement of these various definitions of metabolic syndrome was studied using an agreement index defined as the number of subjects who met both definitions divided by the number of subjects who met either of the 2 definitions. Agreement indices among these various definitions of metabolic syndrome for Japanese were between 0.19 and 0.6 in men and between 0.31 and 0.89 in women. The average agreement index was 0.41 in men and 0.51 in women, and the overall agreement index was 0.15 in men and 0.21 in women. There was considerable disagreement among various definitions of metabolic syndrome for Japanese. Therefore, diagnosis with this syndrome should not be made until a truly consensual definition of metabolic syndrome can be established.

The reduction of pipe leakage is one of the top priorities for water companies, with many investing in greater sensor coverage to improve the forecasting of flow and detection of leaks. The majority of research on this topic is focused on leakage detection through the analysis of sensor data from district metered areas (DMAs), with the aim of identifying bursts after occurrence. This study is a step towards development of ‘self-healing’ water infrastructure systems. In particular, the concepts of machine-learning (ML) and deep-learning (DL) are applied to the forecasting of water flow in DMAs at various temporal scales, thereby aiding in the health monitoring of water distribution systems. This study uses a dataset for ~2500 DMAs in Yorkshire, containing flow time-series recorded at every 15-minute interval over the period of a year. Firstly, the method of isolation forests is used to identify anomalies in the dataset which are verified as corresponding to entries in water mains repair log, indicating the occurrence of bursts. Going beyond leakage detection, this research proposes a hybrid framework of DL models - such as recurrent neural networks (RNNs) and transformer neural networks) - and state-space ML algorithms - such as Kalman filter and autoregressive integrated moving average (ARIMA). The ML algorithms are trained to forecast the stationary component of the expected flow patterns in real-time, which is then combined (through Bayesian updating) with the non-stationary component obtained from DL models. As well as providing expected day-to-day flow demands, this framework aims to issue sufficient early warning for any upcoming anomalous flow or possible leakages. For a given forecast period, the framework can be used to compute the probability of flow exceeding a pre-defined threshold, thus allowing decisions to be made regarding any necessary interventions. This can inform targeted repair strategies which best utilise resources to minimise leakage and disruptions by addressing both detected and predicted burst events.

The state of the near‐surface atmosphere, especially air temperature (AT) and specific humidity (SH), has profound effects on human health, ecosystem function, and global energy flows. Accurate measurements of AT and SH are essential for weather forecasting, climate modeling, data assimilation, and trend assessment. The Atmospheric Infrared Sounder (AIRS) provides global estimates of near‐surface AT and SH estimates, with continuous improvements in accuracy leading to significant reductions in error rates. However, existing studies have not systematically validated AIRS near‐surface products in both temporal and spatial perspectives, especially over oceans. This study aims to address this gap by using the International Comprehensive Ocean–Atmosphere Data Set as a ground‐based reference to evaluate AIRS near‐surface AT and SH over the ocean from the V7 Level 2 product. Our results show an overall underestimation of near‐surface AT and SH. Spatially, higher uncertainties, indicated by high root‐mean‐square error, near land were found. In terms of seasonality and diurnal variation, we found that the products perform better during winter and at night on a global scale, although there are regional exceptions. In terms of temporal variation, the estimation errors show remarkable stability over a 20‐year period, demonstrating the ability of AIRS to capture general temporal characteristics. These findings underscore the importance of validating and understanding the retrieval uncertainties of AIRS near‐surface products, paving the way for improved climatological applications.

A statistical forecasting method of air quality based on meteorological elements with high spatiotemporal resolution simulated by the Weather Research and Forecasting (WRF) model and a back-propagation (BP) neural network was established to predict 72 h PM2.5 mass concentrations over the Yangtze River Delta (YRD) region of eastern China. Short-term statistical forecasting of air quality in 25 major cities in the YRD region was conducted and the PM2.5 forecast was validated using the corresponding surface PM2.5 observational data in this study. Results indicate that the short-term air quality forecasting system has a ability to accurately forecast PM2.5 concentration in the major cities in the YRD region. The average index of agreement (IA) between PM2.5 forecasts and observations in the four seasons ranges from 74% to 77%, and the root mean square error (RMSE) fall between 15.2 µg m–3 and 33.0 µg m–3. The data with PM2.5 concentration greater than 115 µg m–3 are selected to establish the EXP-Polluted model and then used to predict PM2.5 concentration during heavy haze periods in 2017. The RMSEs of PM2.5 forecasts during severe haze periods are improved by 44.1%, which compared to predictions using the EXP-All Time model constructed by the full-year data.