Commercial Microwave Links Research Articles

Rainfall Nowcasting Using Commercial Microwave Links

AbstractAccurate and timely precipitation forecasts are crucial for early warning. Rainfall nowcasting, the process of statistically extrapolating recent rainfall observations, is increasingly used for short‐term forecasting. Nowcasts are generally constructed with high‐resolution radar observations. As a proof of concept, we construct nowcasts with country‐wide rainfall maps estimated from signal level data of commercial microwave links (CMLs) for 12 summer days in the Netherlands. CML nowcasts compare well to radar rainfall nowcasts. Provided well‐calibrated CML rainfall estimates are employed, CML nowcasts can outperform unadjusted real‐time radar nowcasts for high rainfall rates, which are underestimated as compared to a reference. Care should be taken with the sensitivity of the advection field derivation to areas with low CML coverage and the inherent measurement scale of CML data, which can be larger than the application scale. We see potential for rainfall nowcasting with CML data, for example, in regions where weather radars are absent.

Rainfall measurement from mobile telecommunication network and potential benefit for urban hydrology in Africa: a simulation framework for uncertainty propagation analysis

Abstract. Urban floods due to intense precipitation is a major problem in many developing countries, especially in Africa. There are few available solutions for effective and yet affordable flood early warning systems for these regions. Weather radar is common in industrialized countries but too costly for most tropical cities. Satellite provides interesting information in real time but not yet quantitative enough at the space and time scales needed for urban flood monitoring. Rainfall measurement using commercial microwave links (CML) from cellular communication networks is a cost effective alternative to conventional methods. The method is based on measuring rain induced fluctuations between telecommunication antennas; if the operator provides this information rain maps can be produced and used for hydrological prediction. Many CML studies have been carried out in Europe and Israel. Recently IRD implemented pilot sites in Africa in order to test this rainfall estimation technique and quantify the uncertainties. After reviewing the method principles and providing an overview of the current research on CML, we present a simulation framework to analyse the propagation of CML rainfall uncertainties in an urban hydrological model.

Recurrent Neural Network for Rain Estimation Using Commercial Microwave Links

The use of recurrent neural networks (RNNs) to utilize measurements from commercial microwave links (CMLs) has recently gained attention. Whereas previous studies focused on the performance of methods for wet-dry classification, here we propose an RNN algorithm for estimating the rain-rate. We empirically analyzed the proposed algorithm, using real data, and compared it with the traditional power-law (PL)-based algorithm, commonly used for estimating rain from CML attenuation measurements. Our analysis shows that the data-driven RNN algorithm, when properly trained, outperforms the PL algorithm in terms of accuracy. On the other hand, the PL algorithm is simpler and more robust when dealing with a large variety of corruptions and adverse conditions. We then introduced a time normalization (TN) layer for controlling the trade-off between performance and robustness of the RNN methods, and demonstrated its performance.

Rain event detection in commercial microwave link attenuation data using convolutional neural networks

Abstract. Quantitative precipitation estimation with commercial microwave links (CMLs) is a technique developed to supplement weather radar and rain gauge observations. It is exploiting the relation between the attenuation of CML signal levels and the integrated rain rate along a CML path. The opportunistic nature of this method requires a sophisticated data processing using robust methods. In this study we focus on the processing step of rain event detection in the signal level time series of the CMLs, which we treat as a binary classification problem. This processing step is particularly challenging, because even when there is no rain, the signal level can show large fluctuations similar to that during rainy periods. False classifications can have a high impact on falsely estimated rainfall amounts. We analyze the performance of a convolutional neural network (CNN), which is trained to detect rainfall-specific attenuation patterns in CML signal levels, using data from 3904 CMLs in Germany. The CNN consists of a feature extraction and a classification part with, in total, 20 layers of neurons and 1.4×105 trainable parameters. With a structure inspired by the visual cortex of mammals, CNNs use local connections of neurons to recognize patterns independent of their location in the time series. We test the CNN's ability to recognize attenuation patterns from CMLs and time periods outside the training data. Our CNN is trained on 4 months of data from 800 randomly selected CMLs and validated on 2 different months of data, once for all CMLs and once for the 3104 CMLs not included in the training. No CMLs are excluded from the analysis. As a reference data set, we use the gauge-adjusted radar product RADOLAN-RW provided by the German meteorological service (DWD). The model predictions and the reference data are compared on an hourly basis. Model performance is compared to a state-of-the-art reference method, which uses the rolling standard deviation of the CML signal level time series as a detection criteria. Our results show that within the analyzed period of April to September 2018, the CNN generalizes well to the validation CMLs and time periods. A receiver operating characteristic (ROC) analysis shows that the CNN is outperforming the reference method, detecting on average 76 % of all rainy and 97 % of all nonrainy periods. From all periods with a reference rain rate larger than 0.6 mm h−1, more than 90 % was detected. We also show that the improved event detection leads to a significant reduction of falsely estimated rainfall by up to 51 %. At the same time, the quality of the correctly estimated rainfall is kept at the same level in regards to the Pearson correlation with the radar rainfall. In conclusion, we find that CNNs are a robust and promising tool to detect rainfall-induced attenuation patterns in CML signal levels from a large CML data set covering all of Germany.

Deep Learning for an Improved Prediction of Rainfall Retrievals From Commercial Microwave Links

AbstractCommercial microwave links (CMLs) have proven useful for providing rainfall information close to the ground surface. However, large uncertainties are associated with these retrievals, partly due to challenges in the type of data collection and processing. In particular, the most common case is when only minimum and maximum received signal levels (RSLs) over a given time interval (hereafter 15 min) are stored by mobile network operators. The average attenuation and the corresponding rainfall rate are then calculated based on a weighted average method using the minimum and maximum attenuation. In this study, an alternative to using a constant weighted average method is explored, based on a machine learning model trained to produce actual attenuation from minimum/maximum values. A rainfall retrieval deep learning model was designed based on a long short‐term memory (LSTM) model architecture and trained with disdrometer data in a form that is comparable to the data provided by mobile network operators. A first evaluation used only disdrometer data to mimic both attenuation from a CML and corresponding rainfall rates. For the test data set, the relative bias was reduced from 5.99% to 2.84% and the coefficient of determination (R2) increased from 0.86 to 0.97. The second evaluation used this disdrometer‐trained LSTM to retrieve rainfall rates from an actual CML located nearby the disdrometer. A significant improvement in the overall rainfall estimation compared to existing microwave link attenuation models was observed. The relative bias reduced from 7.39% to −1.14% and the R2 improved from 0.71 to 0.82.

Optimal grid resolution for precipitation maps from commercial microwave link networks

Abstract. High-resolution precipitation observation based on signal attenuation in a Commercial Microwave Link (CML) network is an emerging technique that is becoming more and more used. Commonly, the raw data – line measurements from successive time steps – are mapped onto a grid to estimate precipitation fields with a full spatio-temporal coverage. Assuming the CML-estimated precipitation to be accurate, the attainable resolutions in time and space are primarily dependent on two factors: (i) the spatial distribution of the link network and (ii) the spatial correlation properties of the precipitation. Here we outline a pragmatic method for estimating the optimal resolution based on variogram analysis. The method is demonstrated using a CML network and a representative variogram in Stockholm, Sweden. Conceivable applications include preliminary investigations in cities considering starting CML-based precipitation observations.

Rainfall estimation from a German-wide commercial microwave link network: optimized processing and validation for 1 year of data

Abstract. Rainfall is one of the most important environmental variables. However, it is a challenge to measure it accurately over space and time. During the last decade, commercial microwave links (CMLs), operated by mobile network providers, have proven to be an additional source of rainfall information to complement traditional rainfall measurements. In this study, we present the processing and evaluation of a German-wide data set of CMLs. This data set was acquired from around 4000 CMLs distributed across Germany with a temporal resolution of 1 min. The analysis period of 1 year spans from September 2017 to August 2018. We compare and adjust existing processing schemes on this large CML data set. For the crucial step of detecting rain events in the raw attenuation time series, we are able to reduce the amount of misclassification. This was achieved by using a new approach to determine the threshold, which separates a rolling window standard deviation of the CMLs' signal into wet and dry periods. For the compensation for wet antenna attenuation, we compare a time-dependent model with a rain-rate-dependent model and show that the rain-rate-dependent model performs better for our data set. We use RADOLAN-RW, a gridded gauge-adjusted hourly radar product from the German Meteorological Service (DWD) as a precipitation reference, from which we derive the path-averaged rain rates along each CML path. Our data processing is able to handle CML data across different landscapes and seasons very well. For hourly, monthly, and seasonal rainfall sums, we found good agreement between CML-derived rainfall and the reference, except for the winter season due to non-liquid precipitation. We discuss performance measures for different subset criteria, and we show that CML-derived rainfall maps are comparable to the reference. This analysis shows that opportunistic sensing with CMLs yields rainfall information with good agreement with gauge-adjusted radar data during periods without non-liquid precipitation.

Hydrometeorological Monitoring Using Opportunistic Sensing Networks in the Amsterdam Metropolitan Area

AbstractThe ongoing urbanization and climate change urges further understanding and monitoring of weather in cities. Two case studies during a 17-day period over the Amsterdam metropolitan area, the Netherlands, are used to illustrate the potential and limitations of hydrometeorological monitoring using nontraditional and opportunistic sensors. We employ three types of opportunistic sensing networks to monitor six important environmental variables: 1) air temperature estimates from smartphone batteries and personal weather stations, 2) rainfall from commercial microwave links and personal weather stations, 3) solar radiation from smartphones, 4) wind speed from personal weather stations, 5) air pressure from smartphones and personal weather stations, and 6) humidity from personal weather stations. These observations are compared to dedicated, traditional observations where possible, although such networks are typically sparse in urban areas. First, we show that the passage of a front can be successfully monitored using data from several types of nontraditional sensors in a complementary fashion. Also, we demonstrate the added value of opportunistic measurements in quantifying the urban heat island (UHI) effect during a hot episode. The UHI can be clearly determined from personal weather stations, though UHI values tend to be high compared to records from a traditional network. Overall, this study illustrates the enormous potential for hydrometeorological monitoring in urban areas using nontraditional and opportunistic sensing networks.

Microwave Instrument for Simultaneous Wet Antenna Attenuation and Precipitation Measurement

Attenuation data from microwave links can be used to derive important near-ground precipitation estimates which complements observations from rain gauges and weather radars. Wetness on the cover of the microwave link antennas can, however, lead to additional attenuation, resulting in an overestimation of rainfall rates. To investigate the impact of this wet antenna attenuation (WAA) effect, a versatile microwave field experiment was built. It is based on a two-port vector network analyzer and equipped with six dual-polarized commercial microwave link antennas at different frequencies between 18 and 40 GHz. To a measurement frequency range of 150 MHz, range gating enables simultaneous measurement of antenna reflection coefficient and total path attenuation. An optical camera is integrated for visual inspection of the antenna covers. Appropriate housing and active temperature control makes the device weather-proof and enables all-season operation. Several supplementary meteorological instruments at the installation site provide information about the actual weather conditions. They can be used as the ground truth for evaluation. The presented instrument is a unique setup to study the WAA effect, by simultaneously providing measurements of the total path attenuation together with reliable WAA estimations, directly derived from measurements of the antenna reflection coefficient. Its data will give insights into WAA magnitude and temporal evolution in relation to meteorological variables. This will help developing improved WAA estimation algorithms, which will result in improved rainfall estimation from microwave links.

Spatial Reconstruction of Rain Fields From Wireless Telecommunication Networks—Scenario-Dependent Analysis of IDW-Based Algorithms

In the last decade, commercial microwave links (CMLs) have been treated as opportunistic near-ground rain sensors, and successfully used for the retrieval of 2-D near-ground rain fields in several countries. In spite of the path integration of a CML, most studies represent the rainfall measured by a CML as a single virtual rain gauge (VRG) in the center of the path. Here, we study the performance of spatial reconstruction of rain fields by an inverse distance weighting (IDW) spatial interpolation method. We compare the case where each CML is represented by a single VRG with the case where it is represented by several VRGs along its path. A synthetic rain field was produced, simplified to a single rain cell, and sampled by a synthetic CML network that was built according to statistics of actual CMLs. A Monte Carlo simulation study yielded a quantitative and specific set of metrics showing that the rain-retrieval results are scenario-dependent and can be used to design a rain-retrieval system. In particular, we show that if the rain-cell dimensions are in the order of the average length of the CMLs, using several VRG with the iterative algorithm can significantly improve the retrieval performance, whereas the performance gain is small otherwise.

Commercial microwave links for urban drainage modelling: The effect of link characteristics and their position on runoff simulations

Commercial microwave links (CMLs), radio connections widely used in telecommunication networks, can provide path-integrated quantitative precipitation estimates (QPEs) which could complement traditional precipitation observations. This paper assesses the ability of individual CMLs to provide relevant QPEs for urban rainfall-runoff simulations and specifically investigates the influence of CML characteristics and position on the predicted runoff. The analysis is based on a 3-year-long experimental data set from a small (1.3 km2) urban catchment located in Prague, Czech Republic. QPEs from real world CMLs are used as inputs for urban rainfall-runoff predictions and subsequent modelling performance is assessed by comparing simulated runoffs with measured stormwater discharges. The results show that model performance is related to both the sensitivity of CML to rainfall and CML position. The bias propagated into the runoff predictions is inversely proportional to CML path length. The effect of CML position is especially pronounced during heavy rainfalls, when QPEs from shorter CMLs, located within or close to catchment boundaries, better reproduce runoff dynamics than QPEs from longer CMLs extending far beyond the catchment boundaries. Interestingly, QPEs averaged from all available CMLs best reproduce the runoff temporal dynamics. Adjusting CML QPEs to three rain gauges located 2–3 km outside of the catchment substantially reduces the bias in CML QPEs. Unfortunately, this compromises the ability of the CML QPEs to reproduce runoff dynamics during heavy rainfalls. More experimental case studies are necessary to provide specific recommendations on CML preprocessing methods tailored to different water management tasks, catchments and CML networks.

Deep Learning for an improved prediction of rainfall retrievals from commercial microwave links

Deep Learning for an improved prediction of rainfall retrievals from commercial microwave links

Deep Learning for an improved prediction of rainfall retrievals from commercial microwave links

Deep Learning for an improved prediction of rainfall retrievals from commercial microwave links

Excess Attenuation Caused by Antenna Wetting of Terrestrial Microwave Links at 32 GHz

Excess attenuation of microwave radio links caused by the wet antenna effect is addressed. Long-term measured attenuation of two commercial microwave links (816 m 611 m long) at 32 GHz is compared to theoretically predicted rain attenuation. The theoretical prediction is calculated using a kRα relationship where k and α are constants derived from a local disdrometer measurement, and rain rate R is available from rain gauges placed at both ends of the links to obtain the distribution of rain rate along the path length. From this comparison, excess attenuation caused by antenna wetting is derived. A new model describing wet antenna attenuation (WAA) as a function of rain rate is suggested where excess attenuation reaches approx. 3 dB per antenna for a rain rate of 100 mm/h. Statistics of theoretically predicted rain attenuation, including the WAA term, is compared with statistics of measured attenuation on a set of three other links working at the same frequency. Adding the wet antenna term to the rain attenuation calculation significantly improves the accuracy of the rain attenuation prediction.

Modeling of Wet Antenna Attenuation for Precipitation Estimation From Microwave Links

This letter presents an analysis of wet antenna attenuation (WAA) from an electrodynamic perspective. The WAA is a relevant part of the path attenuation determined from the received signal level (RSL). In the case of extracting rain rates from path attenuation, as it is done with data from country-wide commercial microwave link (CML) networks, the WAA is a significant error source and leads to overestimation of rainfall. To handle this effect, a model, based on the three antenna parameters, reflectivity, efficiency, and directivity, is developed. The model allows the variation of those three parameters depending on the degree of wetness and shows that WAA is composed of variations of antenna impedance, antenna efficiency, and antenna directivity. The suitability of the theoretical model was assessed by a full-wave numerical simulation of a pyramidal horn antenna at 18 GHz with homogeneous water film on the surface of the antenna cover. The result of this simulation is a quantification of these three subeffects. For verification of the simulation, an antenna, similar to the antenna in the simulation, was manufactured and a controlled sprinkler device to generate a water film was built. The adjustable water flow rate determines the thickness of the water film. The parameters were generated by measuring the near field and the reflectivity of the antenna in an anechoic chamber, with a near-field scanning system and a vector network analyzer, under different flow rates equaling different water film thicknesses. From these results, the reflectivity, directivity, and efficiency were computed and compared with the results of the full-wave simulation. It is found that the reflectivity is linked to water film thickness in a such way that the WAA can be deduced from monitoring the antenna reflectivity.

Use of the C-Band Microwave Link to Distinguish between Rainy and Dry Periods

As a method that does not require additional cost, precipitation measurement by microwave links (MLs) has quickly attracted the attention of experts in meteorological, hydrological, and other related fields, of which wet-dry classification by MLs is one of the most important methods. Considering that existing commercial MLs are usually single-path, single-polarization, or low-frequency MLs, this paper uses the C-band ML and analyzes the variation in the receive signal level (RSL) of the C-band ML under the conditions of no rain, drizzle, light rain, and moderate rain. The RSL data are analyzed at different time scales by using long short-term memory (LSTM) network techniques, and then the method for distinguishing parts of the precipitation period by using the RSL from low-frequency MLs is proposed and validated. The results show that wet-dry classification is ideal. The accuracy on each day was higher than 60%, and some days had accuracies that were even higher than 98%. MLs below 10 GHz also had the potential to monitor ground rainfall. This study will broaden the range of available equipment for MLs for precipitation measurement.

Rainfall Estimation Accuracy of a Nationwide Instantaneously Sampling Commercial Microwave Link Network: Error Dependency on Known Characteristics

AbstractCommercial microwave links are installed and maintained for the purpose of telecommunication. Hydrometeors between transmitting and receiving antennas cause the microwave signal to be attenuated. From signal attenuation, the path-averaged rainfall intensity can be calculated. A 7-month dataset of instantaneously logged signal powers from almost 2000 unique links in the Netherlands is analyzed. Rainfall intensities are calculated with the RAINLINK package with a novel preprocessing module, enabling the package to be applied on instantaneously logged data from now on. Rainfall intensities per link are validated with the path-averaged rainfall intensities according to a gauge-adjusted radar product. Both the overall performance and the dependence of errors on link characteristics and measurement conditions are evaluated. The coefficient of variation decreases from 3.70 to 2.32 and the correlation increases from 0.30 to 0.63 from instantaneous to daily estimates of rainfall accumulations. The coefficient of variation is also smaller during heavy rainfall. Errors are largest for pathlengths shorter than 2 km, for observations during the late night and early morning, and for observations during colder months (when solid or melting precipitation could occur and dew is more likely to form on the antennas). Comparison of our results with those of earlier studies shows that minimum/maximum sampling (widely employed in network management systems) outperforms instantaneous sampling regarding detection of both quantity and occurrence of rain at a 15-min sampling rate in the Dutch climate.

An evaluation of weather radar adjustment algorithms using synthetic data

Adjustment of weather radar estimates using observed precipitation has been an accepted procedure for decades. Ground observations of precipitation typically come from rain gauges, but can also include data from diverse networks of sensors, with different levels of reliability. This study presents a standardized framework for evaluating adjustment algorithms using synthetically constructed, but realistic, rain grids and weather radar rainfall. Ground observation points are randomly placed throughout the synthetic storm domain and the precipitation for each sensor is extracted from the true rain. Then a subset of the sensors are defined as unreliable, and a log-normal error factor is applied at those locations.This double network of rain sensors could be applicable, for example, when rainfall is derived from signal attenuation between commercial microwave link (CML) antennas. Past research has tested CML observations as a source of precipitation data and validated various radar adjustment algorithms. However, a comprehensive evaluation of adjustment algorithms using accurate gauge data mixed with CML observations at different densities is lacking.Five adjustment algorithms are applied to the synthetic radar grid: Mean Field Bias (MFB), a Multiplicative algorithm, Mixed (additive and multiplicative), Conditional Merge (CondMerge) and Kriging with External Drift (KED). Generation of the synthetic framework, and application of the adjustment algorithms is repeated for 150 realizations. Comparison of coefficient of determination (R2), root mean square error and linear regression for all adjustment procedures over all realizations indicates the following results. Only MFB and KED adjustments performed well when using accurate gauges. The kriging based KED was able to achieve good adjustment also with the addition of error-prone sensors. CondMerge and the Mixed and Multiplicative, however, resulted in poorer adjustments.

Harnessing Crowdsourced Data and Prevalent Technologies for Atmospheric Research

The knowledge garnered in environmental science takes a crucial part in informing decision-making in various fields, including agriculture, transportation, energy, public health and safety, and more. Understanding the basic processes in each of these fields relies greatly on progress being made in conceptual, observational and technological approaches. However, existing instruments for environmental observations are often limited as a result of technical and practical constraints. Current technologies, including remote sensing systems and ground-level measuring means, may suffer from obstacles such as low spatial representativity or a lack of precision when measuring near ground-level. These constraints often limit the ability to carry out extensive meteorological observations and, as a result, the capacity to deepen the existing understanding of atmospheric phenomena and processes. Multi-system informatics and sensing technology have become increasingly distributed as they are embedded into our environment. As they become more widely deployed, these technologies create unprecedented data streams with extraordinary levels of coverage and immediacy, providing a growing opportunity to complement traditional observation techniques using the large volumes of data created. Commercial microwave links that comprise the data transfer infrastructure of cellular communication networks are an example of these types of systems. This viewpoint letter briefly reviews various works on the subject and presents aspects concerning the added value that may be obtained as a result of the integration of these new means, which are becoming available for the first time in this era, for studying and monitoring atmospheric phenomena.

Quantifying Wet Antenna Attenuation in 38-GHz Commercial Microwave Links of Cellular Backhaul

The quantification of wet antenna attenuat-ion (WAA) represents one of the major uncertainties in rainfall retrieval from commercial microwave links (CMLs). We propose identifying WAA distribution and upper limits based on rainfall climatology without the need for nearby rainfall observations. The quantified contributions of antenna wetting due to total loss are derived from 2 years of data collected from eight short (48–497 m) CMLs operating at frequencies of 37.3–39.2 GHz within the cellular backhaul. More specifically, the complementary cumulative distribution functions of measured attenuation are compared with theoretical ones calculated from rainfall data from local rain gauges. The results show that WAA reaches 1.5–2.0 dB and dominates over path attenuation during light rainfalls ( $R mm/h). WAA can, however, reach 2.8–5.3 dB during heavy rainfalls and maximal WAA observed during extreme rainfalls ( $R \approx 70$ –130 mm/h) is between 6–9 dB. The constant value of WAA used in some previous studies may, therefore, lead to a significant overestimation of peak rainfalls retrieved from CMLs.