Radio Frequency Interference Mitigation Research Articles

RFI-DRUnet: Restoring dynamic spectra corrupted by radio frequency interference—Application to pulsar observations

Radio frequency interference (RFI) has been an enduring concern in radio astronomy, particularly for the observations of pulsars which require high timing precision and data sensitivity. In most works of the literature, RFI mitigation has been formulated as a detection task that consists of localizing possible RFI in dynamic spectra. This strategy inevitably leads to a potential loss of information since parts of the signal identified as possibly RFI-corrupted are generally not considered in the subsequent data processing pipeline. Conversely, this work proposes to tackle RFI mitigation as a joint detection and restoration that allows parts of the dynamic spectrum affected by RFI to be not only identified but also recovered. The proposed supervised method relies on a deep convolutional network whose architecture inherits the performance reached by a recent yet popular image-denoising network. To train this network, a whole simulation framework is built to generate large data sets according to physics-inspired and statistical models of the pulsar signals and of the RFI. The relevance of the proposed approach is quantitatively assessed by conducting extensive experiments. In particular, the results show that the restored dynamic spectra are sufficiently reliable to estimate pulsar times-of-arrivals with an accuracy close to the one that would be obtained from RFI-free signals.

TransientX: A high-performance single-pulse search package

Context. Radio interferometers composed of a large array of small antennas possess large fields of view, coupled with high sensitivities. For example, the Karoo Array Telescope (MeerKAT) achieves a gain of up to 2.8 KJy−1 across its >1 deg2 field of view. This capability significantly enhances the survey speed for pulsars and fast transients. It also introduces challenges related to the high data rate, which reaches a few Tb s−1 for MeerKAT, and it requires substantial computing power. Aims. To handle the high data rate of surveys, we have developed a high-performance single-pulse search software called “TransientX”. This software integrates multiple processes into one pipeline, which includes radio-frequency interference mitigation, dedispersion, matched filtering, clustering, and candidate plotting. Methods. In TRANSIENTX, we developed an efficient CPU-based dedispersion implementation using the sub-band dedispersion algorithm. Additionally, TRANSIENTX employs the density-based spatial clustering of applications with noise (DBSCAN) algorithm to eliminate duplicate candidates, using an efficient implementation based on the kd-tree data structure. We also calculate the decrease of signal-to-noise ratio (s/N) resulting from dispersion measure, boxcar width, spectral index, and pulse-shape mismatches. Remarkably, we find that the decrease of S/N resulting from the mismatch between a boxcar-shaped template and a Gaussian-shaped pulse with scattering remains relatively small, at approximately 9%, even when the scattering timescale is ten times that of the pulse width. Additionally, the decrease in the S/N resulting from the spectral index mismatch becomes significant with multi-octave receivers. Results. We have benchmarked the individual processes, including dedispersion, matched filtering, and clustering. Our dedispersion implementation can be executed in real time using a single CPU core on data with 4096 dispersion measure trials, which consist of 4096 channels and have a time resolution of 153 µs. Overall, TRANSIENTX offers the capability for efficient CPU-only real-time single-pulse searching.

Low Complexity Radio Frequency Interference Mitigation for Radio Astronomy Using Large Antenna Array

With the ongoing growth in radio communications, there is an increased contamination of radio astronomical source data, which hinders the study of celestial radio sources. In many cases, fast mitigation of strong radio frequency interference (RFI) is valuable for studying short lived radio transients so that the astronomers can perform detailed observations of celestial radio sources. The standard method to manually excise contaminated blocks in time and frequency makes the removed data useless for radio astronomy analyses. This motivates the need for better RFI mitigation techniques for array of size M antennas. Although many solutions for mitigating strong RFI improves the quality of the final celestial source signal, many standard approaches require all the eigenvalues of the spatial covariance matrix ([Formula: see text]) of the received signal, which has [Formula: see text] computation complexity for removing RFI of size d where [Formula: see text]. In this work, we investigate two approaches for RFI mitigation, (1) the computationally efficient Lanczos method based on the Quadratic Mean to Arithmetic Mean (QMAM) approach using information from previously-collected data under similar radio-sky-conditions, and (2) an approach using a celestial source as a reference for RFI mitigation. QMAM uses the Lanczos method for finding the Rayleigh–Ritz values of the covariance matrix [Formula: see text], thus, reducing the computational complexity of the overall approach to [Formula: see text]. Our numerical results, using data from the radio observatory Long Wavelength Array (LWA-1), demonstrate the effectiveness of both proposed approaches to remove strong RFI, with the QMAM-based approach still being computationally efficient.

Radio Frequency Interference Mitigation in SAR Systems via Multi-Polarization Framework

Radio Frequency Interference Mitigation in SAR Systems via Multi-Polarization Framework

Survey and monitoring of ASKAP’s RFI environment and trends I: Flagging statistics

Abstract We present an initial analysis of Radio Frequency Interference (RFI) flagging statistics from archived Australian SKA Pathfinder (ASKAP) observations for the ‘Survey and Monitoring of ASKAP’s RFI environment and Trends’ (SMART) project. SMART is a two-part observatoryled project combining analysis of archived observations with a dedicated, comprehensive RFI survey. The survey component covers ASKAP’s full 700–1 800 MHz frequency range, including bands not typically used due to severe RFI. Observations are underway to capture a detailed snapshot of the ASKAP RFI environment over representative 24 h periods. In addition to this dedicated survey, we routinely archive and analyse flagging statistics for all scientific observations to monitor the observatory’s RFI environment in near real-time. We use the telescope itself as a very sensitive RFI monitor and directly assess the fraction of scientific observations impacted by RFI. To this end, flag tables are now automatically ingested and aggregated as part of routine ASKAP operations for all science observations, as a function of frequency and time. The data presented in this paper come from processing all archived data for several ASKAP Survey Science Projects (SSPs). We found that the average amount of flagging due to RFI across the routinely used ‘clean’ continuum science bands is 3%. The ‘clean’ mid band from 1 293 to 1 437 MHz (excluding the 144 MHz below 1293 MHz impacted by radionavigation-satellites which is discarded before processing) is the least affected by RFI, followed by the ‘clean’ low band from 742 to 1 085 MHz. ASKAP SSPs lose most of their data to the mobile service in the low band, aeronautical service in the mid band and satellite navigation service in the 1 510–1 797 MHz high band. We also show that for some of these services, the percentage of discarded data has been increasing year-on-year. SMART provides a unique opportunity to study ASKAP’s changing RFI environment, including understanding and updating the default flagging behaviour, inferring the suitability of and calibrating RFI monitoring equipment, monitoring spectrum management compliance in the Australian Radio Quiet Zone – Western Australia (ARQZWA), and informing the implementation of a suite of RFI mitigation techniques.

The RFI Fast Mitigation Algorithm Based on Block LMS Filter

The radio telescope possesses high sensitivity and strong signal collection capabilities. While receiving celestial radiation signals, it also captures Radio Frequency Interferences (RFIs) introduced by human activities. RFI, as signals originating from sources other than the astronomical targets, significantly impacts the quality of astronomical data. This paper presents an RFI fast mitigation algorithm based on block Least Mean Square (LMS) algorithm. It enhances the traditional adaptive LMS filter by grouping L adjacent time-sampled points into one block and applying the same filter coefficients for filtering within each block. This transformation reduces multiplication calculations and enhances algorithm efficiency by leveraging the time-domain convolution theorem. The algorithm is tested using baseband data from the Parkes 64 m radio telescope’s pulsar observations and simulated data. The results confirm the algorithm’s effectiveness, as the pulsar profile after RFI mitigation closely matches the original pulsar profile.

Radio Frequency Interference Mitigation in Synthetic Aperture Radar Data Based on Instantaneous Spectrum Forward Consecutive Mean Excision

Radio frequency interference (RFI) poses major threats to synthetic aperture radar (SAR) systems. Due to the suppression of useful target signals via high-power RFI, the SAR imaging quality is severely degraded. Nevertheless, existing studies on RFI mitigation mainly focus on narrowband filtering, while wideband RFI mitigation methods are relatively lacking and perform non-robustly. In this paper, an RFI mitigation scheme is proposed based on instantaneous spectrum forward consecutive mean excision (FCME), which is suitable for both narrowband and wideband RFI mitigation. The SAR echo signal is first transformed into a time–frequency (TF) domain through a short-time Fourier transform (STFT). On this basis, the instantaneous spectra polluted via RFI are detected via a kurtosis-based statistical test and then filtered via FCME to achieve RFI mitigation. Finally, connected component analysis is applied as a safety measure so as to avoid the unnecessary loss of useful target signal. The combination of FCME and connected component analysis enables the proposed method to thoroughly filter out RFI while retaining more useful target signals compared with other competing methods. The experimental results on real SAR raw data validate the effectiveness of the proposed method.

PulsarX: A new pulsar searching package

Context. Pulsar surveys with modern radio telescopes are becoming increasingly computationally demanding. This is particularly true for wide field-of-view pulsar surveys with radio interferometers and those conducted in real or quasi-real time. These demands result in data analysis bottlenecks that can limit the parameter space covered by the surveys and diminish their scientific return. Aims. In this paper we address the computational challenge of ‘candidate folding’ in pulsar searching, presenting a novel, efficient approach designed to optimise the simultaneous folding of large numbers of pulsar candidates. We provide a complete folding pipeline appropriate for large-scale pulsar surveys that includes radio frequency interference mitigation, de-dispersion, folding, and parameter optimisation. Methods. By leveraging the fast discrete dispersion measure transform (FDMT) algorithm, we have developed an optimised and cache-friendly implementation that we term the pruned FDMT (pFDMT). This implementation is specifically designed for candidate folding scenarios where the candidates are broadly distributed in dispersion measure space. The pFDMT approach efficiently reuses intermediate processing results and prunes the unused computation paths, resulting in a significant reduction in arithmetic operations. In addition, we propose a novel folding algorithm based on the Tikhonov-regularised least squares method that can improve the time resolution of the pulsar profile. Results. We present the performance of its real-world application as an integral part of two major pulsar search projects conducted with the MeerKAT telescope: the MPIfR-MeerKAT Galactic Plane Survey (MMGPS) and the Transients and Pulsars with MeerKAT (TRAPUM) project. In our processing of approximately 500 candidates, the theoretical number of de-dispersion operations can be reduced by a factor of around 50 when compared to brute-force de-dispersion, which scales with the number of candidates.

Bayesian approach to radio frequency interference mitigation

Bayesian approach to radio frequency interference mitigation

A Search for Extraterrestrial Technosignatures in Archival FAST Survey Data Using a New Procedure

The search for extraterrestrial intelligence (SETI) commensal surveys aim to scan the sky to find possible technosignatures from an extraterrestrial intelligence (ETI). The mitigation of radio frequency interference (RFI) is an important step, especially for the most sensitive Five-hundred-meter Aperture Spherical radio Telescope (FAST), which can detect more weak RFI. In this paper, we propose several new techniques for RFI mitigation and use our procedure to search for ETI signals from the archival data of FAST’s first SETI commensal survey. We detect the persistent narrowband RFI by setting a threshold of the signals’ sky separation and detect the drifting RFI (and potentially other types of RFI) using the Hough transform. We also use the clustering algorithms to remove more RFI and select candidates. The results of our procedure are compared to the earlier work on the same FAST data. We find that our methods, though relatively simpler in computation, remove more RFI (99.9912% compared to 99.9063% in the earlier work) but preserve the simulated ETI signals, except for those (5.1%) severely affected by the RFI. We also report more interesting candidate signals, about a dozen of which are new candidates that were not previously reported. In addition, we find that the proposed Hough transform method, with suitable parameters, also has the potential to remove the broadband RFI. We conclude that our methods can effectively remove the vast majority of the RFI while preserving and finding the candidate signals that we are interested in.

Radio Frequency Interference Mitigation via Nonnegative Matrix Factorization for GNSS

A radio frequency interference (RFI) mitigation approach based on nonnegative matrix factorization (NMF) for global navigation satellite systems (GNSS) signals is proposed. The proposed approach employs NMF to separate the interference from the GNSS signals, and it can be deployed in a supervised or semi-blind manner. The supervised NMF framework assumes prior knowledge about the RFI whereas its semi-blind counterpart does not require any <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information about the RFI. Results indicate that both schemes are able to mitigate narrow and wideband RFI signals, outperforming Kalman, notch filter and wavelet-based techniques, enabling GNSS signal acquisition even in scenarios where the interference is 50 dB stronger than the GNSS signals. In addition, the proposed approach is able to mitigate multiple, different types of RFI corrupting the received GNSS signal.

Robust RFI Excision for Pulsar Signals by a Novel Nonlinear M-type Estimator with an Application to Pulsar Timing

Radio frequency interference (RFI) mitigation for pulsar signals is a long perplexing issue in astrophysical measurements. Linear mitigation methods are often criticized for limited RFI excision range and weakness of RFI modeling. Meanwhile, thresholding methods (e.g., the SumThreshold) suffer from empirical factors. In our opinion, the main defect of the current status is the lack of a concise definition to distinguish signals from RFI with the aid of certain techniques, e.g., sparse representation. This point is the root cause of these problems and also forms our motivation. This paper aims to expand the excision range (e.g., the on-pulse and quasiperiodic RFI) and cut down some empirical factors. The main contribution is that we give a definition and derive a widely practicable nonlinear framework for RFI excision. This framework can overcome the susceptibility of the least-square criterion to RFI, and excise almost all types of RFI once and for all. A robust LnCosh criterion based nonlinear maximum likelihood-type (M-type) penalized smoothing estimator is introduced. The novelty is that this estimator is first embedded into the iterative shrinkage-thresholding algorithm (ISTA) and the fast ISTA. Nonlinearity highlights this method. Curvelet sparsity gives satisfying approximation for pulsar signals containing dispersion feature. Finally, useful signal details will be retrieved from the excision residual by a morphological component analysis. This method is applied to the time-frequency signals collected by the Nanshan 26 m Radio Telescope. The numerical experiments can persuasively prove that it has desired application prospects.

An RFI Mitigation Pipeline for CRAFTS Multi-beam Data Based on Signal Cross-Correlation Function and SumThreshold Algorithm

The increasing radio frequency interference (RFI) is a well-recognized problem in radio astronomy research. Pulsars and Fast Radio Bursts (FRBs) are high-priority science targets of the ongoing Commercial Radio Astronomy FAST Survey (CRAFTS). To improve the quality of RFI removal in searches of pulsars and FRBs based on CRAFTS multi-beam data, we here propose an intuitive but powerful RFI mitigation pipeline (CCF-ST). The “CCF-ST” is a spatial filter constructed by signal cross-correlation function (CCF) and Sum-Threshold (ST) algorithm. The RFI marking result is saved in a “mask” file, a binary format for RFI masks in PRESTO. Three known pulsars, PSR B0525-21, PSR B0621-04, and PSR J0943 + 2252 from CRAFTS L-band 19 beams data are used for evaluation of the performance of CCF-ST in comparison with other methods, such as PRESTO’s “rfifind”, ArPLS-ST and ArPLS-SF. The result shows that CCF-ST can reduce effective data loss rate and improves the detected signal-to-noise ratio of the pulsations by ∼26% and ∼18% respectively compared with PRESTO’s “rfifind” and ArPLS-ST. The CCF-ST also has the advantage of low computational cost, e.g., reducing the time consumption by ∼40% and memory consumption by ∼90% compared with ArPLS-SF. We expect that the new RFI mitigation and analysis toolkit (CCF-ST) demonstrated in this paper can be applied to CRAFTS and other multi-beam telescope observations to improve the data quality and efficiency of pulsar and FRB searches.

A Comparative Study on Radio Frequency Interference Suppression and foF2 Scaling for Ionograms

In this paper, we compare the performances of radio frequency interference (RFI) suppression between radio frequency interference mitigation (RFIM) and frequency-domain adaptive clutter suppression (FACS) algorithms that apply to ionograms for foF2 scaling through true height analysis. We find that the use of RFIM and FACS can both effectively suppress RFIs in ionograms to improve the signal-to-clutter (SCR) of the original ionogram. Overall, the RFI clutter suppressed by FACS is about 5–7 dB better than that by RFIM. In addition, the SCR improvement made by FACS with neighboring range cells in a range of 10–40 is also better than that made by RFIM, which is about 5–10 dB for FACS and 4–8 dB for RFIM, respectively. However, the signal-to-noise ratios (SNRs) of the ionospheric echoes of the ionograms with FACS are about 3–5 dB (2–3 dB) smaller than those of RFIM during daytime (nighttime). Intercomparisons reveal that the correlations of the scaled foF2 values between the original and RFIM ionograms are slightly higher than those between the original and FACS ionograms, while the root-mean-square-error (RMSE) and mean bias of the former are slightly larger than the latter. These results indicate that the use of RFIM and FACS algorithms to suppress RFIs in ionograms can conditionally improve the quality of scaled foF2 using Ke et al.’s autoscaling algorithm. Statistical results reveal that the use of FACS algorithms can result in about a 5–12% improvement in uncertainty and a 9–80% improvement in accuracy, provided that 5–20 neighboring range cells are selected. As for the RFIM algorithm with a threshold of 2–2.5 times standard deviation, the improvements in uncertainty and accuracy are, respectively, about 0.9–10% and 11–90%.

Superconducting Notch Filter for RFI Mitigation in Ground-Based Radio Telescope

This article presents a high-rejection, thin-film high-temperature superconductor, microstrip bandstop filter to prevent a local, and high-power radio-frequency interference (RFI) source from interfering with NASA Goddard Geophysical Astronomical Observatory (GGAO)’s very long baseline interferometry (VLBI) global observing system (VGOS) cryogenic receiver. This filter has an excellent 2.7%, 50-dB-fractional-bandwidth, and a center stopband frequency of 9.41 GHz. It does not contain any narrow or interdigital features as found in some designs, which reduces the fringing electric fields and improves its power handling capability. The YBCO films were grown on 435- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mu$</tex-math></inline-formula> m-thick R-plane sapphire substrate and the anisotropic behavior was modeled and simulated with a high degree of accuracy. The device was tested while cooled to 77 K and the measurements agree well with simulation.

An Injection System for the CHIME/FRB Experiment

Dedicated surveys searching for fast radio bursts (FRBs) are subject to selection effects that bias the observed population of events. Software injection systems are one method of correcting for these biases by injecting a mock population of synthetic FRBs directly into the real-time search pipeline. The injected population may then be used to map intrinsic burst properties onto an expected signal-to-noise ratio (S/N), so long as telescope characteristics such as the beam model and calibration factors are properly accounted for. This paper presents an injection system developed for the Canadian Hydrogen Intensity Mapping Experiment Fast Radio Burst Project (CHIME/FRB). The system was tested to ensure high detection efficiency, and the pulse calibration method was verified. Using an injection population of ∼85,000 synthetic FRBs, we found that the correlation between fluence and S/N for injected FRBs was consistent with that of CHIME/FRB detections in the first CHIME/FRB catalog. We noted that the sensitivity of the telescope varied strongly as a function of the broadened burst width, but not as a function of the dispersion measure. We conclude that some of the machine-learning based Radio Frequency Interference mitigation methods used by CHIME/FRB can be retrained using injection data to increase sensitivity to wide events, and that planned upgrades to the presented injection system will allow for determining a more accurate CHIME/FRB selection function in the near future. We also provide the full injection data sets along with usage tutorials.

Modeling and Mitigation of Radio Frequency Interference for Wireless Devices

This article reviews the electromagnetic framework used to model radio frequency interference (RFI) and the resulting development of mitigation methods. With the rise of IoT devices, wireless devices in which RF antennas are integrated with high-performance digital systems in small form factors suffer from electromagnetic interference, known as RF interference or RF desensitization. The simple yet rigorous framework can be used for a systematic RFI-aware design, saving time and effort for trial-and-error troubleshooting.

A Coastal RFI Mitigation Method for Synthetic Aperture Interferometric Radiometer

This letter pays attention to the mitigation of the Radio Frequency Interference (RFI) sources located at coastal regions for Synthetic Aperture Interferometric Radiometer (SAIR). To remove an RFI at the visibility level, it will be necessary to estimate the Brightness Temperature (BT) of the point source, which is currently done by computing the median of the surrounded pixels to get the background BT, and subtracting it from the original contaminated BT. However, this strategy is based on the assumption that the background BT distribution around the RFI is flat, and this is not reasonable for those RFIs from coastal cities, and could result in estimation deviation. A robust RFI mitigation method is proposed without introducing complex computing. Instead of focusing only on the local area, the basic idea of our method is to step into the wide area, and minimize the overall fluctuation of the pixels affected by the RFI in the BT image. The method tests based on SMOS visibility samples validate the correctness of the developed method.

Radio frequency interference detection and mitigation in the DWD C-band weather radar network

Abstract. C-band weather radar data are commonly compromised by interference from external sources even though weather radars are the primary and therefore privileged user of this frequency band. This is also the case for the radar network of the German Meteorological Service (Deutscher Wetterdienst, DWD). Theoretically, dynamic frequency selection (DFS) by devices operating in the C band should prevent any disturbance of the primary user. In practice, this does not always work as intended under the current regulations. As it is not possible to separate a superimposed interference signal from measured weather radar data, the protection of the frequency band is of utmost importance. Currently, the only available option is to discard the compromised portions of the radar data. Therefore, the current best course of action is to shut down radio frequency interference (RFI) sources as quickly as possible. The automated RFI detection algorithm for the German C-band weather radar network, operational since July 2017, makes use of routinely measured radar moments. Built upon data gathered since 2017, an RFI classification with respect to the severity and duration of RFI sources was first implemented in 2019. An independent verification of the RFI detection algorithm was performed by using a commercially available Wi-Fi adapter, which is directly integrated into the radar receiver. Subsequently, a mitigation workflow was implemented to efficiently identify and shut down detected RFI sources by the German Federal Network Agency (Bundesnetzagentur, BNetzA). By following this workflow with great effort, the number of persistent RFI sources has been decreasing since October 2019, while a steady increase in short-lived RFI sources over the last 5 years also exists. In total, 11 889 RFI sources have been identified from July 2017 to May 2022. Most of these (94.8 %) are such short-lived sources that an unambiguous identification by the BNetzA is, in general, not feasible. However, as stated by the C-band regulations, any non-compliant transmitter compromising the operation of a weather radar has to be shut down. This is important, as even these short-lived RFI sources negatively affect meteorological product generation.

Simulating Spectral Kurtosis Mitigation against Realistic Radio Frequency Interference Signals

We investigate the effectiveness of the statistical radio frequency interference (RFI) mitigation technique spectral kurtosis () in the face of simulated realistic RFI signals. estimates the kurtosis of a collection of M power values in a single channel and provides a detection metric that is able to discern between human-made RFI and incoherent astronomical signals of interest. We test the ability of to flag signals with various representative modulation types, data rates, duty cycles, and carrier frequencies. We flag with various accumulation lengths M and implement multiscale , which combines information from adjacent time-frequency bins to mitigate weaknesses in single-scale . We find that signals with significant sidelobe emission from high data rates are harder to flag, as well as signals with a 50% effective duty cycle and weak signal-to-noise ratios. Multiscale with at least one extra channel can detect both the center channel and sideband interference, flagging greater than 90% as long as the bin channel width is wider in frequency than the RFI.