CLEAN Algorithm Research Articles

Bayesian self-calibration and imaging in very long baseline interferometry

Context. Self-calibration methods with the CLEAN algorithm have been widely employed in very long baseline interferometry (VLBI) data processing in order to correct antenna-based amplitude and phase corruptions present in the data. However, human interaction during the conventional CLEAN self-calibration process can impose a strong effective prior, which in turn may produce artifacts within the final image and hinder the reproducibility of final results. Aims. In this work, we aim to demonstrate a combined self-calibration and imaging method for VLBI data in a Bayesian inference framework. The method corrects for amplitude and phase gains for each antenna and polarization mode by inferring the temporal correlation of the gain solutions. Methods. We use Stokes I data of M87 taken with the Very Long Baseline Array (VLBA) at43 GHz, pre-calibrated using the rPICARD CASA-based pipeline. For antenna-based gain calibration and imaging, we use the Bayesian imaging software resolve. To estimate gain and image uncertainties, we use a variational inference method. Results. We obtain a high-resolution M87 Stokes I image at 43 GHz in conjunction with antenna-based gain solutions using our Bayesian self-calibration and imaging method. The core with counter-jet structure is better resolved, and extended jet emission is better described compared to the CLEAN reconstruction. Furthermore, uncertainty estimation of the image and antenna-based gains allows us to quantify the reliability of the result. Conclusions. Our Bayesian self-calibration and imaging method is able to reconstruct robust and reproducible Stokes I images and gain solutions with uncertainty estimation by taking into account the uncertainty information in the data.

Radar Target Radar Cross-Section Measurement Based on Enhanced Imaging and Scattering Center Extraction.

Accurate measurement of a Radar Cross-Section (RCS) is a critical technical challenge in assessing the stealth performance and scattering characteristics of radar targets. Traditional RCS measurement methods are limited by high costs, sensitivity to environmental conditions, and difficulties in distinguishing local scattering features of targets. To address these challenges, this paper proposes a novel RCS measurement method based on enhanced imaging and scattering center extraction. This method integrates sub-aperture imaging with image fusion techniques to improve imaging quality and enhance the detail of target scattering characteristics. Additionally, an improved sequence CLEAN algorithm is employed to effectively suppress sidelobe effects and ensure the accuracy of scattering center extraction. Experimental results demonstrate that this method achieves higher precision in RCS measurement of complex targets and is particularly effective in environments with strong interference, where it successfully separates the scattering contributions of the target from those of the interference sources. This method offers a new technological approach for precise RCS measurement of radar stealth targets in the future.

TWPT: Through-Wall Position Detection and Tracking System Using IR-UWB Radar Utilizing Kalman Filter-Based Clutter Reduction and CLEAN Algorithm

Against the backdrop of rapidly advancing Artificial Intelligence of Things (AIOT) and sensing technologies, there is a growing demand for indoor location-based services (LBSs). This paper proposes a through-the-wall localization and tracking (TWPT) system based on an improved ultra-wideband (IR-UWB) radar to achieve more accurate localization of indoor moving targets. The TWPT system overcomes the limitations of traditional localization methods, such as multipath effects and environmental interference, using the high penetration and high accuracy of IR-UWB radar based on multi-sensor fusion technology. In the study, an improved Kalman filter (KF) algorithm is used for clutter reduction, while the CLEAN algorithm, combined with a compensation mechanism, is utilized to increase the target detection probability. Finally, a three-point localization estimation algorithm based on multi-IR-UWB radar is applied for the precise position and trajectory estimation of the target. Experimental validation indicates the TWPT system achieves a high positioning accuracy of 96.91%, with a root mean square error (RMSE) of 0.082 m and a Maximum Position Error (MPE) of 0.259 m. This study provides a highly accurate and precise solution for indoor TWPT based on IR-UWB radar.

A Novel Two-dimensional Low-redundancy Array Design for Solar Radio Imaging

The radioheliograph is an extensive array of antennas operating on the principle of aperture synthesis to produce images of the Sun. The image acquired by the telescope results from convoluting the Sun’s true brightness distribution with the antenna array’s directional pattern. The imaging quality of the radioheliograph is affected by a multitude of factors, with the performance of the “dirty beam” being simply one component. Other factors such as imaging methods, calibration techniques, clean algorithms, and more also play a significant influence on the resulting image quality. As the layout of the antenna array directly affects the performance of the dirty beam, the design of an appropriate antenna configuration is critical to improving the imaging quality of the radioheliograph. Based on the actual needs of observing the Sun, this work optimized the antenna array design and proposed a two-dimensional low-redundancy array. The proposed array was compared with common T-shaped arrays, Y-shaped arrays, uniformly spaced circular arrays, and three-arm spiral arrays. Through simulations and experiments, their performance in terms of sampling point numbers, UV coverage area, beam-half width, sidelobe level, and performance in the absence of antennas are compared and analyzed. It was found that each of these arrays has its advantages, but the two-dimensional low-redundancy array proposed in this paper performs best in overall evaluation. It has the shortest imaging calculation time among the array types and is highly robust when antennas are missing, making it the most suitable choice.

The Distribution of Pc5 Ultralow-frequency Waves at Geostationary Orbit

Using GOES magnetometer data spanning over 30 yr, we study the distribution of Pc5 ultralow-frequency waves at geosynchronous orbit. The CLEAN algorithm was applied to detect narrowband Pc5 waves. This method was first used in astronomical radio interferometry, but has since been applied to many other areas. The algorithm allows for identification of multiple individual peaks within the same power spectral density. We found close to 30,000 events in each magnetic field component in field-aligned coordinates. With some exceptions, results were in agreement with previous studies. Wave occurrence along magnetic local time (MLT) was higher in the 18–23 MLT sector. However, the largest accumulated wave amplitudes were observed on the dayside, peaking close to local noon. As expected, wave amplitudes were larger during more active solar wind conditions. For the radial and parallel components, amplitudes were maximum at the interplanetary magnetic field clock angle of −30° to −45°. The resultant database of Pc5 ultralow-frequency wave events can be used to constrain radiation belt models, while the CLEAN method provides a unique technique to automatically detect narrowband plasma waves in the magnetosphere and beyond.

Radio Frequency Interference Mitigation in Data and Image Bi-Domains for an Aperture Synthesis Radiometer

For synthetic aperture microwave radiometers, the problem of Radio Frequency Interference (RFI) is becoming more and more serious, which affects both the scientific retrieval of remote sensing data and the imaging quality of brightness temperature (BT) images. In the visibility data domain, the array factor synthesis algorithm is commonly employed to mitigate RFI sources and their Gibbs trailing. In the BT image domain, the CLEAN algorithm is typical applied to mitigate RFI sources and their Gibbs trailing. However, the array factor synthesis algorithm can result in anomalous BT points near the “zero trap” region, and the CLEAN algorithm will miss some BT points below a certain threshold. In this paper, a Bi-domain combined mitigation algorithm is proposed to mitigate RFI sources and their Gibbs trailing. Following initial mitigation in the visibility data domain, dual thresholds are applied to normalize anomalous BT points near the “zero trap” region, thereby enhancing imaging quality. The effectiveness of the Bi-domain combined mitigation algorithm is verified by using both measured data from SMOS L1A and simulated data. The experimental results demonstrate that the Bi-domain combined mitigation algorithm is superior to the array factor synthesis algorithm and the CLEAN algorithm in mitigating RFI sources and their Gibbs trailing.

Human movement detection and classification capabilities using passive Wi-Fi based radar

Human detection and classification via Wi-Fi transmission have received a lot of attention in recent years as crucial facilitators in security and human-computer interaction (HCI). The passive Wi-Fi radar (PWR) system used by previous researchers applied cross-ambiguity function (CAF) and CLEAN algorithms to process the detected signals. This paper explores the feasibility and viability of a PWR system in detecting and classifying human movements without utilizing CAF and CLEAN algorithms. The movements are performed by four participants but with comparable body sizes and heights. Three daily human movements are investigated namely walking, bending, and sitting, with each participant performing each movement 24 times, providing a total of 96 samples per activity. The system is evaluated based on the consistency of the signal pattern in a frequency domain and the percentage accuracy is assessed using an artificial neural network (ANN) classifier and trained using a leave-one-out cross-validation (LOOCV) method. The frequency domain results reveal that the signals are consistent, with no noticeable variations or changes in the voltage intensity or shape of the main lobe. The classification of the movements shows that the classifier has an overall accuracy of 97.6%.

Iterative removal of sources to model the turbulent electromotive force

ABSTRACT We describe a novel method to compute the components of dynamo tensors from direct magnetohydrodynamic (MHD) simulations. Our method relies upon an extension and generalization of the standard Högbom CLEAN algorithm widely used in radio astronomy to systematically remove the impact of the strongest beams on to the corresponding image. This generalization, called the Iterative Removal of Sources (IROS) method, has been adopted here to model the turbulent electromotive force (EMF) in terms of the mean magnetic fields and currents. Analogous to the CLEAN algorithm, IROS treats the time series of the mean magnetic field and current as beams that convolve with the dynamo coefficients which are treated as (clean) images to produce the EMF time series (the dirty image). We apply this method to MHD simulations of galactic dynamos, to which we have previously employed other methods of computing dynamo coefficients such as the test-field method, the regression method, as well as local and non-local versions of the singular value decomposition (SVD) method. We show that our new method reliably recovers the dynamo coefficients from the MHD simulations. It also allows priors on the dynamo coefficients to be incorporated easily during the inversion, unlike in earlier methods. Moreover, using synthetic data, we demonstrate that it may serve as a viable post-processing tool in determining the dynamo coefficients, even when the power of additive noise to the EMF is twice as much the actual EMF.

Identifying synergies between VLBI and STIX imaging

Context. Reconstructing an image from noisy, sparsely sampled Fourier data is an ill-posed inverse problem that occurs in a variety of subjects within science, including data analysis for Very Long Baseline Interferometry (VLBI) and the Spectrometer/Telescope for Imaging X-rays (STIX) with respect to solar observations. The need for high-resolution, high-fidelity imaging fosters the active development of a range of novel imaging algorithms in a variety of different algorithmic settings. However, despite these ongoing, parallel developments, such synergies remain unexplored. Aims. We study, for the first time, the synergies between the data analysis for the STIX instrument and VLBI. In particular, we compare the methodologies that have been developed in both fields and evaluate their potential. In this way, we identify key trends in the performance of several algorithmic ideas and draw recommendations for the future spending of resources in the study and implementation of novel imaging algorithms. Methods. To this end, we organized a semi-blind imaging challenge with data sets and source structures that are typical for sparse VLBI, specifically in the context of the Event Horizon Telescope (EHT) as well as STIX observations. We used 17 different algorithms from both communities, from six different imaging frameworks, in the challenge, making this work the largest scale code comparison for STIX and VLBI to date. Results. We identified strong synergies between the two communities, as proven by the success of the imaging methods proposed for STIX in imaging VLBI data sets and vice versa. Novel imaging methods outperform the standard CLEAN algorithm significantly in every test case. Improvements over the performance of CLEAN offer deeper updates to the inverse modeling pipeline necessary or, consequently, the possibility to replace inverse modeling with forward modeling. Entropy-based methods and Bayesian methods perform best on STIX data. The more complex imaging algorithms utilizing multiple regularization terms (recently proposed for VLBI) add little to no additional improvements for STIX. However, they do outperform the other methods on EHT data, which correspond to a larger number of angular scales. Conclusions. This work demonstrates the great synergy between the STIX and VLBI imaging efforts and the great potential for common developments. The comparison identifies key trends on the efficacy of specific algorithmic ideas for the VLBI and the STIX setting that may evolve into a roadmap for future developments.

RCS measurement and ISAR imaging radar in VHF/UHF radio channels

ABSTRACT The Radar Cross Section (RCS) plays a crucial role in radar systems, serving to detect and characterize radar targets. Moreover, the need to penetrate foliage and capture images of man-made structures like roads and buildings has driven the development of very high frequency/ultra high frequency (VHF/UHF)-based radar systems. In this context, we propose a novel polyphase-coded spread spectrum (PCSS) radar system designed for RCS measurements and ISAR imaging over VHF/UHF radio channels in open environments. To provide a comprehensive understanding, we examined the open range, considering factors such as amplitude distribution function, root mean square (RMS) delay, and coherence bandwidth, followed by a link budget analysis. We also conducted a comparative study of RCS measurements to evaluate the performance of the proposed radar system, demonstrating its effectiveness through theoretical calculations. Furthermore, we assessed the radar imaging capabilities by applying the CLEAN algorithm to improve the quality of the generated images.

Residual Entropy as a Diagnostic and Stopping Metric for CLEAN

We propose the use of entropy, measured from the spatial and flux distribution of pixels in the residual image, as a potential diagnostic and stopping metric for the CLEAN algorithm. Despite its broad success as the standard deconvolution approach in radio interferometry, finding the optimum stopping point for the iterative CLEAN algorithm is still a challenge. We show that the entropy of the residual image, measured during the final stages of CLEAN, can be computed without prior knowledge of the source structure or expected noise levels, and that finding the point of maximum entropy as a measure of randomness in the residual image serves as a robust stopping criterion. We also find that, when compared to the expected thermal noise in the image, the maximum entropy of the residuals is a useful diagnostic that can reveal the presence of data editing, calibration, or deconvolution issues that may limit the fidelity of the final CLEAN map.

A membership-based resampling and cleaning algorithm for multi-class imbalanced overlapping data

Real-world datasets frequently have an imbalanced class distribution, which significantly degrades classification performance. However, some studies have suggested that the adverse effects of class imbalance occur only when datasets have other intrinsic characteristics (such as class overlap, noise, and data scarcity). Noise and class overlap have the greatest effect. To deal with other intrinsic characteristics that affect classification performance in a multi-class environment, such as class overlap, noise, and data scarcity, we propose a method that can directly handle multi-class overlapping data, called the membership-based multi-class resampling and cleaning (MC-MBRC) algorithm. The proposed method divides samples into safe, overlapping, and noisy areas based on their membership degrees, and then according to the influence of the samples in each area on classification performance, it performs various operations such as noise removal, interpolation oversampling, and energy-based cleaning of the overlapping region. An extensive comparison using various datasets shows that, compared with state-of-the-art methods, the proposed method makes significant statistical improvements in different classification performance metrics and is robust to data containing class overlap and label noise. Furthermore, for datasets that do not have complex intrinsic features, MC -MBRC will not significantly degrade classification performance.

Measurement and Investigation of HB-UWB Transmission Link for BAN System

The short-range wireless multimedia is consider used a ultra wideband (UWB) technology for human body mobile and multimedia applications shows promise for wireless multi-media systems. Based on IEEE 802.15.6, wireless body area networks (BAN) require understanding the human body’s effects on channel characteristics. This paper presents how to evaluation of human body ulra-wideband (HB-UWB) transmission with line-of-sight and non-line-of-sight scenario. Our research aims to enhance HB-UWB channel propagation on the body media by employing with CLEAN algorithm to eliminate noise. This research leverage findings from previous studies to facilitate performance comparison. Furthermore, for analyze system performance using the CLEAN algorithm at different body positions. The measurement setup covers band the FCC regulated from 3.0 GHz to 11 GHz. It includes the tested with wideband antenna and vector network analyzer (VNA). HB-UWB characteristics are shows in the path loss and power delay profile are discussed as relevant parameters. This research very useful for design and evaluation of human body mobile network and wireless multimedia systems.

M-STCP: an online ship trajectory cleaning and prediction algorithm using matrix neural networks

Accurate prediction of ship trajectories is crucial to guarantee the safety of maritime navigation. In this paper, a matrix neural network-based online ship track cleaning and prediction algorithm called M-STCP is suggested to forecast ship tracks. Firstly, the GPS-provided historical ship trajectory data is cleaned, and the data cleaning process is finished using the anomaly point algorithm. Secondly, the trajectory is input into the matrix neural network for training and prediction, and the algorithm is improved by using Kalman filtering, which reduces the influence of noise on the prediction results and improves the prediction accuracy. In the end, the effectiveness of the method is verified using real GPS trajectory data, and compared with the GRU model and long-short-term memory networks. The M-STCP method can improve the prediction accuracy of ship trajectory to 89.44%, which is 5.17% higher than LSTM and 1.82% higher than GRU, effectively improving the prediction accuracy and time efficiency.

An Excess Kurtosis People Counting System Based on 1DCNN-LSTM Using Impulse Radio Ultra-Wide Band Radar Signals

As the Artificial Intelligence of Things (AIOT) and ubiquitous sensing technologies have been leaping forward, numerous scholars have placed a greater focus on the use of Impulse Radio Ultra-Wide Band (IR-UWB) radar signals for Region of Interest (ROI) population estimation. To address the problem concerning the fact that existing algorithms or models cannot accurately detect the number of people counted in ROI from low signal-to-noise ratio (SNR) received signals, an effective 1DCNN-LSTM model was proposed in this study to accurately detect the number of targets even in low-SNR environments with considerable people. First, human-induced excess kurtosis was detected by setting a threshold using the optimized CLEAN algorithm. Next, the preprocessed IR-UWB radar signal pulses were bundled into frames, and the resulting peaks were grouped to develop feature vectors. Subsequently, the sample set was trained based on the 1DCNN-LSTM algorithm neural network structure. In this study, the IR-UWB radar signal data were acquired from different real environments with different numbers of subjects (0–10). As indicated by the experimental results, the average accuracy of the proposed 1DCNN-LSTM model for the recognition of people counting reached 86.66% at ROI. In general, a high-accuracy, low-complexity, and high-robustness solution in IR-UWB radar people counting was presented in this study.

SENTIMENT CLASSIFICATION OF TWEETS WITH EXPLICIT WORD NEGATIONS AND EMOJI USING DEEP LEARNING

The widespread use of social media platforms such as Twitter, Instagram, Facebook, and LinkedIn have had a huge impact on daily human interactions and decision-making. Owing to Twitter's widespread acceptance, users can express their opinions/sentiments on nearly any issue, ranging from public opinion, a product/service, to even a specific group of people. Sharing these opinions/sentiments results in a massive production of user content known as tweets, which can be assessed to generate new knowledge. Corporate insights, government policy formation, decision-making, and brand identity monitoring all benefit from analyzing the opinions/sentiments expressed in these tweets. Even though several techniques have been created to analyze user sentiments from tweets, social media engagements include negation words and emoji elements that, if not properly pre-processed, would result in misclassification. The majority of available pre-processing techniques rely on clean data and machine learning algorithms to annotate sentiment in unlabeled texts. In this study, we propose a text pre-processing approach that takes into consideration negation words and emoji characteristics in text data by translating these features into single contextual words in tweets to minimize context loss. The proposed preprocessor was evaluated on benchmark Twitter datasets using four deep learning algorithms: Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), and Artificial Neural Network (ANN). The results showed that LSTM performed better than the approaches already discussed in the literature, with an accuracy of 96.36%, 88.41%, and 95.39%. The findings also suggest that pre-processing information like emoji and explicit word negations aids in the preservation of sentimental information. This appears to be the first study to classify sentiments in tweets while accounting for both explicit word negation conversion and emoji translation.

Extending the Spectral Difference Method with Divergence Cleaning (SDDC) to the Hall MHD Equations

The Hall Magnetohydrodynamic (MHD) equations are an extension of the standard MHD equations that include the “Hall” term from the general Ohm’s law. The Hall term decouples ion and electron motion physically on the ion inertial length scales. Implementing the Hall MHD equations in a numerical solver allows more physical simulations for plasma dynamics on length scales less than the ion inertial scale length but greater than the electron inertial length. The present effort is an important step towards producing physically correct results to important problems, such as the Geospace Environmental Modeling (GEM) Magnetic Reconnection problem. The solver that is being modified is currently capable of solving the resistive MHD equations on unstructured grids using the spectral difference scheme which is an arbitrarily high-order method that is relatively simple to parallelize. The GEM Magnetic Reconnection problem is used to evaluate whether the Hall MHD equations have been correctly implemented in the solver using the spectral difference method with divergence cleaning (SDDC) algorithm by comparing against the reconnection rates reported in the literature.

The effect of artifact rejection on the performance of a convolutional neural network based algorithm for binary EEG data classification

ObjectiveEvaluate the effect of artifact rejection on the performance of a Convolutional Neural Network (CNN) based algorithm for classification of abnormal and normal electroencephalography (EEG) data. MethodsWe developed an automated CNN-based clean versus artifact classification algorithm and applied this to abnormal and normal EEG data. Additionally, algorithms for abnormal versus normal classification were developed with and without artifact rejection beforehand. For each algorithm, five CNNs were trained using 5-fold cross-validation and the majority vote of these was used for classification of test data. We compared bootstrap test accuracies and the number of training epochs required between the scenario in which artifacts were rejected and the scenario in which they were not. ResultsThe clean versus artifact classification algorithm had a test accuracy of 85% (recall:89%, precision:82%). Bootstrap test accuracies of the abnormal versus normal classification algorithms were similar with and without artifact rejection beforehand (mean [95% CI]:84% [77%-91%] and 84% [76%-91%], respectively). For the five cross-validation runs, respectively −8%, 22%, 31%, −94% and 26% less training epochs were required when artifacts were rejected beforehand. ConclusionIn our study, EEG artifact rejection did not improve CNN-based abnormal versus normal classification performance, but did slightly speed up training. SignificanceIn our study, artifact rejection was not required. The possibility to omit this step lowers the threshold for applying CNNs to EEG data and using these in clinical practice. Future studies should confirm whether this step could also be omitted in case other artificial intelligence techniques are applied to different datasets.

Study on the Method of Extracting Plasma Lines Based on Sanya Incoherent Scatter Radar

The plasma lines observed by Sanya incoherent scatter radar (SYISR) are dependent on the enhancement of Langmuir waves due to superthermal photoelectrons generated by solar EUV radiation. The plasma line power spectrum can be obtained using long-pulse and alternating-code transmission signals during the period from sunrise to noon almost every day. For the power spectrum of the long pulse, the CLEAN algorithm that has been applied in this field is used to verify the feasibility of this method for SYISR in only a few cases. However, it is difficult to deal with alternating code with such a low SNR using the general deconvolution method. The irreversible-migration filtering (IMF) method has been developed to separate signal noise from the measurements of the alternating code. Some experimental results from the SYISR measurements validate the excellent performance of the IMF method for alternating code. Additionally, an example observation of the electron density with a high time and range resolution is derived. The results show that plasma line detection can be a powerful new observational capability for SYISR as an ionospheric experimental mode for ionospheric calibration, when possible, which can be simultaneously measured with the ion line for constant radar calibration in the standard fitting of the ion line.

Scalable precision wide-field imaging in radio interferometry: I. uSARA validated on ASKAP data

ABSTRACT As Part I of a paper series showcasing a new imaging framework, we consider the recently proposed unconstrained Sparsity Averaging Reweighted Analysis (uSARA) optimization algorithm for wide-field, high-resolution, high-dynamic range, monochromatic intensity imaging. We reconstruct images from real radio-interferometric observations obtained with the Australian Square Kilometre Array Pathfinder (ASKAP) and present these results in comparison to the widely used, state-of-the-art imager WSClean . Selected fields come from the ASKAP Early Science and Evolutionary Map of the Universe (EMU) Pilot surveys and contain several complex radio sources: the merging cluster system Abell 3391-95, the merging cluster SPT-CL 2023-5535, and many extended, or bent-tail, radio galaxies, including the X-shaped radio galaxy PKS 2014-558 and ‘the dancing ghosts’, known collectively as PKS 2130-538. The modern framework behind uSARA utilizes parallelization and automation to solve for the w -effect and efficiently compute the measurement operator, allowing for wide-field reconstruction over the full field-of-view of individual ASKAP beams (up to ∼3.3° each). The precision capability of uSARA produces images with both super-resolution and enhanced sensitivity to diffuse components, surpassing traditional CLEAN algorithms that typically require a compromise between such yields. Our resulting monochromatic uSARA-ASKAP images of the selected data highlight both extended, diffuse emission and compact, filamentary emission at very high resolution (up to 2.2 arcsec), revealing never-before-seen structure. Here we present a validation of our uSARA-ASKAP images by comparing the morphology of reconstructed sources, measurements of diffuse flux, and spectral index maps with those obtained from images made with WSClean .