Radar Image Sequences Research Articles

Three-Dimensional Geometry Reconstruction Method for Slowly Rotating Space Targets Utilizing ISAR Image Sequence

For a slowly rotating space target (SRST) with a fixed axis, traditional 3D geometry reconstruction methods become invalid as the projection vectors cannot be formed without accurate target rotational parameters. To tackle this problem, we present a new technique for 3D geometry reconstruction by using inverse synthetic aperture radar (ISAR) image sequence energy accumulation (ISEA). Firstly, by constituting the motion model of SRST, an explicit expression is derived to describe the relative geometric relationship between the 3D geometry and ISAR image sequence. Then accurate rotational parameters and the 3D geometry of SRST can be estimated by combining the idea of the ISEA method and quantum-behaved particle swarm optimization (QPSO). Compared with the ISEA method, which can be only applied to triaxial stabilized space targets, the proposed method can achieve 3D geometry reconstruction of SRST. Experimental results based on the simulated point model and simulated electromagnetic computer aided design (CAD) model validate the effectiveness and robustness of the proposed method.

Sea Clutter Suppression and Target Detection Algorithm of Marine Radar Image Sequence Based on Spatio-Temporal Domain Joint Filtering.

In marine radar target detection, sea clutter will cause a large number of missed alarms and false alarms, which will affect the accuracy of target detection. In order to suppress sea clutter effectively, a sea clutter suppression and target detection algorithm of marine radar image sequence based on spatio-temporal domain joint filtering is proposed in this paper. The proposed method is to add a sea clutter suppression link before detecting the target. Firstly, the marine radar image sequence is transformed into three-dimensional frequency wavenumber domain by three-dimensional fast Fourier transform (3D-FFT), and then the three-dimensional image spectrum is obtained. According to the fact that the sea clutter spectrum obtained from the image spectrum satisfies the dispersion relation of linear wave theory in the three-dimensional frequency wavenumber domain, a sea clutter model is established. Then, through the established sea clutter model, a spatio-temporal domain joint sea clutter suppressor is designed to filter the image spectrum. After that, the filtered image spectrum is transformed by three-dimensional inverse fast Fourier transform (3D-IFFT) to obtain the image sequence in which sea clutter is suppressed. Finally, target detection is carried out for sea clutter suppressed image sequence. The method is validated by using the real data of X-band marine radar. Compared with the classical Empirical mode decomposition (EMD) method, the improvement of signal-to-noise ratio (SNR) is more obvious, and SNR can be increased by 15.3 db at most. In addition, compared with target detection on original images directly, the proposed method has excellent detection rate and can increase detection rates by at least 8%.

SEN12MS-CR-TS: A Remote-Sensing Data Set for Multimodal Multitemporal Cloud Removal

About half of all optical observations collected via spaceborne satellites are affected by haze or clouds. Consequently, cloud coverage affects the remote-sensing practitioner’s capabilities of a continuous and seamless monitoring of our planet. This work addresses the challenge of optical satellite image reconstruction and cloud removal by proposing a novel multimodal and multitemporal data set called SEN12MS-CR-TS. We propose two models highlighting the benefits and use cases of SEN12MS-CR-TS: First, a multimodal multitemporal 3-D convolution neural network that predicts a cloud-free image from a sequence of cloudy optical and radar images. Second, a sequence-to-sequence translation model that predicts a cloud-free time series from a cloud-covered time series. Both approaches are evaluated experimentally, with their respective models trained and tested on SEN12MS-CR-TS. The conducted experiments highlight the contribution of our data set to the remote-sensing community as well as the benefits of multimodal and multitemporal information to reconstruct noisy information. Our data set is available at <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><uri>https://patrickTUM.github.io/cloud_removal</uri></i> .

A New Scatterer Trajectory Association Method for ISAR Image Sequence Utilizing Multiple Hypothesis Tracking Algorithm

Scatterer trajectory association is a critical step of 3-D target reconstruction from the inverse synthetic aperture radar (ISAR) image sequence. To cope with the complex scatterer trajectory association of the noncooperative target, a novel method based on multiple hypothesis tracking (MHT) algorithm is proposed. First, the scatterer trajectory association situation is modeled as a multiple hypothesis tree, in which each branch represents a possible association. Then, to generate the hypothesis in each specific branch, a general trajectory motion model is constructed and the parameters are estimated based on the current trajectory association situation. The parameter estimation precision will increase with the growth of the trajectory length. Besides, to eliminate the influence of inaccurate association initialization and direction selection, a fusion algorithm is proposed to merge the forward and backward associated trajectories. Finally, experimental results based on the simulated data and electromagnetic data verify the effectiveness and robustness of the proposed method.

Wavelet-Based 2-D Sea Surface Reconstruction Method From Nearshore X-Band Radar Image Sequences

A 2D Wavelet-based radar analysis method for sea surface reconstruction is presented. The Wavelet-based Sea Surface Reconstruction (WSSR) method resolves the traditional limitations of the Fourier analysis, such as the requirement of the homogeneity and periodicity of the analyzed image sequences which is hardly found in nearshore wave shoaling conditions. The method is based on a filtration of the corresponding 2D wavelet spectra of the radar images in the pseudo wavevector domain and includes band-pass filtration and phase correction as well as the application of an empirical modulation transfer function. In extending the method to 2D images, the main numerical and computational challenge is to realize the inverse wavelet transform. This challenge is overcome by using the Dirac delta-function as a mother wavelet in the synthesis step to reduce the computational requirements of the integration procedures via an analytical solution. A statistical comparison between the original and the reconstructed surface elevations shows mean absolute error on the level of 6-14% of the significant wave height, which is quite an improvement when compared to conventional 2D Fourier-based techniques (≈27% for the tested case). Another main advantage of the WSSR method is its near real-time capabilities, as it can work on a single or a small number of images unlike conventional methods, which require a few minutes integration time. In addition to various synthetic data consistency checks, the WSSR was employed on real radar data collected in Haifa bay, Israel, with good qualitative agreements in comparison with a near wave buoy.

Spatial–Temporal Convolutional Gated Recurrent Unit Network for Significant Wave Height Estimation From Shipborne Marine Radar Data

Spatial–temporal features are extracted from X-band marine radar backscatter image sequences via deep neural networks to estimate sea surface significant wave heights (SWHs). A convolutional neural network (CNN) is first constructed based on the pretrained GoogLeNet to estimate SWH using multiscale deep spatial features extracted from each radar image. Since the CNN-based model cannot analyze the temporal behavior of wave signatures in radar image sequences, a gated recurrent unit (GRU) network is concatenated after the deep convolutional layers from the CNN to build a convolutional GRU (CGRU)-based model, which generates spatial–temporal features for SWH estimation. Both the CNN and CGRU-based models are trained and tested using shipborne marine radar data collected during a sea trial off the East Coast of Canada, while simultaneous SWHs measured by nearby buoys are used as ground truths for model training and reference. Experimental results show that compared to the classic signal-to-noise ratio (SNR)-based method, both models improve estimation accuracy and computational efficiency significantly, with a reduction of RMSD by 0.32 m (CNN) and 0.35 m (CGRU), respectively. It is also found that under rainy conditions, CGRU outperforms SNR and CNN-based models by reducing the RMSD from around 0.90 to 0.54 m.

Reconstruction of Missing Data in Weather Radar Image Sequences Using Deep Neuron Networks

Missing data in weather radar image sequences may cause bias in quantitative precipitation estimation (QPE) and quantitative precipitation forecast (QPF) studies, and also the obtainment of corresponding high-quality QPE and QPF products. The traditional approaches that are used to reconstruct missing weather radar images replace missing frames with the nearest image or with interpolated images. However, the performance of these approaches is defective, and their accuracy is quite limited due to neglecting the intensification and disappearance of radar echoes. In this study, we propose a deep neuron network (DNN), which combines convolutional neural networks (CNNs) and bi-directional convolutional long short-term memory networks (CNN-BiConvLSTMs), to address this problem and establish a deep-learning benchmark. The model is trained to be capable of dealing with arbitrary missing patterns by using the proposed training schedule. Then the performances of the model are evaluated and compared with baseline models for different missing patterns. These baseline models include the nearest neighbor approach, linear interpolation, optical flow methods, and two DNN models three-dimensional CNN (3DCNN) and CNN-ConvLSTM. Experimental results show that the CNN-BiConvLSTM model outperforms all other baseline models. The influence of data quality on interpolation methods is further investigated, and the CNN-BiConvLSTM model is found to be basically uninfluenced by less qualified input weather radar images, which reflects the robustness of the model. Our results suggest good prospects for applying the CNN-BiConvLSTM model to improve the quality of weather radar datasets.

Multi-Temporal Small Baseline Interferometric SAR Algorithms: Error Budget and Theoretical Performance

Multi-temporal interferometric synthetic aperture radar (MT-InSAR) techniques are well recognized as useful tools for detecting and monitoring Earth’s surface temporal changes. In this work, the fundamentals of error noise propagation and perturbation theories are applied to derive the ground displacement products’ theoretical error bounds of the small baseline (SB) differential interferometric synthetic aperture radar algorithms. A general formulation of the least-squares (LS) optimization problem, representing the SB methods implementation’s core, was adopted in this research study. A particular emphasis was placed on the effects of time-uncorrelated phase unwrapping mistakes and time-inconsistent phase disturbances in sets of SB interferograms, leading to artefacts in the attainable InSAR products. Moreover, this study created the theoretical basis for further developments aimed at quantifying the error budget of the time-uncorrelated phase unwrapping mistakes and studying time-inconsistent phase artefacts for the generation of InSAR data products. Some experiments, performed by considering a sequence of synthetic aperture radar (SAR) images collected by the ASAR sensor onboard the ENVISAT satellite, supported the developed theoretical framework.

On the Detection and Long-Term Path Visualisation of A-68 Iceberg

The article presents a methodology for examining a temporal sequence of synthetic aperture radar (SAR) images, as applied to the detection of the A-68 iceberg and its drifting trajectory. Using an improved image processing scheme, the analysis covers a period of eighteen months and makes use of a set of Sentinel-1 images. A-68 iceberg calved from the Larsen C ice shelf in July 2017 and is one of the largest icebergs observed by remote sensing on record. After the calving, there was only a modest decrease in the area (about 1%) in the first six months. It has been drifting along the east coast of the Antarctic Peninsula, and is expected to continue its path for more than a decade. It is important to track the huge A-68 iceberg to retrieve information on the physics of iceberg dynamics and for maritime security reasons. Two relevant problems are addressed by the image processing scheme presented here: (a) How to achieve quasi-automatic analysis using a fuzzy logic approach to image contrast enhancement, and (b) The use of ferromagnetic concepts to define a stochastic segmentation. The Ising equation is used to model the energy function of the process, and the segmentation is the result of a stochastic minimization.

An Energy Spectrum Algorithm for Wind Direction Retrieval From X-Band Marine Radar Image Sequences

In this study, an energy spectrum (ES) algorithm is proposed to retrieve wind direction from X -band marine radar image sequences. This algorithm is based on utilizing the occlusion area zero-pixel percentage to distinguish rain-free and rain-contaminated radar data. And then the rain-contaminated images are detected and discarded. The effect of radar radial attenuation in radar image sequences is modified by the piecewise fitting technique. Wind direction is determined from rain-free and radial correction data, based on the ES of small-scale wind streaks. The ES of small-scale wind streaks is obtained by establishing an ES scale separation filter. Based on the wind streak characteristics, a 2-D fast Fourier transform is used to obtain the ES of radar images. The wind streak characteristics are derived from the distribution of the azimuth normalization radar cross section. The proposed algorithm is tested using data collected from X -band radar images and in situ anemometer data from the coast of the East China Sea. Compared with the anemometer data, after using the proposed algorithm, the root-mean-square difference for wind direction is 12.13°, which is an acceptable result for engineering application.

Space Target Attitude Estimation From ISAR Image Sequences With Key Point Extraction Network

Attitude determination of space target from an inverse synthetic aperture radar (ISAR) image sequence is an important but difficult task, because of the complex electromagnetic scattering characteristics of the target. To autonomously estimate the attitude of the space target from the ISAR image sequence, this letter introduces the key point feature extraction network (KPEN), and proposes a novel method for attitude determination via the key point feature extraction. The geometric relationship between the target attitude parameters and the key point feature is established through range-Doppler (RD) projection matrix. With the help of the key point extraction network, key point feature is extracted from ISAR image. Then the target attitude and its component size are autonomously recovered through key point feature by solving a nonlinear optimization. Extensive experiments show the effectiveness and superiority of the proposal.

Wave Height Estimation From X-Band Nautical Radar Images Using Temporal Convolutional Network

In this article, a temporal convolutional network (TCN)-based model is proposed to retrieve significant wave height ( $H_s$ ) from X-band nautical radar images. Three types of features are first extracted from radar image sequences based on signal-to-noise ratio (SNR), ensemble empirical mode decomposition (EEMD), and gray level cooccurrence matrix methods, respectively. Then, feature vectors are input into the proposed TCN-based regression model to produce $H_s$ estimation. Radar data are collected from a moving vessel at the East Coast of Canada, as well as the simultaneous wave data measured by several wave buoys deployed nearby are used for model training and testing. Experimental results after averaging show that TCN-based model further improves the $H_s$ estimation accuracy, with reductions of root-mean-square errors by 0.33 and 0.10 m, respectively, compared to the SNR-based and the EEMD-based linear fitting methods. It has also been found that under the same feature extraction scheme, TCN outperforms other machine learning-based algorithms including support vector regression and the gated recurrent unit network.

Quantifying the Location Error of Precipitation Nowcasts

In precipitation nowcasting, it is common to track the motion of precipitation in a sequence of weather radar images and to extrapolate this motion into the future. The total error of such a prediction consists of an error in the predicted location of a precipitation feature and an error in the change of precipitation intensity over lead time. So far, verification measures did not allow isolating the extent of location errors, making it difficult to specifically improve nowcast models with regard to location prediction. In this paper, we introduce a framework to directly quantify the location error. To that end, we detect and track scale-invariant precipitation features (corners) in radar images. We then consider these observed tracks as the true reference in order to evaluate the performance (or, inversely, the error) of any model that aims to predict the future location of a precipitation feature. Hence, the location error of a forecast at any lead time Δt ahead of the forecast time t corresponds to the Euclidean distance between the observed and the predicted feature locations at t + Δt. Based on this framework, we carried out a benchmarking case study using one year worth of weather radar composites of the German Weather Service. We evaluated the performance of four extrapolation models, two of which are based on the linear extrapolation of corner motion from t − 1 to t (LK-Lin1) and t − 4 to t (LK-Lin4) and the other two are based on the Dense Inverse Search (DIS) method: motion vectors obtained from DIS are used to predict feature locations by linear (DIS-Lin1) and Semi-Lagrangian extrapolation (DIS-Rot1). Of those four models, DIS-Lin1 and LK-Lin4 turned out to be the most skillful with regard to the prediction of feature location, while we also found that the model skill dramatically depends on the sinuosity of the observed tracks. The dataset of 376,125 detected feature tracks in 2016 is openly available to foster the improvement of location prediction in extrapolation-based nowcasting models.

Precipitation Nowcasting with Weather Radar Images and Deep Learning in São Paulo, Brasil

Precipitation nowcasting can predict and alert for any possibility of abrupt weather changes which may cause both human and material risks. Most of the conventional nowcasting methods extrapolate weather radar echoes, but precipitation nowcasting is still a challenge, mainly due to rapid changes in meteorological systems and time required for numerical simulations. Recently video prediction deep learning (VPDL) algorithms have been applied in precipitation nowcasting. In this study, we use the VPDL PredRNN++ and sequences of radar reflectivity images to predict the future sequence of reflectivity images for up to 1-h lead time for São Paulo, Brazil. We also verify the feasibility for the continuous use of the VPDL model, providing the meteorologist with trends and forecasts in precipitation edges regardless of the weather event occurring. The results obtained confirm the potential of the VPDL model as an additional tool to assist nowcasting. Even though meteorological systems that trigger natural disasters vary by location, a general solution can contribute as a tool to assist decision-makers and consequently issue efficient alerts.

A New 3-D Geometry Reconstruction Method of Space Target Utilizing the Scatterer Energy Accumulation of ISAR Image Sequence

By analyzing the motion characteristics and the radar observation model of triaxial stabilized space targets, a new 3-D geometry reconstruction method is proposed based on the energy accumulation of inverse synthetic aperture radar (ISAR) image sequence. According to the radar line of sight (LOS), we first construct the projection vectors of the 3-D geometry of a space target on the imaging planes. Then, by projecting the 3-D scatterer candidates on each imaging plane, we can accumulate the scattering energy of the corresponding 2-D projection position in each image. The 3-D scatterer candidates occupying the larger accumulated energy will be reserved as the real scatterers. To improve the efficiency, the real 3-D scatterers will be searched by using the particle swarm optimization (PSO) algorithm one by one. Compared with traditional 3-D geometry reconstruction methods, the proposed one never needs the 2-D scatterer extraction and trajectory association, which remains the challenges in ISAR image processing. Experimental results based on the simulated point target and electromagnetic data are presented to validate the effectiveness and robustness of the proposed method.

Dynamic Fractal Texture Analysis for PolSAR Land Cover Classification

Polarimetric response is strongly target orientation dependent. The observed polarimetric matrices from the same target with different orientations can be quite different. The existence of target scattering orientation diversity contains rich information, and leveraging information of target scattering orientation diversity may help to reveal polarimetric properties of different land cover types. In this work, a robust land cover feature descriptor, dynamic fractal texture, is introduced to capture the stochastic self-similarities of land cover scattering responses in both spatial and rotation domains. We extend the polarimetric matrix to the rotation domain by polarimetric basis transformation. Varying polarization orientation angle (POA) or ellipticity angle (EA), polarimetric responses of land cover under a series of orientations can be obtained. Then, the dynamic fractal texture is formulated by serializing received responses as a polarimetric synthetic-aperture radar (PolSAR) image sequence. Finally, the proposed features are combined with random forest (RF)/support vector machine (SVM) classifier to produce the classification maps on real PolSAR data. Experiment results show that dynamic fractal texture has an advantage in indicating rotation domain information. The proposed method has superior performance in land cover classification and yields accurate classification results.

Precession parameter estimation method of the cone‐shaped target based on inverse synthetic aperture radar image sequence analysis

The micro-motion features induced by space targets provide effective information for target classification and recognition. This study introduces the equivalent rotational velocity (ERV) estimation for precession parameter estimation of the cone-shaped target with inverse synthetic aperture radar image sequence. On the basis of the rotation matching relationship of precession cone-shaped target, the estimated ERV is used to build an eigenfunction, and the parameter searching set is obtained. Through theoretical analysis, the coupled parameters of the eigenfunction will concentrate on a curve on the searching plane. Thus, the parameter searching set is reduced to the neighbouring area of this curve, which reduces the computational burden greatly. Finally, the parameters are decoupled by the reconstructed radar observation angle, and the precession parameters such as precession period, precession angle, average observation angle, radial length and bottom radius of the target are precisely estimated. The proposed algorithm requires no prior information and it has high computational efficiency. Simulation results with electromagnetic data illustrate the validity of the proposed method.

Learning to Generate Radar Image Sequences Using Two-Stage Generative Adversarial Networks

While quantitative precipitation estimation (QPE) using weather radar is widely adopted in operation, precipitation data sets are often highly imbalanced. In particular, extreme precipitation usually lacks representation, which may introduce the bottleneck for radar QPE with machine learning models. Discovering the intrinsic characteristic of extreme precipitation with few samples is challenging. In this letter, we focus on the radar reflectivity data and aim to generate synthetic radar image sequences with respect to extreme precipitation. Considering the relatively long interval between continuous radar images due to radar volume scan, traditional methods in video generation are not suitable. In this letter, we propose Two-stage Generative Adversarial Networks (TsGANs) to address the above-mentioned problem. In general, our TsGAN constructs adversarial process between generators and discriminators: the generator produces samples similar to real data, while the discriminator determines whether or not a sample is eligible. In Stage I, we generate an image sequence containing content and motion features. In Stage II, we design an enhanced net structure to enrich the adversarial processes and further improve the motion features. Experimental testing is performed within the radar coverage in Shenzhen, China, on rainfall events in 2014–2016. Results show that our TsGAN is superior to previous works.

Generation of long-term InSAR ground displacement time-series through a novel multi-sensor data merging technique: The case study of the Shanghai coastal area

Ground deformation is one of the most significant challenges faced by many coastal mega-cities, with major societal and economic impacts. In this context, the possibility to monitor the temporal evolution of the ground subsidence processes, which may also last for several years, is of great relevance. This goal can be obtained by applying differential SAR interferometry (DInSAR) techniques to sequences of multiple-satellite synthetic aperture radar (SAR) images. Moreover, the growing availability of large archives of SAR images collected by different SAR instruments nowadays leads to the need of developing new data merging techniques, which may take profit from the complementary information recoverable from every single set of data. In this work, a novel data merging approach for the generation of long-tem ground displacement time-series, based on the use of the modified Quantile-Quantile Adjustment (MQQA) algorithm, is proposed. Specifically, the methodology has successfully been applied to study the long-term evolution of the ground subsidence occurred in the coastal area of Shanghai from February 2007 to April 2017. A cross-comparison analysis between DInSAR and ground truth data has also been carried out, showing that the average root mean square error (RMSE) between the obtained displacement time-series and available ground truth data is of about 3 mm. This outcome confirms the validity of the novel DInSAR-based MQQA combination method.

Estimation of Sea Surface Current from X-Band Marine Radar Images by Cross-Spectrum Analysis

The cross-spectral correlation approach has been used to estimate the wave spectrum from optical and radar images. This work aims to improve the cross-spectral approach to derive current velocity from the X-band marine radar image sequence, and evaluate the application conditions of the method. To reduce the dependency of gray levels on range and azimuth, radar images are preprocessed by the contrast-limited adaptive histogram equalization. Two-dimensional cross-spectral coherence and phase are derived from neighboring X-band marine radar images, and the phases with large coherences are used to estimate the phase velocity and angular frequency of waves, which are first fitted with the theoretical dispersion relation by different least square models, and then the current velocity can be determined. Compared with the current velocities measured by a current meter, the root-mean-square error, correlation coefficient, bias, and relative error are 0.15 m/s. 0.88, –0.05 m/s, and 7.79% for the north-south velocity, and 0.14 m/s, 0.86, 0.06 m/s, and 10.75% for the east-west velocity in the experimental area, respectively. The preprocessing, critical coherence, and the number of images for applying the cross-spectral approach, are discussed.