MMW Imaging Research Articles

Segmentation-Aided Upsampling for High-Resolution Reconstruction of 3-D MMW Images

Segmentation-Aided Upsampling for High-Resolution Reconstruction of 3-D MMW Images

PointNet + + Based Concealed Object Classification Utilizing an FMCW Millimeter-Wave Radar

In the field of millimeter-wave (MMW) imaging, the integration of artificial intelligence (AI) has emerged as a crucial solution for addressing automation challenges. In this study, concealed object classification was successfully achieved on point cloud data from MMW radar high-precision imaging using the PointNet + + deep learning method. The utilized dataset comprises point cloud data generated through the transformation of 3D models and reconstruction of physical objects with an accuracy of less than 1 mm via MMW radar scanning. Classification accuracy was significantly improved by introducing data enhancement techniques, including the generation of homologous data and optimization of sampling points. After several evaluations, 300 epochs of training were conducted using 8192 sampling points, the results showed an accuracy of 0.998 for the training dataset and 0.996 for the test dataset. Moreover, evaluations of samples not included in the original dataset as well as multi-surface scans of concealed objects within the cardboard both resulted in correct predictions, which further validates the effectiveness and reliability of the study and demonstrates the potential of AI applied to MMW imaging.

Towards large-scale single-shot millimeter-wave imaging for low-cost security inspection

Millimeter-Wave (MMW) imaging is a promising technique for contactless security inspection. However, the high cost of requisite large-scale antenna arrays hinders its widespread application in high-throughput scenarios. Here, we report a large-scale single-shot MMW imaging framework, achieving low-cost high-fidelity security inspection. We first analyzed the statistical ranking of each array element through 1934 full-sampled MMW echoes. The highest-ranked elements are preferentially selected based on the ranking, building the experimentally optimal sparse sampling strategy that reduces antenna array cost by one order of magnitude. Additionally, we derived an untrained interpretable learning scheme, realizing robust and accurate MMW image reconstruction from sparsely sampled echoes. Last, we developed a neural network for automatic object detection, and experimentally demonstrated successful detection of concealed centimeter-sized targets using 10% sparse array, whereas all the other contemporary approaches failed at such a low sampling ratio. With the strong detection ability and order-of-magnitude cost reduction, we anticipate that this technique provides a practical way for large-scale single-shot MMW imaging.

Integrated convolutional neural networks for joint super-resolution and classification of radar images

Deep learning techniques have been widely used for two-dimensional (2D) and three-dimensional (3D) computer vision problems, such as object detection, super-resolution (SR) and classification to name a few. Radar images suffer from poor resolution as compared to optical images, hence developing a high-accuracy model to solve computer vision problems, such as a classifier, is a challenge. This is because of the lack of high-frequency details in the input images which makes it difficult for the classifier model to generate accurate predictions. Ways of addressing this challenge include training the learning model with a large dataset or using a more complicated model, such as deeper layer architecture. However, employing such approaches might result in the overfitting of the model, where the model might not generalize well on previously unseen data. Also, generating a large dataset for training the model is a challenging task, especially in the case of radar images. An alternate solution for achieving high accuracy in radar classification problems is provided in this paper wherein a CNN-enabled super-resolution (SR) model is integrated with the classifier model. The SR model is trained to generate high-resolution (HR) millimeter-wave (mmW) images from any input low-resolution (LR) mmW images. These resolved images from the SR model will be used by the classifier model to classify the input images into appropriate classes, consisting of threat and non-threat objects. The training data for the dual CNN layers are generated using a numerical model of a near-field coded-aperture computational imaging (CI) system. This trained dual CNN model is tested with simulated data generated from the CI numerical model wherein a high classification accuracy of 95% and a fast inference time of 0.193 s are obtained, making it suitable for real-time automated threat classification applications. For fair comparison, the developed CNN model is also validated with experimentally generated reconstruction data, in which case, a classification accuracy of 94% is obtained.

Efficient MMW Image Reconstruction Algorithm Based on ADMM Framework for Near-Field MIMO-SAR

Efficient MMW Image Reconstruction Algorithm Based on ADMM Framework for Near-Field MIMO-SAR

Over-the-Air Calibration of mmW Imaging Radars Using Uncorrelated Continuous Wave Signals

Over-the-Air Calibration of mmW Imaging Radars Using Uncorrelated Continuous Wave Signals

Millimeter-Wave 3-D Imaging Using Leaky-Wave Antennas and an Extended Rytov Approximation in a Frequency-Diverse MIMO System

A millimeter-wave (mmW) 3-D imaging system using high scanning-rate leaky-wave antennas in a frequency-diverse multiple-input–multiple-output (MIMO) antenna configuration is proposed. A novelty of the approach is the use of high scanning-rate leaky-wave antennas combined with MIMO to obtain both frequency and spatial diversities. Furthermore, an extended Rytov approximation (xRA), recently shown to provide accurate reconstructions for high permittivity and electrically large-sized low-loss objects, is also included in the approach. Compared with frequency-diversity-only and spatial-diversity-only imaging systems, the proposed system can provide enhanced imaging performance by leveraging both spatial and frequency diversities simultaneously. In addition, the imaging time and hardware cost can be significantly reduced, resulting in a low-cost mmW imaging system. The formulation for the 3-D imaging system is simplified to a scalar form by leveraging the polarization of the leaky-wave antennas and the polarization dominance of the scattered wave components, allowing xRA to be formulated. Sensing capacity and correlation coefficient evaluation methods are used to estimate the performance of the high scanning-rate leaky-wave MIMO imaging system. The scalar formulation also allows us to straightforwardly compare MIMO microwave imaging with MIMO radar to highlight the commonality between them. Numerical and experimental examples demonstrate that the proposed system with xRA can accurately estimate the contrast function amplitude and the positions of dielectric scatterers in a 3-D imaging region using two transmit and 11 receive leaky-wave antennas and 49 frequency samples within the 37.2–42-GHz frequency range.

Millimeter-Wave Image Deblurring via Cycle-Consistent Adversarial Network

Millimeter-wave (MMW) imaging has a tangible prospect in concealed weapon detection for security checks. Typically, a one-dimensional (1D) linear antenna array with mechanical scanning along a perpendicular direction is employed for MMW imaging. To achieve high-resolution imaging, the target under test needs to keep steady enough during the mechanical scanning process since slight movement can induce large phase variation for MMW systems, which will result in a blurred image. However, in the scenario of imaging of a human body, sometimes it is difficult to meet this requirement, especially for the elderly. Such blurred MMW images would reduce the detection accuracy of the concealed weapons. In this paper, we propose a deblurring method based on cycle-consistent adversarial network (Cycle GAN). Specifically, the Cycle GAN can learn the mapping between the blurred MMW images and the focused ones. To minimize the effect of the shaking blur, we introduce an identity loss. Moreover, a mean squared error loss (MSE loss) is utilized to stabilize the training, so as to obtain more refined deblurred results. The experimental results demonstrate that the proposed method can efficiently suppress the blurring effect in the MMW image.

Fast and Enhanced MMW Imaging System Using a Simple Row Detector Circuit with GDDs as Sensor Elements and an FFT-Based Signal Acquisition System.

The relatively high atmospheric propagation of millimeter-waves (MMW) was found to be one of the most critical reasons for the development of reliable sensors for MMW detection. According to previous research works, it has been already shown that incident MMW radiation on a glow discharge detector (GDD) can increase the discharge current. Hence, the electrical mode of detection can be employed to detect the presence of MMW radiation. In this article, a new design of a row detector using GDDs as pixel elements, and the influence of MMW incidence on GDD's discharge current, were acquired using an elementary data acquisition (DAQ) platform. The DAQ system computes the averaged Fast Fourier Transform (FFT) spectrum of the time signal and returns the FFT results as magnitude based on the level of detection. An FFT-based signal acquisition proved to be a better alternative to the lock-in detection that was commonly used in MMW detection systems. This improved detection circuit provides enhanced noise filtering, thereby resulting in better MMW images within a short time. The overhead expense of the entire system is very low, as it can avoid lock-in amplifier stages that were previously used for signal enhancement. A scanning mechanism using a motorized translation stage (step motor) is involved to place and align the row detector in the image plane. The scanning can be carried out vertically to perform the imaging, by configuring the step motor after selecting the desired step size and position. A simplified version of the MMW detection circuit with a dedicated over-voltage protection facility is presented here. This made the detection system more stable and reliable during its operation. The MMW detection circuit demonstrated in this work was found to be a milestone to develop larger focal plane arrays (FPA) with very inexpensive sensor elements.

Suspicious Object Detection for Millimeter-Wave Images with Multi-View Fusion Siamese Network.

Millimeter-wave (MMW) imaging techniques have been widely used in the public security industries for their under-controlled privacy concerns and no health hazards. However, since MMW images are low resolution and most objects are small, reflection-weak, diverse, suspicious object detection in the MMW images is a very challenging task. This paper develops a robust suspicious object detector for the MMW images based on the Siamese network integrated with the pose estimation and image segmentation, which estimates the coordinates of human joints and segments the complete human images into symmetrical body part images. Unlike most existing detectors, which detect and recognize suspicious objects in MMW images and require a complete training set with correct annotations, our proposed model aims to learn the similarity between two symmetrical human body part images segmented from the complete MMW images. Furthermore, to decrease the misdetection caused by the restricted field of view, we further fuse the multi-view MMW images observed from the same person by designing a decision-level fusion strategy and feature-level fusion strategy based on the attention mechanism. Experimental results on the measured MMW images show that our proposed models have favorable detection accuracy and speed in practical application and thus prove their effectiveness.

A Deep Learning Approach for Reconstruction in Millimeter-Wave Imaging Systems

In millimeter-wave (MMW) imaging, the objects of interest are oftentimes modeled as 2-D binary (black and white) shapes with white pixels representing the reflecting interior of the object. However, due to the propagation of the scattered waves, the continuous-domain binary images are convolved with a so-called point-spread function (PSF) before being digitized by means of sampling. As the 2-D PSF is both nonseparable and nonvanishing in the case of MMW imaging, exact recovery is quite complicated. In this communication, we propose a deep-learning approach for image reconstruction. We should highlight that the wave scatterings are suitably represented with complex-valued quantities, while standard deep neural networks (DNNs) accept real-valued inputs. To overcome this challenge, we separate the real and imaginary parts as if we had two imaging modalities and concatenate them to form a real-valued input with a larger size. Fortunately, the network automatically learns how to combine the mutual information between these modalities to reconstruct the final image. Among the advantages of the proposed method are improved robustness against additive noise and mismatch errors of imaging frequency and object to antenna distance; indeed, the method works well in wideband imaging scenarios over a wide range of objects to antenna distances even in the presence of high noise levels without requiring a separate calibration stage. We test the method with synthetic data simulated with software as well as real recordings in the laboratory.

Conical Differential Range Based Back-Projection Algorithm for Concealed Object Detection with Three-Dimensional mmW Imaging

Millimeter wave(mmW) imaging has spread to a wide range of applications in the last quarter. One of the most important research areas of mmW is three-dimensional (3D) imaging systems. In this study, conical differential range-based back-projection (BP) algorithm is proposed for three-dimensional mmW imaging. In the algorithm, the differential range is created using points inside a conical volume, thus the number of interpolation points is considerably reduced. The performance of the algorithm is demonstrated by simulation and experimental studies. Cylindrical scanning is carried out by means of the experimental setup. Experiments are carried out at frequencies of 26.5-40 GHz. The traditional BP algorithm (BPA) and the proposed algorithm are used to reconstruct the images. With the proposed method, it is observed that ISLR for the point target increased by about 5 dB compared to the traditional method. Moreover, the computational complexity is reduced by up to 10 times, depending on the imaging area. Thanks to the proposed method, the image of the concealed weapon under the cloth in an experimental study is more clearly focused compared to the traditional method. Therefore, it can provide images that give more accurate results for applications such as automatic target detection methods.

On-chip SRR and CSRR for millimeter-wave integrated systems: Review and applications

On-chip metadevices have been developed in progress aiming to generate extraordinary performance over conventional components in millimeter-wave (mmW) realms, and their full integration with advanced complementary-metal-oxide-semiconductor (CMOS) circuits pave a new way toward miniature and high-performance integrated systems. This article aims to provide a general overview of split-ring resonators (SRRs) and complementary split-ring resonators (CSRR), from the perspective of fundamental physics, recent progress of on-chip implementations, and their impacts on the development of mmW integrated circuits and systems. SRR/CSRR-based metadevices are discussed in terms of their design challenges, methods, implementation, and testing results. In particular, their applications to CMOS oscillators, modulator and signal source are highlighted. Finally, using the SRR/CSRR, a number of examples are provided to showcase the recent advances of integrated systems toward mmW communication and imaging. Future directions enabled by SRR and CSRR will be envisioned.

Active and passive mixed millimeter wave imaging target recognition method based on multi‐feature fusion

Due to millimeter-wave (MMW) has a strong ability to penetrate clothing, MMW holographic imaging technology can conduct a non-contact inspection of the human body's surface. Therefore, it is of great significance to study the security inspection equipment and target recognition technology based on millimeter wave imaging. In this paper, an active and passive hybrid millimeter wave imaging target recognition method based on multi-feature fusion is proposed, which improves the ability of the imaging system to identify different dangerous targets. First, active and passive millimeter wave imaging techniques are discussed, the reconstruction algorithm of active millimeter wave holographic imaging is derived in detail, and the measured optical photos and data of active and passive imaging are obtained. Secondly, the image preprocessing technology applied to active and passive millimeter wave imaging is studied, which can effectively highlight the target, eliminate background interference, and clearly describe the target contour. Then the methods of image feature extraction, data feature extraction and multi-feature fusion are studied. On this basis, a multi-feature fusion method based on weighted series fusion is proposed to obtain the fusion feature vector form of active and passive MMW imaging. Finally, this paper proposed the target recognition method applied to millimeter wave imaging, and concludes that the fusion feature vector is better than the original feature vector, which provides an idea for the fusion of active and passive millimeter wave imaging at the feature level. It also provides a theoretical basis for the application of security equipment.

3-D Super-Resolution of Coded Aperture Millimeter-Wave Images Using Complex-Valued Convolutional Neural Network

Electromagnetic (EM) waves at millimeter-wave (mmW) frequencies have found applications in a variety of imaging systems, from security screening to defense and automotive radars, with the research and development of mmW imaging systems gaining interest in recent years. Despite their significant advantages, mmW imaging systems suffer from poor resolution compared to higher frequency reconstructions, such as optical images. To improve the resolution of mmW images, various super-resolution (SR) techniques have been introduced. One such technique is the use of machine learning algorithms in the signal processing layer of the imaging system without altering any of the system’s parameters. This article focuses on the use of a convolutional neural network (CNN) architecture to achieve SR when applied to 3-D mmW input images. To exploit the phase information content of the input images along with the magnitude, a complex-valued CNN is designed, which can accommodate complex-valued data. To simplify the learning process, the resolution difference between the input and output images is divided into smaller parts by using subnetworks in the CNN architecture. The trained model is tested on simulated and experimental targets. The average mean square error score and the structural similarity index obtained on a test dataset of 460 samples are 0.0127 and 0.9225, respectively. It can be inferred that the model has the capability to improve the resolution of input mmW images to a high degree of fidelity, hence paving the way for an end-to-end SR imaging system.

Real-Time Millimeter-Wave Imaging for Moving Targets Realized in a Sparse Array Based on InP-HBT MMICs

Millimeter-wave (MMW) imaging has been widely studied for applications, such as personnel screening and industrial noninvasive testing; however, real-time MMW imaging with a subcentimeter resolution and a large field of view (FoV) for a fast-moving target has been rarely reported to date. In this article, we present the design of such an imaging system at the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Ka$ </tex-math></inline-formula> -band utilizing the concept of multistatic sparse array. Microwave monolithic integrated circuits based on an InP-heterojunction bipolar transistor (HBT) technology are used for the transmitter and receiver chipsets. With the help of an improved backprojection algorithm and a computation platform based on field-programmable gate arrays, real-time imaging for a target moving as fast as about 10 m/s is demonstrated at a frame rate of over 20 frames/s. The imaging resolution is measured to be better than 6 mm and the FoV can reach over <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1\times 1\,\,\text{m}^{2}$ </tex-math></inline-formula> . Imaging of a moving person with concealed items is carried out to illustrate the system’s capability for personnel screening. To the best of the authors’ knowledge, this is the first real-time MMW sparse array imaging system built upon InP-HBT chipsets.

Transmit–receive isolation improvement by a switch sequence adjustment for MMW linear-arrays imaging

High transmit-receive isolation between any two adjacent elements in sequentially switched linear arrays guarantees the performance of high-resolution close-range imaging. However, the transmit-receive antenna spacing required in wavelength order makes its isolation deteriorated rapidly in the millimeter wave and higher frequency bands. In this paper, we propose a simple yet effective method to improve transmit-receive isolation by a switch sequence adjustment on the traditional linear arrays. The new switch-mode aims to simply increase the vertical spacing between each transmit-receive pair. To maintain the densely uniform-spatial effective sampling and Fourier transform-based image reconstruction, the principle of equivalent phase center is utilized to analyze the maximum of the possible vertical spacing of each transmit-receive pair is four wavelengths. Experiments verify that the method can efficiently improve transmit-receive isolation by up to 11.5 dB and guarantee the high performance of MMW imaging.

Resolution Enhancement for Millimeter-Wave Radar ROI Image with Bayesian Compressive Sensing.

For millimeter-wave (MMW) imaging security systems, the image resolution promisingly determines the performance of suspicious target detection and recognition. Conventional synthetic aperture radar (SAR) imaging algorithms only provide limited resolution in active MMW imaging, which is limited by the system. In terms of enhancing the resolution of a region of interest (ROI) image containing suspicious targets, super-resolution (SR) imaging is adopted via Bayesian compressive sensing (BCS) implemented by fast Fourier transform (FFT). The spatial sparsity of MMW ROI images is well exploited with BCS to achieve resolution enhancement without computational cost. Both simulated and measured experiments confirm that the proposed scheme effectively improves the resolution of ROI images.

Fast and accurate concealed dangerous object detection for millimeter-wave images

Millimeter-wave (MMW) imaging is a touch-free method for security inspection at railway stations and airports. However, automatic detection of dangerous objects in MMW images is challenging; the characteristics of images are remarkable with variable appearances of an object, different contrasts, and lower resolutions. We propose a fast and accurate concealed dangerous object detection method called MMW image Detection (MMWDet). MMWDet comprises two stages: single-image detection and multi-angle image detection. In the single-image detection stage, we design a more robust feature fusion relationship between adjacent feature maps in a feature pyramid network to achieve a more robust feature representation of an object. Meanwhile, we employ localization confidence rather than category confidence in the classification branch to address the negative sample interference in the training process. Then, multi-angle image detection is proposed to reduce the false and missed detection rate based on a single image by refining the detection results of multi-angle images, which maps and filters the prediction boxes frame by frame based on matching pairs of adjacent image feature points. We collected a MMW dataset that included 38k images. Extensive experiments on the MMW dataset demonstrate the effectiveness and improved performance of the proposed method compared with advanced general detection methods.

Phase-Based Range-Enhanced Millimeter-Wave Imaging Technique for Multistatic Planar Array

A millimeter-wave (mmW) range enhancement imaging technique for multistatic planar arrays is presented. The echo in an active mmW imaging system that contains both amplitude and phase information, and the latter can be potentially exploited for improving imaging performance in the range direction. With the help of a two-dimensional (2-D) phase unwrapping scheme, and the multi-input-multi-output range migration algorithm (MIMO-RMA) for a multistatic planar array, along with the requisite pretreatment processes such as target extraction and phase error correction for the reconstructed image region, a reconstructed image having a finer range profile could be achieved. The underlying principle of the phase information exploitation to achieve finer range distance information is discussed, the range enhancement imaging implementation steps are outlined, and the necessary pretreatment processes in simulation and experiment are introduced. The efficiency of the proposed approach is analyzed theoretically and demonstrated experimentally. Numerical simulation and experimental results show that the reconstructed image using the new range enhancement imaging technique contains the promised finer range distance information.