Passive Millimeter Wave Imaging Research Articles

Object Segmentation Using Polarization Random Feature in Passive Millimeter-Wave Imaging

Object Segmentation Using Polarization Random Feature in Passive Millimeter-Wave Imaging

PMMWMatcher: Weakly Supervised Feature Matching With Descriptor Enhancement for Passive Millimeter-Wave Images

PMMWMatcher: Weakly Supervised Feature Matching With Descriptor Enhancement for Passive Millimeter-Wave Images

PMMWPoint: Weakly Supervised Dense Keypoint Detection and Description for Passive Millimeter-Wave Image Matching

PMMWPoint: Weakly Supervised Dense Keypoint Detection and Description for Passive Millimeter-Wave Image Matching

A Novel Antenna Array Design Method Based on Antenna Grouping for Near-Field Passive Millimeter-Wave Imaging

A Novel Antenna Array Design Method Based on Antenna Grouping for Near-Field Passive Millimeter-Wave Imaging

FA‐UNet: Semantic Segmentation of Passive Millimeter–Wave Images for Concealed Object Detection

Passive millimeter–wave (PMMW) scanners are widely used for personal security screening in public places due to their nonradiation and high real‐time capabilities. However, the images obtained by these scanners frequently exhibit low signal‐to‐noise ratios and contrast, presenting challenges for automated detection systems. To address this issue, we propose an efficient semantic segmentation approach, FA‐UNet, that employs a UNet architecture with a fusion attention mechanism to conduct binary classification (human body vs. background, including objects) for PMMW images. This approach incorporates a spatial attention mechanism into the lateral connections between the encoder and decoder and introduces a channel attention mechanism during the feature fusion process in the decoder. By combining these attention mechanisms, FA‐UNet leads to more precise segmentation outcomes. The segmented image is then fused with the original image using our multistage fusion method, in which, first, the two images are blended in a 1:1 ratio for object detection. Then, a new fused image is obtained by adjusting the ratio within a certain range (0.3–0.5). Finally, the object detection results are overlaid on this fused image to generate a directly displayable image. We evaluate our method using a self‐made dataset. Experimental results demonstrate that FA‐UNet can accurately segment the human body region and preserve object shapes effectively. Using the fused image for object detection helps reduce false detections caused by background noise interference while improving the detection rate of weak targets. Additionally, the fused image aids in manual image interpretation in locations with higher security inspection levels and contributes to protect the privacy of individuals undergoing inspection to the greatest extent possible.

Unifying Convolution and Transformer for Efficient Concealed Object Detection in Passive Millimeter-Wave Images

Passive millimeter-wave (PMMW) imaging is an ideal technique for concealed object detection in non-contact security inspection scenarios, and has been widely used in railway stations and airports. However, there are still several challenges that limit the accuracy of detection in PMMW images: low resolution, small objects and complex background interference. The existing deep learning-based methods mainly adopt two-stage architecture with convolutional neural networks (CNN) as the backbone for feature extraction, while the low speed of two-stage architecture and limited receptive field of CNN impede the further improvement of the intelligent inspection system. In this paper, we propose a one-stage anchor-free detector that combines the merits of CNN and transformer to solve these problems, and we avoid the tradeoff between accuracy and computational complexity that most hierarchical transformers make by constricting self-attention within pre-defined local windows. Specifically, we design a novel backbone to model low-level local and high-level global features at different scales via CNN and transformer. We also introduce object queries to guide the detector to perceive the concealed objects from the background noise, and these learnable queries are further utilized to form a full-size object-aware attention mechanism. Besides, to select the optimal positive samples for the anchor-free detector, we propose a novel label assignment strategy by employing Gaussian distribution to adaptively model the objects with various shapes. Experimental results on our self-developed PMMW imager demonstrate that the proposed method outperforms the state-of-the-art methods in terms of accuracy and speed.

Regional-Based Object Detection Using Polarization and Fisher Vectors in Passive Millimeter-Wave Imaging

Passive millimeter-wave (PMMW) imaging is a powerful approach for detecting hidden objects underneath clothing. The theoretical basis of object detection methods is the contrast of brightness temperature (TB) image. TB differences may be caused by the diversity of material, physical temperature, surface structure, and so on. Existing methods are mainly based on single-polarization and single-pixel processing, which usually generate many discrete pixels of false alarms or missing detections. In this article, we present a regional-based method for hidden object detection using polarization and Fisher vector (FV) features. The necessity of polarization averaging for detection is revealed by theoretical simulation and experimental analyses. Based on the superpixel segmentation of polarization mean image, a modified FV, regional mean FV (RMFV), is created to extract concealed object features. Various imaging experimental data of typical security inspection scenarios are applied to verify the proposed method. The robustness and effectiveness are proved by comparing with several state-of-the-art methods.

Passive millimeter-wave imaging system with variable focal length optical configuration

Passive millimeter-wave imaging system with variable focal length optical configuration

HINRDNet: A half instance normalization residual dense network for passive millimetre wave image restoration

During the imaging process of passive millimeter wave (PMMW), due to the long wavelength of millimeter wave, the small aperture of the antenna, and the introduction of noises, the obtained PMMW image are degraded by blur and noise simultaneously. To address these problems, we propose a passive millimeter-wave image restoration network called as HINRDNet which includes a denoising module (DNM), deblurring and denoising U-Net (DBNU-Net), and an image reconstruction module (RCM). In order to fully utilize local features, the proposed network is designed based on Half Instance Normalization Block, and is mainly composed of the densely connected sum dense HINBlock (SDHB) and the concatenate dense HINBlock (CDHB) block from low step to high step. In addition, the higher level features information is combined with low level features for global feature cross fusion. Finally, we construct remote sensing and infrared simulation datasets to train the network, and test it on simulated datasets and real PMMW degraded images. The experimental results for the simulated degraded images and the real PMMW degraded images demonstrate that our proposed method outperforms the existing state-of-the-art methods in terms of visual perception and evaluation metrics.

Ka Band Low Channel Mutual Coupling Integrated Packaged Phased Array Receiver Front-End for Passive Millimeter-Wave Imaging

This paper presents a Ka band eight-channel integrated packaged phased array receiver front-end for a passive millimeter-wave imaging system. Since multiple receiving channels are integrated in a given package, the mutual coupling issue affecting the channel will deteriorate imaging quality. Therefore, in this study, the influence of channel mutual coupling on the system array pattern and amplitude phase error is analyzed, and the design requirements are proposed according to the results. During the design implementation, the coupling paths are discussed, and passive circuits in the path are modeled and designed to reduce the level of channel mutual coupling and spatial radiation. Finally, an accurate coupling measurement method for a multi-channel integrated phased array receiver is proposed. The receiver front-end achieves a 28~31 dB single channel gain, a 3.6 dB noise figure, less than −47 dB of channel mutual coupling. Furthermore, the array pattern of the two-dimensional 1024 channel system composed of the front end of the receiver is consistent with the simulation, and the receiver’s performance is verified by a human-body-imaging experiment. The proposed coupling analysis, design, and measurement methods are also applicable to other multi-channel integrated packaged devices.

Source-Free Domain Adaptive Detection of Concealed Objects in Passive Millimeter-Wave Images

Passive millimeter-wave (PMMW) imager can detect concealed objects under clothing in a touch-free manner. Existing detectors exhibit promising performance on specific PMMW datasets. However, these detectors will suffer severe degradation due to the inevitable domain shift when the environment changes in real applications. To avoid huge manual efforts in preparing the target domain dataset, unsupervised domain adaption (UDA) is proposed to tackle this issue. UDA holds the assumption that the labeled source data and unlabeled target data are accessible during the adaptation, but it is not always true due to the privacy concern. In this article, we aim to address this challenge by source-free domain adaption (SFDA), which migrates the source-trained model to target domain without the source data. Our method is built on the student–teacher framework and combines the merits of pseudo-label self-training and adversarial learning (AL). We also propose a novel spatio-temporal weighting strategy (STW). It mines the spatial self-correlation in single PMMW image and temporal cross correlation among PMMW images for teacher to produce high-quality pseudo-labels. It guides the student to select strong positive pairs for contrastive learning. In addition, the AL is further employed for feature alignment between teacher and student at image and instance levels, which ensures the student to learn domain-invariant features. Extensive experiments show that our method achieves the state-of-the-art performance. We further demonstrate that the proposed method can work in online settings. To the best of authors’ knowledge, this is the first work to incorporate SFDA for concealed object detection in PMMW images.

Registration and Fusion Using Reflection in Passive Millimeter-Wave Images

Passive millimeter-wave (PMMW) imaging technology is widely used in civilian and military applications. However, there are reflections similar to the optical band in PMMW images, which have a negative influence on target detection and recognition. In this article, we present a reflection-based method to enhance the target features in PMMW images. The dividing line between the target and reflection is obtained by the similarity of brightness temperature (TB). By combining the similarity and reflection principle, we propose a new method to obtain the feature points of the target and reflection for registration. Then, the weighted method based on region TB is used to fusion target and reflection. Finally, to avoid interference with target detection and recognition, the reflection is removed. The experimental results show that the method can obtain higher contrast and more accurate target information.

Concealed Object Detection for Passive Millimeter-Wave Security Imaging Based on Task-Aligned Detection Transformer

Passive millimeter-wave (PMMW) is a significant potential technique for human security screening. Several popular object detection networks have been used for PMMW images. However, restricted by the low resolution and high noise of PMMW images, PMMW hidden object detection based on deep learning usually suffers from low accuracy and low classification confidence. To tackle the above problems, this paper proposes a Task-Aligned Detection Transformer network, named PMMW-DETR. In the first stage, a Denoising Coarse-to-Fine Transformer (DCFT) backbone is designed to extract long- and short-range features in the different scales. In the second stage, we propose the Query Selection module to introduce learned spatial features into the network as prior knowledge, which enhances the semantic perception capability of the network. In the third stage, aiming to improve the classification performance, we perform a Task-Aligned Dual-Head block that decouples the classification and regression tasks, and enhances the interaction between the two heads. Based on our self-developed PMMW security screening dataset, experimental results including comparison with State-Of-The-Art (SOTA) methods and ablation study demonstrate that the PMMW-DETR obtains higher accuracy and classification confidence than previous works, and exhibits robustness to the PMMW images of low quality.

High-resolution 96-GHz (3 mm) passive millimeter-wave imaging system and the effect of scanning resolution on images

A high-resolution 96-GHz (3 mm) passive millimeter-wave imaging system is constructed. The system has a 96-GHz radiometric receiver, which is placed on the focal point of a 50-cm diameter parabolic reflector designed and created. A specially written program for the system controls the antenna via control hardware in desired steps by a 2-axis positioner motor. A balance weight has been calculated and mounted to the rear of this parabolic reflector antenna to prevent possible damage to the rotor of the positioner motor due to the antenna’s weight. The maximum field of view of the system is 120 deg in azimuth and 90 deg in elevation. The angular scan step can be varied from 0.1 deg to 1 deg. Although the system can shoot 1190 × 890 samples (pixels) images, which is close to the HD quality (1280 × 720 pixels) in the full field of view, the actual resolution of the image is limited by the antenna’s beamwidth (BW). Some measurements were performed to investigate the effect of scanning steps on PMMW image quality for different angular scan steps. It is observed that the PMMW image scan with the smallest step gives a detailed image that is the closest to the scene’s optical image as expected. Even though the angular scan step angle is less than half of the antenna’s BW at Nyquist sampling, it is observed that the image quality still improves due to averaging effect of the received signals giving a better estimated S/N. The resolution and scan step values of our system are compared with that of some systems found in the literature and it is shown that ours has the highest value among them. This system can be used to obtain high-resolution images in remote sensing and other applications.

Coupling Effects Analysis and Suppression in a Highly Integrated Ka-Band Receiver Front-End MMIC for a Passive Millimeter-Wave Imager System

This paper presents the coupling effects analysis and suppression of a highly integrated receiver front-end MMIC for a passive millimeter-wave imager system. The receiver MMIC consists of a low-noise amplifier, double-balanced image-reject mixer, frequency quadrupler, and analog phase shifter. In order to integrate these devices into a compact single chip without affecting the core performance, coupling problems need to be solved. We analyze the influence of coupling effects on the image rejection ratio, and propose corresponding solutions for three different coupling paths. (1) The coupling in the LO-RF path of the mixer is solved by designing a double-balanced mixer with high isolation characteristics. (2) The coupling between the LO chain and the LNA from space and dielectric is suppressed by optimizing the two main transmission lines spacing and adding isolation vias. (3) The coupling caused by the line crossing is restrained by designing a differential line crossover structure. The design and implementation of the MMIC are based on 0.15 µm GaAs pHEMT process. The receiver chip has 6.1~8.7 dB conversion gain in 32~36 GHz, less than 3.5 dB of noise figure, and more than 35 dB of image rejection ratio. The measurement results show that the receiver MMIC is especially suitable for high-sensitivity passive millimeter-wave imaging systems.

A Compact Broadband Analog Complex Correlator with High Correlation Efficiency for Passive Millimeter-Wave Imaging System

In this paper, the design, fabrication, and measurement of a compact broadband (4–8 GHz) analog complex correlator for a passive millimeter-wave imaging system are presented. To achieve high sensitivity and high integration of the imaging system, the wideband and miniaturization of the correlator are required. The correlator achieves wide bandwidth by using the add-and-square method, which is composed of a six-port circuit and a detection circuit. In order to realize the miniaturization of the correlator, the six-port circuit is realized on the chip base on the 0.15-μm gallium arsenide (GaAs) process. The influence of mismatch of the detection circuit that employs zero-bias Schottky diodes on the correlator is also analyzed to guide the design of the correlator. The measurement results of the designed chips and detector are consistent with the simulation result. Finally, a Sweep-frequency test is applied to the designed correlator, and the measurement results show that, within the frequency range of 4–8 GHz, the correlation amplitude fluctuation is less than 1.9 dB and the correlation efficiency is larger than 99%, which reveal that the correlator is suited for interferometric passive millimeter-wave imaging applications.

A Near-Field Imaging Algorithm Based on Angular Spectrum Theory for Synthetic Aperture Interferometric Radiometer

For the near-field synthetic aperture interferometric radiometer (SAIR), the Fourier transform relationship between the visibility function and the near-field brightness temperature (BT) distribution is not valid. It is a challenging task for near-field SAIR imaging to realize very close-range accurate imaging with a large field of view (FOV). This article presents a new SAIR near-field imaging algorithm based on the angular spectrum theory to realize the passive millimeter-wave (PMMW) imaging, called synthetic-angular-spectrum imaging (SASI) algorithm. This SASI algorithm mainly addresses data samples of a 4-D visibility function acquired from planar arrays. First, we invert 4-D visibility samples into the angular spectrum domain via the Fourier transformation. The obtained angular spectrum data cannot be directly used to estimate the near-field BT distribution. Second, a dedicated phase factor compensation is adopted for handling this problem. The dimension-reducing accumulation is employed to generate the synthetic angular spectrum (SAS) of near-field BT distribution. Finally, we reconstruct the BT image by using the generated SAS data. Simulation and experiment results show that the presented SASI algorithm has the superiority on image reconstruction quality, especially for the off-axis regions, compared with the existing near-field SAIR imaging methods.

Passive Millimeter-Wave Imaging for Burns Diagnostics under Dressing Materials.

This paper presents a feasibility study of using a passive millimeter-wave imaging (PMMWI) system to assess burn wounds and the potential for monitoring the healing process under dressing materials, without their painful removal. Experimental images obtained from ex vivo porcine skin samples indicate that a ThruVision passive imager operating over the band 232–268 GHz can be used for diagnosing burns and for potentially monitoring the healing under dressing materials. Experimental images show that single and multiple burns are observed throughout dressing materials. As the interaction of millimeter-wave (MMW) radiation with the human body is almost exclusively with the skin, the major outcomes of the research are that PMMWI is capable of discriminating burn-damaged skin from unburned skin, and these measurements can be made through bandages without the imager making any physical contact with the skin or the bandage. This highlights the opportunity that the healing of burn wounds can be assessed and monitored without the removal of dressing materials. The key innovation in this work is in detecting single and multiple burns under dressing materials in noncontact with the skin and without exposing the skin to any type of manmade radiation (i.e., passive sensing technology). These images represent the first demonstration of burns wound under dressing materials using a passive sensing imager.

Transformer-Based Anchor-Free Detection of Concealed Objects in Passive Millimeter Wave Images

Passive millimeter wave (PMMW) imaging has merits of non-radiation and good penetrability to most of clothes, hence it has been a reliable security technique for the detection of concealed objects. At present, deep learning based approaches have shown great advantage in automatic detection. However, low resolution and high background noise of PMMW images make the task tricky, especially for small objects. In this paper, we propose a transformer-based anchor-free detector with integration of local/global information and adaptive label assignment to address the aforementioned issues. Compared with the existing anchor-based methods adopted for PMMW image detection, our detector can further improve the efficiency and remove the handcraft anchor boxes. To be specific, we firstly employ hierarchical transformer architecture as backbone, which has the capacity to model long-range dependencies of the feature at different scales. We propose a new strategy that calculates self-attention within local region/global region in turn, providing detailed and global features of small objects. Secondly, we design a learnable position encoding module to obtain positional information between pixels. We propose an attention weighting module which enables the network to adaptively refine the features and distinguish positive and negative samples. Finally, we propose an adaptive label assignment strategy to dynamically optimize the number of positive samples used for detections. The proposed method is validated on our self-developed PMMW imager. The experimental results show that our method achieves better performance on accuracy and competitive speed compared with the state-of-the-art methods.

A Low- and High-Resolution Terahertz Image Pair Construction Method With Gradient Fusion for Learning-Based Super-Resolution

Quality and feature quantity of low- and high-resolution image pairs for training are directly related to performance of learning-based super-resolution methods. In order to accurately increase learning features to the low- and high-resolution image pairs, we proposed a gradient degradation model of terahertz(THz) images to describe the process of a high-resolution THz image’s gradient map is how degraded to corresponding low-resolution THz image’s gradient map. And with the proposed model, we presented a low- and high-resolution THz image pair construction method with gradient fusion for learning-based super-resolution. As gradient maps are fused to training pairs, performance of learning-based super-resolution methods for THz images could be improved. In addition, we applied the low- and high-resolution THz image pair construction method with gradient fusion to very deep super-resolution(VDSR) method and zero shot super-resolution(ZSSR) method, named VDSR method with gradient fusion and ZSSR method with gradient fusion, respectively. To evaluate the performance of these two improved methods, comparison experiments with passive millimeter-wave images and our THz lab data are presented. The experimental results show that the performance of the VDSR method with gradient fusion and the ZSSR method with gradient fusion have significant improvements in peak signal-to-noise ratio and structural similarity index measure than the VDSR method and the ZSSR method, respectively. It indicates that performance of learning-based super-resolution methods for THz images could be improved by applying the proposed low- and high-resolution THz image pair construction method with gradient fusion.