Concealed Object Detection Research Articles

Hybrid network with difficult–easy learning for concealed object detection in imbalanced terahertz image dataset

Hybrid network with difficult–easy learning for concealed object detection in imbalanced terahertz image dataset

AI-Enabled Sensor Fusion of Time-of-Flight Imaging and mmWave for Concealed Metal Detection.

In the field of detection and ranging, multiple complementary sensing modalities may be used to enrich information obtained from a dynamic scene. One application of this sensor fusion is in public security and surveillance, where efficacy and privacy protection measures must be continually evaluated. We present a novel deployment of sensor fusion for the discrete detection of concealed metal objects on persons whilst preserving their privacy. This is achieved by coupling off-the-shelf mmWave radar and depth camera technology with a novel neural network architecture that processes radar signals using convolutional Long Short-Term Memory (LSTM) blocks and depth signals using convolutional operations. The combined latent features are then magnified using deep feature magnification to reveal cross-modality dependencies in the data. We further propose a decoder, based on the feature extraction and embedding block, to learn an efficient upsampling of the latent space to locate the concealed object in the spatial domain through radar feature guidance. We demonstrate the ability to detect the presence and infer the 3D location of concealed metal objects. We achieve accuracies of up to 95% using a technique that is robust to multiple persons. This work provides a demonstration of the potential for cost-effective and portable sensor fusion with strong opportunities for further development.

SMR–YOLO: Multi-Scale Detection of Concealed Suspicious Objects in Terahertz Images

The detection of concealed suspicious objects in public places is a critical issue and a popular research topic. Terahertz (THz) imaging technology, as an emerging detection method, can penetrate materials without emitting ionizing radiation, providing a new approach to detecting concealed suspicious objects. This study focuses on the detection of concealed suspicious objects wrapped in different materials such as polyethylene and kraft paper, including items like scissors, pistols, and blades, using THz imaging technology. To address issues such as the lack of texture details in THz images and the contour similarity of different objects, which can lead to missed detections and false alarms, we propose a THz concealed suspicious object detection model based on SMR–YOLO (SPD_Mobile + RFB + YOLO). This model, based on the MobileNext network, introduces the spatial-to-depth convolution (SPD-Conv) module to replace the backbone network, reducing computational and parameter load. The inclusion of the receptive field block (RFB) module, which uses a multi-branch structure of dilated convolutions, enhances the network’s depth features. Using the EIOU loss function to assess the accuracy of predicted box localization further optimizes convergence speed and localization accuracy. Experimental results show that the improved model achieved mAP@0.5 and mAP@0.5:0.95 scores of 98.9% and 89.4%, respectively, representing improvements of 0.2% and 1.8% over the baseline model. Additionally, the detection speed reached 108.7 FPS, an improvement of 23.2 FPS over the baseline model. The model effectively identifies concealed suspicious objects within packages, offering a novel approach for detection in public places.

Concealed hazardous object detection for terahertz images with cross-feature fusion transformer

Terahertz (THz) waves can penetrate many non-metallic materials (such as paper, plastic, and clothing), making them highly valuable in security inspection areas. However, the low contrast between target objects and the background in THz images poses problems for object detection and recognition tasks. Though many schemes have been proposed to tackle the problems, the low detection accuracy and high computational complexity remain challenges. To address these challenges, we propose a concealed hazardous object detection method called Cross-Feature Fusion Transformer YOLO (CFT-YOLO) for THz images. We design a Multi-Channel Spatial Feature Module (MCSFM) to achieve cross-channel and cross-spatial information transfer between feature maps. MCSFM also realizes the feature parameter transfer between the backbone and neck of the network. Additionally, we develop a CrossFuse-Transformer (CF-Former) module that leverages self-attention to fully exploit the correlations and structural information between different feature maps. We conduct a series of experiments on public datasets. The results show that CFT-YOLO outperforms the other advanced methods in THz hazardous object detection tasks. Moreover, the CFT-YOLO achieves a better balance between model complexity and detection accuracy compared to YOLOv8s, making it more practical for real-world applications.

Helmholtz–Galerkin Technique in Dipole Field Scattering from Buried Zero-Thickness Perfectly Electrically Conducting Disk

Non-invasive concealed object detection, identification, and discrimination have been of interest to the research community for decades due to the needs to preserve infrastructures and artifacts, guarantee safe conditions for the detection and location of landmines, etc. A modern approach is based on the use of an unmanned aerial vehicle equipped with ground-penetrating radar, which has the advantage of not requiring direct contact with the ground. Moreover, high-resolution underground images are obtained by coherently combining measurements by using a synthetic aperture radar algorithm. Due to the complexity of the real scenario, numerical analyses have always been welcomed to provide almost real-time information to make the best use of the potential of such kinds of techniques. This paper proposes an analysis of the scattering from a zero-thickness perfectly electrically conducting disk buried in a lossy half-space surrounded by air and illuminated by a field generated by a Hertzian dipole located in the air. It is carried out by means of a generalized form of the analytically regularizing Helmholtz–Galerkin technique, introduced and successfully applied by the authors to analyze the plane-wave scattering from a disk and a holed plane in a homogeneous medium. As clearly shown in the numerical results, the proposed method is very effective and drastically outperforms the commercial software CST Microwave Studio 2023.

Automated detection and classification of concealed objects using infrared thermography and convolutional neural networks

This paper presents a study on the effectiveness of a convolutional neural network (CNN) in classifying infrared images for security scanning. Infrared thermography was explored as a non-invasive security scanner for stand-off and walk-through concealed object detection. Heat generated by human subjects radiates off the clothing surface, allowing detection by an infrared camera. However, infrared lacks in penetration capability compared to longer electromagnetic waves, leading to less obvious visuals on the clothing surface. ResNet-50 was used as the CNN model to automate the classification process of thermal images. The ImageNet database was used to pre-train the model, which was further fine-tuned using infrared images obtained from experiments. Four image pre-processing approaches were explored, i.e., raw infrared image, subject cropped region-of-interest (ROI) image, K-means, and Fuzzy-c clustered images. All these approaches were evaluated using the receiver operating characteristic curve on an internal holdout set, with an area-under-the-curve of 0.8923, 0.9256, 0.9485, and 0.9669 for the raw image, ROI cropped, K-means, and Fuzzy-c models, respectively. The CNN models trained using various image pre-processing approaches suggest that the prediction performance can be improved by the removal of non-decision relevant information and the visual highlighting of features.

Improved YOLOv5 for concealed object detection in human body millimeter wave images

Improved YOLOv5 for concealed object detection in human body millimeter wave images

Concealed objects detection and recognition in active transmission sub-terahertz imaging scanner for automatic security screening

Concealed objects detection and recognition in active transmission sub-terahertz imaging scanner for automatic security screening

A Novel Multimodal Fusion Framework Based on Point Cloud Registration for Near-Field 3D SAR Perception

This study introduces a pioneering multimodal fusion framework to enhance near-field 3D Synthetic Aperture Radar (SAR) imaging, crucial for applications like radar cross-section measurement and concealed object detection. Traditional near-field 3D SAR imaging struggles with issues like target–background confusion due to clutter and multipath interference, shape distortion from high sidelobes, and lack of color and texture information, all of which impede effective target recognition and scattering diagnosis. The proposed approach presents the first known application of multimodal fusion in near-field 3D SAR imaging, integrating LiDAR and optical camera data to overcome its inherent limitations. The framework comprises data preprocessing, point cloud registration, and data fusion, where registration between multi-sensor data is the core of effective integration. Recognizing the inadequacy of traditional registration methods in handling varying data formats, noise, and resolution differences, particularly between near-field 3D SAR and other sensors, this work introduces a novel three-stage registration process to effectively address these challenges. First, the approach designs a structure–intensity-constrained centroid distance detector, enabling key point extraction that reduces heterogeneity and accelerates the process. Second, a sample consensus initial alignment algorithm with SHOT features and geometric relationship constraints is proposed for enhanced coarse registration. Finally, the fine registration phase employs adaptive thresholding in the iterative closest point algorithm for precise and efficient data alignment. Both visual and quantitative analyses of measured data demonstrate the effectiveness of our method. The experimental results show significant improvements in registration accuracy and efficiency, laying the groundwork for future multimodal fusion advancements in near-field 3D SAR imaging.

Few-shot concealed object detection in sub-THz security images using improved pseudo-annotations

In this research, we explore the few-shot object detection application for identifying concealed objects in sub-terahertz security images, using fine-tuning based frameworks. To adapt these machine learning frameworks for the (sub-)terahertz domain, we propose an innovative pseudo-annotation method to augment the object detector by sourcing high-quality training samples from unlabeled images. This approach employs multiple one-class detectors coupled with a fine-grained classifier, trained on supporting thermal-infrared images, to prevent overfitting. Consequently, our approach enhances the model’s ability to detect challenging objects (e.g., 3D-printed guns and ceramic knives) when few-shot training examples are available, especially in the real-world scenario where images of concealed dangerous items are scarce.

Dielectric Lens-Based Millimeter Wave Imaging for Concealed Object Detection in Security Applications

Dielectric Lens-Based Millimeter Wave Imaging for Concealed Object Detection in Security Applications

Multi-band passive detection and imaging system for concealed weapon with dual assessment method

Technological advancements in the millimeter-wave (MMW) and terahertz (THz) frequency bands have seen rapid progress, especially in the field of concealed weapon detection. The capability to detect concealed objects under clothing without causing harm to individuals is of utmost importance. Because of this reason, the concept of the variable target ranges in a passive MMW/THz imaging system with the infrared band has been studied in detail. An imaging system, called multi-band passive detection and imaging system (MIS) in this study, with variable target ranges, was designed and tested. A passive MMW/THz imaging system is developed specifically for concealed weapon detection, with a primary focus on variable target ranges. In addition to this, an infrared imaging system has been incorporated into the concept of concealed weapon detection. To integrate these two imaging systems, image fusion techniques have been employed as a conjunction between them. Detailed analyses have been conducted to assess the concealed weapon detection capabilities of this integrated system. The image fusion results, combining infrared and passive MMW/THz bands, have been tested at variable target ranges, and the fused images have been compared in terms of structural similarity index (SSIM) and contrast levels (CL). The SSIM alone is an inadequate method in the concept of concealed weapon detection. The CL method, used with SSIM, eliminates the shortcoming of the SSIM alone. The outcomes of these tests demonstrate that MIS enhances the detectability of concealed weapons with the use of SSIM and CL together. As a result, this study lays a solid foundation for the future detection of concealed objects beneath clothing. The dual assessment method proposed in this study will enable the development of new methods that will increase the detection of specific objects under the clothes in the future.

Swin-YOLO for Concealed Object Detection in Millimeter Wave Images

Concealed object detection in millimeter wave (MMW) images has gained significant attention in the realm of public safety, primarily due to its distinctive advantages of non-hazardous and non-contact operation. However, this undertaking confronts substantial challenges in practical applications, owing to the inherent limitations of low imaging resolution, small concealed object size, intricate environmental noise, and the need for real-time performance. In this study, we propose Swin-YOLO, an innovative single-stage detection model built upon transformer layers. Our approach encompasses several key contributions. Firstly, the integration of Local Perception Swin Transform Layers (LPST Layers) enhanced the network’s capability to acquire contextual information and local awareness. Secondly, we introduced a novel feature fusion layer and a specialized prediction head for detecting small targets, effectively leveraging the network’s shallow feature information. Lastly, a coordinate attention (CA) module was seamlessly incorporated between the neck network and the detection head, augmenting the network’s sensitivity towards critical regions of small objects. To validate the efficacy and feasibility of our proposed method, we created a new MMW dataset containing a large number of small concealed objects and conducted comprehensive experiments to evaluate the effectiveness of overall and partial improvements, as well as computational efficiency. The results demonstrated a remarkable 4.7% improvement in the mean Average Precision (mAP) for Swin-YOLO compared with the YOLOv5 baseline. Moreover, when compared with other enhanced transformer-based models, Swin-YOLO exhibited a superior accuracy and the fastest inference speed. The proposed model showcases enhanced performance and holds promise for advancing the capabilities of real-world applications in public safety domains.

Unifying Convolution and Transformer for Efficient Concealed Object Detection in Passive Millimeter-Wave Images

Passive millimeter-wave (PMMW) imaging is an ideal technique for concealed object detection in non-contact security inspection scenarios, and has been widely used in railway stations and airports. However, there are still several challenges that limit the accuracy of detection in PMMW images: low resolution, small objects and complex background interference. The existing deep learning-based methods mainly adopt two-stage architecture with convolutional neural networks (CNN) as the backbone for feature extraction, while the low speed of two-stage architecture and limited receptive field of CNN impede the further improvement of the intelligent inspection system. In this paper, we propose a one-stage anchor-free detector that combines the merits of CNN and transformer to solve these problems, and we avoid the tradeoff between accuracy and computational complexity that most hierarchical transformers make by constricting self-attention within pre-defined local windows. Specifically, we design a novel backbone to model low-level local and high-level global features at different scales via CNN and transformer. We also introduce object queries to guide the detector to perceive the concealed objects from the background noise, and these learnable queries are further utilized to form a full-size object-aware attention mechanism. Besides, to select the optimal positive samples for the anchor-free detector, we propose a novel label assignment strategy by employing Gaussian distribution to adaptively model the objects with various shapes. Experimental results on our self-developed PMMW imager demonstrate that the proposed method outperforms the state-of-the-art methods in terms of accuracy and speed.

Object Recognition for Millimeter Wave MIMO-SAR Images Based on High-Resolution Feature Recursive Alignment Fusion Network

There are several complex situations in recognizing concealed objects from millimeter wave multiple-input multiple-output synthetic aperture radar (MIMO-SAR) security images, such as incomplete imaging of objects, partially occluded objects, and overlapping objects, which are detrimental to the accurate recognition of concealed objects. To solve these problems, a concealed object detection method based on a high-resolution feature recursive alignment fusion network (HR-FRAFnet) is proposed. The HR-FRAFnet can segment the object area from the grayscale image with complex human background and complete the recognition. The overall architecture of the HR-FRAFnet follows the encoder-decoder framework. Specifically, in the encoder stage, a deep parallel feature extraction network (DPFEN) connects the multi-resolution feature maps in parallel and repeats multi-scale feature fusion. This approach suppresses the background noise flowing and retains more recognizable target characteristics. Then, in the decoder stage, a feature recursive alignment fusion module (FRAFM) is designed to enhance the perception of object edges. The FRAFM effectively improves the segmentation accuracy of objects whereas decreasing the computational complexity of the network. Besides, we employ a combined loss function to alleviate the foreground-background imbalance problem in MIMO-SAR images. Homemade human security screening image datasets are used for evaluation. The experimental results show that the proposed method outperforms existing semantic segmentation methods in Mean Intersection over Union (mIoU) and reduces the incidence of missed and error detection of targets.

Privacy Preserving Contraband Detection Using a Millimeter-Wave Dynamic Antenna Array

We present a privacy preserving technique for contraband detection using a real-time 75 GHz rotational dynamic antenna array that samples only a reduced set of Fourier-domain information to prevent image reconstruction of the screened subject. Complex objects such as handguns have sharp edges that correspond to sharp spatial frequency features in the Fourier domain. These features are dependent on the orientation of the object and manifest at millimeter-wave frequencies through conditions such as being concealed under clothing. The technique combines a dynamically rotated active interferometer and simple arithmetic feature extraction approach supporting a threshold-based classifier to identify when a handgun is concealed under clothing. The active interferometer technique utilizes noise transmitters to illuminate the test subject to produce thermal-like radiation, enabling Fourier-domain signal sampling, while the rotational dynamics support sampling that spans a circle in the 2-D Fourier domain to retrieve sharp-edge responses induced by concealed objects. We experimentally demonstrate the detection of concealed objects under clothing in a laboratory environment using a 75 GHz dynamic antenna array based on simple heuristic features extracted from statistics of the measured responses. The classifier is evaluated based on processing one or more subsequent measurements, showing a classification accuracy of 0.908 and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F$</tex-math> </inline-formula> 1-score of 0.916 using four consecutive measurements.

275–305 GHz FM-CW Radar 3D Imaging for Walk-Through Security Body Scanner

Imaging using millimeter waves (MMW) and terahertz (THz) waves can help inspect hazardous materials hidden beneath clothing in a non-contact and non-invasive manner. A 3D terahertz imaging system for security gate applications in the 275–305 GHz range was developed and experimentally demonstrated to detect concealed objects carried by pedestrians. This system performs 3D measurements by combining depth detection using frequency-modulated continuous wave (FM-CW) radar, vertical scanning of the detection spot using a 1D high-speed mechanical beam scanner, and horizontal movement of the irradiated area and detection spot as the pedestrian walks. The high-speed beam scanner comprises an F-Theta telecentric lens and a polygon mirror. It has a vertical line scan rate of 142 lines/s and spatial resolution of ~10 mm, consistent with the design value, and a depth resolution of ~7 mm, which is 40% larger than the theoretical value estimated from the FM-CW radar principle. The depth-dependent lateral distortion in 3D images, known as telecentricity, measured using the body scanner imaging system, was also evaluated. Consequently, images with the same magnification were obtained at a range of more than 500 mm of focus depth. Finally, the detection of concealed objects carried by pedestrians was demonstrated, showing that the system can work for a pedestrian walking at speeds from 4 km/h to 7 km/h.

Weakly-Supervised Semantic Feature Refinement Network for MMW Concealed Object Detection

The concealed object detection in millimeter-wave human body images is a challenging task due to the noise and dim-small objects. Exploiting the spatial dependencies to mine the difference between the object and the noise is vital for the discrimination of objects. However, most approaches ignore the context around the object. In this paper, a concealed object detection framework based on structural context is proposed to suppress noise interference and refine localizable semantic features. The framework consists of two subnetworks, structural region-based multi-scale weakly supervised feature refinement and local context-based concealed object detection. The multi-scale weakly supervised feature refinement is constructed to learn position-aware semantics of objects of various sizes while suppressing background noises in structural regions. Specifically, a multi-scale pooling method is proposed to better localize objects of different sizes, and an object-activated region enhancement module is designed to strengthen object semantic representations and suppress the background interference. Moreover, an adaptive local context aggregation module is designed to integrate the local context around the bounding box in the concealed object detection, which improves the discrimination of the model for the dim-small objects. Experimental results on the AMMW and the PMMW datasets demonstrate that the proposed approach improves detection performance with lower false alarm rates.

Saliency and superpixel improved detection and segmentation of concealed objects for passive terahertz images

Application of passive terahertz imaging in concealed weapon detection has been looked at, such that the final result is the segmentation of the foreground concealed weapons from the rest of the background. For the same, a fully automatic and completely generic technique, without any learning, has been proposed. It was observed that a simple thresholding step, exploiting varied intensity bands of the tetrahertz images is not enough. Thus, an innovative method to isolate humans and thus improve the region of interest (ROI) has been proposed. Thereafter, saliency has been used to further improve ROI, as these images are quite noisy and the central focusing aspect of saliency could handle the noise around the concealed weapons. It was observed that this step could handle the noise around the concealed weapons but degraded the boundaries of the concealed weapons. To further improve boundary adherence, superpixels are used. Finally, results are evaluated both quantitatively and qualitatively and outperformed the traditional approach.

Weakly Supervised 2D Pose Adaptation and Body Part Segmentation for Concealed Object Detection.

Weakly supervised pose estimation can be used to assist unsupervised body part segmentation and concealed item detection. The accuracy of pose estimation is essential for precise body part segmentation and accurate concealed item detection. In this paper, we show how poses obtained from an RGB pretrained 2D pose detector can be modified for the backscatter image domain. The 2D poses are refined using RANSAC bundle adjustment to minimize the projection loss in 3D. Furthermore, we show how 2D poses can be optimized using a newly proposed 3D-to-2D pose correction network weakly supervised with pose prior regularizers and multi-view pose and posture consistency losses. The optimized 2D poses are used to segment human body parts. We then train a body-part-aware anomaly detection network to detect foreign (concealed threat) objects on segmented body parts. Our work is applied to the TSA passenger screening dataset containing millimeter wave scan images of airport travelers annotated with only binary labels that indicate whether a foreign object is concealed on a body part. Our proposed approach significantly improves the detection accuracy of TSA 2D backscatter images in existing works with a state-of-the-art performance of 97% F1-score, 0.0559 log-loss on the TSA-PSD test-set, and a 74% reduction in 2D pose error.