Large Scene Imaging Research Articles

Multi-scale ship target detection using SAR images based on improved Yolov5

Synthetic aperture radar (SAR) imaging is used to identify ships, which is a vital task in the maritime industry for managing maritime fisheries, marine transit, and rescue operations. However, some problems, like complex background interferences, various size ship feature variations, and indistinct tiny ship characteristics, continue to be challenges that tend to defy accuracy improvements in SAR ship detection. This research study for multiscale SAR ships detection has developed an upgraded YOLOv5s technique to address these issues. Using the C3 and FPN + PAN structures and attention mechanism, the generic YOLOv5 model has been enhanced in the backbone and neck section to achieve high identification rates. The SAR ship detection datasets and AirSARship datasets, along with two SAR large scene images acquired from the Chinese GF-3 satellite, are utilized to determine the experimental results. This model’s applicability is assessed using a variety of validation metrics, including accuracy, different training and test sets, and TF values, as well as comparisons with other cutting-edge classification models (ARPN, DAPN, Quad-FPN, HR-SDNet, Grid R-CNN, Cascade R-CNN, Multi-Stage YOLOv4-LITE, EfficientDet, Free-Anchor, Lite-Yolov5). The performance values demonstrate that the suggested model performed superior to the benchmark model used in this study, with higher identification rates. Additionally, these excellent identification rates demonstrate the recommended model’s applicability for maritime surveillance.

The Adaptive Streaming SAR Back-Projection Algorithm Based on Half-Precision in GPU

The back-projection (BP) algorithm is completely accurate in the imaging principle, but the computational complexity is extremely high. The single-precision arithmetic used in the traditional graphics processing unit (GPU) acceleration scheme has low throughput and its usage of the video memory is large. An adaptive asynchronous streaming scheme for the BP algorithm based on half-precision is proposed in this study, and then it is extended to the fast back-projection (FBP) algorithm. In this scheme, the adaptive loss factors selection strategy ensures the dynamic range of data, the asynchronous streaming structure ensures the efficiency of large scene imaging, and the mixed-precision data processing ensures the imaging quality. The schemes proposed in this paper are compared with BP, FBP, and fast factorized back-projection (FFBP) algorithms of single-precision in GPU. The experimental results show that the two half-precision acceleration schemes in this paper reduce the video memory usage to 74% and 59% of the single-precision schemes with guaranteed image quality. The efficiency improvements of the proposed schemes are almost one and 0.5 times greater than that of the corresponding single-precision scheme, and the advantage can be more obvious when dealing with large computations.

A deep learning semantic template matching framework for remote sensing image registration

We propose a deep learning framework by the probability of the predicting semantic spatial position distribution for remote sensing image registration. Traditional matching methods optimize similarity metrics with pixel-by-pixel searching, which is time consuming and sensitive to radiometric differences. Driven by learning-based methods, we take the reference and template images as inputs and map them to the semantic distribution position of the corresponding reference image. We apply the affine invariant to obtain a correspondence between the overlap of the barycenter position of the semantic template and the center pixel point, which transforms the semantic boundary alignment into a point-to-point matching problem. Additionally, two loss functions are proposed, one for optimizing the subpixel matching position and the other for determining the semantic spatial probability distribution of the matching template. In this work, we explore the influence of the template radius size, the filling form of training labels, and the weighted combination of loss function on the matching accuracy. Our experiments show that the trained model is robust to template deformation while still operating orders of magnitude faster. Furthermore, our proposed method implements high matching accuracy in four large scene images with radiometric differences. This ensures the improved speed of remote sensing image analysis and pipeline processing while facilitating novel directions in learning-based registration. Our code is freely available at https://github.com/liliangzhi110/semantictemplatematching.

Deep social grouping network for large scenes with multiple subjects

In computer vision, more attention has been paid to group analysis, and the group detection in images becomes a key technology of human analysis on groups. The existing social grouping methods only focus on small scenes with fixed number of persons and cannot deal with large scene images in the real world. This paper proposes the first fine-grained social grouping framework for gigapixel large scene images based on deep learning, which consists of a graph-guided global-to-local partition strategy and a deep grouping network that learns an implicit respresentation for social pairs. The framework has achieved accurate grouping on large scene images. Our method is also applicable to small scene images, and has outperformed the existing methods. The relevant code and the training dataset will be released soon.

Extended Polar Format Algorithm for Large-Scene High-Resolution WAS-SAR imaging

The polar format algorithm (PFA) is a frequency-domain algorithm with high computational efficiency especially for the high resolution imaging applications in wide-angle staring synthetic aperture radar (WAS-SAR). Because of the image distortion and defocus caused by plane wave assumption, the effective imaging scene is limited to a small region which is defined as the defocus-negligible region. In this article, a novel sub-block imaging algorithm which first applied to WAS-SAR is proposed. This method combines sub-block imaging and spatially variant post-filtering (SVPF) which can not only effectively extend the depth of focus of PFA but also greatly reduces the computational burden. What's more, the method of mosaic image is proposed in this article which can eliminate discontinuity between adjacent sub-blocks. First, the large scene is segmented into multiple regions along range direction and the size of each region is still larger than that of the defocus-negligible region. Second, refocusing is implemented by a phase compensation of the already de-chirped signal with the center of sub-blocks, respectively, and each sub-block image is formed by PFA. Third, the region in each sub-block that exceeds the defocus-negligible region is corrected by SVPF and the image distortion can be corrected by 2-D interpolation. Finally, the mosaic image method is applied to get the large-scene image with high quality. The algorithm proposed in this article is efficient and solves the problems in the process of SAR image mosaic such as dislocation and grayscale differences. The simulated and experimental data results demonstrate the effectiveness and practicability of the proposed algorithm.

Instance Segmentation for Large, Multi-Channel Remote Sensing Imagery Using Mask-RCNN and a Mosaicking Approach

Instance segmentation is the state-of-the-art in object detection, and there are numerous applications in remote sensing data where these algorithms can produce significant results. Nevertheless, one of the main problems is that most algorithms use Red, Green, and Blue (RGB) images, whereas Satellite images often present more channels that can be crucial to improve performance. Therefore, the present work brings three contributions: (a) conversion system from ground truth polygon data into the Creating Common Object in Context (COCO) annotation format; (b) Detectron2 software source code adaptation and application on multi-channel imagery; and (c) large scene image mosaicking. We applied the procedure in a Center Pivot Irrigation System (CPIS) dataset with ground truth produced by the Brazilian National Water Agency (ANA) and Landsat-8 Operational Land Imager (OLI) imagery (7 channels with 30-m resolution). Center pivots are a modern irrigation system technique with massive growth potential in Brazil and other world areas. The round shapes with different textures, colors, and spectral behaviors make it appropriate to use Deep Learning instance segmentation. We trained the model using 512 × 512-pixel sized patches using seven different backbone structures (ResNet50- Feature Pyramid Network (FPN), Resnet50-DC5, ResNet50-C4, Resnet101-FPN, Resnet101-DC5, ResNet101-FPN, and ResNeXt101-FPN). The model evaluation used standard COCO metrics (Average Precision (AP), AP50, AP75, APsmall, APmedium, and AR100). ResNeXt101-FPN had the best results, with a 3% advantage over the second-best model (ResNet101-FPN). We also compared the ResNeXt101-FPN model in the seven-channel and RGB imagery, where the multi-channel model had a 3% advantage, demonstrating great improvement using a larger number of channels. This research is also the first with a mosaicking algorithm using instance segmentation models, where we tested in a 1536 × 1536-pixel image using a non-max suppression sorted by area method. The proposed methodology is innovative and suitable for many other remote sensing problems and medical imagery that often present more channels.

An Efficient Pattern Matching Approach Using Double Measures of Correlation and Rank Reduction

This paper discusses an efficient pattern matching approach on the use of K-nearest neighbour (K-NN) based rank order reduction and Haar transform in order to detect a pattern in a large scene image. To accomplish the task, scene image is divided into a number of candidate windows and both input pattern and candidate windows are characterised by Haar transform. This characterisation seeks to determine distinctive coefficients known as Haar projection values (HPVs). To obtain more relevant and useful representation of HPVs, rectangle sum is computed and further, sum of absolute (SAD) correlation measure is applied as successive measures between the input pattern and candidate windows. This leads to increase the possibility of finding the object in the scene image before being detected and localised. The proposed pattern matching approach is tested on COIL-100 database and the matching accuracy proves the efficacy of the proposed algorithm.

Mental Retrieval of Remote Sensing Images via Adversarial Sketch-Image Feature Learning

Searching the targets of interest in large-scale remote sensing images is a fundamental problem, which becomes a very challenging issue when there is no relevant example at hand but a mental picture in mind. Hand-drawn sketch as a precise and convenient expression of the mental picture makes sketch-based remote sensing image retrieval (SBRSIR) an ideal choice to cope with this issue. However, the accuracy of SBRSIR algorithm, which is critical to effective retrieval, is still far behind classic query-by-example image retrieval. Two central limiting factors for this performance gap are: 1) the lack of effective cross-domain representations for bridging the domain gap between sketches and remote sensing images and 2) the absence of large sketch/remote sensing image data sets for developing, evaluating, and comparing the SBRSIR approaches. In this article, we first develop a novel SBRSIR model to learn a deep joint embedding space with discriminative losses, where adversarial training is used for the embedding space to learn domain-invariant representations. Then, we contribute a sketch/remote sensing image data set specifically for SBRSIR and provide a benchmark for subsequent researchers. Extensive experiments on the data set and large scene images demonstrate the effectiveness and superiority of the method for both the seen and unseen categories.

A Generalized Wavefront-Curvature-Corrected Polar Format Algorithm to Focus Bistatic SAR Under Complicated Flight Paths

Bistatic synthetic aperture radar (BiSAR) imaging is faced with two major challenges: large scene imaging and adaptability to unideal platform motion in practice. In order to deal with these two problems, a generalized wavefront-curvature-corrected polar format algorithm (PFA) is proposed in this article. The traditional PFA is little restricted on geometry configuration and platform motion, but its application to large scene imaging is limited by the far-field planar wavefront assumption. To solve this limitation, this article derives the phase error caused by wavefront curvature and analyzes its influence on both geometric distortion and defocusing effect in detail. Based on the analysis, we present a wavefront curvature completely correcting method through space-variant phase compensation using the analytical wavefront curvature phase in wavenumber-domain, which is derived through method of series reversion. What's more, an efficient realization of the space-variant phase compensation based on two-stage image division is given to avoid high overlap rate in the traditional image division method. The proposed method can obtain well focused and geometric undistorted image for BiSAR under complicated flight paths, and it also keeps the logarithmic complexity of traditional PFA. The effectiveness of the proposed method is verified by numerical simulations.

Arbitrary-oriented target detection in large scene sar images

Target detection in the field of synthetic aperture radar (SAR) has attracted considerable attention of researchers in national defense technology worldwide, owing to its unique advantages like high resolution and large scene image acquisition capabilities of SAR. However, due to strong speckle noise and low signal-to-noise ratio, it is difficult to extract representative features of target from SAR images, which greatly inhibits the effectiveness of traditional methods. In order to address the above problems, a framework called contextual rotation region-based convolutional neural network (RCNN) with multilayer fusion is proposed in this paper. Specifically, aimed to enable RCNN to perform target detection in large scene SAR images efficiently, maximum sliding strategy is applied to crop the large scene image into a series of sub-images before RCNN. Instead of using the highest-layer output for proposal generation and target detection, fusion feature maps with high resolution and rich semantic information are constructed by multilayer fusion strategy. Then, we put forwards rotation anchors to predict the minimum circumscribed rectangle of targets to reduce redundant detection region. Furthermore, shadow areas serve as contextual features to provide extraneous information for the detector identify and locate targets accurately. Experimental results on the simulated large scene SAR image dataset show that the proposed method achieves a satisfactory performance in large scene SAR target detection.

A Novel Strategy for Retrieving Large Scale Scene Images Based on Emotional Feature Clustering

Due to complicated data structure, image can present rich information, and so images are applied widely at different fields. Although the image can offer a lot of convenience, handling such data consume much time and multi-dimensional space. Especially when users need to retrieve some images from larger-scale image datasets, the disadvantage is more obvious. So, in order to retrieve larger-scale image data effectively, a scene images retrieval strategy based on the MapReduce parallel programming model is proposed. The proposed strategy first, investigates how to effectively store large-scale scene images under a Hadoop cluster parallel processing architecture. Second, a distributed feature clustering algorithm MeanShift is introduced to implement the clustering process of emotional feature of scene images. Finally, several experiments are conducted to verify the effectiveness and efficiency of the proposed strategy in terms of different aspects such as retrieval accuracy, speedup ratio and efficiency and data scalability.

Multi-Scale Image Block-Level F-CNN for Remote Sensing Images Object Detection

Nowadays, with the development of remote sensing technology, very-high-resolution (VHR) remote sensing image object detection technology attracts more and more attention. However, various challenges still exist in remote sensing image object detection field, such as the complex and varied appearances, the expensive manual annotation, and difficult in fast detecting the large scene image. In this paper, we propose a multi-scale image block-level fully convolutional neural network (MIF-CNN) for remote sensing image object detection, which can solve the above problems to a certain degree. First, the training data sets which only require class labels and do not need the bounding box label can reduce the spend of manual annotation during training. Second, the MIF-CNN is designed to extract the multi-scale-based high-level feature which can better represent the various types of objects. The image block-level fully convolutional network contributes to improving computing efficiency and can directly detect any size of the input image, including a large scene remote sensing image. In the testing phase, the large scene image is directly input to MIF-CNN model, and the detection results are generated from the MIF-CNN output maps which are improved by the proposed bounding boxes modification strategy and local re-recognized strategy. The experiments on NWPU VHR-10 [1] and two Airports data sets demonstrate the effectiveness of the proposed method.

Research on large scene image acquisition technology for distant oblique reconnaissance aerial cameras

Research on large scene image acquisition technology for distant oblique reconnaissance aerial cameras

SAR Target Recognition in Large Scene Images via Region-Based Convolutional Neural Networks

In this paper, a new Region-based Convolutional Neural Networks (RCNN) method is proposed for target recognition in large scene synthetic aperture radar (SAR) images. To locate and recognize the targets in SAR images, there are three steps in the traditional procedure: detection, discrimination, classification and recognition. Each step is supposed to provide optimal processing results for the next step, but this is difficult to implement in real-life applications because of speckle noise and inefficient connection among these procedures. To solve this problem, the RCNN is applied to large scene SAR target recognition, which can detect the objects while recognizing their classes based on its regression method and the sharing network structure. However, size of the input images to RCNN is limited so that the classification could be accomplished, which leads to a problem that RCNN is not able to handle the large scene SAR images directly. Thus, before the RCNN, a fast sliding method is proposed to segment the scene image into sub-images with suitable size and avoid dividing targets into different sub-images. After the RCNN, candidate regions on different slices are predicted. To locate targets on large scene SAR images from these candidate regions on small slices, the Non-maximum Suppression between Regions (NMSR) is proposed, which could find the most proper candidate region among all the overlapped regions. Experiments on 1476 × 1784 simulated MSTAR images of simple scenes and complex scenes show that the proposed method can recognize all targets with the best accuracy and fastest speed, and outperform the other methods, such as constant false alarm rate (CFAR) detector + support vector machine (SVM), Visual Attention+SVM, and Sliding-RCNN.

Integrated Localization and Recognition for Inshore Ships in Large Scene Remote Sensing Images

Automatic inshore ship recognition, which includes target localization and type recognition, is an important and challenging task. However, existing ship recognition methods mainly focus on the classification of ship samples or clips. These methods rely deeply on the detection algorithm to complete localization and recognition in large scene images. In this letter, we present an integrated framework to automatically locate and recognize inshore ships in large scene satellite images. Different from traditional object recognition methods using two steps of detection-classification, the proposed framework could locate inshore ships and identify types without the detection step. Considering ship size is a useful feature, a novel multimodel method is proposed to utilize this feature. And an Euclidean-distance-based fusion strategy is used to combine candidates given by models. This fusion strategy could effectively separate side-by-side ships. To handle large scene images efficiently, scale-invariant feature transform registration is also integrated into the framework to utilize geographic information. All of these make the framework an end-to-end fashion which could automatically recognize inshore ships in large scene satellite images. Experiments on Quickbird images show that this framework could achieve the actual applied requirements.

Visual Saliency Models for Text Detection in Real World.

This paper evaluates the degree of saliency of texts in natural scenes using visual saliency models. A large scale scene image database with pixel level ground truth is created for this purpose. Using this scene image database and five state-of-the-art models, visual saliency maps that represent the degree of saliency of the objects are calculated. The receiver operating characteristic curve is employed in order to evaluate the saliency of scene texts, which is calculated by visual saliency models. A visualization of the distribution of scene texts and non-texts in the space constructed by three kinds of saliency maps, which are calculated using Itti's visual saliency model with intensity, color and orientation features, is given. This visualization of distribution indicates that text characters are more salient than their non-text neighbors, and can be captured from the background. Therefore, scene texts can be extracted from the scene images. With this in mind, a new visual saliency architecture, named hierarchical visual saliency model, is proposed. Hierarchical visual saliency model is based on Itti's model and consists of two stages. In the first stage, Itti's model is used to calculate the saliency map, and Otsu's global thresholding algorithm is applied to extract the salient region that we are interested in. In the second stage, Itti's model is applied to the salient region to calculate the final saliency map. An experimental evaluation demonstrates that the proposed model outperforms Itti's model in terms of captured scene texts.

SPARSE ARRAY MICROWAVE 3-D IMAGING: COMPRESSED SENSING RECOVERY AND EXPERIMENTAL STUDY

Microwave array 3-D imaging is an emerging technique capable of producing a 3-D map of scattered electric flelds. Its all-weather and large scene imaging features make it an attractive powerful tool for target detection and feature extraction. Typically, a microwave array 3-D imaging system based on the classical sampling theory requires a large dense 2-D antenna array, which may sufier from a very high cost. To reduce the number of the antenna array elements, this paper surveys the use of compressed sensing recovery and sparse measurement strategies for microwave array 3-D imaging. Combining with the typical spatial sparsity of the underlying scene, we pose the sparse array microwave 3-D imaging as flnding sparse solutions to under-determined linear equations. Further, to reduce the computational of the compressed sensing recovery with the large- scale echoes data, we divide the underlying 3-D scene into a series of equal-range 2-D slices, and deal with these slices separately using the orthogonal matching pursuit (OMP) algorithm. Lastly, the performance of the presented compressed sensing approach is verifled by an X-band microwave array 3-D imaging system. The experimental results demonstrate that the compressed sensing approach can produce a better resolution 3-D image of the observed scatterers compared with the conventional method, especially in the case of very sparse activate antenna array.

A Large Scene Imaging Algorithm for Missile-borne Side-looking SAR

A Large Scene Imaging Algorithm for Missile-borne Side-looking SAR

The study on automatic registration of InSAR images

Interferometric Synthetic Aperture Radar (InSAR) allows production of high resolution DEM and detection of small earth motions using multiple pass SAR data sets obtained by remote sensing satellite. But the whole process has not yet reached sufficient robustness to warrant automated DEM production as commonly produced by stereovision with optical images. The automatic algorithm for precision registration is one of the bottlenecks in InSAR data processing. In this paper, an automatic approach with multi-step image matching algorithm is presented. All procedures are automatically implemented. The experiment is carried out successfully with SIR-C/L-band InSAR data. The triangular piecewise rectification is also advanced in reducing local distortion between the images and processing the large scene image. The primary result has prospect in the precision registration for the repeat-track InSAR data and reveals the potential of the presented automatic strategy.