Spatial Position Information Research Articles

Index Your Position: A Novel Self-Supervised Learning Method for Remote Sensing Images Semantic Segmentation

Learning effective visual representations without human supervision is a critical problem for the task of semantic segmentation of remote sensing images (RSIs), where pixel-level annotations are difficult to obtain. Self-supervised learning (SSL), which learns useful representations by creating artificial supervised learning problems, has recently emerged as an effective method to learn from unlabelled data. Current SSL methods are generally trained on ImageNet through image-level prediction tasks. We argue that this is suboptimal for application in semantic segmentation of RSIs since it does not take into account spatial position information between objects, which is critical for segmentation of RSIs characterized by multi-object. In this study, we propose a novel self-supervised dense representation learning method, IndexNet, for the semantic segmentation of RSIs. On the one hand, considering the multi-object characteristics of RSIs, IndexNet learns pixel-level representations by tracking object positions while maintaining sensitivity to object position changes to ensure that no mismatches are caused. On the other hand, by combining image-level contrast and pixel-level contrast, IndexNet can learn spatiotemporal invariant features. Experimental results show that our method works better than ImageNet pre-training and outperforms state-of-the-art (SOTA) self-supervised learning methods. Code and pre-trained models will be available at: https://github.com/pUmpKin-Co/offical-IndexNet.

BS2T: Bottleneck Spatial–Spectral Transformer for Hyperspectral Image Classification

Convolutional Neural Networks (CNNs) have been extensively applied to hyperspectral (HS) image classification tasks and achieved promising performance. However, for CNN based HS image classification methods, it is hard to depict the dependencies among HS image pixels in long-range distanced positions and bands. Moreover, the limited receptive field of the convolutional layers extremely hinders the development of the CNN structure. To tackle these problems, in this paper, the novel Bottleneck Spatial-Spectral Transformer (BS2T) is proposed to depict the long-range global dependencies of HS image pixels, which can be regarded as a feature extraction module for HS image classification networks. More specifically, inspired by Bottleneck Transformer in computer vision, for HS image feature extraction, the proposed BS2T is incorporated with a feature contraction module, a multi-head spatial-spectral self-attention (MHS2A) module and a feature expansion module. In this way, convolutional operations are replaced by the MHS2A to capture the long-range dependency of HS pixels regardless of their spatial position and distance. Meanwhile, in the MHS2A module, to highlight the spectral features of HS images, we introduce the spectral information and content spatial positional information to classical multi-head self-attentions to make the attentions more positional aware and spectral aware. On this basis, a dual-branch HS image classification framework based on 3D CNN and BS2T is defined for jointly extracting the local-global features of HS images. Experimental results on three public HS image classification datasets show that the proposed classification framework achieves a significant improvement when comparing with the state-of-the-art methods. The source code of the proposed framework can be downloaded from https://github.com/srxlnnu/BS2T.

Hyperspectral Image Classification Based on Multibranch Attention Transformer Networks

Deep learning has become a mainstream method of hyperspectral image (HSI) classification. Many DL-based methods exploit spatial-spectral features to achieve better classification results. However, due to the complex backgrounds in HSIs, existing methods usually show unsatisfactory performance for the class pixels located on the land-cover category boundary area. In large part, this is because the network is susceptible to interference by the irrelevant information around the target pixel in the training stage, resulting in inaccurate feature extraction. In this paper, a new multibranch transformer architecture (SST-M) that assembles spatial attention and extracts spectral features is proposed to address this problem. The transformer model has a global receptive field and thus can integrate global spatial position information in the HSI cube. Meanwhile, we design a spatial sequence attention model to enhance the useful spatial location features and weaken invalid information. Considering that HSIs contain considerable spectral information, a spectral feature extraction model is designed to extract discriminative spectral features, replacing the widely used PCA method and obtaining better classification results than it. Finally, inspired by semantic segmentation, a mask prediction model is designed to classify all of the pixels in the HSI cube; this guides the neural network to learn precise pixel characteristics and spatial distributions. To verify the effectiveness of our algorithm (SST-M), quantitative experiments were conducted in three well-known datasets, namely, IP, PU, and KSC. The experimental results demonstrate that the proposed model achieves better performance than the other state-of-the-art methods.

Optimized Design of 3D Spatial Images Based on Kalman Filter Equation

This paper takes the advantageous ability of Kalman filter equation as a means to jointly realize the accurate and reliable extraction of 3D spatial information and carries out the research work from the extraction of 3D spatial position information from multisource remote sensing optical stereo image pairs, recovery of 3D spatial structure information, and joint extraction of 3D spatial information with optimal topological structure constraints, respectively. Taking advantage of the stronger effect capability of Wiener recovery and shorter computation time of Kalman filter recovery, Wiener recovery is combined with Kalman filter recovery (referred to as Wiener-Kalman filter recovery method), and the mean square error and peak signal-to-noise ratio of the recovered image of this method are comparable to those of Wiener recovery, but the subjective evaluation concludes that the recovered image obtained by the Wiener-Kalman filter recovery method is clearer. To address the problem that the Kalman filter recovery method has the advantage of short computation time but the recovery effect is not as good as the Wiener recovery method, an improved Kalman filter recovery algorithm is proposed, which overcomes the fact that the Kalman filter recovery only targets the rows and columns of the image matrix for noise reduction and cannot utilize the pixel point information among the neighboring rows and columns. The algorithm takes the first row of the matrix image as the initial parameter of the Kalman filter prediction equation and then takes the first row of the recovered image as the initial parameter of the second Kalman filter prediction equation. The algorithm does not need to estimate the degradation function of the degradation system based on the degraded image, and the recovered image presents the image edge detail information more clearly, while the recovery effect is comparable to that of the Wiener recovery and Wiener-Kalman filter recovery method, and the improved Kalman filter recovery method has stronger noise reduction ability compared with the Kalman filter recovery method. The problem that the remote sensing optical images are seriously affected by shadows and complex environment detail information when 3D spatial structure information is extracted and the data extraction feature edge is not precise enough and the structure information extraction is not stable enough is addressed. A global optimal planar segmentation method with graded energy minimization is proposed, which can realize the accurate and stable extraction of the topological structure of the top surface by combining the edge information of remote sensing optical images and ensure the accuracy and stability of the final extracted 3D spatial information.

Ship Detection via Dilated Rate Search and Attention-Guided Feature Representation

Due to the complexity of scene interference and the variability of ship scale and position, automatic ship detection in remote sensing images makes for challenging research. The existing deep networks rarely design receptive fields that fit the target scale based on training data. Moreover, most of them ignore the effective retention of position information in the feature extraction process, which reduces the contribution of features to subsequent classification. To overcome these limitations, we propose a novel ship detection framework combining the dilated rate selection and attention-guided feature representation strategies, which can efficiently detect ships of different scales under the interference of complex environments such as clouds, sea clutter and mist. Specifically, we present a dilated convolution parameter search strategy to adaptively select the dilated rate for the multi-branch extraction architecture, adaptively obtaining context information of different receptive fields without sacrificing the image resolution. Moreover, to enhance the spatial position information of the feature maps, we calculate the correlation of spatial points from the vertical and horizontal directions and embed it into the channel compression coding process, thus generating the multi-dimensional feature descriptors which are sensitive to direction and position characteristics of ships. Experimental results on the Airbus dataset demonstrate that the proposed method achieves state-of-the-art performance compared with other detection models.

DDA-Net: Unsupervised cross-modality medical image segmentation via dual domain adaptation

Background and Objective: Deep convolutional networks are powerful tools for single-modality medical image segmentation, whereas generally require semantic labelling or annotation that is laborious and time-consuming. However, domain shift among various modalities critically deteriorates the performance of deep convolutional networks if only trained by single-modality labelling data.Methods: In this paper, we propose an end-to-end unsupervised cross-modality segmentation network, DDA-Net, for accurate medical image segmentation without semantic annotation or labelling on the target domain. To close the domain gap, different images with domain shift are mapped into a shared domain-invariant representation space. In addition, spatial position information, which benefits the spatial structure consistency for semantic information, is preserved by an introduced cross-modality auto-encoder.Results: We validated the proposed DDA-Net method on cross-modality medical image datasets of brain images and heart images. The experimental results show that DDA-Net effectively alleviates domain shift and suppresses model degradation.Conclusions: The proposed DDA-Net successfully closes the domain gap between different modalities of medical image, and achieves state-of-the-art performance in cross-modality medical image segmentation. It also can be generalized for other semi-supervised or unsupervised segmentation tasks in some other field.

CDUNet: Cloud Detection UNet for Remote Sensing Imagery

Cloud detection is a key step in the preprocessing of optical satellite remote sensing images. In the existing literature, cloud detection methods are roughly divided into threshold methods and deep-learning methods. Most of the traditional threshold methods are based on the spectral characteristics of clouds, so it is easy to lose the spatial location information in the high-reflection area, resulting in misclassification. Besides, due to the lack of generalization, the traditional deep-learning network also easily loses the details and spatial information if it is directly applied to cloud detection. In order to solve these problems, we propose a deep-learning model, Cloud Detection UNet (CDUNet), for cloud detection. The characteristics of the network are that it can refine the division boundary of the cloud layer and capture its spatial position information. In the proposed model, we introduced a High-frequency Feature Extractor (HFE) and a Multiscale Convolution (MSC) to refine the cloud boundary and predict fragmented clouds. Moreover, in order to improve the accuracy of thin cloud detection, the Spatial Prior Self-Attention (SPSA) mechanism was introduced to establish the cloud spatial position information. Additionally, a dual-attention mechanism is proposed to reduce the proportion of redundant information in the model and improve the overall performance of the model. The experimental results showed that our model can cope with complex cloud cover scenes and has excellent performance on cloud datasets and SPARCS datasets. Its segmentation accuracy is better than the existing methods, which is of great significance for cloud-detection-related work.

EPIsHilbert: Prediction of Enhancer-Promoter Interactions via Hilbert Curve Encoding and Transfer Learning.

Enhancer-promoter interactions (EPIs) play a significant role in the regulation of gene transcription. However, enhancers may not necessarily interact with the closest promoters, but with distant promoters via chromatin looping. Considering the spatial position relationship between enhancers and their target promoters is important for predicting EPIs. Most existing methods only consider sequence information regardless of spatial information. On the other hand, recent computational methods lack generalization capability across different cell line datasets. In this paper, we propose EPIsHilbert, which uses Hilbert curve encoding and two transfer learning approaches. Hilbert curve encoding can preserve the spatial position information between enhancers and promoters. Additionally, we use visualization techniques to explore important sequence fragments that have a high impact on EPIs and the spatial relationships between them. Transfer learning can improve prediction performance across cell lines. In order to further prove the effectiveness of transfer learning, we analyze the sequence coincidence of different cell lines. Experimental results demonstrate that EPIsHilbert is a state-of-the-art model that is superior to most of the existing methods both in specific cell lines and cross cell lines.

Short Literature Review for Visual Scene Understanding

Abstract Individuals are highly accurate for visually understanding natural scenes. By extracting and extrapolating data we reach the highest stage of scene understanding. In the past few years it proved to be an essential part in computer vision applications. It goes further than object detection by bringing machine perceiving closer to the human one: integrates meaningful information and extracts semantic relationships and patterns. Researchers in computer vision focused on scene understanding algorithms, the aim being to obtain semantic knowledge from the environment and determine the properties of objects and the relations between them. For applications in robotics, gaming, assisted living, augmented reality, etc a fundamental task is to be aware of spatial position and capture depth information. First part of this paper focuses on deep learning solutions for scene recognition following the main leads: low-level features and object detection. In the second part we present extensively the most relevant datasets for visual scene understanding. We take into consideration both directions having in mind future applications.

Integrating object proposal with attention networks for video saliency detection

Video saliency detection is an active research issue in both information science and visual psychology. In this paper, we propose an efficient video saliency-detection model, based on integrating object-proposal with attention networks, for efficiently capturing salient objects and human attention areas in the dynamic scenes of videos. In our algorithm, visual object features are first exploited from individual video frame, using real-time neural networks for object detection. Then, the spatial position information of each frame is used to screen out the large background in the video, so as to reduce the influence of background noises. Finally, the results, with backgrounds removed, are further refined by spreading the visual clues through an adaptive weighting scheme into the later layers of a convolutional neural network. Experimental results, conducted on widespread and commonly used databases for video saliency detection, verify that our proposed framework outperforms existing deep models.

General Purpose Low-Level Reinforcement Learning Control for Multi-Axis Rotor Aerial Vehicles.

This paper proposes a multipurpose reinforcement learning based low-level multirotor unmanned aerial vehicles control structure constructed using neural networks with model-free training. Other low-level reinforcement learning controllers developed in studies have only been applicable to a model-specific and physical-parameter-specific multirotor, and time-consuming training is required when switching to a different vehicle. We use a 6-degree-of-freedom dynamic model combining acceleration-based control from the policy neural network to overcome these problems. The UAV automatically learns the maneuver by an end-to-end neural network from fusion states to acceleration command. The state estimation is performed using the data from on-board sensors and motion capture. The motion capture system provides spatial position information and a multisensory fusion framework fuses the measurement from the onboard inertia measurement units for compensating the time delay and low update frequency of the capture system. Without requiring expert demonstration, the trained control policy implemented using an improved algorithm can be applied to various multirotors with the output directly mapped to actuators. The algorithm’s ability to control multirotors in the hovering and the tracking task is evaluated. Through simulation and actual experiments, we demonstrate the flight control with a quadrotor and hexrotor by using the trained policy. With the same policy, we verify that we can stabilize the quadrotor and hexrotor in the air under random initial states.

A real-time multi-vehicle tracking framework in intelligent vehicular networks

In this paper, we provide a new approach for intelligent traffic transportation in the intelligent vehicular networks, which aims at collecting the vehicles' locations, trajectories and other key driving parameters for the time-critical autonomous driving's requirement. The key of our method is a multi-vehicle tracking framework in the traffic monitoring scenario. Our proposed framework is composed of three modules: multi-vehicle detection, multi-vehicle association and miss-detected vehicle tracking. For the first module, we integrate self-attention mechanism into detector of using key point estimation for better detection effect. For the second module, we apply the multi-dimensional information for robustness promotion, including vehicle re-identification (Re-ID) features, historical trajectory information, and spatial position information For the third module, we re-track the miss-detected vehicles with occlusions in the first detection module. Besides, we utilize the asymmetric convolution and depth-wise separable convolution to reduce the model's parameters for speed-up. Extensive experimental results show the effectiveness of our proposed multi-vehicle tracking framework.

PCNet: partial convolution attention mechanism for image inpainting

Image inpainting is an essential issue in the field of computer vision. The basic purpose is to automatically recover the lost content according to the known content in the image. Although significant progress has been made in the completion of the regular missing image, the completion of the irregular image is still challenging. The completed images generated by the previous methods were different from the surrounding areas, considering the low efficiency of convolution in processing spatial position information. Based on this study, a partial convolution attention mechanism for image inpainting was proposed (PCNet), and an attention mechanism was introduced to extract long-distance and irregular image content which is a self-supervised module capable of focusing on global information. In addition, attentive normalization is introduced to model the long-distance relationship on the conditional image generation task. Experiments show that the results generated by our method are more natural and real, and the completed parts demonstrate more connected consistency.

A study on attention-based LSTM for abnormal behavior recognition with variable pooling

Behavior recognition is a well-known computer vision mobile technology. It has been used in many applications such as video surveillance, motion detection on devices, human-computer interaction and sports video, etc. However, most of the existing works ignored the depth and spatio-temporal information so that they resulted in over-fitting and inferior performance. Consequently, a novel framework for behavior recognition is proposed in this paper. In this framework, we propose a target depth estimation algorithm to calculate the 3D spatial position information of the target, and take this information as the input of the behavior recognition model. Simultaneously, in order to obtain more Spatio-temporal information and better handle long-term video, combining with the idea of attention mechanism, we propose a skeleton behavior recognition model which is based on spatio-temporal convolution and attention-based LSTM (ST-CNN & ATT-LSTM). The deep spatial information is merged into each segment, and the model focuses on the key information extraction, which is essential for improving behavior recognition performance. Meanwhile, we use a feature compression method based on variable pooling to solve the problem of inconsistent input sizes caused by multi-person behavior recognition, so that the network can flexibly recognize multi-person skeleton sequences. Finally, the proposed framework is evaluated with real-world surveillance video data, and the results indicate that our framework is superior to existing methods.

Single-Object Tracking Algorithm Based on Two-Step Spatiotemporal Deep Feature Fusion in a Complex Surveillance Scenario

Object tracking has been one of the most active research directions in the field of computer vision. In this paper, an effective single-object tracking algorithm based on two-step spatiotemporal feature fusion is proposed, which combines deep learning detection with the kernelized correlation filtering (KCF) tracking algorithm. Deep learning detection is adopted to obtain more accurate spatial position and scale information and reduce the cumulative error. In addition, the improved KCF algorithm is adopted to track and calculate the temporal information correlation of gradient features between video frames, so as to reduce the probability of missing detection and ensure the running speed. In the process of tracking, the spatiotemporal information is fused through feature analysis. A large number of experiment results show that our proposed algorithm has more tracking performance than the traditional KCF algorithm and can efficiently continuously detect and track objects in different complex scenes, which is suitable for engineering application.

Registration Method Between Phase-Contrast Magnetic Resonance Angiography and Time-of-Flight Magnetic Resonance Angiography—A Preliminary Study

Purpose: To evaluate a new method that registers phase-contrast magnetic resonance angiography images to time-of-flight magnetic resonance angiography images. Methods: Magnetic resonance angiography datasets of 10 healthy volunteers obtained by using two modalities (phase-contrast magnetic resonance angiography and time-of-flight magnetic resonance angiography) were preprocessed. Specifically, vessel regions were extracted using the region growing method with a threshold of 10% of the signal intensity maximum or 50% of the signal intensity maximum for phase-contrast magnetic resonance angiography images and time-of-flight magnetic resonance angiography images, respectively. Then, the normalized mutual information method was used to determine spatial positions, and registration between non-preprocessed phase-contrast magnetic resonance angiography and time-of-flight magnetic resonance angiography was performed using the spatial position information. Misalignment of 3 anatomical points was used to compare the accuracy of registration in this data group (the proposed method group) to that in the data group without registration (the non-registration group) and that in the data group subjected to normalized mutual information-based registration without preprocessing (the non-preprocessing group). Results: The mean misalignment of 3 anatomical points ± standard error was 1.69 ± 0.11 mm in the proposed method group, and 2.77± 0.13 mm and 90.28 ± 8.24 mm in the non-registration group and non-preprocessing group, respectively. The mean misalignment of 3 anatomical points was significantly smaller in the proposed method group than in the non-registration group (p = 0 001). Conclusions: The proposed preprocessing and registration method improved the accuracy of normalized mutual information-based registration between phase-contrast magnetic resonance angiography images and time-of-flight magnetic resonance angiography images.

Ship Trajectory Online Compression Algorithm Considering Handling Patterns

With the widespread use of shipborne Automatic Identification System (AIS) and the establishment of shore-based AIS networks in the past twenty years, a large amount of AIS trajectory data has been generated and accumulated in the shore-based systems. However, big data size increases the cost of storing, querying, and processing AIS data in practical applications. AIS data includes not only the ship spatial position information but also the course, speed, heading information et al . In other words, the knowledge of ship handling behavior is hidden in AIS data. In fact, many practical applications, such as maritime accident investigation and evidence collection, route extraction, ship behavior analysis, will require a compression algorithm to retain the ship key handling points in the original trajectory. To address these research challenges, the ship trajectory compression algorithm considering handling patterns is proposed in this paper. In order to explain the implicit handling patterns of AIS trajectories, the suggested method adopts the rate of course change (ROCC) and the rate of speed change (ROSC) in the sliding window as the criterion of whether the current trajectory point can be simplified. Numerical experiments are performed to verify the effectiveness of the proposed algorithm. Compared with Douglas-Peucker (DP) algorithm, Sliding Window (SW) algorithm, Opening Window Time Ratio (OPW-TR) algorithm, the results show that the proposed algorithm can efficiently compress trajectories by considering ship behavior patterns under application requirement.

A Multi-Level Convolution Pyramid Semantic Fusion Framework for High-Resolution Remote Sensing Image Scene Classification and Annotation

High spatial resolution (HSR) imagery scene classification has become a hot research topic in remote sensing. Scene classification method based on the handcrafted features, such as the bag-of-visual-words (BoVW) model, describes an image by extracting local features of the scene and mapping them to the dictionary space, but usually uses a shallow structure and loses the spatial distribution characteristics of the scene. The method based on deep learning extracts hierarchical features to describe the scene, which can maintain the spatial position information well. However, deep features in different levels have scale recognition restrictions for multi-scale ground objects, and cannot understand complex scenes well. In this paper, the multi-level convolutional pyramid semantic fusion (MCPSF) framework is proposed for HSR imagery scene classification. Differing from previous scene classification methods, which integrate the feature of different levels directly, of which the fusion features have large differences in both sparsity and eigenvalue magnitude, MCPSF integrates multi-level semantic features extracted by BoVW model and convolutional neural network (CNN) model. In MCPSF, two convolution pyramid feature expression strategies are proposed to enhance the ability of capturing multi-scale land objects, i.e., local and convolutional pyramid based BoVW (LCPB) model and local and convolutional pyramid based pooling-stretched (LCPP) model. The effectiveness of the proposed method is verified on 21-class UC Merced (UCM) dataset and 30-class Aerial Image Dataset (AID). The framework was also transferred toa case study of scene annotation in Wuhan. The proposed framework significantly improves the performance when compared with other state-of-the-art methods.

The Use of Intelligent Sensing Algorithm for Internet of Things and Hash Spatial Information Location Technology

This exploration is aimed at quickly obtaining the spatial position information of microseismic focal points and increasing the accuracy of microseismic rapid positioning, to take timely corresponding measures. A microseismic focal point location system completely different from the traditional microseismic location method is proposed. The search engine technology is introduced into the system, which can locate the microseismic focal point quickly and accurately. First, the propagation characteristics of microseismic signals in coal and rock layers are analyzed, and the focal position information is obtained. However, the collected microseismic signal of the coal mine contains noise, so it is denoised at first. Then, a waveform database is established for the denoised waveform data and focal point position. The structure and mathematical model of the location‐sensitive hash (LSH) based on P stable distribution are introduced and improved, and the optimized algorithm multiprobe LSH is obtained. The microseismic location model is established according to the characteristics of microseismic data. The values of three parameters, hash table number, hash function family dimension, and interval size, are determined. The experimental data of the parameters of the search engine algorithm are analyzed. The results show that when the number of hash tables is 6, the dimension k of the hash function family is 14, and the interval size W is 8000, the retrieval time reaches a relatively small value, the recall rate reaches a large value, and the proportion of retrieved candidates is large; the parameters of the search engine algorithm of the measured coal mine microseismic data are analyzed. It is obtained that when the number of hash tables is 4, the dimension k of the hash function family is 6, and the interval size W is 500, the retrieval time reaches a relatively small value, the recall rate obtains a large value, and the proportion of retrieved candidates is large. The contents studied are of great significance to the evaluation of destructive mine earthquakes and impact risk.

Robot dynamic trajectory tracking control algorithm based on steady-state closed-loop learning

In order to improve the stability control ability of flexible lower limb exoskeleton robot, a dynamic trajectory tracking control algorithm of flexible lower limb exoskeleton robot based on steady-state closed-loop learning is proposed. Gyroscope and rangefinder are used as information sensors of flexible lower limb exoskeleton robot to collect position information of flexible lower limb exoskeleton robot, fuse collected positioning information of flexible lower limb exoskeleton robot, fuse physical information and measure parameters of flexible lower limb exoskeleton robot by using dynamic information measurement method, and obtain mapping feature set of Cartesian space according to trajectory components of flexible lower limb exoskeleton robot. The pose information of the flexible lower limb exoskeleton robot is obtained through the forward kinematics, and the information is enhanced according to the spatial position information. The steady-state closed-loop learning method is adopted to realize the adaptive learning of the robot dynamic trajectory tracking control. The simulation results show that this method is adaptive to the dynamic trajectory tracking control of the robot, and the positioning control ability of the robot is strong.