Spatial Geometric Information Research Articles

XDL-ESI: Electrophysiological Sources Imaging via explainable deep learning framework with validation on simultaneous EEG and iEEG

Electroencephalography (EEG) or Magnetoencephalography (MEG) source imaging aims to estimate the underlying activated brain sources to explain the observed EEG/MEG recordings. Solving the inverse problem of EEG/MEG Source Imaging (ESI) is challenging due to its ill-posed nature. To achieve a unique solution, it is essential to apply sophisticated regularization constraints to restrict the solution space. Traditionally, the design of regularization terms is based on assumptions about the spatiotemporal structure of the underlying source dynamics. In this paper, we propose a novel paradigm for ESI via an Explainable Deep Learning framework, termed as XDL-ESI, which connects the iterative optimization algorithm with deep learning architecture by unfolding the iterative updates with neural network modules. The proposed framework has the advantages of (1) establishing a data-driven approach to model the source solution structure instead of using hand-crafted regularization terms; (2) improving the robustness of source solutions by introducing a topological loss that leverages the geometric spatial information applying varying penalties on distinct localization errors; (3) improving the reconstruction efficiency and interpretability as it inherits the advantages from both the iterative optimization algorithms (interpretability) and deep learning approaches (function approximation). The proposed XDL-ESI framework provides an efficient, accurate, and interpretable paradigm to solve the ESI inverse problem with satisfactory performance in both simulated data and real clinical data. Specially, this approach is further validated using simultaneous EEG and intracranial EEG (iEEG).

Bilinear Distance Feature Network for Semantic Segmentation in PowerLine Corridor Point Clouds.

Semantic segmentation of target objects in power transmission line corridor point cloud scenes is a crucial step in powerline tree barrier detection. The massive quantity, disordered distribution, and non-uniformity of point clouds in power transmission line corridor scenes pose significant challenges for feature extraction. Previous studies have often overlooked the core utilization of spatial information, limiting the network's ability to understand complex geometric shapes. To overcome this limitation, this paper focuses on enhancing the deep expression of spatial geometric information in segmentation networks and proposes a method called BDF-Net to improve RandLA-Net. For each input 3D point cloud data, BDF-Net first encodes the relative coordinates and relative distance information into spatial geometric feature representations through the Spatial Information Encoding block to capture the local spatial structure of the point cloud data. Subsequently, the Bilinear Pooling block effectively combines the feature information of the point cloud with the spatial geometric representation by leveraging its bilinear interaction capability thus learning more discriminative local feature descriptors. The Global Feature Extraction block captures the global structure information in the point cloud data by using the ratio between the point position and the relative position, so as to enhance the semantic understanding ability of the network. In order to verify the performance of BDF-Net, this paper constructs a dataset, PPCD, for the point cloud scenario of transmission line corridors and conducts detailed experiments on it. The experimental results show that BDF-Net achieves significant performance improvements in various evaluation metrics, specifically achieving an OA of 97.16%, a mIoU of 77.48%, and a mAcc of 87.6%, which are 3.03%, 16.23%, and 18.44% higher than RandLA-Net, respectively. Moreover, comparisons with other state-of-the-art methods also verify the superiority of BDF-Net in point cloud semantic segmentation tasks.

Dual encoding DDS‐UNet liver tumour segmentation based on multi‐scale deep and shallow feature fusion

AbstractThe fusion and utilization of multi‐scale deep and shallow features are of great significance in liver tumour segmentation. This study proposes a dual encoding DDS‐UNet liver tumour segmentation method based on multi‐scale deep and shallow feature fusion, aiming to fully achieve the fusion and utilization of deep and shallow features and achieve accurate segmentation. The proposed method mainly consists of residual convolution module fusion residual convolution (FRC), dual encoding end fusion module dual encoding end fusion (DEF), and skip connection fusion module jump connection fusion module (JCF). In the residual convolution module, a layer by layer fused residual convolution is used instead of traditional convolution to achieve better training. In the dual encoding end fusion module, multi‐scale feature fusion at the end provides more comprehensive contextual information, solves the loss of spatial geometric information. The skip connection fusion module reduces the interference of invalid features by changing the weights of spatial attention and channel attention on important features. The Dice coefficient, average intersection to union ratio, accuracy, recall, and accuracy indicators tested on the LiTS dataset were 90.37%, 90.16%, 93.78%, 94.91%, and 98.84%, respectively, which are superior to many advanced liver tumor segmentation methods.

Development of a mixed reality assisted escape system for underground mine- based on the mine water-inrush accident background

The underground mine is an essential infrastructure for national resources and economy, and its safety system is an important issue, especially the miners’ safety. They should not only enhance their preventive awareness of disasters but also enhance their response capability. When miners are in danger with invalid escape lighting, it is difficult for them to overcome panic. Therefore, the automatic assisted escape system has a particular engineering significance in underground space. This study attempts to develop a reference technical framework for emergency evacuation of underground space workers. A human-environment-computer interaction system is developed, which combines the A*-3D Time algorithm with mixed reality (MR) technology. It is constrained by the spatial geometric information of the mine and the escape time. Firstly, the target mine’s geographic information and spatial structure are surveyed on the field, and a 3D digital model is built by the 3DMine; the BIM is used to integrate 3DMine information (data and material) and design the spatial structure. Secondly, based on the A* algorithm in 3D space, the escape time is introduced to evaluate the escape capability and construct the A*-3D Time model. Thirdly, the timeliness of mine water-inrush scenario is simulated by the Comsol; the interaction method and navigation map are developed by the OpenXR and MR Toolkit (MRTK) in Game Engine; the A*-3D Time model is introduced into the C# script and assigned to the digital objects (Mine and Miners). Finally, the performance of this system is analyzed based on the underground metal mine. Based on game thinking to solve safety problems can be used as a reference for underground space engineering.

Neighborhood Manifold Preserving Matching for Visual Place Recognition

This paper proposes an effective and efficient visual place recognition (VPR) approach, which can make full use of semantic, sequential and spatial geometric information in VPR tasks. Rather than previous methods focusing on extracting discriminative and compact features to represent images, we improve VPR performance from candidate selection and geometric verification. To this end, we propose a fast feature matching algorithm for real-time geometrical verification of candidate places, termed neighborhood manifold preserving matching (NMP). To generate high-quality candidates, we design a dynamic sequence partitioning strategy based on NMP, which is able to utilize the inherently sequential nature of spatial data to cluster images into places. By sequence-to-sequence matching, the ambiguity of single frame matching can be reduced. Extensive experiments demonstrate that our VPR method outperforms the current state-of-the-art methods, and our geometric verification and candidate selection strategies are easily plugged into other VPR pipelines to significantly improve the VPR performance.

Proactive cognitive control, mathematical cognition and functional activity in the frontal and parietal cortex in primary school children: An fNIRS study

Understanding how children acquire mathematical abilities is fundamental to planning mathematical schooling. This study focuses on the relationships between mathematical cognition, cognition in general and neural foundation in 8 to 9-year-old children. We used additive mathematics tests, cognitive tests determining the tendency for proactive and reactive problem solving and functional near-infrared spectroscopy (fNIRS) for functional brain imaging. The ability to engage in proactive control had a stronger association with mathematical performance than other cognitive abilities, such as processing speed, sustained attention and pattern recognition. The fNIRS method identified differences between proactive and reactive control, i.e., the more proactive the children were, the greater the increase in oxygenated hemoglobin in the left lateral prefrontal cortex during reactive beneficiary situations. During a text-based task involving additive reasoning, increased activity in the dorsal medial prefrontal cortex was detected compared to a similar task with supportive spatial-geometric information.

Visual odometry algorithm based on geometric prior for dynamic environments

Simultaneous localization and mapping (SLAM) is considered to be an important way for some smart devices to perform automatic path planning, and many successful SLAM systems have been developed in the past few years. Most existing approaches rely heavily on static world assumptions, and such strong assumptions limit the application of most vSLAM (visual SLAM) in complex dynamic reality environments, where dynamic objects often lead to incorrect data association in tracking, which reduces the overall accuracy and robustness of the system and causes tracking crashes. The dynamic objects in the map may change over time; thus, distinguishing dynamic information in a scene is challenging. In order to solve the interference problem of dynamic objects, most point-based visual odometry algorithms have concentrated on feature matching or direct pixel intensity matching, disregarding an ordinary but crucial image entity: geometric information. In this article, we put forward a novel visual odometry algorithm based on dynamic point detection methods called geometric prior and constraints. It removes the moving objects by combining the spatial geometric information of the image and depends on the remaining features to estimate the position of the camera. To the best of our knowledge, our proposed algorithm achieves superior performance over existing methods on a variety of public datasets.

Comparison of Machine Learning Methods Applied on Multi-Source Medium-Resolution Satellite Images for Chinese Pine (Pinus tabulaeformis) Extraction on Google Earth Engine

Chinese pine has tremendous applications in many fields. Mapping the distribution of Chinese pine is of great importance for government decision-making and forest management. In order to extract Chinese pine on a large scale, efficient algorithms and open remote-sensing datasets are needed. It is widely believed that machine learning algorithms and medium-resolution remote-sensing datasets can work well for this purpose. Unfortunately, their performance for Chinese pine extraction has remained unclear until now. Therefore, this study aims to explore the ability of the different machine learning algorithms and open remote-sensing datasets for Chinese pine extraction over large areas on Google Earth Engine (GEE). So, based on the combination of three typical machine learning algorithms, namely deep neural network (DNN), support vector machine (SVM), random forest (RF), and three open medium-resolution remote-sensing datasets, namely Sentinel-2, Gaofen-1, and Landsat-8 OLI, 27 models are constructed and GEE, with its powerful computing ability, is used. The main findings are as follows: (1) DNN has the highest accuracy for Chinese pine extraction, followed by SVM and RF; DNN is more sensitive to spatial geometric information, while SVM and RF algorithms are more sensitive to spectral information. (2) Spectral indexes are helpful for improving the extraction accuracy of Chinese pine. The extraction accuracy by using Gaofen-1 dataset increases 7.6% after adding spectral indexes, while the accuracies by using Sentinel-2 and Landsat-8 datasets increase 1.8% and 1.9% after adding spectral indexes, respectively. (3) The extraction accuracy by using DNN and Sentinel-2 dataset with spectral indexes is the highest, with an overall accuracy of 94.4%. (4) The area of Chinese pine is 153.73 km2, accounting for 5.06% of the administrative area of Karaqin Banner, and it is convenient to extract Chinese pine on a large scale by using GEE.

A Unified Pansharpening Method With Structure Tensor Driven Spatial Consistency and Deep Plug-and-Play Priors

Pansharpening is to generate a high resolution multispectral (HRMS) image by preserving the spectral information from a low resolution multispectral (LRMS) image and the spatial content from a panchromatic (PAN) image. This article proposes a unified pansharpening method with structure tensor driven spatial consistency and deep plug and play priors. First, the spectral fidelity constraint between HRMS and LRMS is imposed for preserving spectral information. Second, the structure tensor is applied to characterize the spatial geometric information of HRMS and PAN images, thus the structure tensor driven spatial consistency prior between HRMS and PAN is particularly exploited for preserving spatial content. Moreover, by generalizing the convolution neural network (CNN) fusion method into a unified variational framework, a novel CNN-based deep plug and play prior between the HRMS and CNN-based fused MS images is also proposed to generate more image characteristics for further preserving spectral information and spatial content. Besides, the proposed model is solved by the alternating direction method of multipliers (ADMM) algorithm. Finally, extensive experiments on both reduced and full resolution by comparing with various representative approaches exhibit the excellent performance of the proposed method.

Multisource high-resolution optical remote sensing image registration based on point–line spatial geometric information

High-resolution multisource optical remote sensing images often have considerable non-linear radiation and scale differences that become more prominent as the image resolution increases. Since this problem may cause low registration accuracy, we propose an automatic registration algorithm based on point–line spatial geometric information (PLSGI). First, we propose an improved scale invariant feature transform algorithm using the quadtree uniformization, the Bhattacharyya distance, and the slope constraint to obtain many correct point pairs. Then we generate a new global line descriptor using PLSGI. In particular, the descriptor can resist the non-linear radiation and scale differences of multisource remote sensing images. Finally, we use a piecewise linear model to warp the sensed images based on a set of tie points that consist of corresponding features and corresponding intersections. The effectiveness of the proposed algorithm was validated using multiple groups of high-resolution remote sensing images. The experimental results indicated that our algorithm is highly generalized and robust for high-resolution multisource remotely sensed images.

SCN: Dilated silhouette convolutional network for video action recognition

Human action is a spatio-temporal motion sequence where strong inter-dependencies between the spatial geometry and temporal dynamics of motion exist. However, in existing literature for human action recognition from a video, there is a lack of synergy in investigating spatial geometry and temporal dynamics in a joint representation and embedding space. In this paper, we propose a dilated Silhouette Convolutional Network (SCN) for action recognition from a monocular video. We model the spatial geometric information of the moving human subject using silhouette boundary curves extracted from each frame of the motion video. The silhouette curves are stacked to form a 3D curve volume along the time axis and resampled to a 3D point cloud as a unified spatio-temporal representation of the video action. With the dilated silhouette convolution, the SCN is able to learn co-occurrence features from low-level geometric shape boundaries and their temporal dynamics jointly, and construct a unified convolutional embedding space, where the spatial and temporal properties are integrated effectively. The geometry-based SCN significantly improves the discrimination of learned features from the shape motions. Experiment results on the JHMDB, HMDB, and UCF101 datasets demonstrate the effectiveness and superiority of our proposed representation and deep learning method.

Land-Use/Land-Cover Change Detection Based on Class-Prior Object-Oriented Conditional Random Field Framework for High Spatial Resolution Remote Sensing Imagery

High spatial resolution (HSR) remote sensing images can reflect more subtle changes and more specific types of land use and land cover (LULC) due to the abundant spatial geometric information. In this article, a class-prior object-oriented conditional random field (COCRF) framework consisting of a binary change detection (CD) task and a multiclass CD task is proposed to fill the application gap. In the proposed framework, the class-prior knowledge is used to improve the construction of the unary potential in both the binary and multiclass CD tasks, to reduce the influence of spectral variability. The binary CD result provides a constraint to the multiclass CD result. As a result, both parts have effective interaction. The class posterior probability images of two dates can be obtained automatically with the class-prior knowledge by sample migration. Furthermore, an object constraint described by the class dispersion within the objects is added to improve the smoothness in local objects, while the pairwise potential improves the smoothness of the whole area by using the eight-neighborhood spectral information of the center pixel. By integrating the above approaches, the problems of error accumulation and the manual intervention required in the traditional multiclass CD methods can be relieved. An adaptive parameter estimation strategy is also adopted in the proposed framework, to save the time required for manual parameter setting. The proposed COCRF framework was validated on two HSR remote sensing image data sets, where it achieved a better performance than the other state-of-the-art CD methods.

The geometry as an eyed fish feels it in spontaneous and rewarded spatial reorientation tasks

Disoriented human beings and animals, the latter both sighted and blind, are able to use spatial geometric information (metric and sense properties) to guide their reorientation behaviour in a rectangular environment. Here we aimed to investigate reorientation spatial skills in three fish species (Danio rerio, Xenotoca eiseni, Carassius auratus) in an attempt to discover the possible involvement of extra-visual senses during geometric navigation. We observed the fish’s behaviour under different experimental procedures (spontaneous social cued task and rewarded exit task), providing them different temporal opportunities to experience the environmental shape (no experience, short and prolonged experience). Results showed that by using spontaneous social cued memory tasks, fishes were not able to take advantage of extra-visual senses to encode the spatial geometry, neither allowing them short time-periods of environmental exploration. Contrariwise, by using a reference memory procedure, during the rewarded exit tasks, thus providing a prolonged extra-visual experience, fishes solved the geometric task, showing also differences in terms of learning times among species.

Preliminary Study on Green Building Design Review System Based on BIM

Sustainable development is a common concern in today’s world. In the field of urban construction, it has also begun to develop from traditional high-consumption buildings to green and efficient buildings. Green buildings have become the main trend of world architectural development. From the perspective of design and application of architectural design review system, combined with the concept of BIM technology, green building design and related requirements, this paper proposes to use computer technology to build BIM building information model, and to construct spatial geometric information, functional information, building materials and Integrated management of equipment. Besides other related data provides relevant basis for the design of green buildings. At the same time, the formation of the green building design review system can realize the green building evaluation function by visualizing the three-dimensional data information.

A model-based approach to convert a building BIM-IFC data set model into CityGML

ABSTRACTDigital building modelling faces issues related to inconsistent integration/interoperability, in particular for a BIM GIS convergence. A consistent data conversion approach should consider semantic and spatial geometric modelling, therefore potentially leading to a loss of spatial geometric information. This paper simplifies semantic solving, and concentrates on the accuracy of the spatial geometric representation to avoid conflicting 3D spatial mismatches. To solve the geometry, original spatial specifications in 3D described in the IFC data schema are considered for all objects present in the model. The method is explored with a main BIM-IFC model, and additionally tested with two other models.

A new method for 3D individual tree extraction using multispectral airborne LiDAR point clouds

Characterization of individual trees is essential for many applications in forest management and ecology. Previous studies relied on single tree detection from monochromatic wavelength airborne laser scanning (ALS) systems and they focused on the use of the geometric spatial information of the point clouds (i.e., X, Y, and Z coordinates). However, there is quite often a difficulty dealing with clumped trees when only the geometric spatial information is considered. The emergence of multispectral LiDAR sensors provides a new solution for individual tree structure acquisition. The aim of this paper is to investigate the performance of multispectral ALS data for delineating individual trees which are challenging by using the monochromatic wavelength ALS system. The proposed workflow utilizes the mean shift segmentation method on different feature spaces for crown isolation. In addition, both spatial domain and multispectral domain are used to refine the under-segmentation crown segments. Ten plots (2 sets of different structural complexity) located in the dense coniferous forest area in Tobermory, Ontario, Canada are selected as experiment data. Results show that the developed method correctly detects 88% and 82% of the dominant trees with and without multispectral information, respectively. Compared with segmentation using geometric spatial information solely, the main improvements are achieved for clumped tree segment with the distinguished multispectral features. This study demonstrates that multispectral airborne laser scanning data is more capable for individual tree delineation than monochromatic wavelength laser scanning data in dealing with forests with clumped crowns in dense forests.

An automated approach for wood-leaf separation from terrestrial LIDAR point clouds using the density based clustering algorithm DBSCAN

The terrestrial light detection and ranging (LiDAR) technique has been recently used to provide 3D structural information of forest canopy at the individual tree level. However, the operational use of Terrestrial Laser Scanner (TLS) for canopy characterization of broadleaf non-deciduous forests needs further investigations. The estimation of wood volume, above-ground woody biomass, tree canopy characteristics and leaf area index often requires separation of photosynthetically active material and non-photosynthetically active material. This article describes an automated wood-leaves separation method, based on spatial geometric information of TLS point clouds, for broad leaved non-deciduous trees. Scans of seven individuals of Quercus suber L. trees were acquired by using the TLS phase-based Leica HDS6100. Point clouds were partitioned in cubic volumes (voxels) that were used as input to generate clusters through the point density algorithm DBSCAN. The clustering process led to the identification of wood and non-wood voxels. A specific automatic routine was written to process data from the point clouds to the visualization of clustering results. The analysis of results showed good performance for this approach, with the overall accuracy in classifying wood components of trees ranging from 95% to 97%. The largest accuracies were observed for branches larger than 5 cm in diameter whereas the accuracy of classification dropped, as expected, for branches with diameter lower than 3 cm.The results suggest that the proposed method can be conveniently used to extract woody components from point clouds of broad leaved non-deciduous trees.

A Novel Generalized Intensity-Hue-Saturation (GIHS) Based Pan-Sharpening Method With Variational Hessian Transferring

The popular intensity-hue-saturation (IHS) pan-sharpening method can provide high spatial quality while suffering from some spectral distortion, mainly due to the fact that it cannot estimate an accurate intensity image to substitute the original intensity image in the IHS space. To overcome this drawback, in this paper, we particularly use the modeling idea of variational complementary data fusion, and propose a novel yet highly effective pan-sharpening method with variational Hessian transferring in the generalized IHS (GIHS) transform domain, which aims to estimate a more accurate intensity image. More specifically, the novelty of proposed method consists in building a variational Hessian transferring model in the GIHS transform domain to transfer the Hessian-based spatial geometric information of panchromatic (Pan) image to the new intensity image and meanwhile take the spectral information preserving into consideration. Finally, the experimental results demonstrate the effectiveness of the proposed method which can perform higher spectral and spatial qualities, and higher efficiency than some state-of-the-art methods.

Combined discriminative global and generative local models for visual tracking

It is a challenging task to develop an effective visual tracking algorithm due to factors such as pose variation, rotation, and so on. Combined discriminative global and generative local appearance models are proposed to address this problem. Specifically, we develop a compact global object representation by extracting the low-frequency coefficients of the color and texture of the object based on two-dimensional discrete cosine transform. Then, with the global appearance representation, we learn a discriminative metric classifier in an online fashion to differentiate the target object from its background, which is very important to robustly indicate the changes in appearance. Second, we develop a new generative local model that exploits the scale invariant feature transform and its spatial geometric information. To make use of the advantages of the global discriminative model and the generative local model, we incorporate them into Bayesian inference framework. In this framework, the complementary models help the tracker locate the target more accurately. Furthermore, we use different mechanisms to update global and local templates to capture appearance changes. The experimental results demonstrate that the proposed approach performs favorably against state-of-the-art methods in terms of accuracy.

Infrared‐visible video fusion based on motion‐compensated wavelet transforms

Video data fusion should simultaneously take into account both temporal and spatial dimensions, and therefore a novel spatiotemporal video-fusion algorithm based on motion compensation in the wavelet-transform domain is proposed in this study. The fusion method incorporates motion compensation and the wavelet transform, thus making full use of spatial geometric information and inter-frame temporal information of input videos. The proposed method improves the temporal stability and consistency of the fused video compared to other existing individual frame-based fusion methods. The algorithm first decomposes the image frames which have been processed by an optic flow motion-compensation approach and then develops a spatiotemporal energy-based fusion rule to merge input videos. Experimental results demonstrate that the proposed fusion algorithm has superior visual and quantitative performance to traditional individual-frame-based and state-of-the-art three-dimensional-transform-based methods.