Spatiotemporal Big Data Research Articles

Functional urban land use recognition integrating multi-source geospatial data and cross-correlations

The significance of urban function recognition has stimulated the need for multi-source geospatial data fusion, especially the fusion between remote sensing images and spatiotemporal big data. In previous studies, the natural correspondence across multi-source geospatial data has often been ignored in the description of one object, which would influence the performance of data fusion. Therefore, this study introduces the cross-correlations mechanism to achieve the natural correspondence by taking remote sensing images, point of interest (POI), and real-time social media users as an example. It proposes a new cross-correlations based functional urban land use (CC-FLU) model to infer urban functions. The presented model extracts physical and human semantic features from multi-source geospatial data, then maps them to their common subspaces to obtain their cross-correlations respectively. These semantic features and their cross-correlations are integrated together to classify urban functions. An experiment in Shenzhen, China was implemented to evaluate the performance of the presented model at a fine scale. The results show that the proposed CC-FLU model achieved a better performance than previous methods, yielding OA and Kappa values of 0.851 and 0.812, respectively. The results of the presented approach outperform those of methods using one single source geospatial data. The results demonstrate that the skilled information of each type of geospatial data is fully melted into the data fusion model, and simultaneously achieves the natural correspondence across multi-source geospatial data. Moreover, this study resolves the possible disadvantages of models using one single source data and fusion methods by sequentially concatenating multi-source features. The results will benefit urban planners and urban policy-makers.

Measuring spatio-temporal autocorrelation in time series data of collective human mobility

ABSTRACTMassive spatio-temporal big data about human mobility have become increasingly available. Revealing underlying dynamic patterns from these data is essential for understanding people’s behavior and urban deployment. Spatio-temporal autocorrelation analysis is an exploratory approach to recognizing data distribution in space and time. The most widely used spatial autocorrelation measurements, such as Moran’s I and local indicators of spatial association (LISA), only apply to static data, so are powerless to spatio-temporal big data about human mobility. Thus, we proposed a new method by extending Moran’s I to measure the spatial autocorrelation of time series data. Then the method was applied to taxi ride data in Beijing, China to reveal the spatial pattern of collective human mobility. The result shows that there is strong positive spatio-temporal autocorrelation within the 5th Ring Road, weak negative spatio-temporal autocorrelation nearby the Sixth Ring Road, and almost no spatio-temporal autocorrelation between the roads. Local spatial patterns of taxi travel were also recognized. This method is useful for discovering underlying patterns from spatio-temporal big data to understand human mobility.

SOVAS: a scalable online visual analytic system for big climate data analysis

ABSTRACTBig climate data offers great opportunities for scientific discovery but demands efficient and effective analytics to investigate unknown and complex patterns. Most existing online processing and analytics systems for climate studies only support fixed user interface with predefined functions. These systems are often not scalable to handle massive climate data that could easily accumulate terabytes daily. To address the major limitations of existing online systems for climate studies, this paper presents a scalable online visual analytic system, known as SOVAS, to balance both usability and flexibility. SOVAS, enabled by a set of key techniques, supports large-scale climate data analytics and knowledge discovery in a scalable and sharable environment. This research not only contributes to the community an efficient tool for analyzing big climate data but also contributes to the literature by providing valuable technical references for tackling spatiotemporal big data challenges.

HadoopTrajectory: a Hadoop spatiotemporal data processing extension

The recent advances in location tracking technologies and the widespread use of location-aware applications have resulted in big datasets of moving object trajectories. While there exists a couple of research prototypes for moving object databases, there is a lack of systems that can process big spatiotemporal data. This work proposes HadoopTrajectory, a Hadoop extension for spatiotemporal data processing. The extension adds spatiotemporal types and operators to the Hadoop core. These types and operators can be directly used in MapReduce programs, which gives the Hadoop user the possibility to write spatiotemporal data analytics programs. The storage layer of Hadoop, the HDFS, is extended by types to represent trajectory data and their corresponding input and output functions. It is also extended by file splitters and record readers. This enables Hadoop to read big files of moving object trajectories such as vehicle GPS tracks and split them over worker nodes for distributed processing. The storage layer is also extended by spatiotemporal indexes that help filtering the data before splitting it over the worker nodes. Several data access functions are provided so that the MapReduce layer can deal with this data. The MapReduce layer is extended with trajectory processing operators, to compute for instance the length of a trajectory in meters. This paper describes the extension and evaluates it using a synthetic dataset and a real dataset. Comparisons with non-Hadoop systems and with standard Hadoop are given. The extension accounts for about 11,601 lines of Java code.

Detecting and Evaluating Urban Clusters with Spatiotemporal Big Data.

The design of urban clusters has played an important role in urban planning, but realizing the construction of these urban plans is quite a long process. Hence, how the progress is evaluated is significant for urban managers in the process of urban construction. Traditional methods for detecting urban clusters are inaccurate since the raw data is generally collected from small sample questionnaires of resident trips rather than large-scale studies. Spatiotemporal big data provides a new lens for understanding urban clusters in a natural and fine-grained way. In this article, we propose a novel method for Detecting and Evaluating Urban Clusters (DEUC) with taxi trajectories and Sina Weibo check-in data. Firstly, DEUC applies an agglomerative hierarchical clustering method to detect urban clusters based on the similarities in the daily travel space of urban residents. Secondly, DEUC infers resident demands for land-use functions using a naïve Bayes’ theorem, and three indicators are adopted to assess the rationality of land-use functions in the detected clusters—namely, cross-regional travel index, commuting direction index, and fulfilled demand index. Thirdly, DEUC evaluates the progress of urban cluster construction by calculating a proposed conformance indicator. In the case study, we applied our method to detect and analyze urban clusters in Wuhan, China in the years 2009, 2014, and 2015. The results suggest the effectiveness of the proposed method, which can provide a scientific basis for urban construction.

Review of Power Spatio-Temporal Big Data Technologies for Mobile Computing in Smart Grid

Mobile computing adds computing, storage, processing and other functions in wireless network ends to provide customized and differentiated services, so that it can be widely applied in the fields of internet of things, video, medical treatment, retail and so on. Recently, power spatio-temporal big data (PSTBD) technology of smart grid based on mobile computing has experienced explosive growth. This paper emphasizes the specific requirements, technologies, applications, and challenges of the current PSTBD for mobile computing in smart grid. Based on current development status of PSTBD companies in representative countries in the world, we introduce PSTBD technology based on the characteristics of mobile computing based smart grid, and conduct a comprehensive investigation and analysis of relevant articles in this field. After comparing the differences between the traditional and the PSTBD based platform in the aspects of important features, platform goal, and platform architecture, we describe the key technologies and algorithms of the current PSTBD in detail. Then, based on the requirements of each link and field of the power grid, the typical application of PSTBD technology in various aspects of smart grid application based on mobile computing is discussed. Finally, the development direction and challenges of PSTBD are given. Through data analysis and technical discussion, it hopes to provide technical support and decision support for relevant practitioners in the PSTBD field.

An integrated GIS platform architecture for spatiotemporal big data

With the increase in smart devices, spatiotemporal data has grown exponentially. To deal with challenges caused by an increase data requires a scalable and efficient architecture that can store, query, analyze, and visualize spatiotemporal big data. This paper describes a Cloud-terminal integrated GIS platform architecture designed to meet the requirements of processing and analyzing spatiotemporal big data. Cloud-terminal Integration GIS is developed according to the architecture. Extensive experiments deployed on the internal organization cluster using real-time datasets showed that the SuperMap GIS spatiotemporal big data engine achieved excellent performance.

RESEARCH AND PRACTICE ON SPATIO-TEMPORAL BIG DATA CLOUD PLATFORM OF THE BELT AND ROAD INITIATIVE

Abstract. Spatio-temporal big data cloud platform is an important spatial information infrastructure that can provide different period spatial information data services, various spatial analysis services and flexible API services. Activities of policy coordination, facilities connectivity and unimpeded trade on the Belt and Road Initiative (B&R) will create huge demands to the spatial information infrastructure. This paper focuses on researching a distributed spatio-temporal big data engine and an extendable cloud platform framework suits for the B&R and some key technologies to implement them. A distributed spatio-temporal big data engine based on Cassandra™ and an extendable 4-tier architecture cloud platform framework is put forward according to the spirit of parallel computing and cloud service. Four key technologies are discussed: 1) a storage and indexing method for distributed spatio-temporal big data, 2) an automatically collecting, processing, mapping and updating method of authoritative spatio-temporal data for web mapping service, 3) a schema of services aggregation based on nodes registering and services invoking based on view extension, 4) a distributed deployment and extension method of the cloud platform. We developed a distributed spatio-temporal big data centersoftware and founded the main node platform portal with MapWorld™ map services and some thematic information services inChina and built some local platform portals for those countries in the B&R area. The management and analysis services for spatio-temporal big data were built in flexible styles on this platform. Practices show that we provide a flexible and efficient solution tobuild the distributed spatio-temporal big data center and cloud platform, more node portals can be aggregated to the main portal bypublishing their own web services and registering them in the aggregation schema. The data center and platform can support thestorage and management of massive data well and has higher fault tolerance and better scalability.

TEMPORAL GIS AND SPATIOTEMPORAL DATA SOURCES

The recent technological advances in geospatial data collection have created massive data sets with better spatial and temporal resolution than ever. To properly deal with these data sets, geographical information systems (GIS) must evolve to represent, access, analyze and visualize big spatiotemporal data in an efficient and integrated way. In this paper, we highlight challenges in temporal GIS development and present a proposal to overcome one of them: how to access spatiotemporal data sets from distinct kinds of data sources. Our approach uses Semantic Web techniques and is based on a data model that takes observations as basic units to represent spatiotemporal information from different application domains. We define a RDF vocabulary for describing data sources that store or provide spatiotemporal observations.

Uncovering patterns of ties among regions within metropolitan areas using data from mobile phones and online mass media

Given the abundant quantities of big spatiotemporal geographic data that are available, interactions among spatial entities can now be extracted from various perspectives. This research investigates the spatial interactions within the metropolis of Beijing quantitatively. Two methods of quantifying the interactions are proposed. These interactions can be calculated from either individual trajectories extracted from mobile phone records or the co-occurrence of the toponyms of administrative units mentioned in online news items. By fitting these two types of data with a gravity model and comparing the results, we determine that the distance decay effect exists in both data sets, and this effect is more obvious in the interactions computed from the human trajectories. The spatial interactions and connections quantified from the two data sources display greater numbers of mutual patterns in the central urban areas, whereas more diversity is observed in the suburban areas. We conclude that the choice of assumptions as to which data can adequately represent spatial interactions significantly affects the results; therefore, rigorous examination of specific problems is needed to redefine the problems in a more specific way.

Low-Dimensional Models for Traffic Data Processing Using Graph Fourier Transform

The reliability of services offered by intelligent transportation systems is attributed to the accuracy and timely availability of road-network traffic information. However, in the present era of big data, compliance with anticipated service quality requirements mandates consistent real-time processing of big spatiotemporal traffic data. Thus, development of low-dimensional models is a crucial challenge in traffic data processing. The authors developed such representations using data graph framework and graph Fourier transform (GFT) approaches. Experimental results on California daily network traffic data showed that, even with a 15:1 compression ratio, GFT-based models offered less than 6 percent reconstruction error (RE), instigating a less than 2 percent increase in mean absolute percentage error of corresponding predictions. The authors also proposed a 3D graph framework, which reduced RE by almost 2 percent compared to its 2D counterpart.

Principles for statistical inference on big spatio-temporal data from climate models

The vast increase in size of modern spatio-temporal data sets has prompted statisticians working in environmental applications to develop new and efficient methodologies that are still able to achieve inference for nontrivial models within an affordable time. Climate model outputs push the limits of inference for Gaussian processes, as their size can easily be larger than 10 billion data points. Drawing from our experience in a set of previous work, we provide three principles for the statistical analysis of such large data sets that leverage recent methodological and computational advances. These principles emphasize the need of embedding distributed and parallel computing in the inferential process.

ELASTIC CLOUD COMPUTING ARCHITECTURE AND SYSTEM FOR HETEROGENEOUS SPATIOTEMPORAL COMPUTING

Abstract. Spatiotemporal computation implements a variety of different algorithms. When big data are involved, desktop computer or standalone application may not be able to complete the computation task due to limited memory and computing power. Now that a variety of hardware accelerators and computing platforms are available to improve the performance of geocomputation, different algorithms may have different behavior on different computing infrastructure and platforms. Some are perfect for implementation on a cluster of graphics processing units (GPUs), while GPUs may not be useful on certain kind of spatiotemporal computation. This is the same situation in utilizing a cluster of Intel's many-integrated-core (MIC) or Xeon Phi, as well as Hadoop or Spark platforms, to handle big spatiotemporal data. Furthermore, considering the energy efficiency requirement in general computation, Field Programmable Gate Array (FPGA) may be a better solution for better energy efficiency when the performance of computation could be similar or better than GPUs and MICs. It is expected that an elastic cloud computing architecture and system that integrates all of GPUs, MICs, and FPGAs could be developed and deployed to support spatiotemporal computing over heterogeneous data types and computational problems.

SPATIOTEMPORAL DATA ORGANIZATION AND APPLICATION RESEARCH

Abstract. Organization and management of spatiotemporal data is a key support technology for intelligence in all fields of the smart city. The construction of a smart city cannot be realized without spatiotemporal data. Oriented to support intelligent applications，this paper proposes an organizational model for spatiotemporal data, and details the construction of a spatiotemporal big data calculation, analysis, and service framework for highly efficient management and intelligent application of spatiotemporal data for the entire data life cycle.

Geographic spatiotemporal big data correlation analysis via the Hilbert–Huang transformation

As a typical representative of big data, geographic spatiotemporal big data present new features especially the non-stationary feature, bringing new challenges to mine correlation information. However, representation of instantaneous information is the main bottleneck for non-stationary data, but the traditional non-stationary analysis methods are limited by Heisenberg's uncertainty principle. Therefore, we firstly represent instantaneous frequency of geographic spatiotemporal big data based on Hilbert–Huang transform to overcome traditional methods' weakness. Secondly, we propose absolute entropy correlation analysis method based on KL divergence. Finally, we select five geographic factors to certify that the absolute entropy correlation analysis method is effective and distinguishable.

A High Performance, Spatiotemporal Statistical Analysis System Based on a Spatiotemporal Cloud Platform

With the increase in size and complexity of spatiotemporal data, traditional methods for performing statistical analysis are insufficient for meeting real-time requirements for mining information from Big Data, due to both data- and computing-intensive factors. To solve the Big Data challenges in geostatistics and to support decision-making, a high performance, spatiotemporal statistical analysis system (Geostatistics-Hadoop) is proposed in this paper. The proposed system has several features: (1) Hadoop is enhanced to handle spatial data in a native format and execute a number of parallelized spatial analysis algorithms to solve practical geospatial analysis problems; (2) the Oozie-based workflow system is utilized to ease the operation and sharing of spatial analysis services; and (3) a private cloud platform based on Eucalyptus is leveraged to provide on-the-fly and elastic computing resources. Experimental results show that Geostatistics-Hadoop efficiently conducts rapid information mining and analysis of big spatiotemporal data sets, with the support of elastic computing resources from a cloud platform. The adoption of cloud computing and the Hadoop cluster to parallelize statistical calculations significantly improves the performance of Big Data analyses.

A NoSQL–SQL Hybrid Organization and Management Approach for Real-Time Geospatial Data: A Case Study of Public Security Video Surveillance

With the widespread deployment of ground, air and space sensor sources (internet of things or IoT, social networks, sensor networks), the integrated applications of real-time geospatial data from ubiquitous sensors, especially in public security and smart city domains, are becoming challenging issues. The traditional geographic information system (GIS) mostly manages time-discretized geospatial data by means of the Structured Query Language (SQL) database management system (DBMS) and emphasizes query and retrieval of massive historical geospatial data on disk. This limits its capability for on-the-fly access of real-time geospatial data for online analysis in real time. This paper proposes a hybrid database organization and management approach with SQL relational databases (RDB) and not only SQL (NoSQL) databases (including the main memory database, MMDB, and distributed files system, DFS). This hybrid approach makes full use of the advantages of NoSQL and SQL DBMS for the real-time access of input data and structured on-the-fly analysis results which can meet the requirements of increased spatio-temporal big data linking analysis. The MMDB facilitates real-time access of the latest input data such as the sensor web and IoT, and supports the real-time query for online geospatial analysis. The RDB stores change information such as multi-modal features and abnormal events extracted from real-time input data. The DFS on disk manages the massive geospatial data, and the extensible storage architecture and distributed scheduling of a NoSQL database satisfy the performance requirements of incremental storage and multi-user concurrent access. A case study of geographic video (GeoVideo) surveillance of public security is presented to prove the feasibility of this hybrid organization and management approach.

A Framework for Discovering Evolving Domain Related Spatio-Temporal Patterns in Twitter

In massive Twitter datasets, tweets deriving from different domains, e.g., civil unrest, can be extracted to constitute spatio-temporal Twitter events for spatio-temporal distribution pattern detection. Existing algorithms generally employ scan statistics to detect spatio-temporal hotspots from Twitter events and do not consider the spatio-temporal evolving process of Twitter events. In this paper, a framework is proposed to discover evolving domain related spatio-temporal patterns from Twitter data. Given a target domain, a dynamic query expansion is employed to extract related tweets to form spatio-temporal Twitter events. The new spatial clustering approach proposed here is based on the use of multi-level constrained Delaunay triangulation to capture the spatial distribution patterns of Twitter events. An additional spatio-temporal clustering process is then performed to reveal spatio-temporal clusters and outliers that are evolving into spatial distribution patterns. Extensive experiments on Twitter datasets related to an outbreak of civil unrest in Mexico demonstrate the effectiveness and practicability of the new method. The proposed method will be helpful to accurately predict the spatio-temporal evolution process of Twitter events, which belongs to a deeper geographical analysis of spatio-temporal Big Data.

The synthetic geo-ecological environmental evaluation of a coastal coal-mining city using spatiotemporal big data: A case study in Longkou, China

The geological and ecological (geo-ecological) environment of the coastal coal-mining city is a basic element for human subsistence, and it connects the regional economy with social sustainable development. Using Longkou, a coastal mining city, as an example, a geological and ecological environmental quality assessment was conducted based on spatiotemporal big data. Remote sensing images, a digital elevation model (DEM), and precipitation and interpolation processing were used to generate factor layers. A synthetic evaluation index system was set up, including physical geography, geological conditions, mining intensity, ecological environmental recovery and geological hazards associated with mining. Moreover, an analytical hierarchy process was used to calculate the factor weight of each evaluation factor, and a consistency check was performed to build an assessment model of the geo-ecological environment of Longkou. The results indicate that multi-factor spatiotemporal big data provide a scientific assessment of the geo-ecological environmental quality with indispensable data and methods. The spatial distribution of geo-ecological environmental quality presented clear specialization of zonality, showing poor quality in the coastal coal mine ore concentration area and good quality in the inland and mountainous areas of Nanshan Mountain. The geo-ecological environmental quality of Longkou was divided into 5 levels as worst, poor, middle, good and better districts. The good and better districts accounted for 76.763% of the total area of the assessment region, indicating that the geo-ecological environmental quality of the study area was in good condition. The mining intensity and ecological environment recovery were major factors in determining the regional variation of the geo-ecological environment of Longkou. The possible causes inducing uncertainties and limitations in evaluation of the geo-ecological environmental quality were discussed. The model combining AHP with GIS proposed in this paper is an effective means of evaluating regional geo-ecological environmental quality.

Discussion on geological science big data and its applications

The geological big data is a kind of spatio-temporal big data whose characteristics has both similarities and marked differences with the general big data. Due to long-term evolution of geological objects with huge space and influence of various geological process, its development undergoes a complicated process. Geological bodies are buried in deep underground so that geological data has the feature of incomplete information of parameter, structure, relationship and evolution, which has high computation complexity with high dimension and significant uncertainty. In order to address those challenges, the theories, methods and techniques of big data should be introduced to integrate and utilize geological science big data. It is necessary to set up unified spatial reference system for geological spatio-temporal big data to realize data consistency processing and fusion with regards to spatial standard benchmark, tense, scale and semantics. Various kinds of integrated storage and management methods for static geological exploration data and dynamic distributed geological observation data will be developed. On the one hand, current big data techniques can be adapted, such as realtime data storage and management, big data storage and management architecture, and analysis and processing architecture. On the other hand, specialized big data techniques based on actual situation should be developed, such as distributed parallel spatio-temporal indexing for geological data, and pre-dispatch method considering spatio- temporal relationship and geological semantics. To judge a traditional algorithm whether is suitable for being transformed into big data environment, main criteria include algorithm task decomposability, data decomposability and data flow segmented relevance. Hence, the transformation of traditional geological data processing methods will be from these three aspects to excavate the task parallelism, geometry decomposability and reduce the data flow segmented relevance as much as possible to adapt to the distributed computing and high performance computing of big data environment. The big data technique, which may be a breakthrough in geological science, can mine knowledge from massive spatio-temporal and text data directly in spite of the limits in geological sampling methods such as the randomness and sparsity of sample space, the judge relied on inadequate observational data and fixed mode, and traditional data analysis methods. A series of mathematical geology methods and spatial data mining methods which are frequently used nowadays can be used in geological spatio-temporal big data mining after being remolded. The key science and technology issues referred in geological science development in big data era can be listed as follows—the integrated storage management and handling of structured, semi-structured and unstructured data, big and small data, hybrid and precision data, model and data, static exploration model and dynamic monitoring model, the combination of data mining and data analysis, the unification of correlation and causality, deep mining and visualization of geological science big data, and so on. Taking metal mineral metallogenic prediction as an example, geological big data utilization and analysis are studied. In terms of a global view, Glass Earth is the underground extention of Digital Earth. Glass Earth is a kind of 3D visualized virtual shallow crust, which is an aggregation of geological and geographical information and stored on the computer network to be accessed by multi-user and provide analysis service and decision making to the research and applications of geology, resources and environment. The Glass Earth is the effective vehicle of geological science big data and the construction of the Glass Earth is one of the best ways to solve all the aforementioned scientific and technical problems.