A Latent Feature-Based Multimodality Fusion Method for Theme Classification on Web Map Service

Zelong Yang,Zhipeng Gui,Wenwen Li,Huayi Wu

doi:10.1109/access.2019.2954851

Abstract

Massive maps have been shared as Web Map Service (WMS) from various providers, which could be used to facilitate people's daily lives and support space analysis and management. The theme classification of maps could help users efficiently find maps and support theme-related applications. Traditionally, metadata is usually used in analyzing maps content, few papers use maps, especially legends. In fact, people usually considers metadata, maps and legends together to understand what maps tell, however, no study has tried to exploit how to combine them. This paper proposes a method to fuse them with the purpose of classifying map themes, named latent feature based multimodality fusion for theme classification (LFMF-TC). Firstly, a multimodal dataset is created that supports the supervised classification on map themes. Secondly, textual and visual features are designed for metadata, maps, and legends using some advanced techniques. Thirdly, a latent feature based fusion method is proposed to fuse the multimodal features on the feature level. Finally, a neural network classifier is implemented using supervised learning on the multimodal dataset. In addition, a web-based collaboration platform is developed to facilitate users in labeling multimodal samples through an interactive Graphical User Interface (GUI). Extensive experiments are designed and implemented, whose results prove that LFMF-TC could significantly improve the classification accuracy. In theory, the LFMF-TC could be used for other applications with few modifications.

Highlights

Maps enable people to intuitively sense geospatial entities’ morphology, distribution and spatial relationships by visualizing them using some creative efforts, which could serve for people’s daily lives, space analysis and management
In the Web Map Service (WMS), a map layer consists of metadata, map and legend, and each of them contains some information related to the map theme
As the deep learning (DL) develops, especially the convolutional neural network (CNN), some salient and hierarchical visual features could be automatically learnt from raw images [8]–[10], which have motivated researchers to exploit them in variouse applications

Summary

Introduction

Maps enable people to intuitively sense geospatial entities’ morphology, distribution and spatial relationships by visualizing them using some creative efforts (e.g. symbolization, generalization), which could serve for people’s daily lives, space analysis and management. Features extraction plays a key role in the theme classification. In the WMS, a map layer consists of metadata, map and legend, and each of them contains some information related to the map theme. Most studies have exploited in extracting textual features from the metadata to describe maps using some natural language process (NLP) techniques [2], [3]. Few papers tried to understand maps using maps or legends due to the semantic gap between low-level visual features and themes. As the deep learning (DL) develops, especially the convolutional neural network (CNN), some salient and hierarchical visual features could be automatically learnt from raw images [8]–[10], which have motivated researchers to exploit them in variouse applications.

Methods

Findings

Discussion

Conclusion