Parallel t-SNE Applied to Data Visualization in Smart Cities

Maximiliano Araujo Da Silva Lopes,Allan De Medeiros Martins,Adriao D Doria Neto

doi:10.1109/access.2020.2964413

Maximiliano Araujo Da Silva Lopes, Allan De Medeiros Martins + Show 1 more

Open Access

https://doi.org/10.1109/access.2020.2964413

Copy DOI

Abstract

The growth of smart city applications is increasingly around the world, many cities invest in the development of these systems intending to improve the management and life of their residents. This increase is mainly due to the emergence of new technologies such as Big Data and Internet of Things (IoT). Some of the biggest challenges in applying these systems, relate to the processing, visualization, and analysis of the generated data, since most systems tend to work connected, thus generating a large mass of data that deviates from the standard of previously used systems. For data visualization, one of the main devices used is the reduction of dimensionality, in an attempt to bring data from one dimension N to two or three dimensions and thus be noticeable to human eyes. There are several algorithms used for dimensionality reduction, the linear ones that as the name implies, solve linearly separable problems and so these are very limited and the nonlinear ones, that solve more complex problems, but usually have an excessive runtime, making them or often inappropriate to apply. This article presents the parallel implementation of the nonlinear dimension reduction algorithm t-Distributed Stochastic Neighbor Embedding (t-SNE), showing better results than its conventional version in terms of runtime, thus showing that parallelism can make the problem of dimensionality reduction treatable, bringing greater scalability and delivering results within an acceptable time frame.

Highlights

The change in computational paradigms from traditional desktop work at office desks and moving to distributed computing with multiple devices spread across an environment, all communicating with each other, increases the computational possibilities of the systems
The data of police occurrences, when well analyzed, can optimize the management, organization and governance of cities, contributing to the reduction of violence and improving the quality of life of the population, fitting the framework presented by Chourabi et al [13], which defines the involvement of a city with the concept of smart city
By checking the results presented by the execution of the algorithms, it can be observed that there was an improvement in the processing times of the t-Distributed Stochastic Neighbor Embedding (t-Stochastic Neighbor Embedding (SNE)) algorithm when the parallelization process was applied to it, and this improvement continued to exist with the increase in the number of processors. two to four, but with a lower significance

Summary

Introduction

The change in computational paradigms from traditional desktop work at office desks and moving to distributed computing with multiple devices spread across an environment, all communicating with each other, increases the computational possibilities of the systems. Smart environments soon began to emerge, such as homes in Khan et al [1] and Rehman & Gruhn [2] and doctor’s offices in Abatal et al [3] and Alamri [4]. The expansion of this concept to meet the demands of a city was a natural transition where an investment of 80 billion dollars was expected for 2018 worldwide, with an increase in this investment to about $ 135 billion in 2021 [5].

Objectives

Results

Conclusion