A Middleware Framework for Scalable Management of Linked Streams

Abstract

The Web has long exceeded its original purpose of a distributed hypertext system and has become a global, data sharing and processing platform. This development is confirmed by remarkable milestones such as the Semantic Web, Web services, social networks and mashups. In parallel with these developments on the Web, the Internet of Things (IoT), i.e., sensors and actuators, has matured and has become a major scientific and economic driver. Its potential impact cannot be overestimated – for example, in logistics, cities, electricity grids and in our daily life, in the form of sensor-laden mobile phones – and rival that of the Web itself. While the Web provides ease of use of distributed resources and a sophisticated development and deployment infrastructure, the IoT excels in bringing real-time information from the physical world into the picture. Thus a combination of these players seems to be the natural next step in the development of even more sophisticated systems of systems. While only starting, there is already a significant amount of sensor-generated, or more generally dynamic information, available on the Web. However, this information is not easy to access and process, depends on specialised gateways and requires significant knowledge on the concrete deployments, for example, resource constraints and access protocols. To remedy these problems and draw on the advantages of both sides, we try to make dynamic, online sensor data of any form as easily accessible as resources and data on the Web, by applying well-established Web principles, access and processing methods, thus shielding users and developers from the underlying complexities. In this paper we describe our Linked StreamMiddleware, which makes it easy to integrate time-dependent data with other Linked Data sources, by enriching both sensor sources and sensor data streams with semantic descriptions, and enabling complex SPARQL-like queries across both dataset types through a novel query processing engine, along with means to mashup the data and process results. Most prominently, LSM provides (1) extensible means for real-time data collection and publishing using a cloud-based infrastructure, (2) a Web interface for data annotation and visualisation, and (3) a SPARQL endpoint for querying unified Linked Stream Data and Linked Data. We describe the system architecture behind LSM, provide details how Linked Stream Data is generated, and demonstrate the benefits and efficiency of the platform by showcasing some experimental evaluations and the system’s interface.