Moriarty: Improving ‘Time To Market’ in big data and Artificial intelligence applications

Abstract

The objective of this paper is to present the Moriarty framework and show one use case of the recommendation of entertainment events. Moriarty is a tool that can generate Big Data near real-time analytics solutions (Streaming Analytics). This new tool makes possible the collaboration among the data scientist and the software engineer. Through Moriarty, they join forces for the rapid generation of new software solutions. The data scientist works with algorithms and data transformations using a visual interface, while the software engineer works with the idea of services to be invoked. The underlying idea is that a user can build projects of Artificial Intelligence and Data Analytics without having to make any line of code. The main power of the tool is to reduce the ‘time to market’ in an application which embeds complex algorithms of Artificial Intelligence. It is based on different Artificial Intelligence algorithms (like Deep Learning, Natural Language Processing and Semantic Web) and Big Datamodules (Spark as a distributed data engine and access to NoSQL databases). Moriarty is divided into several layers; its core is a BPMN engine, which executes the processing and defines data analytics process, called workflows. Each workflow is defined by the standard BPMN model and is linked to a set of reusable functions or Artificial Intelligence algorithms written following a service-oriented architecture. An example of service presented is a recommendation application of restaurants, concerts, entertainment and events in general, where information is collected from social networks and websites, is processed by Natural Language Processingalgorithms and finally introduced into a graph database.