Odyssey

Abstract

We present a data analytics system, Odyssey, that is being developed at NEC Labs in collaboration with NEC’s commercial business units and our academic collaborators. The design principles of the system are based on the business requirements identified through extensive surveys and communications with the practitioners and customers. Most notable high-level requirements are: 1) The analytics system should be able to effectively use both structured and unstructured data sources. 2) Business requirements are not captured in a single simple metric but combination of metrics, such as value of data, performance, monetary costs, and they are dynamic. The system should manage data by observing constantly changing metric values. 3) Time-to-insight is very important, so the system should enable immediate exploratory querying of data without heavy prerequisite processes. 4) The system should efficiently support both ad-hoc queries and application workloads. 5) Very often there are already data analytics solutions / products in place (such as a traditional data warehouse), hence the system should be able to incorporate the existing settings. The data analysis is performed in a rapidly changing way and includes the new role of data scientist who is tasked with finding the benefit in big data, which come from disparate sources. The trend of collecting ever-growing data with unknown and unproven benefits, and the nature of the exploratory queries that are posed on these datasets represent an emerging type of data analysis. The fluid nature of this analysis is that the analyst may start by posing simple questions on the data but then evolve towards more sophisticated reasoning as well as apply sophisticated techniques. The evolutionary nature of the investigation is due to the analyst who may not initially be able to express her goals well, thus modifying her workflow slightly and iteratively refining it until achieving the intent. The evolutionary process may also require incorporating more data sources into the analysis to obtain richer and more confident answers. We call this iterative process by which an analyst finds bene-