Table2Vec-automated universal representation learning of enterprise data DNA for benchmarkable and explainable enterprise data science

Longbing Cao,Chengzhang Zhu

doi:10.1038/s41598-021-03443-0

Abstract

Enterprise data typically involves multiple heterogeneous data sources and external data that respectively record business activities, transactions, customer demographics, status, behaviors, interactions and communications with the enterprise, and the consumption and feedback of its products, services, production, marketing, operations, and management, etc. They involve enterprise DNA associated with domain-oriented transactions and master data, informational and operational metadata, and relevant external data. A critical challenge in enterprise data science is to enable an effective ‘whole-of-enterprise’ data understanding and data-driven discovery and decision-making on all-round enterprise DNA. Accordingly, here we introduce a neural encoder Table2Vec for automated universal representation learning of entities such as customers from all-round enterprise DNA with automated data characteristics analysis and data quality augmentation. The learned universal representations serve as representative and benchmarkable enterprise data genomes (similar to biological genomes and DNA in organisms) and can be used for enterprise-wide and domain-specific learning tasks. Table2Vec integrates automated universal representation learning on low-quality enterprise data and downstream learning tasks. Such automated universal enterprise representation and learning cannot be addressed by existing enterprise data warehouses (EDWs), business intelligence and corporate analytics systems, where ‘enterprise big tables’ are constructed with reporting and analytics conducted by specific analysts on respective domain subjects and goals. It addresses critical limitations and gaps of existing representation learning, enterprise analytics and cloud analytics, which are analytical subject, task and data-specific, creating analytical silos in an enterprise. We illustrate Table2Vec in characterizing all-round customer data DNA in an enterprise on complex heterogeneous multi-relational big tables to build universal customer vector representations. The learned universal representation of each customer is all-round, representative and benchmarkable to support both enterprise-wide and domain-specific learning goals and tasks in enterprise data science. Table2Vec significantly outperforms the existing shallow, boosting and deep learning methods typically used for enterprise analytics. We further discuss the research opportunities, directions and applications of automated universal enterprise representation and learning and the learned enterprise data DNA for automated, all-purpose, whole-of-enterprise and ethical machine learning and data science.

Highlights

Data science plays a pivotal scientific role in transforming enterprise innovation and thinking[1,2,3]
These baselines are chosen because (1) they are mostly used for enterprise data science; (2) ensembles like random forests, GBoost and XGBoost are typically applied for improving performance; (3) they cover different learning mechanisms including tree models, ensembles, distance learning, probabilistic learning, and deep learning; and (4) a state-of-the-art deep neural network (DNN) is compared
The data is complex as it consists of heterogeneous features[41], some of which are dynamic, all of them are sparse with a significant proportion of features having missing values, and the labels provided by the bank are binary without semantic differentiation

Summary

Introduction

Data science plays a pivotal scientific role in transforming enterprise innovation and thinking[1,2,3]. Of interest and tasks, e.g., related to customers, products, services, productivity, marketing, competition, new product promotion, and interactions with customers This subject-oriented enterprise analytics has been on the standard agenda of many enterprises for their data strategies and enterprise data science, becoming increasingly adopted to achieve better business performance, transformation, and decision-making. The existing approaches and results only capture partial enterprise intelligence, domain intelligence, human intelligence, and data intelligence[1], serve partial purposes and do not address enterprise-wide goals and challenges well This scenario applies to related areas including cloud analytics and distributed and federated learning, where analytical silos remain on specific learning objectives, businesses, tasks and data by customized m odels[8]. We discuss the gaps in the related work on learning from enterprise data for ‘whole-of-enterprise’ learning objectives and tasks

Methods

Results

Discussion

Conclusion