Abstract
The realm of big data has brought new venues for knowledge acquisition, but also major challenges including data interoperability and effective management. The great volume of miscellaneous data renders the generation of new knowledge a complex data analysis process. Presently, big data technologies provide multiple solutions and tools towards the semantic analysis of heterogeneous data, including their accessibility and reusability. However, in addition to learning from data, we are faced with the issue of data storage and management in a cost-effective and reliable manner. This is the core topic of this paper. A data lake, inspired by the natural lake, is a centralized data repository that stores all kinds of data in any format and structure. This allows any type of data to be ingested into the data lake without any restriction or normalization. This could lead to a critical problem known as data swamp, which can contain invalid or incoherent data that adds no values for further knowledge acquisition. To deal with the potential avalanche of data, some legislation is required to turn such heterogeneous datasets into manageable data. In this article, we address this problem and propose some solutions concerning innovative methods, derived from a multidisciplinary science perspective to manage data lake. The proposed methods imitate the supply chain management and natural lake principles with an emphasis on the importance of the data life cycle, to implement responsible data governance for the data lake.
Highlights
With the realm of big data as a source of new knowledge extraction through data analysis and mining techniques, machine learning, correlation, and cluster analysis techniques, data heterogeneity and interoperability are common challenges
Our approach is based on a comparison of the dynamics, life cycles, and operations within those two systems with those needed for data lakes. We show that such perspectives provide paradigms for optimizing data lake performance, and we describe some methods for sustainable data governance
We have proposed some multidisciplinary approaches, which are natural manner and systematic manner, for data governance in data lake and argued that supply chain strategies and natural principals could be the effective sources of inspiration for data governance in order to assess the life cycle of data from the moment they enter the data lake until they are destroyed
Summary
With the realm of big data as a source of new knowledge extraction through data analysis and mining techniques, machine learning, correlation, and cluster analysis techniques, data heterogeneity and interoperability are common challenges. Some data governance policies or regulation methodologies have been extracted from systematic approaches or natural mechanisms to preserve and destroy data throughout their life cycle This viewpoint provides enormous capabilities to govern a data lake effectively. A strategy frequently used in the supply chain such as “agile management”, will improve the responsibility and flexibility of the data lake with regard to user requirements with high quality of service even in critical situations [23] Those two frameworks can be viewed as effective paradigms for managing the data life cycle, and its governance to ensure its viability. We present a general analogy and comparison between supply chain management, natural lakes, and data lakes and identify similar aspects and components Based on those similarities we propose new methodologies to improve data lake’s validity
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
