Big Data Management Challenges in a Meteorological Organisation

Abstract

Data management practices strongly impact enterprise performance, especially for e-science organisations dealing with big data. This study identifies the key challenges and issues facing information system managers in growing demand for big data operations to deliver timely meteorological products. Data was collected from in-depth interviews with five MetService information system managers, including the CIO. Secondary data sources include internal documents and relevant literatures. The study revealed the pressing and challenging big data management issues can broadly be classified as data governance, infrastructure management, and workflow management. The study identifies a gap in adopting effective workflow management system and coordinated outsourcing plan within the organisation. Although the study is limited by its sample size and generalisation, the findings are useful for other IT managers and practitioners of data-intensive organisations to examine their data management practices on the need to balance the demand for efficient scientific operations and sustainable business growth. This study recognised that although the organisation is implementing up-to-date and practical solutions to meet these challenges, effort is needed to harmonise and align these solutions with business growth strategies to sustain future growth. This study enhanced societies’ understanding to the current practices of a real world organization.