Big Data Management Research Articles

Towards building a comprehensive big data management maturity framework

Towards building a comprehensive big data management maturity framework

Retracted: Construction of a Public Service Cloud Platform for Disabled People Based on the Big Data Management Model of the Internet of Things.

[This retracts the article DOI: 10.1155/2022/3153454.].

УПРАВЛІННЯ ТА ІНТЕГРАЦІЯ ДАНИХ В УМОВАХ ЦИФРОВІЗАЦІЇ ЕКОНОМІЧНИХ ПРОЦЕСІВ: ВИКЛИКИ ТА ПЕРСПЕКТИВИ

Objective. The purpose of the article is to clarify the theoretical and methodological aspects, analyze data management methods in the context of digitalization of economic processes, and choose the priority method of integrating corporate information systems depending on the tasks to be solved in each case. Methods. The paper uses a set of data integration methods: application integration method (EAI), method of extracting data from external sources, transforming them in the appropriate structure and forming data warehouses (ETL); method of real-time integration of incomparable data types from different sources (EI). Results. The paper proves that data management includes the formation and analysis of data architecture, integration of the database management system; data security, identification, segregation and storage of data sources. Data integration refers to the process of combining data from different sources into a single, holistic system and aims to provide access to a complete, updated and easy-to-analyze data set. Data integration is especially important in the areas of e-commerce, logistics and supply chains, where it is necessary to combine data from different sources to optimize processes, in the field of business intelligence, where processing large amounts of data and combining them allows you to identify useful information and certain patterns. Integration of enterprise information systems is the process of combining several IS and individual applications into a single, holistic system that works together to achieve a common goal, aimed at increasing the efficiency of the company, reducing duplication of efforts and streamlining processes. The main functional components of a corporate information system are identified: Business Process Automation IS, Financial Management IS, Customer Relationship Management IS, Supply Chain Management IS, Human Resources Management IS, Business Intelligence IS, Communication IS, and Data Security and Protection IS. Within a corporate information system, several narrowly focused software products operate simultaneously, capable of successfully solving a certain range of tasks. At the same time, some of them may not involve interaction with other information systems. The main approaches to data integration include universal access to data and data warehouses. Universal access technologies allow for equal access to data from different information systems, including on the basis of the concept of data warehouses - a database containing data collected from databases of different information subsystems for further analysis and use. It is proved that the most holistic approach to the integration of information systems is integration at the level of business processes. As part of the integration of business processes, there is an integration of applications, data integration, and integration of people involved in this business process. The article substantiates the feasibility of using three methods of big data management and integration: integration of corporate applications, integration of corporate information, and software for obtaining, transforming, and downloading data. As a result of comparing integration methods and building a generalized scheme for integrating heterogeneous IS, a number of situations have been identified in which the use of a specific integration method is preferable or the only possible one. The scientific novelty of the study is to identify the problems of integrating big data and corporate information systems. Approaches to choosing a method for integrating data and applications based on a generalized scheme for integrating heterogeneous information systems are proposed. Practical significance. The results of the analysis allow optimizing the methods of data integration within a corporate information system. The principles of integration inherent in the considered methods are used to solve a wide range of tasks: from real-time integration to batch integration and application integration. Implementation of the proposed methods of big data integration will make information more transparent; obtain additional detailed information about the efficiency of production and technological equipment, which stimulates innovation and improves the quality of the final product; use more efficient, accurate analytics to minimize risks and identify problems in advance before catastrophic consequences; more effectively manage supply chains, forecast demand, carry out comprehensive business planning, organize cooperation

Design and Development of a Big Data Platform for Disease Burden Based on the Spark Engine.

This study attempts to build a big data platform for disease burden that can realize the deep coupling of artificial intelligence and public health. This is a highly open and shared intelligent platform, including big data collection, analysis, and result visualization. Based on data mining theory and technology, the current situation of multisource data on disease burden was analyzed. Putting forward the disease burden big data management model, functional modules, and technical framework, Kafka technology is used to optimize the transmission efficiency of the underlying data. This will be an efficient and highly scalable data analysis platform through embedding embedded Sparkmlib in the Hadoop ecosystem. With the concept of "Internet + medical integration," the overall architecture design of the big data platform for disease burden management was proposed based on the Spark engine and Python language. The main system composition and application scenarios are given at four levels: multisource data collection, data processing, data analysis, and the application layer, according to application scenarios and use requirements. The big data platform of disease burden management helps to promote the multisource convergence of disease burden data and provides a new path for the standardized paradigm of disease burden measurement. Provide methods and ideas for the deep integration of medical big data and the formation of a broader standard paradigm.

Discussion on the Reform of Ideological and Political Teaching in Major Courses of Big Data Management— Taking the Operations Research Course as an Example

Discussion on the Reform of Ideological and Political Teaching in Major Courses of Big Data Management— Taking the Operations Research Course as an Example

Towards building a comprehensive big data management maturity framework

Towards building a comprehensive big data management maturity framework

DATA MANAGEMENT AND INTEGRATION IN THE CONTEXT OF DIGITALIZATION OF ECONOMIC PROCESSES: CHALLENGES AND PROSPECTS

Objective. The purpose of the article is to clarify the theoretical and methodological aspects, analyze data management methods in the context of digitalization of economic processes, and choose the priority method of integrating corporate information systems depending on the tasks to be solved in each case. Methods. The paper uses a set of data integration methods: application integration method (EAI), method of extracting data from external sources, transforming them in the appropriate structure and forming data warehouses (ETL); method of real-time integration of incomparable data types from different sources (EI). Results. The paper proves that data management includes the formation and analysis of data architecture, integration of the database management system; data security, identification, segregation and storage of data sources. Data integration refers to the process of combining data from different sources into a single, holistic system and aims to provide access to a complete, updated and easy-to-analyze data set. Data integration is especially important in the areas of e-commerce, logistics and supply chains, where it is necessary to combine data from different sources to optimize processes, in the field of business intelligence, where processing large amounts of data and combining them allows you to identify useful information and certain patterns. Integration of enterprise information systems is the process of combining several IS and individual applications into a single, holistic system that works together to achieve a common goal, aimed at increasing the efficiency of the company, reducing duplication of efforts and streamlining processes. The main functional components of a corporate information system are identified: Business Process Automation IS, Financial Management IS, Customer Relationship Management IS, Supply Chain Management IS, Human Resources Management IS, Business Intelligence IS, Communication IS, and Data Security and Protection IS. Within a corporate information system, several narrowly focused software products operate simultaneously, capable of successfully solving a certain range of tasks. At the same time, some of them may not involve interaction with other information systems. The main approaches to data integration include universal access to data and data warehouses. Universal access technologies allow for equal access to data from different information systems, including on the basis of the concept of data warehouses - a database containing data collected from databases of different information subsystems for further analysis and use. It is proved that the most holistic approach to the integration of information systems is integration at the level of business processes. As part of the integration of business processes, there is an integration of applications, data integration, and integration of people involved in this business process. The article substantiates the feasibility of using three methods of big data management and integration: integration of corporate applications, integration of corporate information, and software for obtaining, transforming, and downloading data. As a result of comparing integration methods and building a generalized scheme for integrating heterogeneous IS, a number of situations have been identified in which the use of a specific integration method is preferable or the only possible one. The scientific novelty of the study is to identify the problems of integrating big data and corporate information systems. Approaches to choosing a method for integrating data and applications based on a generalized scheme for integrating heterogeneous information systems are proposed. Practical significance. The results of the analysis allow optimizing the methods of data integration within a corporate information system. The principles of integration inherent in the considered methods are used to solve a wide range of tasks: from real-time integration to batch integration and application integration. Implementation of the proposed methods of big data integration will make information more transparent; obtain additional detailed information about the efficiency of production and technological equipment, which stimulates innovation and improves the quality of the final product; use more efficient, accurate analytics to minimize risks and identify problems in advance before catastrophic consequences; more effectively manage supply chains, forecast demand, carry out comprehensive business planning, organize cooperation

Vertical Pod Autoscaling in Kubernetes for Elastic Container Collaborative燜ramework

Kubernetes is an open-source container management tool which automates container deployment, container load balancing and container(de)scaling, including Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA). HPA enables flawless operation, interactively scaling the number of resource units, or pods, without downtime. Default Resource Metrics, such as CPU and memory use of host machines and pods, are monitored by Kubernetes. Cloud Computing has emerged as a platform for individuals beside the corporate sector. It provides cost-effective infrastructure, platform and software services in a shared environment. On the other hand, the emergence of industry 4.0 brought new challenges for the adaptability and infusion of cloud computing. As the global work environment is adapting constituents of industry 4.0 in terms of robotics, artificial intelligence and IoT devices, it is becoming eminent that one emerging challenge is collaborative schematics. Provision of such autonomous mechanism that can develop, manage and operationalize digital resources like CoBots to perform tasks in a distributed and collaborative cloud environment for optimized utilization of resources, ensuring schedule completion. Collaborative schematics are also linked with Bigdata management produced by large scale industry 4.0 setups. Different use cases and simulation results showed a significant improvement in Pod CPU utilization, latency, and throughput over Kubernetes environment.

Paradigm Shift of Statistical Big Data in Healthcare: Management, Analysis and Future Prospects

‘Big data’ is vast describing volumes of information that can work miracles. Multiple attempts designed at highlighting the relationship between big data analytics and benefits for healthcare organizations have been raised. The big data influence on health organization management is still not clear due to the relationship’s multi-disciplinary nature. It has become a topic of special interest for the past two decades because of a great potential that is hidden in it. Numerous public and private sector industries generate, store, and analyze big data with an aim to improve the services they provide. In the healthcare industry, various sources for big data include hospital records, medical records of patients, results of medical checkups, and devices that are a part of internet of things. Biomedical research also produces a significant share of big data pertinent to public healthcare. Big Data and healthcare are precarious to discourse the risk of hospitalization. This data requires proper management and analysis in order to derive expressive communication. There are various encounters associated with each step of handling big data which can only be surpassed by using high-end computing solutions for big data analysis. Providing relevant results for improving public health, healthcare providers are required to be fully equipped with appropriate infrastructure to systematically generate and analyze big data. A well-organized management, analysis, and interpretation of big data can transformation the game by opening new avenues for contemporary healthcare. With a strong integration of biomedical and healthcare data, modern healthcare institutes can probably revolutionize the medical therapies and personalized medicine.

Evolutionary computation-based machine learning for Smart City high-dimensional Big Data Analytics

Science and technology development promotes Smart City Construction (SCC) as a most imminent problem. This work aims to improve the comprehensive performance of the Smart City-oriented high-dimensional Big Data Management (BDM) platform and promote the far-reaching development of SCC. It comprehensively optimizes the calculation process of the BDM platform through Machine Learning (ML), reduces the dimension of the data, and improves the calculation effect. To this end, this work first introduces the concept of SCC and the BDM platform application design. Then, it discusses the design concept of using ML technology to optimize the calculation effect of the BDM platform. Finally, the Tensor Train Support Vector Machine (TT-SVM) model is designed based on dimension reduction data processing. The proposed model can comprehensively optimize the BDM platform, and the model is compared with other models and evaluated. The research results show that the accuracy of the reduced dimension classification of the TT-SVM model is more than 95. The lowest average processing time for the model’s reduced dimension classification is about 1ms. The model’s highest data processing accuracy is about 98%, and the average processing time is between 1.0–1.5ms. Compared with traditional models and BDM platforms, the proposed model has a breakthrough performance improvement, so it plays an important role in future SCC. This work has achieved a great breakthrough in big data processing, and innovatively improved the application mode of high-dimensional big data technology by integrating multiple technologies. Therefore, the finding provides targeted technical reference for algorithms in BDM platform and contributes to the construction and improvement of Smart City.

“Challenges Of Big Data Management”

“Challenges Of Big Data Management”

Remote Big Data Management Tools, Sensing and Computing Technologies, and Visual Perception and Environment Mapping Algorithms in the Internet of Robotic Things

The purpose of our systematic review was to inspect the recently published research on Internet of Robotic Things (IoRT) and harmonize the assimilations it articulates on remote big data management tools, sensing and computing technologies, and visual perception and environment mapping algorithms. The research problems were whether robotic manufacturing processes and industrial wireless sensor networks shape IoRT and lead to improved product quality by use of remote big data management tools, whether IoRT devices communicate autonomously regarding event modeling and forecasting by leveraging machine learning and clustering algorithms, sensing and computing technologies, and image processing tools, and whether smart connected objects, situational awareness algorithms, and edge computing technologies configure IoRT systems and cloud robotics in relation to distributed task coordination through visual perception and environment mapping algorithms. A Shiny app was harnessed for Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines to configure the flow diagram integrating evidence-based gathered and processed data (the search outcomes and screening procedures). A quantitative literature review of ProQuest, Scopus, and the Web of Science databases was carried out throughout June and October 2022, with search terms including “Internet of Robotic Things” + “remote big data management tools”, “sensing and computing technologies”, and “visual perception and environment mapping algorithms”. Artificial intelligence and intelligent workflows by use of AMSTAR (Assessing the Methodological Quality of Systematic Reviews), Dedoose, DistillerSR, and SRDR (Systematic Review Data Repository) have been deployed as data extraction tools for literature collection, screening, and evaluation, for document flow monitoring, for inspecting qualitative and mixed methods research, and for establishing robust outcomes and correlations. For bibliometric mapping by use of data visualization, Dimensions AI was leveraged and with regards to layout algorithms, VOSviewer was harnessed.

The role of big data management in improving the decision-making of banking organizations (Case study of Sepah Bank)

در سالهاي اخیر، اندازه داده‌ها در جهان به طور چشم‌گیري به صورت نمایی به چندین برابر افزایش پیدا کرده است. بانک‌ها نیز در فرایندهای خود مقادیر زیادی داده ایجاد و گاه هزینه‌های گزافی برای جمع‌آوری و نگهداری آن‌ها صرف می‌کنند. برخی از کارشناسان صنعت بانکی رشد هفت برابری حجم موجود داده‌ها را تا سال ۲۰۲۰ برآورد می‌کنند. امروزه فناوری کلان‌داده به ‌عنوان راه‌حلی برای بهره‌برداری و استفاده این حجم از اطلاعات مطرح است. اما بررسی و پردازش داده‌های بزرگ و نیز بررسی اثربخشی به کارگیری آن در حوزه بانکداری به عنوان یک چالش تلقی می‌گردد. از این رو، در این تحقیق به بررسی نقش مدیریت کلان داده در بهبود تصمیم گیری سازمان های بانکی (مطالعه موردی بانک سپه) پرداخته شد. بدین منظور جامعه آماری این تحقیق شامل 130 نفر از کلیه کارشناسان بخش اداره فناوری اطلاعات بانک سپه میباشد که به دلیل محدودیت جامعه آماری نمونه‌گیری صورت نگرفت. پرسشنامه‌های استاندارد مدیریت کلان داده 20 گویه‌ای، پرسشنامه توانمندسازی بهبود تصمیم‌گیری 22 گویه‌ای و پرسشنامه کیفیت تصمیم‌گیری10 گویه‌ای با استناد بر تحقیقات شمیم و همکاران (2019)، بعد از بومی‌سازی مبنای تحقیق قرار گرفت. نتایج توصیفی و استنباطی داده‌های پژوهش با استفاده از نرم‌افزار 19SPSS وPLS ، مورد بررسی و تحلیل قرار گرفت. نتایج تحقیق، حاکی از آن بوده است که فرهنگ سازمانی با ضریب 0.446 بیشترین رابطه مثبت و معناداری را بر توانمندسازی دارد. همچنین توانمندسازی رابطه مثبت و معنادار با ضریب 0.645 با اثربخشی تصمیم گیری و ضریب 0.884 با کارایی تصمیم‌گیری دارد.

Global output on artificial intelligence in the field of nursing: A bibliometric analysis and science mapping.

To analyze the AI research in the field of nursing, to explore the current situation, hot topics, and prospects of AI research in the field of nursing, and to provide a reference for researchers to carry out related studies. We used the VOSviewer 1.6.17, SciMAT, and CiteSpace 5.8.R3 to generate visual cooperation network maps for the country, organizations, authors, citations, and keywords and perform burst detection, theme evolution, and so forth. A total of 9318 articles were obtained from the Web of Science Core Collection database. Four hundred and thirty-one AI research related to the field of nursing was published by 855 institutions from 54 countries. CIN-Computers Informatics Nursing was the top productive journal. The United States was the dominant country. The transnational cooperation between authors from developed countries was closer than that between authors from developing countries. The main hot topics included nurse rostering, nursing diagnosis, nursing decision support, disease risk factor prediction, nursing big data management, expert system, support vector machine, decision tree, deep learning, natural language processing, and nursing education. Machine learning represented one of the cutting-edge and most applicable branches of artificial intelligence in the field of nursing, and deep learning was the hottest technology among many machine learning methods in recent years. One of the most cited papers was published by Burke in 2004 and cited 500 times, which critically evaluated AI methods to deal with nurse scheduling problems. Although AI has been paid more and more attention to the field of nursing, there is still a lack of high-yielding authors who have been engaged in this field for a long time. Most of the high contribution authors and institutions came from developed countries; therefore, more transnational and multi-disciplinary cooperation is needed to promote the development of AI in the nursing field. This bibliometric analysis not only provided a comprehensive overview to help researchers to understand the important articles, journals, potential collaborators, and institutions in this field but also analyzed the history, hot spots, and future trends of the research topic to provide inspiration for researchers to choose research directions.

The Concept of Big Data Management with Various Transportation Systems Sources as a Key Role in Smart Cities Development

An increasing number of devices and their communication with each other generates huge amounts of data. The efficiency of processing such large and heterogeneous data is crucial for extracting the reliable and consistent information that is needed for the effective management of smart cities within the field of transport. Data heterogeneity and volume as well as its integration and analytics are big challenges for decision-makers. The development of urban agglomerations is largely dependent on the proper management of such data. Therefore, this paper explores the role of these data repositories, their acquisition from different sources, and the ways to combine them. The main goal of this paper is to propose a concept of Smart City management based on Big Data Analytics and technology related to UAVs (Unmanned Aerial Vehicle) which may reduce costs and resource consumption. The presented concept includes successive data generation and collection, data type identification, problem and requirement identification, filtering, classification, pre-processing, and data optimization, as well as decision support analysis. A key part of this analysis utilizes computer algorithms, such as Speeded Up Robust Features (SURF) and Thresholding and Blob detection, to develop a multi-camera image recognition system for freight transport management and logistics in smart cities. The objective is to design a system that optimizes the route planning and time of vehicle passage on selected road sections, ultimately leading to the reduction of emissions. During the study, data obtained from multiple sources were compared, and the analysis uncovered different results for the same assumptions. We discuss the reasons for these variances. Overall, the results obtained in the analysis indicated that it is necessary to correct the predictions of the multi-camera image recognition system with additional methods and algorithms.

INSPIRE PROJECT: INTEGRATED TECHNOLOGIES FOR SMART BUILDINGS AND PREDICTIVE MAINTENANCE

Abstract. Applying integrated digital technologies for the management and maintenance of the existing built heritage appears to be one of the main current challenges for the definition and application of digitisation protocols for the construction supply chain. Key enabling technologies, collaborative platforms, Big Data management and information integration in a BIM environment are areas of increasing experimentation. In the field of intervention on the built heritage, it is the boundaries and opportunities offered by the integration of many different information sources that constitutes the main challenge. Furthermore, the study of the accessibility and usability of data and information from sources such as the three-dimensional terrestrial survey, existing databases, sensor networks, and satellite technologies make it possible to investigate both different ways of data modelling, even with a view to the development of predictive algorithms, and of visualisation and information management. The study illustrates part of the results of the InSPiRE project, an industrial research project financed with European structural funds and carried out in a public-private partnership by four universities and public research bodies, an innovation centre and six companies, SMEs, large enterprises, and start-ups. Specifically, the project highlights the growing importance of BIM-based modelling as a tool to lead users, both experts and non-experts, through the multiple information paths resulting from the relation between data and metadata.

Real Time Feature Influence Analysis Based ETL Framework for Efficient Big Data Management

The increased volume of big data introduces various challenges for its maintenance and analysis. There exist various approaches to the problem, but they fail to achieve the expected results. To improve the big data management performance, an efficient real time feature influence analysis based Extraction, Transform, and Loading (ETL) framework is presented in this article. The model fetches the big data and analyses the features to find noisy records by preprocessing the data set. Further, the method performs feature extraction and applies feature influence analysis to various data nodes and the data present in the data nodes. The method estimates Feature Specific Informative Influence (FSII) and Feature Specific Supportive Influence (FSSI). The value of FSII and FSSI are measured with the support of a data dictionary. The class ontology belongs to various classes of data. The value of FSII is measured according to the presence of a concrete feature on a tuple towards any data node, whereas the value of FSSI is measured based on the appearance of supportive features on any data point towards the data node. Using these measures, the method computes the Node Centric Transformation Score (NCTS). Based on the value of NCTS the method performs map reduction and merging of data nodes. The NCTS_FIA method achieves higher performance in the ETL process. By adapting feature influence analysis in big data management, the ETL performance is improved with the least amount of time complexity.

Smart aquaponics (SMARTAP project): small/ medium farm economic improvement model, environmental sustainability, monitoring and simplified management of big data

ISHS XXXI International Horticultural Congress (IHC2022): International Symposium on Urban Horticulture for Sustainable Food Security (UrbanFood2022) Smart aquaponics (SMARTAP project): small/ medium farm economic improvement model, environmental sustainability, monitoring and simplified management of big data

$\tau\text{JOWL}$: A Systematic Approach to Build and Evolve a Temporal OWL 2 Ontology Based on Temporal JSON Big Data

Nowadays, ontologies, which are defined under the OWL 2 Web Ontology Language (OWL 2), are being used in several fields like artificial intelligence, knowledge engineering, and Semantic Web environments to access data, answer queries, or infer new knowledge. In particular, ontologies can be used to model the semantics of big data as an enabling factor for the deployment of intelligent analytics. Big data are being widely stored and exchanged in JavaScript Object Notation (JSON) format, in particular by Web applications. However, JSON data collections lack explicit semantics as they are in general schema-less, which does not allow to efficiently leverage the benefits of big data. Furthermore, several applications require bookkeeping of the entire history of big data changes, for which no support is provided by mainstream Big Data management systems, including Not only SQL (NoSQL) database systems. In this paper, we propose an approach, named <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\tau \text{JOWL}$</tex> (temporal OWL 2 from temporal JSON), which allows users (i) to automatically build a temporal OWL 2 ontology of data, following the Closed World Assumption (CWA), from temporal JSON-based big data, and (ii) to manage its incremental maintenance accommodating the evolution of these data, in a temporal and multi-schema environment.

Service Strategy Analysis of Big Data Management in Small and Medium-sized Enterprises

As the basic force to promote the development of national economy, small and medium-sized enterprises face many difficulties in their development process. And big data technology can help them improve the efficiency of enterprise management and enhance the viability of enterprises. The rise of the Internet, relying on big data technology, effectively alleviates the financing problems of SMEs. At the same time, it has a great impact on the traditional credit business of commercial banks. The legal service platform for small and medium-sized enterprises, with the development of emerging information technologies such as Internet, big data and sharing economy, has unique service advantages, which can achieve economies of scale and scope in the process of legal services, and has important practical significance for the healthy and rapid development of small and medium-sized enterprises. However, the innovation and development of small and medium-sized technology-based enterprises are difficult to achieve only by themselves, and they need to be provided with targeted and professional knowledge services by internal knowledge service departments, intermediary service agencies, and government departments, so as to improve the knowledge reserves of small and medium-sized technology-based enterprises and promote their development and innovation. This paper analyzes the relationship between SMEs and big data technology, puts forward problems in terms of awareness, technology, policy and talent, and gives corresponding strategies to help SMEs develop.