Massive Data Management Research Articles

IEEE Access Special Section Editorial: Urban Computing and Intelligence

Urban computing utilizes unobtrusive and ubiquitous sensing technologies, advanced data management and analytics models, and novel visualization methods to create win-win-win solutions which intelligently improve people’s lives, urban environments, and city operation systems. With the help of cloud computing, the Internet of Things, device-to-device (D2D) communication, artificial intelligence (AI), big data, and urban computing and intelligence will bridge the gap of ubiquitous sensing, intelligent computing, cooperative communication, and mass data management technologies, to create novel solutions that improve urban environments, human life quality, and smart city systems. Thus, urban computing and intelligence has recently attracted significant attention from industry and academia for building smart cities.

Las perspectivas de estudiantes universitarios sobre el Big Data y su papel en el sistema educativo

Hablar hoy de Big Data en educación es subrayar todas las potencialidades del manejo masivo de datos para la mejora de la calidad educativa, aunque de forma algo tardía. En este artículo se ofrecen los resultados más relevantes de un estudio realizado en un contexto universitario con estudiantes de la Universidad de Málaga. El objetivo del estudio ha sido conocer y comprender las percepciones, opiniones, conocimientos y actitudes relacionadas con el Big Data en el sistema educativo en una muestra de 104 estudiantes. La muestra del estudio fue no probabilística de carácter incidental. Para ello, se llevó a cabo un estudio de encuesta, utilizando la escala VABIDAE para recoger los datos. Se hizo un estudio psicométrico de la escala con los datos recogidos, así como un análisis estadístico descriptivo. Los resultados indican que los estudiantes tienen bastante desconocimiento sobre el Big Data y sus usos que se le puede dar en el campo educativo, aunque con una percepción positiva sobre su posible impacto. Ahora bien, se muestran críticos con los beneficios y perjuicios que puede llegar a tener en la calidad educativa.

Ontology-Based Decision Support Tool for Automatic Sleep Staging Using Dual-Channel EEG Data

Sleep staging has attracted significant attention as a critical step in auxiliary diagnosis of sleep disease. To avoid subjectivity of doctor’s manual sleep staging, and to realize scientific management of massive physiological data, an ontology-based decision support tool is proposed. The tool implements an automated procedure for sleep staging using dual-channel electroencephalogram (EEG) signals. First of all, it encodes EEG features, sleep-related concepts and other contextual information to “EEG-Sleep ontology”. Secondly, a rule-set is constructed based on a data mining technique. Finally, the first two steps are processed in a reasoning engine which is automatically assign each 30 s epoch (segment) sleep stage to one of five possible sleep stages: WA, NREM1, NREM2, SWS and REM. The rule set is obtained using EEG data taken from the Sleep-EDF database [EXPANDED] according to the random forest algorithm (RF), we prove that the performance of the proposed method with 89.12% accuracy, and 0.81 Kappa statistics is superior to other algorithms such as Bayesian network, C4.5, support vector machine, and multilayer perceptron. Additionally, our proposed approach improved performance when compared to other studies using a small subset of the Sleep-EDF database [EXPANDED].

Fog data management: A vision, challenges, and future directions

Cloud computing with its key facets and its inherent advantages still faces several challenges in the Internet of Things (IoT) ecosystem. The distance among the IoT end devices and cloud computing might be a problem for latency-sensitive applications such as catastrophe management and content transference applications. Fog computing is a novel paradigm to address such issues that playacts a significant role in massive and real-time data management systems in an IoT environment. Particularly IoT data management by fog computing is one important phase for latency reduction in latency-sensitive applications and necessary to generate more skilled knowledge and intelligent decisions. In this study, we used the SLR (systematic literature review) method to survey fog data management to understand the various topics and main contexts in this domain that have been newly offered. The target of this article is classifying and analyzing the researches about the fog data management domain which has been released from 2014 to 2019. A context-based taxonomy is offered for fog data management including data processing, data storage and data security based on the context of papers that are elected with the SLR method in our study. Based on presented technical taxonomy, the grouped papers in any context are compared with each other pursuant to some metrics of fog data management reference model. Then, for any selected research, the new findings, advantages, and weaknesses are debated. Finally, based on studies the open issues in fog data management and their related challenges for future researches are highlighted.

Enabling Massive XML-Based Biological Data Management in HBase.

Publishing biological data in XML formats is attractive for organizations who would like to provide their bioinformatics resources in an extensible and machine-readable format. In the era of big data, massive XML-based biological data management is emerged as a challengeable issue. With the continuous growth of the XML-based biological data sets, it is usually frustrating to use traditional declarative query languages to provide efficient query capabilities in terms of processing speed and scale. In this study, we report a novel platform to store and query massive XML-based biological data collections. A prototype tool for constructing HBase tables from XML-based biological data collections is first developed, and then a formal approach to transform the XML query model into the MapReduce query model is proposed. Finally, an evaluation of the query performance of the proposed approach on the existing XML-based biological databases is presented, showing that the performance advantages of the proposed solution. The source code of the massive XML-based biological data management platform is freely available at https://github.com/lyotvincent/X2H.

A Reinforcement Learning Based Data Storage and Traffic Management in Information-Centric Data Center Networks

Data Center Networks (DCN), a core infrastructure of cloud computing, place heavy demands on efficient storage and management of massive data. The data storage scheme, which decides how to assign data to nodes for storage, has a significant impact on the performance of the data center. However, most of the existing solutions focus on where to store the data (i.e., the selection of storage node) but have not considered how to store them (i.e., the traffic management such as routing and transmission rate adjustment). By leveraging the Information-Centric Networks (ICN) architecture, this paper tackles the data storage and traffic management issue in Information-Centric Data Center Networks (ICDCN) based on Reinforcement Learning (RL) method, since RL has been developed as a promising solution to address dynamic network issues. We present a global optimization of joint traffic management and data storage and then solve it by the distributed multi-agent Q-learning. In ICDCN, the data is routed based on the data’s name, which achieves better routing scalability by decoupling the data and its physical location. Compared with IP’s stateless forwarding plane, the stateful forwarding information maintained at every node supports adaptively routing and hop-by-hop traffic control by using the Q-learning method. We evaluate our proposal on an NS-3-based simulator, and the results show that the proposed scheme can effectively reduce transmission time and increase throughput while achieving load-balanced among servers.

Massive Data Management and Sharing Module for Connectome Reconstruction.

Recently, with the rapid development of electron microscopy (EM) technology and the increasing demand of neuron circuit reconstruction, the scale of reconstruction data grows significantly. This brings many challenges, one of which is how to effectively manage large-scale data so that researchers can mine valuable information. For this purpose, we developed a data management module equipped with two parts, a storage and retrieval module on the server-side and an image cache module on the client-side. On the server-side, Hadoop and HBase are introduced to resolve massive data storage and retrieval. The pyramid model is adopted to store electron microscope images, which represent multiresolution data of the image. A block storage method is proposed to store volume segmentation results. We design a spatial location-based retrieval method for fast obtaining images and segments by layers rapidly, which achieves a constant time complexity. On the client-side, a three-level image cache module is designed to reduce latency when acquiring data. Through theoretical analysis and practical tests, our tool shows excellent real-time performance when handling large-scale data. Additionally, the server-side can be used as a backend of other similar software or a public database to manage shared datasets, showing strong scalability.

A Spatiotemporal Trajectory Data Index Based on the Hilbert Curve Code

Massive trajectory data have been accumulated with the rapid development of global positioning technology and the popularisation of intelligent mobile terminal. However, the generation of massive data does not necessarily lead to the increase in effective data. An index method meeting the efficient management requirements of spatiotemporal trajectory data needs to be designed for the efficient spatiotemporal analysis and calculation of data. This paper proposes a spatiotemporal index method based on the Hilbert curve code to solve this problem. Firstly, the method constructs a multi-scale spatiotemporal grid model covering the whole world by dividing the three-dimensional space composed of time, latitude and longitude. Secondly, the grid cell codes are designed based on the Hilbert curve for hierarchical organisation of the trajectory data. Finally, the corresponding query process based on the code index is proposed in accordance with the different spatiotemporal query requirements of trajectory data. The comparison experiments show that the proposed method is more efficient than the existing spatiotemporal index method and can effectively support the management of massive multi-scale trajectory data.

SASC: Secure and Authentication-Based Sensor Cloud Architecture for Intelligent Internet of Things.

Nowadays, the integration of Wireless Sensor Networks (WSN) and the Internet of Things (IoT) provides a great concern for the research community for enabling advanced services. An IoT network may comprise a large number of heterogeneous smart devices for gathering and forwarding huge data. Such diverse networks raise several research questions, such as processing, storage, and management of massive data. Furthermore, IoT devices have restricted constraints and expose to a variety of malicious network attacks. This paper presents a Secure Sensor Cloud Architecture (SASC) for IoT applications to improve network scalability with efficient data processing and security. The proposed architecture comprises two main phases. Firstly, network nodes are grouped using unsupervised machine learning and exploit weighted-based centroid vectors for the development of intelligent systems. Secondly, the proposed architecture makes the use of sensor-cloud infrastructure for boundless storage and consistent service delivery. Furthermore, the sensor-cloud infrastructure is protected against malicious nodes by using a mathematically unbreakable one-time pad (OTP) encryption scheme to provide data security. To evaluate the performance of the proposed architecture, different simulation experiments are conducted using Network Simulator (NS3). It has been observed through experimental results that the proposed architecture outperforms other state-of-the-art approaches in terms of network lifetime, packet drop ratio, energy consumption, and transmission overhead.

Discussion on Application of BIM Technology in Aluminum Alloy Formwork Support System

With the development of science and technology, society has put forward more stringent requirements for the construction industry. The traditional two-dimensional design method can no longer meet the requirements of architectural design. The proposal of BIM technology and its application in the construction industry have solved the problems existing in traditional architectural design methods and adapted to the long-term development needs of architectural design. In the design of aluminum alloy formwork support system, BIM technology can realize efficient processing and management of massive data, realize informationization and integrated management of project design, and effectively improve the quality of project design. In view of this, the author believes that it is necessary to analyze the application of BIM technology in the design of aluminum alloy template support system. Based on the overview of this new technology, this paper will analyze and discuss the application and practical application of BIM technology in the design of aluminum alloy template support system.

A Security Transmission and Storage Solution about Sensing Image for Blockchain in the Internet of Things.

With the rapid development of the Internet of Things (IoT), the number of IoT devices has increased exponentially. Therefore, we have put forward higher security requirements for the management, transmission, and storage of massive IoT data. However, during the transmission process of IoT data, security issues, such as data theft and forgery, are prone to occur. In addition, most existing data storage solutions are centralized, i.e., data are stored and maintained by a centralized server. Once the server is maliciously attacked, the security of IoT data will be greatly threatened. In view of the above-mentioned security issues, a security transmission and storage solution is proposed about sensing image for blockchain in the IoT. Firstly, this solution intelligently senses user image information, and divides these sensed data into intelligent blocks. Secondly, different blocks of data are encrypted and transmitted securely through intelligent encryption algorithms. Finally, signature verification and storage are performed through an intelligent verification algorithm. Compared with the traditional IoT data transmission and centralized storage solution, our solution combines the IoT with the blockchain, making use of the advantages of blockchain decentralization, high reliability, and low cost to transfer and store users image information securely. Security analysis proves that the solution can resist theft attacks and ensure the security of user image information during transmission and storage.

A new system for massive RDF data management using Big Data query languages Pig, Hive, and Spark

A new system for massive RDF data management using Big Data query languages Pig, Hive, and Spark

DDR-ESC: A Distributed and Data Reliability Model for Mobile Edge-Based Sensor-Cloud

Cloud computing gained a lot of growth in the development of scientific and industrial domains due to its scalability, data analysis, portability, and storage abilities. The sensors and actuators facilitate wireless communication with real-time data gathering and local preprocessing. However, the huge amount of data collection and storage for such low powered nodes incurs the problem of reliability and efficient resource management. The technology of cloud computing provides an emerging paradigm for the sensors-based applications to improve processing, management, and storage of massive data. However, sensor nodes are organized and communicate in a distributed environment, which is harmful to network reliability and privacy. Recently, many solutions are presented for data transmissions in distributed systems through link-aware routing, but efficient utilization of network resources is still an open research issue. Also, most of the proposed solutions provide secure transmissions on the cost of computation and routing overhead. In this work, we propose a distributed and data reliability model for mobile edge-based sensor-cloud (DDR-ESC) to grow the data delivery performance and preserve the control of security. Moreover, edge-based communications with the cloud systems make the transmission intelligently and decrease the threshold of packets level overhead with high dependability. The extensive experiments validate the proposed model as compared to the state-of-the-art schemes.

Introducing and Comparing Recent Clustering Methods for Massive Data Management in the Internet of Things

The use of wireless sensor networks, which are the key ingredient in the growing Internet of Things (IoT), has surged over the past few years with a widening range of applications in the industry, healthcare, agriculture, with a special attention to monitoring and tracking, often tied with security issues. In some applications, sensors can be deployed in remote, large unpopulated areas, whereas in others, they serve to monitor confined busy spaces. In either case, clustering the sensor network’s nodes into several clusters is of fundamental benefit for obvious scalability reasons, and also for helping to devise maintenance or usage schedules that might greatly improve the network’s lifetime. In the present paper, we survey and compare popular and advanced clustering schemes and provide a detailed analysis of their performance as a function of scale, type of collected data or their heterogeneity, and noise level. The testing is performed on real sensor data provided by the UCI Machine Learning Repository, using various external validation metrics.

AES-GCTR Mode-based Efficient Hybrid Encryption Scheme in Cloud-CCTV Service Environment

In the environment of Cloud-CCTV surveillance, encryption of images for the protection of privacy is essential. Yet, the existing encryption method has overhead regarding the management of mass data and streaming processing. As encrypted vectors are created and applied in a bucket unit in the proposed method, it has more efficient performance in the process of encryption and decoding compared to AES-CTR mode, and it makes an analysis attack difficult according to the randomness of a bucket size. In this paper, AES-GCTR mode that has improved the existing AES-CTR operation mode was suggested. In this process, encryption of CCTV image data is required. Yet, overhead of encryption itself exists in quality, so an efficient encryption method proper for mass data is necessary. The proposed method can make encryption and decoding performance more efficient, and is expected to be easily applied to the image surveillance system based on CCTVs that demands high capacity processing and real-time streaming.

An Evaluation and Comparative study of massive RDF Data management approaches based on Big Data Technologies

An Evaluation and Comparative study of massive RDF Data management approaches based on Big Data Technologies

Research on Smart Grid Planning and Construction Based on GIS Resource Allocation Technology

Smart grid planning and construction is a very complex engineering project that requires intelligent technology to protect systems such as grid lines and substations to control grid stability and power quality. The smart grid GIS resource allocation technology has designed a new system architecture for the needs of building smart grids. The article expounds the key technologies and typical applications of smart grid GIS resource allocation technology, and describes its broad application prospects in smart grid. As a public service platform for graphical display applications, GIS platform constructs a spatial geographic information data center, which is intelligent. Applications such as grid integration, data mining, and intensive resource utilization provide a massive data storage, management, and analysis environment. The grid GIS platform provides geospatial information sharing and business application integration for smart grid power generation, transmission, substation, power distribution, power consumption, dispatching, etc., providing planning methods and visual analysis services for the construction of smart grid systems.

Research and Application of BIM-based Specification-compliant Field Quality Management for Lean Construction

The construction quality is always the focus and aim of a project, and how to control and manage the quality of the project is an eternal theme. With the endless stream of new architectural design, increasingly diversity in functions and complexity in professional system, and massive tedious data information, quality and management are facing stern challenge in the new political, economic, social and technological environment. The emergence and popularization of BIM (Information Modeling Building) not only changed the production tools of practitioners, but also provided a new way for engineering quality management. This paper will conduct research and discussion in aspect of the construction quality management in the time of opportunity and challenge.

Water contamination monitoring system based on big data: a case study

Water plays a vital role in people's life, and individuals cannot survive without it. However, water contamination has become a serious issue with the development of industry and agriculture, and has become a threat to people's daily life. Moreover, the amount of data people need to process becomes excessively complex and huge in big data era. Hence, data management is increasingly a difficult task. There is an urgent need to develop a system to identify major changes of water quality through monitoring and managing these water quality variables. In this paper, we develop a data monitoring system named monitoring and managing data centre (MMDC) for monitoring, downloading, sharing, and time-series analysis based on big data technology. In order to reflect the real hydrological ecosystem, water quality variable data collected from Taihu Lake in China is used to verify the effectiveness of MMDC. Results show that MMDC brings effectiveness for monitoring and management of massive data. Although this investigation is focused on Taihu Lake, it is applicable as a general monitoring system for other similar natural resources.

Research on massive information query and intelligent analysis method in a complex large-scale system.

With the rapid growth of big data and network information, it is particularly important to perform information query and intelligent analysis on unstructured massive data in large-scale complex systems. The existing methods of directly collating, sorting, summarizing, and storing retrieval of documents cannot meet the needs of information management and rapid retrieval of massive data. This paper takes the standardized storage, effective extraction and standardized database construction of massive resume information in social large-scale complex systems as an example, and proposes a massive information query and intelligent analysis method. The method utilizes the semi-structured features of the resume document, constructs the extraction rule model of various resume data to extract the massive resume information. On the basis of HBase distributed storage, with the help of parallel computing technology to optimize the storage and query efficiency, which ensures the intelligent analysis and retrieval of massive resume information. The experimental results show that this method not only greatly improves the extraction accuracy and recall rate of resume information data, but also compared with the traditional methods, there are obvious improvements in the three aspects of massive information retrieval methods, query usage efficiency, and the intelligent analysis of complex systems.