Big Data Environment Research Articles

Enhance the Transfer Capacity of Multiplex Networks

Most complex real systems are found to have multiple layers of connectivity and required to be modelled as multiplex networks. One of the extremely critical problems is to reduce the congestion and enhance the transfer capacity, especially in real communication networks with a big data environment. A novel and effective strategy to improve traffic and control congestion is proposed by adding edges according to their weights which are defined by the topology structural properties. Furthermore, which layer is more effective when our strategy is applied is discussed based on its topology structure. Adding edges between nodes whose product of multiplex network betweenness is the highest is confirmed to be more effective, particularly in the layer with stronger community structure. Simulation experiments on both computer-generated and real-world networks demonstrate that our strategy can enhance the transfer capacity of multiplex networks significantly, which is in good agreement with our analysis.

Industrial Internet of Learning (IIoL): IIoT based pervasive knowledge network for LPWAN\u2014concept, framework and case studies

Industrial Internet of Things (IIoT) is performed based on the multiple sourced data collection, communication, management and analysis from the industrial environment. The data can be generated at every point in the manufacturing production process by real-time monitoring, connection and interaction in the industrial field through various data sensing devices, which creates a big data environment for the industry. To collect, transfer, store and analyse such a big data efficiently and economically, several challenges have imposed to the conventional big data solution, such as high unreliable latency, massive energy consumption, and inadequate security. In order to address these issues, edge computing, as an emerging technique, has been researched and developed in different industries. This paper aims to propose a novel framework for the intelligent IIoT, named Industrial Internet of Learning (IIoL). It is built using an industrial wireless communication network called Low-power wide-area network (LPWAN). By applying edge computing technologies in the LPWAN, the high-intensity computing load is distributed to edge sides, which integrates the computing resource of edge devices to lighten the computational complexity in the central. It cannot only reduce the energy consumption of processing and storing big data but also low the risk of cyber-attacks. Additionally, in the proposed framework, the information and knowledge are discovered and generated from different parts of the system, including smart sensors, smart gateways and cloud. Under this framework, a pervasive knowledge network can be established to improve all the devices in the system. Finally, the proposed concept and framework were validated by two real industrial cases, which were the health prognosis and management of a water plant and asset monitoring and management of an automobile factory.

Provenance Method of Electronic Archives Based on Knowledge Graph in Big Data Environment

With the advent of the era of big data, the Provenance Method of electronic archives based on knowledge graph under the environment of big data has produced a large number of electronic archives due to the development of science and technology. How to guarantee the credential characteristics of electronic archives in the big data environment has attracted wide attention of the academic community. Provenance is an important technical means to guarantee the certification of electronic archives. In this paper, knowledge graph technology is used to provide the concept provenance of electronic archives in large data environment. It not only enriches the provenance method, but also guarantees the certification of electronic archives in the large data environment.

Media Coverage, CEO Age and Corporate Performance in Big Data Environment

This study analyzes the influence of CEO age on corporate performance under the big data environment and the role of media coverage in this relationship by taking the A-share listed companies from 2009 to 2019 as research objects. Our results show that in the low-speed developing enterprises, the older the CEO, the higher the level of corporate performance. Positive and neutral media reports positively affect corporate performance, whereas negative media reports negatively affect corporate performance. Media reports (including positive, negative, and neutral media reports) weaken the influence of CEO age on corporate performance

Ensemble Pruning of RF via Multi-Objective TLBO Algorithm and Its Parallelization on Spark

Ensemble learning has been widely used in various fields. Still, too many base classifiers will affect the classification time of the ensemble classifier under the big data environment, while reducing base classifiers will affect the classification accuracy of the ensemble classifier. Therefore, the multi-objective teaching-learning-based optimization (MO-TLBO) algorithm is used to carry out ensemble pruning of random forest (RF) to improve the classification accuracy and speed of RF. MO-TLBO algorithm aims at maximizing classification accuracy and minimizing classification time, and it can find a sub-forest with higher classification accuracy and faster classification speed. In addition, considering the vast computational time of ensemble pruning of RF via MO-TLBO algorithm under the big data environment, a vote set is constructed to improve the fitness evaluation process. In the Spark platform, the RF improved by the MO-TLBO algorithm (MO-TLBO-RF) is parallelized based on data parallelism. The Shuffle optimization strategy is proposed to reduce the number of Shuffles in the execution of parallel MO-TLBO-RF. The proposed MO-TLBO-RF is applied to rolling bearing fault diagnosis. The experimental results show that the algorithm can obtain an RF with high fault diagnosis accuracy and fast fault diagnosis speed. The results also prove that the ensemble pruning time can be greatly reduced via the vote set and parallelization of MO-TLBO-RF.

Design of the Piano Score Recommendation Image Analysis System Based on the Big Data and Convolutional Neural Network.

In the era of big data, the problem of information overload is becoming more and more obvious. A piano music image analysis and recommendation system based on the CNN classifier and user preference is designed by using the convolutional neural network (CNN), which can realize accurate piano music recommendation for users in the big data environment. The piano music recommendation system based on the CNN is mainly composed of user modeling, music feature extraction, recommendation algorithm, and so on. In the recommendation algorithm module, the potential characteristics of music are predicted by the regression model, and the matching degree between users and music is calculated according to user preferences. Then, music that users may be interested in is generated and sorted in order to recommend new piano music to relevant users. The image analysis model contains four “convolution + pooling” layers. The classification accuracy and gradient change law of the CNN under RMSProp and Adam optimal controllers are compared. The image analysis results show that the Adam optimal controller can quickly find the direction, and the gradient decreases greatly. In addition, the accuracy of the recommendation system is 55.84%. Compared with the traditional CNN algorithm, this paper uses the convolutional neural network (CNN) to analyze and recommend piano music images according to users' preferences, which can realize more accurate piano music recommendation for users in the big data environment. Therefore, the piano music recommendation system based on the CNN has strong feature learning ability and good prediction and recommendation ability.

Research on Modularization and Association of Elderly Care Services in Big Data Environment

At this stage, China is in the stage of building a well-off society in an all-round way, and the application of big data has positive significance for the grasp of China’s national conditions and the laws of scientific development. At the same time, big data applications can also provide support for the development of the elderly care service industry. This article summarizes the modular content of elderly care services based on previous work experience. The author discusses the data association analysis of distributed elderly care services in a big data environment from four aspects: the construction of distributed association classifiers, the incremental mining of association classifiers based on constraints, the running process of the distributed association classifier model, and the evaluation of actual elderly care services.

A Study of Lung Disease Using Image Processing in Big Data Environment

A In the medical field, the Image dispensation techniques are extensively employed for image amelioration in finding and treatment of lung disease in the big data environment, where the point in time feature is very paramount to determine the idiosyncrasy issues in intention images, particularly in lung disease such as cancer, pneumonia, COVID-19 etc, for early detection and treatment stages of lungs disease, Image processing technique are widely used for identification of genetic as well as environmental factors are very important in developing a novel method of lung disease prevention. The core factors of this research are quality, time, and precision of the dataset. The modification and evaluation of image quality depend on the segmentation techniques, an improved area of the object that is utilized as a rudimentary substructure of feature extraction is obtained and comparison is made on relying feature. Medical images are analyzed by different segmentation techniques of image processing. The segmentation techniques are used dataset to find patterns and retrieve information from the dataset for processing. The goal of this study is discussed various image processing techniques and big data analytics tools for lung disease has been given in the tabular form and provides comparative study. This study provides minutiae of big data analytics tools and image processing techniques, specifically discussed in the context of lung disease images.

Information Security Monitoring and Management Method Based on Big Data in the Internet of Things Environment

As an important part of the new generation of information technology, the Internet of Things (IoT), with its ubiquitous connection and service characteristics, has penetrated into various fields of application and played an important role. In this paper, based on the study of the basic technology of the environmental Internet of Things, combined with the service-oriented technology architecture SOA, J2EE, multi-level system architecture MVC, real-time database and other technologies and project practice experience, summarized and proposed a kind of environmental quality monitoring integrated management platform design and implementation feasibility scheme. Firstly, the background of the era of big data is described in detail, the urgency and necessity of information security monitoring under the background of big data is clarified, and the three elements of information security monitoring mechanism, namely network monitoring personnel, environment and technology, are proposed, and the three elements as the starting point to establish the information security monitoring mechanism; Starting from the relevant monitoring strategies and safety monitoring technologies, this paper explains the basic principles of constructing the evaluation index system, and establishes the evaluation index system according to the key influencing factors of enterprise information security level in the environment of big data. AHP fuzzy comprehensive evaluation method is chosen on the basis of analyzing various comprehensive evaluation methods, and the weight of each evaluation index is determined and the comprehensive evaluation model is constructed. The establishment of information security monitoring and evaluation system, the use of information security monitoring and evaluation system, for information security monitoring work to provide reference standards. Finally, on the basis of the foregoing, relevant strategies for information security monitoring are proposed, and necessary suggestions are provided for information security work.

Research on ABAC Access Control Based on Big Data Platform

In the environment of big data, the traditional access control lacks effective and flexible access mechanism. Based on attribute access control, this paper proposes a HBMC-ABAC big data access control framework. It solves the problems of difficult authority change, complex management, over-authorization and lack of authorization in big data environment. At the same time, binary mapping codes are proposed to solve the problem of low efficiency of policy retrieval in traditional ABAC. Through experimental analysis, the results show that our proposed HBMC-ABAC model can meet the current large and complex environment of big data.

Knowledge Big Graph Fusing Ontology with Property Graph: A Case Study of Financial Ownership Network

The scale of know­ledge is growing rapidly in the big data environment, and traditional know­ledge organization and services have faced the dilemma of semantic inaccuracy and untimeliness. From a know­ledge fusion perspective-combining the precise semantic superiority of traditional ontology with the large-scale graph processing power and the predicate attribute expression ability of property graph-this paper presents an ontology and property graph fusion framework (OPGFF). The fusion process is divided into content layer fusion and constraint layer fusion. The result of the fusion, that is, the know­ledge representation model is called know­ledge big graph. In addition, this paper applies the know­ledge big graph model to the ownership network in the China’s financial field and builds a financial ownership know­ledge big graph. Furthermore, this paper designs and implements six consistency inference algorithms for finding contradictory data and filling in missing data in the financial ownership know­ledge big graph, five of which are completely domain agnostic. The correctness and validity of the algorithms have been experimentally verified with actual data. The fusion OPGFF framework and the implementation method of the know­ledge big graph could provide technical reference for big data know­ledge organization and services.

Study on network security intrusion target detection method in big data environment

In view of the traditional network security intrusion target detection method cannot effectively estimate the trend of the intrusion target, resulting in poor detection accuracy, a new network secu...

Research on Comprehensive Evaluation of Data Source Quality in Big Data Environment

Research on Comprehensive Evaluation of Data Source Quality in Big Data Environment

Research on Improving Computer Teaching Quality in Colleges and Universities in the Context of Big Data

With the continuous development of information technology, big data has entered all walks of life. As an important part of college education, computer teaching is very significant for enhancing students’ computer application ability and promoting comprehensive development. However, there are still many problems in actual computer teaching in Chinese colleges and universities, and the teaching effect is not ideal. Based on this, this article will discuss the characteristics of big data, analyze the problems in computer teaching in colleges and universities, and propose measures to improve the quality under the big data environment, hoping to have certain reference.

Binary Classification of Network-Generated Flow Data Using a Machine Learning Algorithm

This study proposes a model for building intrusion detection systems. The dataset used, CICIDS 2017, contains 14 different attacks with 85 features for each attack. This high dimensionality of the data is a major challenge when building efficient intrusion detection systems, especially in today's big data environment, since a lot of the features are redundant. The main goal in this paper was to reduce the number of features and present a detailed discussion of the important features. For feature selection, information gain was used in an iterative way, and for classification, a machine learning algorithm, the J48 decision tree algorithm, was used. The important features for the classification of each attack were identified, and the features that were important for classifying multiple attacks were also identified and discussed.

Study on network security intrusion target detection method in big data environment

Study on network security intrusion target detection method in big data environment

Вплив аналітики великих даних та інноваційних інформаційних технологій на якість аудиту

The article is aimed at examining and systematizing the factors influencing innovative information technologies on the auditor’s professional judgment, efficiency and quality of audit, as well as studying the impact of big data analytics on the skills and competence required by auditors to conduct an audit in a big data environment. The state of use of data analytics instruments in auditing is characterized. Based on the analysis of the views of scientists, experts and practicing auditors, the main tasks of the audit are systematized, in which the effective use of big data analytics is possible, among which the following are distinguished: assessment of continuity of the enterprise’s activities along with bankruptcy forecasting; detection of financial fraud and assessment of the effectiveness of control; use of visualization dashboards to form a professional judgment of the auditor on the scope of control, the uniformity of the the sample set and size, the threshold of materiality. The main factors that hinder the use of big data analytics in the audit are highlighted, namely: inconsistency in assessing the reliability and accuracy of data in the sets; ensuring the confidentiality and integrity of the client’s data; uncertainty about several methodological bases of the audit, in particular sampling-based testing, as well as the lack of skills and competencies among auditors regarding the use of big data analytics instruments. Prospects for further research in this direction are to identify the possibility of eliminating inconsistencies in the use of big data at the level of standards, as well as to improve educational and professional training programs for accounting and audit specialists with a focus on obtaining skills in working with big data.

Big Data Knowledge Pricing Schemes for Knowledge Recipient Firms

Big data knowledge, such as customer demands and consumer preferences, is among the crucial external knowledge that firms need for new product development in the big data environment. Prior research has focused on the profit of big data knowledge providers rather than the profit and pricing schemes of knowledge recipients. This research addresses this theoretical gap and uses theoretical and numerical analysis to compare the profitability of two pricing schemes commonly used by knowledge recipients: subscription pricing and pay-per-use pricing. We find that: (1) the subscription price of big data knowledge has no effect on the optimal time of knowledge transaction in the same pricing scheme, but the usage ratio of the big data knowledge affects the optimal time of knowledge transaction, and the smaller the usage ratio of big data knowledge the earlier the big data knowledge transaction conducts; (2) big data knowledge with a higher update rate can bring greater profits to the firm both in subscription pricing scheme and pay-per-use pricing scheme; (3) a knowledge recipient will choose the knowledge that can bring a higher market share growth rate regardless of what price scheme it adopts, and firms can choose more efficient knowledge in the pay-per-use pricing scheme by adjusting the usage ratio of knowledge usage according to their economic conditions. The model and findings in this paper can help knowledge recipient firms select optimal pricing method and enhance future new product development performance.

A prediction and analysis model of complex system based on extension neural network-Taking the prediction and analysis of terrorist events in the big data environment as an example

A prediction and analysis model of complex system based on extension neural network-Taking the prediction and analysis of terrorist events in the big data environment as an example

Trust Evaluation Model Based on PSO and LSTM for Huge Information Environments

Due to the challenge of increasing data volume, the traditional trust model is unable to manage data with high efficiency and effectively extract useful information hidden in big data. To fully utilize big data and combine machine learning with trust evaluation, a trust evaluation model based on Long short-term memory (LSTM) is presented. The powerful learning ability, expressive ability and dynamic timing of LSTM can be applied to study data while avoiding the vanishing and exploding gradient phenomena of traditional Recurrent neural networks (RNNs) to ensure that the model can learn sequences of random length and provide accurate trust evaluation. Targeting the performance instability caused by the LSTM model's random initialization of weights and thresholds, Particle swarm optimization (PSO), one of the intelligent algorithms, is introduced to find global optimal initial weights and thresholds. Experiments proved that the trust model proposed in this paper has high accuracy and contributes a new idea for trust evaluation in big data environments.