Big Data Environment Research Articles

An effective co-evolutionary algorithm based on artificial bee colony and differential evolution for time series predicting optimization

Non-linear model optimization for predicting time series is a challenge problem. In Intelligent Transportation Systems (ITS) application, the indispensable short-term traffic flow prediction with big data makes the problem worst. To improve the prediction accuracy and ensure real-time performance in the big data environment, we propose a novel co-evolutionary artificial bee colony (ABC) improved by differential evolution (DE) optimization algorithm combined with a traffic flow predicting model trained by extreme learning machine (ELM) neural network. The proposed model can inherit the better generalization performance and the less training time consumption of the standard ELM, and can achieve a more balanced search strategy with the optimized weights and biases to overcome the random initialization deficiency of the typical ELM, and successfully obtain higher prediction accuracy compared with state-of-the-art methods. To verify the efficiency of the proposed model, we apply it to Lozi and Tent chaotic time series simulations and measured traffic flow time series experiments. Simulation and experimental results demonstrate that the proposed model has superior performance and competitive computational efficiency.

A Novel Method to Protect Private Information in Big Data Environment

In the big data environment, through the analysis and induction of massive data, mining out potential models and studying the market operation rules, can help enterprises adjust strategies, reduce risks and improve efficiency. However, with the wide application of new technology, the sensitive information of users may be disclosed while mining rules. In the big data environment, the amount of data related to users is large. If the privacy information in these data is leaked, it will cause great harm to users. In order to avoid the disclosure of sensitive information in the process of users accessing services in the big data environment, our research will propose new method to protect the privacy data of users in big data environment. Our research design a new approach based on private matching protocol using block chain technology to protect private data in big data environment.

An Air Quality Prediction Model Based on a Noise Reduction Self-Coding Deep Network

Aiming at remedying the problem of low prediction accuracy of existing air pollutant prediction models, a denoising autoencoder deep network (DAEDN) model that is based on long short-term memory (LSTM) networks was designed. This model created a noise reduction autoencoder with an LSTM network to extract the inherent air quality characteristics of original monitoring data and to implement noise reduction processing on monitoring data to improve the accuracy of air quality predictions. The LSTM network structure in the DAEDN model was designed as bidirectional LSTM (Bi-LSTM) to solve the problem of a lag in the unidirectional LSTM prediction results and thereby to further improve the prediction accuracy of the prediction model. Using air pollutant time series data, the DAEDN model was trained using hourly PM2.5 concentration data collected in Beijing over 5 years. The experimental results show that the DAEDN model can extract more stable features from the noisy input after training was completed. The models were evaluated using RMSE and MAE, and the results show that the indexes are 15.504 and 6.789; compared with unidirectional LSTM, it is reduced by 7.33% and 5.87%, respectively. In addition, the new prediction model essentially considered the time series properties of the prediction of the concentration of spatial pollutants and the fully integrated environmental big data, such as air quality monitoring, meteorological monitoring, and forecasting.

Deep Correlation Mining Based on Hierarchical Hybrid Networks for Heterogeneous Big Data Recommendations

The advancement of several significant technologies, such as artificial intelligence, cyber intelligence, and machine learning, has made big data penetrate not only into the industry and academic field but also our daily life along with a variety of cyber-enabled applications. In this article, we focus on a deep correlation mining method in heterogeneous big data environments. A hierarchical hybrid network (HHN) model is constructed to describe multitype relationships among different entities, and a series of measures are defined to quantify the internal correlations within one specific layer or external correlations between different layers. An intelligent router based on deep reinforcement learning framework is designed to generate optimal actions to route across the HHN. An improved random walk with the restart-based algorithm is then developed with the intelligent router, based on the hierarchical influence across network associated with multiple correlations. An intelligent recommendation mechanism is finally designed and applied to support users' collaboration works in scholarly big data environments. Experiments based on DBLP and ResearchGate data show the practicability and usefulness of our model and method.

USER LOCATION PREDICTION USING HYPERGRAPH IMPACT FACTOR IN TWITTER WITH GLOBAL DATA COMMUNICATION

Twitter is one of the most prominent online media that acts as a global network for sharing sensitive real-time information like earthquake alerts, political news, product review, personality identification, criminal detection etc. along with regular usage, which is why knowing the location of a user in twitter gets at most important even though they do not tend to disclose it. In this paper, we propose a technique to detect the name of the locations for the twitter users. This technique involves a hypergraph-based map-reduce concept to represent the user tweets with their locations. The Helly property of the hypergraph was used to remove less potential words and the Impact Factor measure (IF) was introduced to calculate the score of each location for a particular user. The algorithm (HIF) was implemented in a big data environment provided by Hadoop and found to give an average accuracy of 78% which is well ahead of the existing methodologies. This method gives appreciable results, with high values of precision and recall for all locations.

The Impact of Big Data Analysis on Consumer Behavior

Based on the new environment of big data, this paper expounds the connotation and characteristics of big data, and analyzes the characteristics of consumer behavior under the application background of big data analysis technology. With the help of AISAS model, which is used to analyze consumer behavior in the network economy, and in combination with the influence mechanism of big data analysis on consumer behavior decision-making process, we have constructed a consumer behavior model under the background of big data and tested it by means of questionnaire survey. The results show that the factors that affect consumer decision-making include external factors and internal perception, while big data affects consumer internal perception through the impact of external factors, thus affecting consumer decision-making. At the same time, consumer information sharing is also conducive to improving the accuracy of big data analysis.

A Data Fusion Framework of Multi-Source Heterogeneous Network Security Situational Awareness Based on Attack Pattern

In the context of the era of big data, how to quickly and accurately detect attack events from massive amounts of heterogeneous data and form effective response on time has become the main challenge facing network security today. This paper systematically summarizes the attack behavior with both attack technology mechanisms and characteristics of the attack target, thereby constructing a scalable attack behavior model. Based on the attack behavior model, a data fusion framework of the multi-source heterogeneous network security situation is constructed. The framework normalizes multi-source heterogeneous security data into threat events with an attack pattern as the core and determine the attack state by the causal chain. Finally, the feasibility and effectiveness of the framework are verified by analyzing data in real business scenarios. This framework abstracts the multi-source heterogeneous data into an analyzable attack event, which greatly reduces the amount of data to be analyzed and improves the credibility of the network security situation data, realizing the identification of attack behaviors in the big data environment.

Research on the Construction of Knowledge Service Model of Port Supply Chain Enterprise in Big Data Environment

In the context of the rapid development of big data and artificial intelligence, with the help of the knowledge service theory system and big data technology to build a smart port supply chain knowledge service model to provide a personalized, intelligent, and diversified knowledge-based service system platform solution to port supply chain enterprises, which is of great significance to realization of port supply chain transformation and upgrading and intelligent integrated operation. This paper analyzes and summarizes the research status of knowledge service demand and port supply chain knowledge service during the development and operation of port supply chain, and applies big data and artificial intelligence technologies such as knowledge matching, knowledge fusion, and natural language processing. The port supply chain knowledge service model including knowledge acquisition, knowledge organization and knowledge service modules is constructed. Ontology method is used to construct the ontology knowledge base of port supply chain, and based on this, computational reasoning experiments are performed. The experiments show that the ontology technology shows good effectiveness and superiority in constructing the knowledge service system model of the port supply chain in terms of knowledge representation and knowledge reasoning.

Research and Design of Network Situation Awareness System Based on Big Data

Network situational awareness has become a new hotspot in the research of network security. Based on the analysis of existing network situational awareness models, this paper establishes a network situational awareness model based on the SimHash algorithm in a big data environment, which provides reference value for the next step of the network situational awareness system.

Study on the Unified Data of Power Regulation System for Guizhou Power Grid

The grid structure of the Guizhou Power Grid is becoming more and more complex, and the deepening of regulatory integration has made the information and work of regulators more and more onerous. Combining artificial intelligence, big data technology and grid control business is an inevitable trend of development. In this paper, a unified data model of the power system is established based on the Guizhou power grid’s regulatory operation requirements. It can analyze the impact of big data on dispatching business and define the power business demand model in the big data environment. This model also integrates and manages the multi-source heterogeneous data of power grid business by reflecting data and relationships of power regulation business. The research results are of great significance to improve the level of safe and stable operation of the power grid, and to promote intelligent operation of regulation and control, and also lay a solid foundation for intelligent dispatch.

La tecnología Big Data en la salud pública.

El Big Data es una realidad. Su implantación e influencia en todos los ámbitos supondrá una nueva revolución social, económica y tecnológica. Al respecto, el ámbito de la salud no es ajeno a sus posibilidades. En la actualidad, como consecuencia de la crisis sanitaria originada por la COVID-19, el sistema Big Data está demostrando ser una herramienta especialmente útil para prever la expansión del virus, y establecer estrategias de control. No obstante, el entorno Big Data también presenta importantes retos jurídico-económicos que el Derecho debe afrontar. 
 

An efficient clustering mechanism in big data framework for data preprocessing and management

An efficient data handling mechanism has been applied based on epoch-based k-means associated fuzzy clustering (EKFC). In the first phase weights have been assigned to individual data segment presented based on the classification key metrics. It has been assigned automatically. Then weight preprocessing has been done in such manner to prune the unwanted weights. It has been pruned in such way to filter the weights which are not scalable. Then epoch-based k-means associated fuzzy clustering (EKFC) approach has been applied for data arrangement. First different epochs have been considered for the calculation of initial seeds values. These seeds have been considered after considering 100 epochs. After 100 epochs seeds have been determined. These seeds values have been used as the initial centroid for the k-means clustering. After the complete validation similar clusters from the two clustering approaches have been considered. In the next phase operational clustering has been performed. In the final phase threshold ranking has been performed. It has been performed for the final classification based on the above clusters. It will arrange in the order of threshold values. It will be used for the determination of the priority of the task in the big data environment. The results are found to be prominent in terms of classification accuracy.

Machine Learning Based Method for Prediction of Heart Disease in Big Data Environment

Prediction of diseases is one of the challenging tasks in healthcare domain. Conventionally the heart diseases were diagnosed by experienced medical professional and cardiologist with the help of medical and clinical tests. With conventional method even experienced medical professional struggled to predict the disease with sufficient accuracy. In addition, manually analysing and extracting useful knowledge from the archived disease data becomes time consuming as well as infeasible. The advent of machine learning techniques enables the prediction of various diseases in healthcare domain. Machine learning algorithms are trained to learn from the existing historical data and prediction models are being created to predict the unknown raw data. For the past two decades, machine learning techniques are extensively employed for disease prediction. Despite the capability of machine algorithm on learning from huge historical data which is stored in data mart and data warehouses using traditional database technologies such as Oracle OnLine Analytical Processing (OLAP). The conventional database technologies suffer from the limitation that they cannot handle huge data or unstructured data or data that comes with speed. In this context, big data tools and technologies plays a major role in storing and facilitating the processing of huge data. In this paper, an approach is proposed for prediction of heart diseases using Support Vector Algorithm in Spark environment. Support Vector Machine algorithm is basically a binary classifier which classifies both linear and non-linear input data. It transforms the non-linear data into hyper plan with the help of different kernel functions. Spark is a distributed big data processing platform which has a unique feature of keeping and processing a huge data in memory. The proposed approach is tested with a benchmark dataset from UCI repository and results are discussed.

An intelligent system for energy management in smart cities based on big data and ontology

PurposeThis research paper aims at proposing a framework based on semantic integration in Big Data for saving energy in smart cities. The presented approach highlights the potential opportunities offered by Big Data and ontologies to reduce energy consumption in smart cities.Design/methodology/approachThis study provides an overview of semantics in Big Data and reviews various works that investigate energy saving in smart homes and cities. To reach this end, we propose an efficient architecture based on the cooperation between ontology, Big Data, and Multi-Agent Systems. Furthermore, the proposed approach shows the strength of these technologies to reduce energy consumption in smart cities.FindingsThrough this research, we seek to clarify and explain both the role of Multi-Agent System and ontology paradigms to improve systems interoperability. Indeed, it is useful to develop the proposed architecture based on Big Data. This study highlights the opportunities offered when they are combined together to provide a reliable system for saving energy in smart cities.Practical implicationsThe significant advancement of contemporary applications (smart cities, social networks, health care, IoT, etc.) requires a vast emergence of Big Data and semantics technologies in these fields. The obtained results provide an improved vision of energy-saving and environmental protection while keeping the inhabitants’ comfort.Originality/valueThis work is an efficient contribution that provides more comprehensive solutions to ontology integration in the Big Data environment. We have used all available data to reduce energy consumption, promote the change of inhabitant’s behavior, offer the required comfort, and implement an effective long-term energy policy in a smart and sustainable environment.

A New Method for Long-Distance Data Reconstruction in Big Data Environment

In a big data environment, the data from long-term monitoring are of great importance in intelligent diagnoses, prognostic and health management of industrial components. However, in practice, it is difficult to ensure high reconstructive accuracy making the preservation performance of low-frequency information (e.g., periodic fault impulses) stable. To address this issue, this article proposes a new data reconstruction method that combines a smoothing sparse low-rank matrix (SSLRM) with online dictionary learning. This article first presents the design of SSLRM model for decomposing the raw data into a low resonance component (LRC) and a high resonance component (HRC), such that the periodic fault impulse can be isolated and preserved in advance. Then, both LRC and HRC are, respectively, reconstructed using an online dictionary learning algorithm, i.e., Hankel K-singular value decomposition; thus, the reconstructed data can be obtained accordingly. Both simulation case and engineering case are applied to show the effectiveness and practicality of the proposed approach.

Environmental Activism and Big Data: Building Green Social Capital in China

The rapid development of information and communication technologies, coupled with the significant progress in the areas of environmental policy and public participation, has led to the advent of environmental big data in China recently. This article applies social capital theory as an analytical lens to shed light on how Chinese environmental non-government organizations (ENGOs) adopt big data to promote environmental governance. This study conducts case studies focusing on two ENGOs: The Institute of Public and Environmental Affairs (IPE) and Green Hunan. Combining a qualitative approach with quantitative analysis, this research examines two big data-induced initiatives: The first involves green supply chain management in the IPE, brand-sensitive multinational corporations (MNCs), and Chinese suppliers of the MNCs, while the second involves the mobile data-based Riverwatcher Action Network of Green Hunan and numerous volunteers nationwide. This study found that big data adoption by ENGOs contributes effectively to building green social capital, including social networks and pro-environmental social norms. Green social capital has important implications for governance in terms of fostering coordination and cooperation across the boundaries of the public, private, and voluntary sectors. This study highlighted the finding that empowerment by big data helps Chinese ENGOs play the role of a change agent in sustainability transitions.

Handling Missing Values in Information Systems Research: A Review of Methods and Assumptions

In today’s big data environment, missing values continues to be a problem that harms the data quality. The bias caused by missing values raises the highest concern as it cannot be eliminated simply by increasing the sample size. Although the statistics literature has developed approaches to handling missing values and formulated assumptions regarding when these approaches generate valid statistical inferences, these prescriptions have yet to be broadly accepted by many social science disciplines including the Information Systems (IS) discipline. By reviewing recently published empirical research in information systems, we find that missing values is indeed an important and pervasive problem. We believe that a review of missing value theory is necessary for the IS community to understand the nature of missing values and to promote more rigorous research practice when missing values is often unavoidable. In addition, the not missing at random (NMAR) mechanism brings in challenges in parameter estimation. We contribute to research practice by proposing and demonstrating the superior performance of a Monte Carlo likelihood approach in correcting bias in parameter estimation. We conclude by suggesting that research validity can be enhanced through reasoned adoption of missing value handling method and missing value reporting practice.

The application of big data network crawler technology for architectural culture and environment protection

SummaryIn today's big data environment, we get the Maonan residential information of Maonan Nationality in Huanjiang through the web crawler technology, and further study the protection and renewal of the local architectural culture and environment, in order to promote the social civilization and sustainable development. With the continuous development and improvement of Internet technology, information acquisition technology of Web crawler based on large data is particularly important. It can collect network information intelligently and efficiently. Taking the architectural protection as the research object, Maonan Ethnic Group in the area is chosen as the object of study. Based on the web crawler technology, the protection and development of the local residential architecture culture and environment are analyzed. Through data collection and investigation of big data crawler technology, we find that the Maonan Ethnic Group has unique local customs and cultural details. Therefore, how to carry out sustainable development according to the unique local cultural characteristics and by means of information technology will become the focus of further research.

Chemometrics modelling of temporal changes of ozone half hourly concentrations in different monitoring stations

Principal Component Analysis and Multivariate Curve Resolution (MCR) chemometrics soft modelling approaches are proposed for the simultaneous analysis of the half hourly tropospheric ozone concentrations over a continuous period of six years at three different monitoring stations. The proposed approach is based on the direct multilinear modelling of experimental data without needing any prior physicochemical, geographical nor meteorological assumptions. By application of the proposed modelling approach, a summarized description of the changes of ozone concentrations due to a reduced number of contributions is achieved. Each one of these contributions is described by a set of diurnal, seasonal, year and location profiles whose interpretation is then possible using the available knowledge. The use of the proposed chemometrics approach can be generalized for the analysis and interpretation of environmental big data sets generated in continuous on-line monitoring of multiple environmental parameters over long time periods and for a large number of monitoring stations.

Construction of Digital Library Precision Service System under Big Data Environment

With the promotion of Internet communication technology and information technology, university digital libraries are also in the process of continuous reform and development. With the passage of time, the information resources of digital libraries accumulate more and more, coupled with the continuous update of computer technology and social networks, users will generate more and more information resources. The rapid development of communication and Internet technology in the era of big data brings opportunities to digital library’s accurate service, and at the same time, it also makes it face huge challenges. The library should rely on the massive book resources platform, combine the big data processing technology with the digital library service concept, and actively explore the data acquisition, sorting and storage technology, so as to provide more effective and humanized services for users. This paper introduces the goal of precision service of Digital Library Based on big data, expounds the characteristics of precision service of Digital Library Based on big data, and puts forward the construction method of precision service system of Digital Library under big data environment.