Big Data Environment Research Articles

The Influence of Music Appreciation Courses on The Formation of College Students’ ideological Quality under The Environment of Big Data

At present, with the deepening of China’s education system reform, China’s music education is facing a rare opportunity to develop, music education developing the students’ quality comprehensively, promoting the construction of socialist spiritual civilization will play a greater role. This article will music combined with large data environment in the school, to explore big data, and the practicality of music education in education, some problems and try to combine big data with innovative ideas to solve these problems, in order to enrich the content of music of music on college students’ thought positive impact of sustainability. Finally, after expounding the relevant concepts and theories, this paper demonstrates the feasibility of bringing positive influence of music appreciation course on the ideological quality of college students behind the use of big data technology from the aspects of music education methods, approaches, evaluation, contents and objects.

Innovation of Family Business Management Mode under the Environment of Big Data and Artificial Intelligence

In today’s era, the emergence and application of artificial intelligence and big data have become the main means for enterprises to improve economic benefits. It can be said that the application of artificial intelligence and big data in family business management is an information and digital enterprise revolution. Of course, in such an era of rapid development of new technology, the management model of family business has also been some unprecedented impact and impact. The traditional family business management mode cannot meet the needs of the further development of the enterprise at all. In order to promote the development of the family business, it is necessary to use the innovative thinking of artificial intelligence and big data to improve related work. The purpose of this paper is to take the management mode of family business as the research object, analyze and study how the management mode of family business can be innovated to meet the needs of social development under the background of artificial intelligence and big data. This paper will use case analysis, comparative analysis, etc., specifically analyze the family business in the management mode of the shortcomings and problems, so as to find a new approach to these problems. The research results show that family business management model does have its advantages and characteristics, but it is undeniable that in the context of big data and artificial intelligence, the shortcomings of the traditional management model are constantly exposed. How to innovate the traditional management model is the key to the sustainable development of family business.

Research on satellite urban transportation and land spatial planning in big data environment

With the continuous development of computer technology, big data application has become a hot research area. Big data has the ability of high-speed and accurate data analysis, which provides great help for many research work. With the help of big data technology, we can provide data reference for the spatial planning of satellite urban traffic. Satellite towns are different from ordinary cities. Satellite town refers to the small city around the big city, which is a small town that provides residence for workers in the big city who are unable to buy houses in the big city. Compared with the general large and medium-sized cities, the space planning of satellite towns is relatively simple. Therefore, the complexity of traffic structure and function area planning of satellite towns is not as complex as that of big cities. Firstly, this paper analysis the characteristics of satellite urban traffic under big data. Then, this paper analysis the spatial planning of big data land. Finally, this paper puts forward the satellite town planning mode based on the people-oriented core concept under the big data.

Research on Remote Control Method of English Multimedia Online Teaching System In Big Data Environment

In order to ensure the effectiveness and security of remote control of the teaching system, the remote control method of English multimedia online teaching system based on large data environment is proposed. Firstly, the remote control module of online teaching system is optimized, configuration parameters are improved, the remote control module is operated effectively, and the remote control rules of multimedia online teaching system are designed, which further ensures the security of remote control. According to the control rules, the remote control steps are simplified, and the optimization of remote control method for English multimedia online teaching system is finally realized. Finally, the experiment proves that the remote control method of English multimedia online teaching system in large data environment is more effective and safe than the traditional method.

Applied Research on Agricultural Big Data

With the continuous advancement of digital informationization, the agricultural field has naturally entered the era of big data. At present, big data technology has achieved remarkable results in the Internet industry, but it is still in its infancy in the agricultural field. Due to the complexity of the agricultural field itself and the obvious diversity and heterogeneity of data, the agricultural Internet of Things in the big data environment is facing many difficulties and challenges, especially the collection and storage of massive data. The purpose of this article is to study the application of agricultural big data. This article briefly introduces the background of the development of agricultural big data and the significance of developing agricultural big data, and summarizes the meaning of big data. At the same time, this article summarizes and analyzes the major applications of agricultural big data in production process management, agricultural resource management, and agricultural ecology environmental management, agricultural products and food safety management. In the experimental part, this paper designed and implemented an agricultural big data system platform based on Hadoop. This article simulated the query of 10, 100 and 1000 records respectively for 10,000, 100,000, 1 million and 10 million records, and the average response time was 56ms. The experimental results show that the performance of the agricultural big data platform designed in this paper can meet the needs.

Realization of Smart Home Information System with High-accuracy Data Import Based on Big Data

To solve the problem of low data import accuracy of intelligent Internet of Things (IoT) information system in the existing smart home application scenarios, this article develops a scheme of smart home information system based on big data with high-accuracy data import. The scheme is based on the system architecture, combined with the related hardware system of smart home, which can successfully complete the accurate import and stable data preprocessing of a large number of data Manage and classify data. By comparing the existing smart home data import performance, big data simulation and comparative experiments are carried out. The experimental results show that the smart home information system can ensure the accuracy and reliability of information import when importing a large number of data. The validity of the scheme is verified. The smart home information system realized by the scheme can operate accurately and reliably in the big data environment, which also shows that the scheme can be extended to the whole society.

Meteorological Application Based on Big Data Streaming Computation Method

Big data analysis in the form of streaming computation has always been a problem to be solved at present, with relatively few research results and practical experience. The random forest method is currently the most extensively used classification algorithm. However, in the application scenario of streaming computation, real-time, volatility, and disorder features presented by data will lead to a gradual reduction in the accuracy of the algorithm. In this paper, the characteristics of the random forest algorithm are analyzed, and the idea of random forest pruning based on the accuracy of the decision tree is proposed. Meanwhile, to adapt to the changes in meteorological data, the concept of accuracy interval is combined to propose a method for the generation, verification, and supplementation of a new decision tree. Finally, a random forest that can be constantly updated with data is established to meet the requirements of the streaming big data environment for the algorithm. Actual meteorological data are used to verify the feasibility of the improved method. The results show that the new method has a higher classification accuracy in the real streaming big data scenario.

Application of security reference architecture to Big Data ecosystems in an industrial scenario

SummaryBig Data environments are typically very complex ecosystems; this means that implementing them is complicated. One possible technique with which to address this complexity is the use of abstraction. Reference architecture (RA) can be useful for an improved understanding of the main components of Big Data. Herein, we propose a security RA that includes the management of security concerns and provides the main elements of a Big Data ecosystem. Application of this architecture to real‐world scenarios facilitates its refinement and improves its usefulness. In this article, we present a case study of a real‐world Big Data ecosystem implemented in a banking environment. This ecosystem was developed by everis, an NTT company with which we collaborated for this study. To conduct this validation case study, a map was established between the elements of the Big Data ecosystem implemented and our proposal. Consequently, a series of valuable lessons that can improve both our architecture and the security of the Big Data environment were obtained. These include recommendations for a set of best practices such as the use of security patterns.

Enhancing Hadoop Performance in Homogenous and Heterogeneous Big Data Environments by Dynamic Slot Configuration

Hadoop is one of the most famous platform solutions for processing large volume and scale of data in parallel processing in Cloud computing. A Hadoop system can be characterized based on three main factors: cluster, workload and user. Each of these factors can be described as either heterogeneous or homogenous, which reflects the heterogeneity degree of the Hadoop systemThe objective of this proposed research work is to investigate the degree of influence of heterogeneity for each of these factors on the performance of Hadoop based on different schedulers. Three schedulers are considered with different levels of Hadoop heterogeneity and are tested and analyzed: the first algorithm considered is the FIFO (First in First out), the second is the Fair sharing, and the final is the COSHH (Classification and Optimization based Scheduler for Heterogeneous Hadoop). Performance issues are related to Hadoop schedulers and comparative performance analysis between different cases of jobs submission. These jobs are processed in different homogenous or heterogeneous data environments and under fixed or reconfigurable slot between map and reduce tasks for Hadoop MapReduce java programming clustering model. The results showed that when assigning tunable knob between map and reduce tasks under certain schedulers like FIFO algorithm, the performance enhanced significantly especially in cases of heterogeneity environment where the workload decreased significantly and the utilization of computational resources increase was obvious.

Bias in composite indexes of CSR practice: An analysis of CUR matrix decomposition

AbstractThe main objective of this research is to propose the best aggregate index of corporate social responsibility practice at the organisational level. To achieve this, we analyse the consistency of the different aggregated measures that researchers use in their analyses through a robust statistical technique, the CUR matrix, framed in the big data environment for selecting individuals. Accordingly, we use an international sample of 2,675 large listed companies. The results show that the CUR leverage identifies greater consistency of the different aggregate measures of CSR, confirming its coherence with a correlation analysis. In this sense, it is possible to affirm that the composite indexes used in academia do not introduce any bias into the analysis of CSR practices. In addition, we demonstrate the utility of this technique to identify the most powerful companies, analysing their CSR commitment at the country and industry levels, namely, Norsk Hydro in the metal and mining industry in Norway, Stora Enso in forestry and paper in Finland, Akzo Nobel in chemical products in the Netherlands, BMW in automobiles and parts in Germany, Generali in finance in Italy, Novartis in pharmaceuticals in Switzerland, the BT Group in telecommunications in the United Kingdom and Inditex in textiles in Spain. Moreover, the results of the CUR study confirm that companies adapt to the demands or pressures from the stakeholders in different areas of interest, which are specific to each country and industry. The availability of these data allows the identification of the structural drivers of their growth and the establishment of priorities that allow the design of more effective sustainable development momentum policies.

Big Data for Energy Management and Energy-Efficient Buildings

European buildings are producing a massive amount of data from a wide spectrum of energy-related sources, such as smart meters’ data, sensors and other Internet of things devices, creating new research challenges. In this context, the aim of this paper is to present a high-level data-driven architecture for buildings data exchange, management and real-time processing. This multi-disciplinary big data environment enables the integration of cross-domain data, combined with emerging artificial intelligence algorithms and distributed ledgers technology. Semantically enhanced, interlinked and multilingual repositories of heterogeneous types of data are coupled with a set of visualization, querying and exploration tools, suitable application programming interfaces (APIs) for data exchange, as well as a suite of configurable and ready-to-use analytical components that implement a series of advanced machine learning and deep learning algorithms. The results from the pilot application of the proposed framework are presented and discussed. The data-driven architecture enables reliable and effective policymaking, as well as supports the creation and exploitation of innovative energy efficiency services through the utilization of a wide variety of data, for the effective operation of buildings.

An Analysis of Data Object Traceability Guarantee Method in Big Data Environment

An Analysis of Data Object Traceability Guarantee Method in Big Data Environment

A Convolutional Deep Self-Organizing Map Feature extraction for machine learning

In this work we propose a new Unsupervised Deep Self-Organizing Map (UDSOM) algorithm for feature extraction, quite similar to the existing multi-layer SOM architectures. The principal underlying idea of using SOMs is that if a neuron is wins n times, these n inputs that activated this neuron are similar. The basic principle consists of an alternation of phases of splitting and abstraction of regions, based on a non-linear projection of high-dimensional data over a small space using Kohonen maps following a deep architecture. The proposed architecture consists of a splitting process, layers of alternating self-organizing, a rectification function RELU and an abstraction layer (convolution-pooling). The self-organizing layer is composed of a few SOMs with each map focusing on modelling a local sub-region. The most winning neurons of each SOM are then organized in a second sampling layer to generate a new 2D map. In parallel to this transmission of the winning neurons, an abstraction of the data space is obtained after the convolution-pooling module. The ReLU is then applied. This treatment is applied more than once, changing the size of the splitting window and the displacement step on the reconstructed input image each time. In this way, local information is gathered to form more global information in the upper layers by applying each time a convolution filter of the level. The architecture of the Unsupervised Deep Self-Organizing Map is unique and retains the same principle of deep learning algorithms. This architecture can be very interesting in a Big Data environment for machine learning tasks. Experiments have been conducted to discuss how the proposed architecture shows this performance.

An analysis of environmental big data through the establishment of emotional classification system model based on machine learning: focus on multimedia contents for portal applications

With the advent of the Internet, there have been many changes. As existing means of off-line communication have transferred to the Internet, a wide range of multimedia services are emerging. And unlike in the past, these work as tremendous sources from which people’s opinions and attitudes can be obtained. Particularly, diverse opinions and reviews posted on the web in real time help corporations or the State establish the directions of policies. In this context, this study developed machine learning-based environment issue sentiment classifier in order to find out, from comments posted below news, how people recognize issues about climate change in terms of environment. Based on training data constructed by this study, a sentiment classification algorithm was constructed by applying SVM (support vector machine) and Naive Bayes, which are machine learning techniques, and a sentiment classifier was constructed by applying CNN (convolutional neural networks) and Bi-LSTM (bidirectional long-short term memory) techniques among deep learning techniques recently researched vigorously; and then their performance was compared.

SFEEC: Provably Secure Signcryption-Based Big Data Security Framework for Energy-Efficient Computing Environment

Digital networks connect an increasing number of users, sensors, and devices, which communicate, access, share, and analyze data. This phenomenon of diverse data generation is recognized as Big data. While people are benefiting from the convenience added by Big data, they also meet direct threats to privacy and data security. As Big data is massive, it needs an effective security framework with energy-efficient computing environment. Signcryption technique combines both encryption and signature to present an efficient solution. To address threats to data security, chosen cipher security and chosen message security are two important attributes. This means that it is very important to design an efficient framework where we can address these issues. This article introduces an identity-based signcryption technique by combining encryption as well as signature, which provides a solution for secure and authenticated communication in the Big data environment, called SFEEC (security framework for energy-efficient computing). SFEEC fulfills the requirements of less computation and communication overheads by offering pairing-free computation at the user end. SFEEC is also proven under the “indistinguishable against chosen ciphertext” and “secure against chosen message” attacks in the standard model.

Big Earth Observation Data Integration in Remote Sensing Based on a Distributed Spatial Framework

The arrival of the era of big data for Earth observation (EO) indicates that traditional data management models have been unable to meet the needs of remote sensing data in big data environments. With the launch of the first remote sensing satellite, the volume of remote sensing data has also been increasing, and traditional data storage methods have been unable to ensure the efficient management of large amounts of remote sensing data. Therefore, a professional remote sensing big data integration method is sorely needed. In recent years, the emergence of some new technical methods has provided effective solutions for multi-source remote sensing data integration. This paper proposes a multi-source remote sensing data integration framework based on a distributed management model. In this framework, the multi-source remote sensing data are partitioned by the proposed spatial segmentation indexing (SSI) model through spatial grid segmentation. The designed complete information description system, based on International Organization for Standardization (ISO) 19115, can explain multi-source remote sensing data in detail. Then, the distributed storage method of data based on MongoDB is used to store multi-source remote sensing data. The distributed storage method is physically based on the sharding mechanism of the MongoDB database, and it can provide advantages for the security and performance of the preservation of remote sensing data. Finally, several experiments have been designed to test the performance of this framework in integrating multi-source remote sensing data. The results show that the storage and retrieval performance of the distributed remote sensing data integration framework proposed in this paper is superior. At the same time, the grid level of the SSI model proposed in this paper also has an important impact on the storage efficiency of remote sensing data. Therefore, the remote storage data integration framework, based on distributed storage, can provide new technical support and development prospects for big EO data.

A stochastic disaster-resilient and sustainable reverse logistics model in big data environment

In this paper, a mixed-integer linear programming model is discussed to provide joint decision making for facility location and production–distribution across countries for both forward and reverse logistics. A hybrid facility network is considered for cost-cutting and equipment sharing where the facilities of forward logistics are also equipped to provide reverse logistics services. The model considers the dynamic production and storage capacity of the facilities which can be expanded if required. Furthermore, the effectiveness of the model is tested to deal with disruptions due to man-made or natural disasters. The dynamic facility allocation enables the model to withstand the demand/supply disruptions in a disaster-affected zone. Besides this, the model considers carbon emissions caused due to manufacturing, remanufacturing, repair, storage and transportation. These emissions are regulated using cap and trade policy Thus, the proposed model balances resilience and sustainability under uncertain market demand and product returns. The chance-constrained approach is used to obtain the deterministic equivalence of the stochastic demand and returns. The paper also investigates the changes in emission and production level in each country under demand and supply disruptions. The parameters of the model are mapped with the various dimensions of big data such as volume, velocity and variety. The proposed model is solved using randomly generated data sets having realistic parameters with essential big data characteristics.

AAA-WSN: Anonymous access authentication scheme for wireless sensor networks in big data environment

The data capturing and access process is an important stage in the big data applications. Most of these applications are exploited the wireless sensor networks (WSNs) to accomplish this process through the sensor nodes that are deployed in unsecure and unattended environments. Therefore, these applications are suffered from numerous security weaknesses, an adversary may exploit such weaknesses to break user’s privacy, confidentiality of sensor nodes, and control the communication channel between the network components. Consequently, the security issues for this process have attracted much interest of the researchers with the increasing spread of use such applications. The majority of the proposed authentication schemes fail to solve all existing security weaknesses simultaneously. Thus, the authentication scheme is a critical issue in the WSNs. This paper proposes anonymous access authentication scheme for wireless sensor networks in big data environments (AAA-WSN) to achieve appealing security services. Comparing with the recent WSNs authentication schemes, the AAA-WSN scheme cannot only achieve strong security services such as user anonymity and mutual authentication, but also performs the perfect forward secrecy feature with high level of efficiency. The security analysis shows that the AAA-WSN scheme is resistant to the current known attacks. Moreover, the performance analysis in terms of the storage, computations and communications costs demonstrates that the AAA-WSN scheme achieves high level of security with desirable level of efficiency comparing the recent WSNs authentication schemes.

Identification Cold start Problem Start in Matrix Factorization Method for Online Web Recommendation Systems

Online prediction based on user profile or personal history is very essential to increase the revenue in E-commerce. To identify the potential users from a large amount of data and provide effective recommendation it is a very tedious task in data mining approach. Various researchers have already done dissimilar systems using various classification mining algorithms. The basic problem all the systems having a space complexity and heavy resources required to mine that much large data. Sometimes recommendations also generate different problems life redundant recommendations, cold start Matrix factorization, duplicate frequent itemset generation, etc. To eliminate such problems we propose a detection strategy of cold start Matrix factorization method during recommendation. This approach basically designs to increase online business using effective recommendations. This research also illustrates how the system carried out from business to business as well as business to customer recommendation using the proposed recommendation algorithm. Experiment analysis has done with a large amount of transactional data and generates the recommendation to the individual user. To calculate the effectiveness of the system using different Kappa statistical analyses and shows the accuracy of recommendation in Big Data environment. After identifying previous work gaps, we proposed effective recommendations to runtime users and generate the recommendation from real-world data from actual customer-product interaction events. This research basically a combination of a data mining approach and e-commerce management with a recommender system.

Integrating Sentiment Analysis on Hybrid Collaborative Filtering Method in a Big Data Environment

Most of the traditional recommendation systems are based on user ratings. Here, users provide the ratings towards the product after use or experiencing it. Accordingly, the user item transactional database is constructed for recommendation. The rating based collaborative filtering method is well known method for recommendation system. This system leads to data sparsity problem as the user is unaware of other similar items. Web cataloguing service such as tags plays a significant role to analyse the user’s perception towards a particular product. Some system use tags as additional resource to reduce the data sparsity issue. But these systems require lot of specific details related to the tags. Existing system either focuses on ratings or tags based recommendation to enhance the accuracy. So these systems suffer from data sparsity and efficiency problem that leads to ineffective recommendations accuracy. To address the above said issues, this paper proposed hybrid recommendation system (Iter_ALS Iterative Alternate Least Square) to enhance the recommendation accuracy by integrating rating and emotion tags. The rating score reveals overall perception of the item and emotion tags reflects user’s feelings. In the absence of emotional tags, scores found in rating is assumed as positive or negative emotional tag score. Lexicon based semantic analysis on emotion tags value is adopted to represent the exclusive value of tag. Unified value is represented into Iter_ALS model to reduce the sparsity problem. In addition, this method handles opinion bias between ratings and tags. Experiments were tested and verified using a benchmark project of MovieLens dataset. Initially this model was tested with different sparsity levels varied between 0%-100 percent and the results obtained from the experiments shows the proposed method outperforms with baseline methods. Further tests were conducted to authenticate how it handles opinion bias by users before recommending the item. The proposed method is more capable to be adopted in many real world applications