Big Data Prediction Research Articles

Hybrid model of generative adversarial network and Takagi‐Sugeno for multidimensional incomplete hydrological big data prediction

SummaryThe processing of rainfall‐runoff transmission in the catchment area is a very complex phenomenon, such as temporal and spatial changes of the catchment characteristics and uncertainties of rainfall patterns. To handle this challenge, data‐driven hydrologic models emerge rapidly for the rainfall runoff prediction. However, the incomplete hydrological data constrain the development of digital hydrologic model. This article proposes a rainfall‐runoff prediction method of coupling generative adversarial network (GAN) and Takagi‐Sugeno (T‐S) fuzzy model, in which the GAN is used to generate the data for completing the incomplete hydrological data, and the T‐S fuzzy model is utilized for forecasting the rainfall runoff. The presented method is examined by real data in Huaihe basin just considering the precipitation and streamflow. Experiments show that the model combined GAN and T‐S can achieve satisfactory prediction results.

Big Data Analysis and Prediction of Traffic in Los Angeles

Big Data Analysis and Prediction of Traffic in Los Angeles

Research on the Personal Privacy in the Health Big Data

Data mining, data prediction and all-round digital monitoring of health big data make the dilemma of personal privacy control prominent. The weakening of the control of personal privacy by big data technology, the people’s data belief, the diversity of interests and the conflict of interests are the main causes of the problem of personal privacy in the context of the application of health big data. Therefore, in the application of health big data, the solution to the problem of personal privacy in the context of health big data application is to enhance the value transparency of big data technology, return and reshape humanism, and explore common values to reduce conflicts of interest.

Scope of Field: On Defining Big Data Analytics for Field Development in Research & Curricular Design

Big data increasingly drives business models, marketing methods, regulatory techniques, novel intrusion methods and national security practices. Is big data analytics a “real discipline?” If so, it is appropriate to design an ontology, essentially a comprehensive schema acknowledging sub-topics of concern and their relationships. Taxonomies are the foundation for inspiring future research and designing curricula by identifying issues, themes and inevitable gaps. Technical disciplines are primary drivers defining current practice and inspiring innovation. However, technologists devote scant attention to public policy ramifications or predictable regulatory counter-measures. Big data is the primary raw material/feedstock to machine learning and artificial intelligence (AI). AI algorithms are generally opaque, black box decision-makers operating outside traditional democratic transparency. Further, Intellectual property (IP) is under-researched in metering how big data flows through the information supply chain. Big data analytics lie at the heart of financial technology (FinTech) and regulatory technologies (RegTech). Most technical innovations cause regulatory lag (RegLag) evidenced by delayed and reactionary public policy response: (i) phenomena discovery, (ii) theorizing, (iii) interpretation of control mechanism efficacy, and (iv) deployment of adjustments to rights, duties and opportunities. This paper reviews authoritative sources enabling synthesis of big data experience and predictions of future scholarly public policy research. Structural implications emerge for curricula design in public policy incidents of data science, big data analytics, AI and machine learning.

Trends Prediction of Big Data: A Case Study based on Fusion Data

Abstract Big data is developing rapidly these years, leading people gradually to realize the enormous value and various utilities hidden behind data. Therefore, it is vital to form a deep and detailed knowledge of big data field. In this paper, we mainly collect data from different sources, cite the conclusions and discoveries from other scholars in order to get more accurate answers and predictions. Meanwhile, we analyze some fields that may be closely related to big data and may obviously influence it, such as cloud computing and AI, and hope to find the inner connection between their development and big data development. Plus, we analyze the history and current situation of big data, and make reasonable predictions towards the development of its future. Based on data, thesis from other scholars and conclusions from reports, we speculate our prediction towards the future of big data, and it can strongly support the judgement of big data development tendency.

An Efficient Parameter Adaptive Support Vector Regression Using K-Means Clustering and Chaotic Slime Mould Algorithm

Support vector regression (SVR) performs satisfactorily in prediction problems, especially for small sample prediction. The setting parameters (e.g., kernel type and penalty factor) profoundly impact the performance and efficiency of SVR. The adaptive adjustment of the parameters has always been a research hotspot. However, the substantial time cost and forecast accuracy of parameter adjustment are challenging to many scholars. The contradiction of big data prediction is especially prominent. In the paper, an SVR-based prediction approach is presented using the K-means clustering method (KMCM) and chaotic slime mould algorithm (CSMA). Eight high- and low-dimensional benchmark datasets are applied to obtain appropriate key parameters of KMCM and CSMA, and the forecast accuracy, stability performance and computation complexity are evaluated. The proposed approach obtains the optimal (joint best) forecast accuracy on 6 datasets and produces the most stable output on 3 datasets. It ranks first with a score of 0.024 in the overall evaluation. The outcomes reveal that the proposed approach is capable of tuning the parameters of SVR. KMCM, CSMA and SVR are skillfully integrated in this work and perform well. Although the performance is not outstanding in terms of stability, the proposed approach exhibits very strong performance with respect to prediction accuracy and computation complexity. This work validated the tremendous potential of the proposed approach in the prediction field.

Technology on the margins: AI and global migration management from a human rights perspective

Experiments with new technologies in migration management are increasing. From Big Data predictions about population movements in the Mediterranean, to Canada's use of automated decision-making in immigration and refugee applications, to artificial-intelligence lie detectors deployed at European borders, States are keen to explore the use of new technologies, yet often fail to take into account profound human rights ramifications and real impacts on human lives. These technologies are largely unregulated, developed and deployed in opaque spaces with little oversight and accountability. This paper examines how technologies used in the management of migration impinge on human rights with little international regulation, arguing that this lack of regulation is deliberate, as States single out the migrant population as a viable testing ground for new technologies. Making migrants more trackable and intelligible justifies the use of more technology and data collection under the guise of national security, or even under tropes of humanitarianism and development. The way that technology operates is a useful lens that highlights State practices, democracy, notions of power, and accountability. Technology is not inherently democratic and human rights impacts are particularly important to consider in humanitarian and forced migration contexts. An international human rights law framework is particularly useful for codifying and recognising potential harms, because technology and its development is inherently global and transnational. More oversight and issue-specific accountability mechanisms are needed to safeguard fundamental rights of migrants such as freedom from discrimination, privacy rights and procedural justice safeguards such as the right to a fair decision-maker and the rights of appeal.

A Granular GA-SVM Predictor for Big Data in Agricultural Cyber-Physical Systems

The connection of physical agriculture with corresponding cyber systems is helpful to achieve precision agriculture. Real-time data from agriculture sensors can provide decision supports to improve the yields and quality of agricultural products, but also bring about challenges one of which is how to mine useful information from these vast amounts of data at acceptable computation costs. To deal with the dimension disaster problem faced by most conventional mining algorithms, in this paper we combine granulation techniques and genetic algorithm (GA) with a support vector machine (SVM) to propose a granular GA-SVM. In the integrated predictor, three granulation methods, that is, Min–Median–Max granulation, Quartile–Median granulation, and fuzzy granulation, are introduced to break down big data in agricultural cyber-physical systems into small-scale granules, and GA is used to find the optimal values of SVM penalty parameter and kernel parameter from the reduced granules. Internet of Things (IoT) data from Luochuan Apple Experimental Demonstration Station in Shaanxi Province, China, verified that the proposed granular GA-SVM predictor is effective to make big data prediction with reduced computation time and equivalent accuracy. Moreover, the predicted environment information could provide guidance for growers achieving precise management of apple planting.

Financial risk assessment model based on big data

Conventional financial risk assessment is not accurate and its adaptive assessment ability is low. In order to solve this problem, a financial risk assessment model based on big data is proposed. In this method, the quantitative analysis method is adopted to analyze the explanatory variable model and the control variable model of financial risk assessment. The market-to-book ratio, asset–liability ratio, cash flow ratio and financing structure model are adopted as constraint parameters to construct a big data analysis model for financial risk assessment. On this basis, the adaptive fuzzy weighted control method is adopted for information fusion of financial risk assessment data and big data classification, and the asset income control and innovative evaluation model are adopted for linear planning and square fitting during financial risk assessment. Based on the intervention factors of financial market participants, quantitative regression analysis is performed, and according to the economic game theory, big data analysis and prediction of financial risk assessment are performed through the regression analysis method. Then the big data fusion and clustering algorithms are adopted for financial risk assessment. The simulation results show that this method can provide a relatively high accuracy in financial risk assessment, and has relatively strong adaptive evaluation capability to the risk coefficient, so it has a good application value in the prevention and control of risk factors in financial systems.

A Novel on Transmission Line Tower Big Data Analysis Model Using Altered K-means and ADQL

This study sought to propose a big data analysis and prediction model for transmission line tower outliers to assess when something is wrong with transmission line tower big data based on deep reinforcement learning. The model enables choosing automatic cluster K values based on non-labeled sensor big data. It also allows measuring the distance of action between data inside a cluster with the Q-value representing network output in the altered transmission line tower big data clustering algorithm containing transmission line tower outliers and old Deep Q Network. Specifically, this study performed principal component analysis to categorize transmission line tower data and proposed an automatic initial central point approach through standard normal distribution. It also proposed the A-Deep Q-Learning algorithm altered from the deep Q-Learning algorithm to explore policies based on the experiences of clustered data learning. It can be used to perform transmission line tower outlier data learning based on the distance of data within a cluster. The performance evaluation results show that the proposed model recorded an approximately 2.29%~4.19% higher prediction rate and around 0.8% ~ 4.3% higher accuracy rate compared to the old transmission line tower big data analysis model.

Average Restrain Divider of Evaluation Value (ARDEV) in data stream algorithm for big data prediction

Today, big data processing has become a challenging task due to the amount of data collected using various sensors increasingly significantly. To build knowledge and predict the data, traditional data mining methods calculate all numerical attributes into the memory simultaneously. The data stream method is a solution for processing and calculating data. The method streams incrementally in batch form; therefore, infrastructure memory is sufficient to develop knowledge. The existing method for data stream prediction is FIMT-DD (Fast Incremental Model Tree-Drift Detection). Using this method, knowledge is developed in tree form for every instance. In this paper, enhanced FIMT-DD is proposed using ARDEV (Average Restrain Divider of Evaluation Value). ARDEV utilizes the Chernoff bound approach with error evaluation, improvement in learning rate, modification of perceptron rule calculation, and utilization of activation function. Standard FIMT-DD separates the tree formation process and perceptron prediction. The proposed method evaluates and connects the development of the tree for knowledge formation and the perceptron rule for prediction. The prediction accuracy of the proposed method is measured using MAE, RMSE and MAPE. From the experiment performed, the utilization of ARDEV enhancement shows significant improvement in terms of accuracy prediction. Statistically, the overall accuracy prediction improvement is approximately 6.99 % compared to standard FIMT-DD with a traffic dataset.

Building energy consumption forecasting algorithm based on piecewise linear fusion and exponential spectrum analysis

In order to solve the problem of large error in traditional statistical prediction methods, a large data prediction method based on pie chart is proposed. Linear fusion and exponential spectrum analysis methods are proposed. The method establishes the target model of building energy consumption prediction and carries out nonlinear exponential sequence analysis. Game analysis of building energy consumption, segmentation linear fusion method is used to decompose the characteristics of building energy consumption map, and statistical analysis is carried out. According to the evolution of feature decomposition and learning trends, the analysis and accurate prediction of building energy consumption big data is realised. The simulation results show that the method reduces energy consumption, is conducive to building energy-saving emission reduction and green building, and provides a new idea for building energy conservation. Provide scientific support for the development of building energy conservation and environmental protection.

Tensor-Train Fuzzy Deep Computation Model for Citywide Traffic Flow Prediction

Accuracy is extensively considered a key issue for traffic big data prediction in a vehicular cyber-physical system (VCPS). Deep learning with super performance has been successfully applied to traffic prediction for feature learning. However, uncertain traffic big data pose a remarkable challenge on current deep learning models, which work in a vector space in a deterministic manner and fail to learn the features of uncertain traffic data. This study solves the problem of citywide traffic flow prediction to satisfy the accuracy requirement of the VCPS from the perspective of users. In this study, a tensor-train fuzzy deep convolution (TFDC) approach is first proposed to satisfy the accuracy requirements of traffic flow prediction. Moreover, the Tucker deep computation (T-TFDC) approach for TFDC is proposed to satisfy the prediction accuracy requirements with low computational complexity. The TFDC model is built upon the fuzzy deep convolutional network, which uses unified tensor data representation for spatio-temporal traffic flow data. The key idea of the T-TFDC is to introduce Tucker decomposition into the TFDC model to compress parameters for traffic flow feature learning. Furthermore, learning algorithms for training TFDC and T-TFDC model parameters are devised on the basis of the back-propagation strategy. Experimental results on the TaxiBJ and BikeNYC data sets verify the effectiveness and efficiency of the proposed approaches over state-of-the-art methods.

Big data‐driven machine learning‐enabled traffic flow prediction

AbstractReal‐time effective traffic flow big data prediction network has important application significance. Over the past few years, traffic flow data have been exploding and we have entered the big data era. The key challenge of traffic flow prediction network is how to construct an adaptive model relying on historical data. Existing big data‐driven traffic flow prediction networking approaches mainly use shallow learning, and there are unsatisfying for many realistic applications, which inspire us to rethink the traffic flow big data prediction problem with deep learning. In this paper, we propose a novel prediction approach based on machine learning. In addition to the minimum prediction error as the goal, we present the long short‐term memory model, which is a typical machine learning algorithm with deep learning network. This method is applied into the real‐world traffic big data from performance measurement system. Experimental results show that the proposed machine learning algorithm has more applicability and higher performance, compared with shallow machine learning prediction network.

Exploring the participate propensity in cyberspace collective actions: The 5‰ rule

The Internet and Big data have become indispensable in people’s daily life. The whole participation spectrum of cyberspace collective actions contains four stages, such as Access, Browse, Participate, and Offline. There exist three transition probabilities within four stages. This paper focuses on the ratio (second transition probability) between numbers of browse and participate, which is defined as the participate propensity PBP. In the real world, amounts of browse (millions) and participation (tens of thousands) are huge, and they take on irregular distributions. However, it is discovered in this paper that this participate propensity is stable and takes on the regularity of 5‰, i.e. the participation propensity is slightly under 5‰ for most times, while sometimes it is slightly over 5‰. The empirical big data of 310 online collective actions are collected from a famous BBS (Tianya.cn) in China. This 5‰ rule not only holds true for the total 310 cases, but also for the yearly, quarterly, and other subgroups. Furthermore, we check distributive traits of PBP. It is not normally distributed as it has longer right tails. It is verified by the empirical big data that the propensity follows the lognormal distribution, which is relatively robust in the total and subgroups of 310 cyberspace collective actions. Given the distributive regularity of the participate propensity PBP, its probability density function can be obtained. Combined with the known amount of browsers, big data prediction of participation would be possible.

A Big Data Prediction Framework for Weather Forecast Using MapReduce Algorithm

A Big Data Prediction Framework for Weather Forecast Using MapReduce Algorithm

Big data prediction of durations for online collective actions based on peak’s timing

Peak Model states that each collective action has a life circle, which contains four periods of “prepare”, “outbreak”, “peak”, and “vanish”; and the peak determines the max energy and the whole process. The peak model’s re-simulation indicates that there seems to be a stable ratio between the peak’s timing (TP) and the total span (T) or duration of collective actions, which needs further validations through empirical data of collective actions. Therefore, the daily big data of online collective actions is applied to validate the model; and the key is to check the ratio between peak’s timing and the total span. The big data is obtained from online data recording & mining of websites. It is verified by the empirical big data that there is a stable ratio between TP and T; furthermore, it seems to be normally distributed. This rule holds for both the general cases and the sub-types of collective actions. Given the distribution of the ratio, estimated probability density function can be obtained, and therefore the span can be predicted via the peak’s timing. Under the scenario of big data, the instant span (how long the collective action lasts or when it ends) will be monitored and predicted in real-time. With denser data (Big Data), the estimation of the ratio’s distribution gets more robust, and the prediction of collective actions’ spans or durations will be more accurate.

A basis of spatial big data analysis with map-matching system

The data collection of vehicle trajectories becomes the basis of big data analysis and prediction for a variety of purposes, such as vehicle navigation and movement analysis. A digital tachograph (DTG) is pre-installed on most commercial vehicles in South Korea and is highly valuable for analyzing eco-driving metrics such as safe driving and fuel consumption estimates. In order to properly analyze a large amount of GPS location information, it is necessary to find the exact match of the location data in space to the link in the digital road network data. We previously discovered the road information of the GPS coordinates using the commonly utilized map-matching technique. However, such a navigation map-matching technique requires a lot of supplementary corrections in order to rapidly and accurately navigate a large amount of data. In this study, we applied enhanced map-matching logics with Geohash as spatial index, long link vertex division, speed filtering, azimuth filtering, and map-matching weight logics. Also, we established and implemented a distributed analysis environment for the big data map-matching with HBase (a Hadoop-based NoSQL DB). This paper shows a spatial analysis system using the map-matching logics on the Hadoop MapReduce mechanism, which improved its performance.

Development of Advanced Manufacturing Cloud of Things (AMCoT)—A Smart Manufacturing Platform

As semiconductor manufacturing processes are becoming more and more sophisticated, how to maintain their feasible production yield becomes an important issue. Also, how to build a smart manufacturing platform that can facilitate realizing smart factories is essential and desirable for current manufacturing industries. Aimed at addressing the above-mentioned two issues, in this letter, a five-stage approach for enhancing and assuring yield is proposed. Also, a smart manufacturing platform- Advanced Manufacturing Cloud of Things (AMCoT) based on Internet of Things, cloud computing, big data analytics, cyber-physical systems, and prediction technologies is designed and implemented to realize the proposed five-stage approach of yield enhancement and assurance. Finally, AMCoT is applied to a bumping process of a semiconductor company in Taiwan to conduct industrial case studies. Testing results demonstrate that AMCoT possesses capabilities of conducting total inspection in production, providing prognosis, and predictive maintenance on equipment, finding the root cause of yield loss, and storing and handling big production data, which as a whole is promising to achieve the goal of zero defects.

Big Data & Litigation: Analyzing The Expectation of Lawyers to Provide Big Data Predictions when Advising Clients

"This article intends to provide a background of big data and law, and to provide insights on the interaction between professional legal ethics and big data analytics, i.e. whether a lawyer can be disciplined for failing to use big data analytics in litigation cases. While most references in this article will be made to developments in the US legal technology/legal industry scene, this article will also provide a short segment on general developments of big data and law in the developing world. Ultimately, this article hopes to shed light on what litigators may expect from the use of this technology that is gaining traction in the legal industry."