Large Amount Of Real-time Data Research Articles

Low carbon energy industry and network economy prediction based on sensors and real-time data processing

The widespread use of sensors provides a large amount of real-time data for enterprises and decision-makers, providing more accurate information support for the prediction and decision-making of the network economy. With the help of Internet of Things technology, the data collected by sensors is transmitted in real time to data centers or cloud platforms. Real time data processing technology is used to clean, denoise, and analyze the data in real time, ensuring the accuracy and timeliness of the data. Perform pattern recognition and trend analysis on historical data, discover hidden patterns and correlations in the data, construct predictive and decision-making models to predict future economic trends and make reasonable decisions, continuously optimize and adjust the model to adapt to real-time data changes and dynamic changes in the economic environment, and improve the accuracy and efficiency of the model. The experimental results show that the network economy prediction and decision-making model based on sensor networks and Internet of Things technology can more accurately predict economic development trends, improve decision-making efficiency and accuracy. The large amount of data provided by sensor networks provides sufficient support for the construction and optimization of models.

Multi-dimensional early warning of the entire supply chain of power materials based on RFID technology

Early warning system needs to process a large amount of real-time data, and carry out in-depth analysis and mining of these data to identify potential risks and hidden dangers. However, existing data processing and analysis capabilities may not be able to meet this demand. To this end, a multi-dimensional early warning of the entire supply chain of power materials based on RFID technology is designed. According to the real-time failure probability of power materials, the probability of early warning accidents is calculated and classified, and the risk is graded based on these probabilities. Through the methods of bonus points, deduction points and grade evaluation, the risk early warning indicators of different stages in the whole supply chain of power materials were quantified. The objective and rationality of risk assessment can be ensured by means of comprehensive weight. A multi-dimensional early warning system based on RFID technology is established, combining multiple linear regression model and particle swarm optimization algorithm to determine the time window of multi-dimensional early warning, and carry out dynamic monitoring and early warning of supply chain. The experimental results show that the early warning effect of the design method can reach 95% and the highest early warning effect can reach 98% at 10 seconds. The average warning error is only 2.91%, and the average warning time is only 1.34 seconds, which is more accurate in identifying the number of first-level risks, second-level risks and third-level risks.

Task offloading scheme of vehicular cloud edge computing based on Digital Twin and improved A3C

In traditional vehicle edge computing research, the limited computing resources of edge servers are not fully considered. At the same time, when using deep reinforcement learning (DRL) to solve vehicle edge computing problems, it is not fully considered that DRL requires a large amount of real-time data and is prone to gradient explosion or disappearance, over-fitting and local optimal solution problems. Therefore, this article proposes a digital twin (DT)-assisted vehicular cloud edge computing offloading solution. To solve the problem of limited edge server computing resources, this article proposes a vehicle, cloud, and edge server collaboration solution. In response to the problem that DRL requires a large amount of real-time data, this paper proposes a real-time data acquisition method based on DT technology. Aiming at the gradient explosion or disappearance, overfitting, and local optimal solution problems of the asynchronous dominant actor–critic (A3C) algorithm, this paper proposes an improved algorithm of A3C, which introduces the ɛ-greedy strategy and dynamic baseline to increase the possibility of exploration and increase the generalization ability of the model through Dropout technology. During the training process, this article also uses the stochastic gradient descent (SGD) algorithm to accelerate the training process and reduce the computational complexity. The simulation results show that compared with other solutions, the improved algorithm proposed in this paper can effectively reduce the delay and energy consumption of offloading tasks at the vehicular edge cloud.

A novel dynamic risk assessment method for hazardous chemical warehouses based on improved SVM and mathematical methodologies

Effective dynamic risk assessment is crucial for identifying process hazards and preventing accidents. The rapid development of modern technology urgently requires the development of new dynamic risk assessment methods to address current societal needs, especially in situations where a large amount of real-time data is available. Consequently, this manuscript presents an improved support vector machine model (SVM), integrated mathematical modeling, to facilitate dynamic risk assessment for hazardous chemical warehouses (HCWs). Firstly, an indicator framework and a quantitative risk factor table are crafted. Then a mathematical model using knowledge graph and DEMATEL-variable weight theory (VWT) is established, which combines accident information with VWT to optimize the weight calculation and provides reliable prior knowledge for the improved SVM model. Ultimately, the improved SVM model is crafted by the establishment of an improved hybrid kernel of polynomial and radial basis function kernel, fine-tuning its hyperparameters with particle swarm optimization, and utilizing sensitivity analysis to simplify the model. The model is formulated to accommodate the intricate non-linear relationships within indicators and dynamic risk, thereby establishing dynamic risk assessment rules. Through a case study, the improved SVM model demonstrates higher precision in dynamic risk prediction compared to original models. The results substantiate that the model can provide innovative insight and methodology for the precise evaluation of dynamic risk in HCWs.

An Access Control Method Against Unauthorized and Noncompliant Behaviors of Real-Time Data in Industrial IoT

There is a large amount of real-time data, e.g. measurement data and instructions, among controllers, sensors, and actuators in the industrial IoT. These data are vulnerable to unauthorized access and tampering. In addition, once the controller is controlled by malicious code, it may send out dangerous instruction that is not compliant with the preset control process, which seriously interferes with the industrial control process. To achieve correct and undisturbed control based on real-time data, we propose an attribute-based access control method for real-time data in the industrial IoT to mitigate unauthorized access and tampering, and noncompliant operation. First, we analyze the abnormal behaviors of real-time data interaction in the industrial IoT. Second, we propose the multi-level hash identity authentication method to identify and block unauthorized access and tampering with real-time data. And then we model the timing relationship and task logic relationship of the control process into the attribute fields of the attribute-based access control method to identify and block non-compliant operations. Further, we design an access control module and display the lightweight deployment under the availability constraints of the control service. Finally, the proposed access control method is analyzed, proved, and experimented with. The results show that the proposed method can prevent unauthorized and non-compliant behaviors to field real-time data, meanwhile, it has a controllable delay and better scalability.

DEEPFAKE DETECTION USING DEEP LEARNING (CNN+LSTM)

Artificial intelligence advancements have led to the development of deepfake technology, which seriously jeopardises the integrity of visual media material. Robust detection algorithms are becoming more and more necessary as deepfake creation techniques become more complex. This study combines Long Short-Term Memory (LSTM) networks with Convolutional Neural Networks (CNNs) to present a novel method for deepfake identification. The suggested CNN+LSTM architecture makes use of LSTMs' temporal modelling capabilities and CNNs' spatial feature extraction capabilities. While the LSTM component analyses the temporal connections between frames to identify patterns suggestive of deepfake manipulation, the CNN component concentrates on capturing local features and patterns in individual frames. The combination of these two networks improves the model's capacity to identify minute anomalies and inconsistencies that are indicative of deepfake content. To extract frame-level characteristics, we use Res-Next Convolutional Neural Networks. These attributes are then used to train a Recurrent Neural Network (RNN) based on Long Short-Term Memory (LSTM) to determine whether a video has been manipulated, i.e., whether it is a deepfake or a genuine video. We intend to train our deepfake detection model on a varied set of public datasets in order to improve its real-time performance. We improve the model's adaptability by learning features from different photos. Face-Forensic++, Deepfake Detection Challenge, and Celeb-DF datasets are used to extract videos. Furthermore, to assure competitive performance in real-world scenarios, our model will be assessed against a large amount of real-time data, including the YouTube dataset. Key Words: Temporal modelling, Deepfake technology , Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNNs)

Assessment of Groundwater Potential in a Mountainous Area Using Machine Learning and GIS, Rherhaya Basin, High Atlas, Morocco

Mountainous regions are vital for recharging aquifers in plain areas downstream, but understanding the geological, hydrogeological, and climatic factors is crucial to comprehend groundwater processes in these regions. Several parameters, including lithology, topography, secondary porosity, geological structures, and climatic conditions, affect the potential of groundwater in mountainous aquifers. Traditional groundwater modeling tools face several challenges in handling large amounts of real-time data, such as extracting useful features, quantifying uncertainty, and identifying links between different variables. Recent technological advances in artificial intelligence, particularly machine learning, provide solutions for hydrogeological research and applications. This paper focuses on modeling potential zones of groundwater sources using various methodologies based on GIS, spatial remote sensing, and machine learning. The study evaluated three models, Random Forest, Support Vector Machine, and Logistic Regression, in identifying potential groundwater zones in the Rherhaya watershed. More than 200 localized spring points were needed to ensure efficient model learning. The Support Vector Machine model demonstrated the highest performance during the 70/30% split, with a ROC-AUC of 84.4% for the test data. The study identified four critical conditioning factors of groundwater potentiality, including Topographic position index, River Distance, Valley Depth, and Plane Curvature. The models also highlighted the distance to rivers as a significant factor, particularly in the upstream portion of the watershed. The very low potentiality class occupied the largest area (over 32%), followed by low (between 24 and 29%), moderate (12–19%), high (10–14%), and very high (only 9–12%) classes. Only the Support Vector Machine model predicted that 12% of the catchment area had high potential for groundwater resources, indicating its superior performance in identifying high-potential zones. The results offer valuable insights that can aid decision-makers in effectively managing water resources in vulnerable areas.

Self-Organizing Interval Type-2 Fuzzy Neural Network Compensation Control Based on Real-Time Data Information Entropy and Its Application in n-DOF Manipulator.

In order to solve the high-precision motion control problem of the n-degree-of-freedom (n-DOF) manipulator driven by large amount of real-time data, a motion control algorithm based on self-organizing interval type-2 fuzzy neural network error compensation (SOT2-FNNEC) is proposed. The proposed control framework can effectively suppress various types of interference such as base jitter, signal interference, time delay, etc., during the movement of the manipulator. The fuzzy neural network structure and self-organization method are used to realize the online self-organization of fuzzy rules based on control data. The stability of the closed-loop control systems are proved by Lyapunov stability theory. Simulations show that the algorithm is superior to a self-organizing fuzzy error compensation network and conventional sliding mode variable structure control methods in control performance.

A Distributed Sensor System Based on Cloud-Edge-End Network for Industrial Internet of Things

The Industrial Internet of Things (IIoT) refers to the application of the IoT in the industrial field. The development of fifth-generation (5G) communication technology has accelerated the world’s entry into the era of the industrial revolution and has also promoted the overall optimization of the IIoT. In the IIoT environment, challenges such as complex operating conditions and diverse data transmission have become increasingly prominent. Therefore, studying how to collect and process a large amount of real-time data from various devices in a timely, efficient, and reasonable manner is a significant problem. To address these issues, we propose a three-level networking model based on distributed sensor self-networking and cloud server platforms for networking. This model can collect monitoring data for a variety of industrial scenarios that require data collection. It enables the processing and storage of key information in a timely manner, reduces data transmission and storage costs, and improves data transmission reliability and efficiency. Additionally, we have designed a feature fusion network to further enhance the amount of feature information and improve the accuracy of industrial data recognition. The system also includes data preprocessing and data visualization capabilities. Finally, we discuss how to further preprocess and visualize the collected dataset and provide a specific algorithm analysis process using a large manipulator dataset as an example.

Privacy-Preserving-Enabled Lightweight COVID-19 Simulation Model for Mobile Intelligent Application

In order to control the first wave of COVID-19 pandemic in 2020, many models have shown effectiveness in predicting the spread of new coronary pneumonia and the different interventions. However, few models can collect large amounts of high-quality real-time data faster under the premise of protecting privacy, considering the impact of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variant and the mass vaccination program as a new intervention. Therefore, we developed a mobile intelligent application that can collect a large amount of real-time data while protecting privacy and conducted a feasibility study by defining a new COVID-19 mathematical model SEMCVRD. By simulating different intervention measures, the prediction model of the mobile intelligent application used in this article simulates the epidemic situation in the U.K. as an example. The findings are as below: the optimal intervention strategy is to suppress the intervention at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$P=3$ </tex-math></inline-formula> (intervention intensity: the average number of contacts per person per day) before the end of March 2021, then gradually release the intervention intensity at a rate of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$P+2$ </tex-math></inline-formula> , and finally release the intensity to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$P=9$ </tex-math></inline-formula> in June 2021. The COVID-19 pandemic will end at the end of June 2021, when the total number of deaths will reach 128772. This strategy will be able to balance the tradeoff between loss of life and economic loss. Compared with the official statistics released by the U.K. government on May 31, 2021, our model can accurately predict the relative error rate of the total number of cases is less than 6.9%, and the relative error rate of the total number of deaths is less than 1%. Furthermore, the model is also suitable for collecting data from countries/regions around the world.

Events Sourcing and Command Query Responsibility Segregation Based Fast Data Architecture

With the advent of Big Data, the fields of Statistics and Computer Science coexist in current information systems. In addition to this, technological advances in embedded systems, in particular Internet of Things technologies, make it possible to develop real-time applications. These technological developments are disrupting Software Engineering because the use of large amounts of real-time data requires advanced thinking in terms of software architecture. The purpose of this article is to propose an architecture unifying not only Software Engineering and Big Data activities, but also batch and streaming architectures for the exploitation of massive data. This architecture has the advantage of making possible the development of applications and digital services exploiting very large volumes of data in real time; both for management needs and for analytical purposes. This architecture was tested on COVID-19 data as part of the development of an application for real-time monitoring of the evolution of the pandemic in Côte d’Ivoire using PostgreSQL, ELasticsearch, Kafka, Kafka Connect, NiFi, Spark, Node-Red and MoleculerJS to operationalize the architecture.

Data-Representation Aware Resource Scheduling for Edge Intelligence

As a large amount of real-time data generated by billions of mobile devices nowadays, how to efficiently upload and process these data to server to realize ubiquitous intelligence at network edge is critical. A key challenge is to use limited radio resources to transmit massive data samples. Motivated by active learning, a communication-efficient way is to select most informative data samples for model training. In this article, we develop a new data scheduling scheme based on data representation. Specifically, the devices would like to transmit the most representative data samples, while the server prefers to schedule the least representative ones. A scheduling scheme is designed for such value asymmetry of data samples. The experimental results demonstrate that the proposed scheme can significantly improve the performance of machine learning.

Unsupervised Prognostics based on Deep Virtual Health Index Prediction

Prediction of the Remaining Useful Life (RUL) for industrial systems has been facilitated by the acquisition of large amounts of real-time data and the use of deep learning methods. However, the vast majority of these methods rely on the availability of extensive RUL-labeled data, which is not the case for most of real industrial applications. The goal of this paper is to show how unsupervised learning can provide alternative ways to address this issue. The proposed method is essentially made of two steps. First, a Virtual Health Index (VHI) is extracted in an unsupervised manner from the raw sensor data using a Deep Convolutional Neural Network (CNN) autoencoder. Secondly, an Long-Short Term Memory (LSTM) Encoder-Decoder predicts the future values of the VHI, until an End-of-Life (EOL) pattern is recognized (using a sliding window DTW algorithm). The suggested method is tested on the C-MAPSS dataset and offers promising results with a great potential to be applicable on real-life use cases.

An Intelligent IoT Sensing System for Rail Vehicle Running States Based on TinyML

Real-time identification of the running state is one of the key technologies for a smart rail vehicle. However, it is a challenge to accurately real-time sense the complex running states of the rail vehicle on an Internet-of-Things (IoT) edge device. Traditional systems usually upload a large amount of real-time data from the vehicle to the cloud for identification, which is laborious and inefficient. In this paper, an intelligent identification method for rail vehicle running state is proposed based on Tiny Machine Learning (TinyML) technology, and an IoT system is developed with small size and low energy consumption. The system uses a Micro-Electro-Mechanical System (MEMS) sensor to collect acceleration data for machine learning training. A neural network model for recognizing the running state of rail vehicles is built and trained by defining a machine learning running state classification model. The trained recognition model is deployed to the IoT edge device at the vehicle side, and an offset time window method is utilized for real-time state sensing. In addition, the sensing results are uploaded to the IoT server for visualization. The experiments on the subway vehicle showed that the system could identify six complex running states in real-time with over 99% accuracy using only one IoT microcontroller. The model with three axes converges faster than the model with one. The model recognition accuracy remained above 98% and 95%, under different installation positions on the rail vehicle and the zero-drift phenomenon of the MEMS acceleration sensor, respectively. The presented method and system can also be extended to edge-aware applications of equipment such as automobiles and ships.

Map-Reduce based Ensemble Intrusion Detection System with Security in Big Data

In Many domains, data is increasing day by day. Moreover, the large amount of data creates the best challenges for the users. Big data is one of the high focus domains in data science. In many organizations, big data plays a major role in processing large datasets and extracts useful information from the data. Security becomes more important to all the domains and applications. Many applications use different types of security approaches to secure sensitive data from attackers. Many real-time applications are providing security for applications such as banking, trading, and e-commerce applications using better security algorithms. Processing a large amount of real-time data gets a lot of benefits to analyzing the threats in cyber-security systems. The data is collected from online real-time sources and this cyber security data consists of network information, sensor data, threat information, analysis of intrusion detection, and identifying the sensitive data in real-time applications. In this collected information various vulnerabilities and attacks are becoming prevalent and develop security solutions accordingly. In this paper, the Map-reduce-based Ensemble Intrusion Detection System (MR-EIDS) is developed to detect intruders and attackers from the real-time datasets. Two benchmark datasets are used to analyze the intrusion data. The proposed system analyzes the dataset and finds interesting patterns in the datasets. Performance is also measured by showing improved results.

A Lightweight Authenticated Key Agreement Protocol Using Fog Nodes in Social Internet of Vehicles

Recently, there has been rapid growth in the Internet of things, the Internet of vehicles, fog computing, and social Internet of vehicles SIoV , which can generate large amounts of real-time data. Now, researchers have begun applying fog computing to the SIoV to reduce the computing pressure on cloud servers. However, there are still security challenges in SIoV . In this paper, we propose a lightweight and authenticated key agreement protocol based on fog nodes in SIoV . The protocol completes the mutual authentication between entities and generates the session key for subsequent communication. Through a formal analysis of the Burrows–Abadi–Needham (BAN) logic, real-oracle random (ROR) model, and ProVerif, the security, validity, and correctness of the proposed protocol are demonstrated. In addition, informal security analysis shows that our proposed protocol can resist known security attacks. We also evaluate the performance of the proposed protocol and show that it achieves better performance in terms of computing power and communication cost.

Designing a Secure Software-Defined Radio Transceiver using the Logistic Map

The need to exchange large amounts of real-time data is constantly increasing in wireless communication. While traditional radio transceivers are not cost-effective and their components should be integrated, software-defined radio (SDR) ones have opened up a new class of wireless technologies with high security. This study aims to design an SDR transceiver was built using one type of modulation, which is 16 QAM, and adding a security subsystem using one type of chaos map, which is a logistic map, because it is a very simple nonlinear dynamical equations that generate a random key and EXCLUSIVE OR with the originally transmitted data to protect data through the transmission. At the receiver, the data will be recovered using the same key, the received data is the same at the sender, so this result shows that the transceiver work normally and calculates the BER at the end each time the SNR changed. Our results indicate that the designed transmitter and receiver perform as usual and finally calculate the bit error rate when the signal to noise ratio is changed. The evaluation of BER indicates that simulated values are smaller than theoretical ones obtained using the BER tool. when SNR is equivalent to 7 dB, the theoretical and simulated BER obtained is 0.01695 and 0.00199, respectively. Designing soft defined radio transceiver using MATLAB R 2011a and system generator ISE14.1/SIMULINK.

Air Pollution Prediction Using an Ensemble of Dynamic Transfer Models for Multivariate Time Series

Entering a new era of big data, analysis of large amounts of real-time data is important, and air quality data as streaming time series are measured by several different sensors. To this end, numerous methods for time-series forecasting and deep-learning approaches based on neural networks have been used. However, they usually rely on a certain model with a stationary condition, and there are few studies of real-time prediction of dynamic massive multivariate data. Use of a variety of independent variables included in the data is important to improve forecasting performance. In this paper, we proposed a real-time prediction approach based on an ensemble method for multivariate time-series data. The suggested method can select multivariate time-series variables and incorporate real-time updatable autoregressive models in terms of performance. We verified the proposed model using simulated data and applied it to predict air quality measured by five sensors and failures based on real-time performance log data in server systems. We found that the proposed method for air pollution prediction showed effective and stable performance for both short- and long-term prediction tasks. In addition, traditional methods for abnormality detection have focused on present status of objects as either normal or abnormal based on provided data, we protectively predict expected statuses of objects with provided real-time data and implement effective system management in cloud environments through the proposed method.

Flexible Data Integrity Checking With Original Data Recovery in IoT-Enabled Maritime Transportation Systems

Internet of things (IoT) has emerged as a promising technology that can be widely used in various industries to realize real-time information collection, so as to improve production efficiency and reduce running costs. By combining the technology of IoT, maritime transportation systems (MTS) can prevent vessels collision, improve the efficiency of maritime transportation and reduce the loss of revenue for ports and shipbuilders. The large amount of real-time data generated in IoT-enabled MTS can be efficiently utilized to predict the future trajectories and hotspots of vessels on the sea combined with historical data. However, the maritime traffic data in MTS cannot be effectively processed in traditional big data analysis methods, and the integrity of it needs to be checked before being used to achieve the prediction of trajectories and high-density areas of vessels. In this paper, we propose a flexible data integrity checking scheme with original data recovery in IoT-enabled MTS. In the proposed scheme, the data blocks of vessels are encoded based on the technology of erasure coding. To ensure the availability of the historical data, the existence and the integrity of the data elements stored in the cloud can be checked. Moreover, the original data blocks can be recovered efficiently if the encoded data elements have been corrupted or deleted. Security analysis demonstrates that the proposed scheme can be proved to be correct and is secure against malicious attacks. Performance analysis shows that our scheme is more efficient than the previous schemes.

Securing Multimedia by Using DNA-Based Encryption in the Cloud Computing Environment

Today, the size of a multimedia file is increasing day by day from gigabytes to terabytes or even petabytes, mainly because of the evolution of a large amount of real-time data. As most of the multimedia files are transmitted through the internet, hackers and attackers try to access the users’ personal and confidential data without any authorization. Thus, maintaining a strong security technique has become a significant concerned to protect the personal information. Deoxyribonucleic Acid (DNA) computing is an advanced field for improving security, which is based on the biological concept of DNA. A novel DNA-based encryption scheme is proposed in this article for protecting multimedia files in the cloud computing environment. Here, a 1024-bit secret key is generated based on DNA computing and the user's attributes and password to encrypt any multimedia file. To generate the secret key, the decimal encoding rule, American Standard Code for Information Interchange value, DNA reference key, and complementary rule are used, which enable the system to protect the multimedia file against many security attacks. Experimental results, as well as theoretical analyses, show the efficiency of the proposed scheme over some well-known existing schemes.