Data Recovery Accuracy Research Articles

A Reinforcement Learning-Based Dynamic Clustering of Sleep Scheduling Algorithm (RLDCSSA-CDG) for Compressive Data Gathering in Wireless Sensor Networks

Energy plays a major role in wireless sensor networks (WSNs), and measurements demonstrate that transmission consumes more energy than processing. Hence, organizing the transmission process and managing energy usage throughout the network are the main goals for maximizing the network’s lifetime. This paper proposes an algorithm called RLDCSSA-CDG, which is processed through the 3F phases: foundation, formation, and forwarding phases. Firstly, the network architecture is founded, and the cluster heads (CHs) are determined in the foundation phase. Secondly, sensor nodes are dynamically gathered into clusters for better communication in the formation phase. Finally, the transmitting process will be adequately organized based on an adaptive wake-up/sleep scheduling algorithm to transmit the data at the “right time” in the forwarding phase. The MATLAB platform was utilized to conduct simulation studies to validate the proposed RLDCSSA-CDG’s effectiveness. Compared to a very recent work called RLSSA and RLDCA for CDG, the proposed RLDCSSA-CDG reduces total data transmissions by 22.7% and 63.3% and energy consumption by 8.93% and 38.8%, respectively. It also achieves the lowest latency compared to the two contrastive algorithms. Furthermore, the proposed algorithm increases the whole network lifetime by 77.3% and promotes data recovery accuracy by 91.1% relative to the compared algorithms.

Enzymes as green and sustainable tools for DNA data storage.

DNA is considered as an ideal supramolecular material for information storage with high storage density and long-term stability. Enzymes, as green and sustainable tools, offer several unique advantages for DNA-based information storage. These advantages include low cost and reduced generation of hazardous wastes during DNA synthesis, as well as the improvements in data reading speed and data recovery accuracy. Moreover, enzymes could achieve scalable data steganography. In this review, we introduced the exciting application strategies of enzymatic tools in each step of DNA information storage (writing, storing, retrieval and reading). We further address the challenges and opportunities associated with enzymatic tools for DNA information storage, aiming at developing new techniques to overcome these obstacles.

Mobile forensics tools and techniques for digital crime investigation: a comprehensive review

<p>Extracting and analyzing data from smartphones, IoT devices, and drones is crucial for conducting digital crime investigations. Effective cyberattack mitigation necessitates the use of advanced Android mobile forensics techniques. The investigation necessitates proficiency in manual, logical, hex dump, chip-off, and microread methodologies. This paper provides a comprehensive overview of Android mobile forensics tools and techniques for digital crime investigation, as well as their use in gathering and analyzing evidence. Forensic software tools like Cellebrite UFED, Oxygen Forensic Detective, XRY by MSAB, Magnet AXIOM, SPF Pro by SalvationDATA, MOBILedit Forensic Express, and EnCase Forensic employ both physical and logical techniques to retrieve data from mobile devices. These advanced tools offer a structured approach to tackling digital crimes effectively. We compare dependability, speed, compatibility, data recovery accuracy, and reporting. Mobile-network forensics ensures data acquisition, decryption, and analysis success. Conclusions show that Android mobile forensics tools for digital crime investigations are diverse and have different capabilities. Mobile forensics software offers complete solutions, but new data storage and encryption methods require constant development. The continuous evolution of forensic software tools and a comprehensive tool classification system could further enhance digital crime investigation capabilities.</p>

Volatility-based diversity awareness for distributed data storage of Mobile Crowd Sensing

With the widespread utilization of sensors and the rapid development of Mobile Crowd Sensing (MCS), the framework of distributed data storage, which makes use of the limited storage capacity of sensing devices, is providing an alternative to cloud-based data storage. Since the construction of the distributed storage framework for MCS has to be completed by Compressed Sensing (CS) theory, a reasonable measurement allocation becomes a no-brainer if we want to obtain more recovery precision of data without wasting measurement resources. To more efficiently address the above problems, we present a novel strategy of Volatility-Based Diversity Awareness for CS measurement allocation called VBDA, which collectively achieves lower computational complexity, adaptive sampling rate allocation for non-visual data, and high-quality reconstruction. We first process the target monitoring region in blocks. Then, we calculate the degree of data diversity of each area using the volatility, which is used to assess the importance of the different areas. Worthy note, based on the volatility calculation, we can obtain the magnitude of the variation in diversity that is only present in the real data, even when using very ambiguous recovered data. This is primarily due to the volatility unique design concept, which attempts to offset partially identical errors in adjacent recovered data using volatility calculations. Finally, to rationally allocate measurement resources, a volatility-based sampling rate allocation scheme is proposed. We further provide an implementation for practical deployments of VBDA in various related contexts. Numerous experimental show that the VBDA performs excellently. In comparison to the current state-of-the-art strategy, it improves data recovery accuracy by 12.2% without the use of any prior knowledge, while being adaptable to many different distribution types of data.

Abnormal data detection and recovery of sensors network based on spatiotemporal deep learning methodology

This paper proposes a novel deep-learning-enabled framework for abnormal data detection and recovery considering spatial-temporal correlation among sensors. The wavelet scattering transform is adopted to replace the first layer of one-dimension convolutional neural network classification model. The multiple typical sensor failure modes of sensor are identified including bias, drifting, gain, precision degradation, and three complete faults (constant; constant + noise; noise). The bidirectional long-short time memory (BLSTM) network is developed to recover abnormal data through positive and negative propagation. The BLSTM cell involving forget, input, and output gate is utilized to adaptively transmit valuable information of long-term data. Different sensor configurations are conducted to select the optimal input sensors of BLSTM model. The proposed models are then validated by three engineering cases. The results show that the failure mode diagnosis and recovery accuracy are over 90 %. Sensor selection has a large impact on the data reconstruction, and selecting sensors with higher correlation can achieve a higher accuracy. In addition, the data recovery accuracy of the proposed BLSTM model surpasses that of traditional long-short time memory models.

A reinforcement learning-based sleep scheduling algorithm for compressive data gathering in wireless sensor networks

Compressive data gathering (CDG) is an adequate method to reduce the amount of data transmission, thereby decreasing energy expenditure for wireless sensor networks (WSNs). Sleep scheduling integrated with CDG can further promote energy efficiency. Most of existing sleep scheduling methods for CDG were formulated as centralized optimization problems which introduced many extra control message exchanges. Meanwhile, a few distributed methods usually adopted stochastic decision which could not adapt to variance in residual energy of nodes. A part of nodes were prone to prematurely run out of energy. In this paper, a reinforcement learning-based sleep scheduling algorithm for CDG (RLSSA-CDG) is proposed. Active nodes selection is modeled as a finite Markov decision process. The mode-free Q learning algorithm is used to search optimal decision strategies. Residual energy of nodes and sampling uniformity are considered into the reward function of the Q learning algorithm for load balance of energy consumption and accurate data reconstruction. It is a distributed algorithm that avoids large amounts of control message exchanges. Each node takes part in one step of the decision process. Thus, computation overhead for sensor nodes is affordable. Simulation experiments are carried out on the MATLAB platform to validate the effectiveness of the proposed RLSSA-CDG against the distributed random sleep scheduling algorithm for CDG (DSSA-CDG) and the original sparse-CDG algorithm without sleep scheduling. The simulation results indicate that the proposed RLSSA-CDG outperforms the two contrast algorithms in terms of energy consumption, network lifetime, and data recovery accuracy. The proposed RLSSA-CDG reduces energy consumption by 4.64% and 42.42%, respectively, compared to the DSSA-CDG and the original sparse-CDG, prolongs life span by 57.3%, and promotes data recovery accuracy by 84.7% compared to the DSSA-CDG.

Load image inpainting: An improved U-Net based load missing data recovery method

Dealing with large percentage data missing is always a challenge for load data recovery. This paper, drawing on ideas from image inpainting, formulates load missing data recovery problem as load image inpainting and the improved U-Net is proposed to restore the load image. First, the 1-dimensional load data is constructed into 2-dimensional load image. Additionally, missing data presents as irregular black holes on the load image. Then, the improved U-Net which introduces residual network (ResNet) and convolutional block attention module (CBAM) is proposed to specifically restore the incomplete load image. Meanwhile, mean absolute error (MAE) and structural similarity (SSIM) are utilized to evaluate data recovery accuracy and load pattern recovery similarity respectively. Load missing data recovery results based on the actual industrial load dataset are presented to verify the effectiveness of the proposed method.

Traffic Data Recovery From Corrupted and Incomplete Observations via Spatial-Temporal TRPCA

Traffic information can be used for real-time traffic management and long-term transportation planning to increase traffic efficiency and safety. However, data containing both missing and deviating values, can seriously affect the accuracy of traffic information, even leading to incorrect results in traffic data analysis. In this paper, we propose a novel tensor-based data recovery method named spatial-temporal tensor robust principal component analysis (ST-TRPCA) to recover traffic data from corrupted and incomplete observations. Specifically, we not only fully account for the spatial-temporal properties of traffic data to increase the data recovery accuracy, but also utilize tensor factorization and its low-dimensional representation to improve computational efficiency. The extensive experimental results performed on real-world traffic dataset under various scenarios show that ST-TRPCA outperforms other state-of-the-art methods in both missing data recovery and anomaly detection, especially when the traffic data are severely corrupted.

Probabilistic Statistics-Based Endurance Life Prediction of Bridge Structures.

With the massive construction of bridge infrastructures, bridge health monitoring systems have gradually matured in application and research, but previous research has primarily focused on structural damage detection and bridge safety warnings based on valid data. The structural details of steel bridge panels and structural systems are determined by the coupling effects of many intrinsic and extrinsic uncertainties, such as material properties, structural characteristics, manufacturing processes, and random traffic loads. The evaluation of fatigue is a difficult task. This article first builds a big data platform, utilizing its high-efficiency parallel computing capability and highly fault-tolerant distributed file system to achieve second-level monitoring data processing; ensuring real-time data cleaning, data analysis, and safety warning; and building a big data analysis and processing platform with high reliability, high availability, high storage efficiency, and high scalability of bridge health monitoring. The big data platform chooses HDFS for offline data storage and Spark for data analysis and modelling after comparing and analysing the benefits and drawbacks of various big data technologies. Kafka is used for caching real-time data, and Spark-streaming is used for reading data and real-time processing. Finally, the platform's superiority and reliability in terms of offline computing performance, real-time online performance, scalability, and fault tolerance are confirmed through experimental analysis; the optimal data cleaning method is derived by comparing and analysing monitoring data noise, jump point, and drift phenomena. This part of the research is based on bridge temperature data with stable signals and bridge strain data with fluctuating signals, taking into account the influence of different data types; the corresponding data missing repair algorithms are proposed for different types of data to form a complete and general data patching method process. The probabilistic fracture mechanics theory, in comparison to the traditional deterministic fatigue assessment method, can better reflect the essential uncertainty of fatigue problems and is an effective way to assess the fatigue performance of orthotropic steel bridge decks. The goal of data patching is to ensure data recovery accuracy of over 90%, with no patching repair required for monitoring data with too much missing data. The endurance life of bridge structures is predicted using a big data probabilistic statistics approach based on a variety of factors such as material properties, construction characteristics, manufacturing processes, and random traffic loads.

A Recovery Algorithm of Power Quality Big Data Based on Improved Differential Kriging

The integrity of power quality big data directly affects the state sensing accuracy and operation safety of the power system. Therefore, the recovery algorithm of power quality big data based on improved differential Kolding is studied to improve the big data recovery effect. The trend turning point is used to divide the time series of power quality big data, and the characteristic matrix of time series is constructed. The recovery model of power quality big data is built according to the characteristic matrix. By improving the differential Kriging solution model, the estimated value of the data to be recovered is obtained and the big data recovery is completed. Experimental results show that the convergence speed is the fastest when the initial scaling factor is 0.3. The algorithm can effectively recover the big data of random missing and continuous missing. In different fault recovery scenarios, the signal-to-noise ratio (SNR) is high, the structure similarity value is high, the data recovery accuracy is accurate, and the integrity of the restored data is better.

Optimum wavelet selection for nonparametric analysis toward structural health monitoring for processing big data from sensor network: A comparative study

A critical problem encountered in structural health monitoring of civil engineering structures, and other structures such as mechanical or aircraft structures, is how to convincingly analyze the nonstationary data that is coming online, how to reduce the high-dimensional features, and how to extract informative features associated with damage to infer structural conditions. Wavelet transform among other techniques has proven to be an effective technique for processing and analyzing nonstationary data due to its unique characteristics. However, the biggest challenge frequently encountered in assuring the effectiveness of wavelet transform in analyzing massive nonstationary data from civil engineering structures, and in structural health diagnosis, is how to select the right wavelet. The question of which wavelet function is appropriate for processing and analyzing the nonstationary data in civil engineering structures has not been clearly addressed, and no clear guidelines or rules have been reported in the literature to show how the right wavelet is chosen. Therefore, this study aims to address an important question in this regard by proposing a new framework for choosing a proper wavelet that can be customized for massive nonstationary data analysis, disturbances separation, and extraction of informative features associated with damage. The proposed method takes into account data type, data and wavelet characteristics, similarity, sharing information, and data recovery accuracy. The novelty of this study lies in integrating multi-criteria which are associated directly with features that correlated well with change in structures due to damage, including common criteria such as energy, entropy, linear correlation index, and variance. Also, it introduces and considers new proposed measures, such as wavelet-based nonlinear correlation such as cosh spectral distance and mutual information, wavelet-based energy fluctuation, measures-based recovery accuracy, such as sensitive feature extraction, noise reduction, and others to evaluate various base wavelets’ function capabilities for appropriate decomposition and reconstruction of structural dynamic responses. The proposed method is verified by experimental and simulated data. The results revealed that the proposed method has a satisfactory performance for base wavelet selection and the small order of Daubechies and Symlet provide the best results, especially order 3. The idea behind our proposed framework can be applied to other structural applications.

Contention-based nonorthogonal massive access with massive MIMO

The sporadic communication character of massive machine-type communication systems provides natural advantages to utilize the principle of compressive sensing (CS). However, due to the high computational complexity of CS algorithms, CS-based contention-free access schemes have limited scalability and high computational complexity for massive access with user-specific pilots. To address these problems, in this paper, we propose a new contention-based scheme for CS-based massive access, which can support the sporadic access of massive devices (more than one million devices) with limited resources. Furthermore, an advanced receiver algorithm is designed to solve the optimal solutions for the proposed scheme, which utilizes various prior information to enhance the performance. In specific, the joint sparsity between the channel and data is used to improve the accuracy of pilot detection, and the information of modulation and cyclic redundancy check is exploited for channel correction to improve the performance of data recovery. The simulation results show that the proposed scheme can achieve improved active user detection performance and data recovery accuracy than existing methods.

Accurate and Fast Recovery of Network Monitoring Data With GPU-Accelerated Tensor Completion

Monitoring the performance of a large network would involve a high measurement cost. To reduce the overhead, sparse network monitoring techniques may be applied to select paths or time intervals to take the measurements, while the remaining monitoring data can be inferred leveraging the spatial-temporal correlations among data. The quality of missing data recovery, however, highly relies on the specific inference technique adopted. Tensor completion is a promising technique for more accurate missing data inference by exploiting the multi-dimensional data structure. However, data processing for higher dimensional tensors involves a large amount of computation, which prevents conventional tensor completion algorithms from practical application in the presence of large amount of data. This work takes the initiative to investigate the potential and methodologies of performing parallel processing for high-speed and high accuracy tensor completion over Graphics Processing Units (GPUs). We propose a GPU-accelerated parallel Tensor Completion scheme (GPU-TC) for accurate and fast recovery of missing data. To improve the data recovery accuracy and speed, we propose three novel techniques to well exploit the tensor factorization structure and the GPU features: grid-based tensor partition, independent task assignment based on Fisher-Yates shuffle, sphere facilitated and memory-correlated scheduling. We have conducted extensive experiments using network traffic trace data to compare the proposed GPU-TC with the state of art tensor completion algorithms and matrix-based algorithms. The experimental results demonstrate that GPU-TC can achieve significantly better performance in terms of two relative error ratio metrics and computation time.

Gradient-based adaptive modeling for IoT data transmission reduction

Spatial and temporal correlation between sensor observations in an Internet of Things environment can be exploited to eliminate unnecessary transmissions. Transmitting less data certainly contributes to meet the growing need for energy-saving and robust transmissions, thus prolong the lifespan of the entire WSN. Spatiotemporal correlation-based dual prediction (DP) and data compression (DC) schemes aim to reduce the amount of data transmission while ensuring data accuracy. In practice, however, the existing methods restrict the stability of the system when the model hyper-parameters are uncertain. Thus adaptive model has lately attracted extensive attention for the development of resource-constrained WSN. In this paper, we propose a gradient-based adaptive model that implements both schemes in a two-tier data reduction framework. To the best of our knowledge, the proposed scheme is the first attempt to introduce adaptiveness into both the DP and DC schemes by using a simple gradient optimization method. Gradient-based Optimal Step-size LMS (GO-LMS) is introduced to make the DP aspects adaptive, while a Gradient-based Adaptive PCA (GA-PCA) approach is used for the DC aspects. The Barzilai–Borwein method is incorporated into the gradient optimization to enable adaptive computation of the step-size for each iteration. Through extensive simulations, the developed framework was found to outperform other state-of-the-art schemes in terms of both the transmission reduction ratio and data recovery accuracy.

Accurate and Fast Recovery of Network Monitoring Data: A GPU Accelerated Matrix Completion

Gaining a full knowledge of end-to-end network performance is important for some advanced network management and services. Although it becomes increasingly critical, end-to-end network monitoring usually needs active probing of the path and the overhead will increase quadratically with the number of network nodes. To reduce the measurement overhead, matrix completion is proposed recently to predict the end-to-end network performance among all node pairs by only measuring a small set of paths. Despite its potential, applying matrix completion to recover the missing data suffers from low recovery accuracy and long recovery time. To address the issues, we propose MC-GPU to exploit Graphics Processing Units (GPUs) to enable parallel matrix factorization for high-speed and highly accurate Matrix Completion. To well exploit the special architecture features of GPUs for both task independent and data-independent parallel task execution, we propose several novel techniques: similar OD (origin and destination) pairs reordering taking advantage of the locality-sensitive hash (LSH) functions, balanced matrix partition, and parallel matrix completion. We implement the proposed MC-GPU on the GPU platform and evaluate the performance using real trace data. We compare the proposed MC-GPU with the state of the art matrix completion algorithms, and our results demonstrate that MC-GPU can achieve significantly faster speed with high data recovery accuracy.

PMU Missing Data Recovery Using Tensor Decomposition

The paper proposes a new approach for the recovery of missing data from phasor measurement units (PMUs). The approach is based on the application of tensor decomposition to PMU data organized as three-dimensional tensors with respect to time, location and type of variables. The organization of the PMU data is in the form of a bus-oriented data structure and a branch-oriented data structure. Two versions of the approach are introduced. The first version uses polyadic tensor decomposition and the second version uses Tucker tensor decomposition. Several case studies are conducted validating the proposed approaches including measured PMU data in the New York transmission system. Comparison with existing approaches is performed to demonstrate the improvement in missing data recovery accuracy achieved by the new approach.

Edge Computing-Empowered Large-Scale Traffic Data Recovery Leveraging Low-Rank Theory

Intelligent Transportation Systems (ITSs) have been widely deployed to provide traffic sensing data for a variety of smart traffic applications. However, the inevitable and ubiquitous missing data potentially compromises the performance of ITSs and even undermines the traffic applications. Therefore, accurate and real-time traffic data recovery is crucial to ITSs and its related services, especially for large-scale traffic networks. To leverage the characteristics in transportation networks for data recovery, we first conduct experimental explorations on a large-scale traffic dataset of an ITS and further quantify the spatiotemporal correlations of traffic data. Inspired by the observation results, we propose GTR , an edGe computing-empowered system for large-scale Traffic data recovery with low-Rank theory. GTR leverages the decentralized computing power of edge nodes to process massive traffic data from hundreds of traffic stations for accurate and real-time recovery. Specifically, we first propose a suboptimal edge node deployment algorithm with a theoretical performance guarantee, by exploiting the supermodularity in the NP-hard joint-optimization problem. Furthermore, to leverage the low-rank nature of traffic data, we transform the data recovery problem into a low-rank minimization problem, then utilize the fixed-point continuation iterative scheme to capture spatiotemporal correlations for accurate traffic recovery. Finally, the extensive trace-driven evaluations show that GTR only needs at most 5.7% extra total cost compared to the optimal deployment, while outperforming four baseline methods by 63.8% improvement in terms of traffic data recovery accuracy.

Accurate Recovery of Missing Network Measurement Data With Localized Tensor Completion

The inference of the network traffic data from partial measurements data becomes increasingly critical for various network engineering tasks. By exploiting the multi-dimensional data structure, tensor completion is a promising technique for more accurate missing data inference. However, existing tensor completion algorithms generally have the strong assumption that the tensor data have a global low-rank structure, and try to find a single and global model to fit the data of the whole tensor. In a practical network system, a subset of data may have stronger correlation. In this work, we propose a novel localized tensor completion model (LTC) to increase the data recovery accuracy by taking advantage of the stronger local correlation of data to form and recover sub-tensors each with a lower rank. Despite that it is promising to use local tensors, the finding of correlated entries faces two challenges, the data with adjacent indexes are not ones with higher correlation and it is difficult to find the similarity of data with missing tensor entries. To conquer the challenges, we propose several novel techniques: efficiently calculating the candidate anchor points based on locality-sensitive hash (LSH), building sub-tensors around properly selected anchor points, encoding factor matrices to facilitate the finding of similarity with missing entries, and similarity-aware local tensor completion and data fusion. We have done extensive experiments using real traffic traces. Our results demonstrate that LTC is very effective in increasing the tensor recovery accuracy without depending on specific tensor completion algorithms.

Layered adaptive compression design for efficient data collection in industrial wireless sensor networks

Existing compressed sensing (CS)-based spatiotemporal data compression schemes can significantly decrease communication consumption for data collection; however, they ignore data correlation among different clusters over spatial dimensions. To explore data correlation among different clusters and satisfy the requirement of high data precision in industrial applications, in this paper, we propose a layered adaptive compression design for efficient data collection (LACD-EDC) in industrial wireless sensor networks (IWSNs). In the proposed scheme, first, we design a multilayer network architecture to support the exploration of spatiotemporal correlations, especially spatial correlation among different clusters. Then, we construct specific projection methods for exploring temporal correlation in sensory nodes, spatial correlation (intracluster) in cluster heads and spatial correlation (intercluster) in processing nodes. In addition, a detailed solution method is developed to recover the original data and achieve approximate data collection in the sink node. Subsequently, sparsifying dictionaries are trained for adapting different types of data and obtaining better sparse representations, which further improves the data recovery accuracy. Our simulation results indicate that the proposed layered adaptive compression scheme offers better recovery performance than conventional clustered compression schemes (i.e., achieving efficient data collection with high quality).

ComNet: Combination of Deep Learning and Expert Knowledge in OFDM Receivers

In this article, we propose a model-driven deep learning (DL) approach that combines DL with the expert knowledge to replace the existing orthogonal frequency-division multiplexing (OFDM) receiver in wireless communications. Different from the data-driven fully connected deep neural network (FC-DNN) method, we adopt the block-by-block signal processing method that divides the receiver into channel estimation subnet and signal detection subnet. Each subnet is constructed by a DNN and uses the existing simple and traditional solution as initialization. The proposed model-driven DL receiver offers more accurate channel estimation comparing with the linear minimum mean-squared error (LMMSE) method and exhibits higher data recovery accuracy comparing with the existing methods and FC-DNN. Simulation results further demonstrate the robustness of the proposed approach in terms of signal-to-noise ratio and its superiority to the FC-DNN approach in the computational complexities or the memory usage.