Redundant Information In Data Research Articles

Application of sensor technology in grasping and preprocessing of network hotspot information propagation

The rapid dissemination of hot information on the Internet has become a common phenomenon in today's society. Traditional methods of information capture and preprocessing often require a lot of manpower and material resources, and the captured information has low timeliness and accuracy. The purpose of this paper was to use sensor technology to find and locate network hotspots in time. By collecting user generated content, social media data, news reports, etc., the data is analyzed and mined to identify popular topics and events. In terms of information capture, sensor technology can monitor and understand user activities, the popularity of posts, emotional tendencies, user attention, user interaction, etc., through information monitoring. Network data was collected, such as network latency, data loss rate, and bandwidth utilization. Sensor technology was used to collect social media data to understand the level of public attention to hot events. In information preprocessing, sensor technology was used to remove noise and redundant information in data to ensure data quality. The data were labeled and classified, and the information dissemination rules of network hotspot were analyzed in depth. The average capture accuracy of Method 1 for Hotspot 1, Hotspot 2, and Hotspot 3 was 72.11%, 71.81%, and 72.54%, respectively. The average capture accuracy of Method 2 for Hotspot 1, Hotspot 2, and Hotspot 3 was 82.55%, 83.14%, and 82.91%, respectively. When the data was 40, 80, and 120, the preprocessing times of Method 1 for Post 1 were 8.81 s, 15.47 s, and 18.77 s, respectively. The preprocessing times of Method 2 for Post 1 were 5.97 s, 7.80 s, and 9.25 s, respectively. The application of sensor technology in the capture and preprocessing of network hot information dissemination has brought a variety of innovations, including multi-modal data acquisition, real-time monitoring and analysis, user behavior analysis, data cleaning and integration, anomaly detection and early warning, intelligent recommendation and personalized service, etc., thus improving the accuracy, real-time and personalized degree of information acquisition.

Neural-Network-Assisted Packet Accelerators for Internet of Things Network Systems

Major device nodes within the internet of things (IoT) system collects and store information in bit forms of 0’s and 1’s regardless of its repetition. The nodes do not possess the capability of processing redundant data information except for outright rejection/replacement of packets of similar sizes. This becomes a research problem since high volumes of packet redundancy are prevalent owing to repetitive information. Many optimal solutions have been provided to reprocess redundant packets and to store them in edge server for accessibility by other connected IoT systems and networks servers. To do so, major IoT platforms implements tier-based network layers which primarily aid seamless communication among nodes. These network layers perform near-similar tasks of guaranteeing packet sensing and exchange although at often-higher energy requirement. To mitigate the energy concerns, packets are clustered and compressed, allowing exchange of essential information only. But the approach continues to present heavy packet-losses and/or redundancies. In this paper, two-tier layered network | where packet exchange is conducted at the top layer in order to lower energy consumption and promote system-reliability is investigated. All packets are first segregated into multiple clusters using the Voronoi cell-based correlation cluster formation (VC3F) technique. Cluster heads (CHs) are identified by their multi-path (M-Score) value, thus, assuming sole responsibility of redundant packet removal within each cluster. The redundant packets are then moved to the edge-tier layer using optimized multi-objective flower pollination (MO-FPO) routing, and finally processed using hybrid models of novel fast-fully connected neural network (F2CNN) accelerator and Lempel-Ziv-Welch (LZW) data compression. The F2CNN and LZW models are harmonized to further collectively explore potential benefits of both models. These benefits include the neural capability to work on the sensitivity level of the packets determined by the packet classification and validation approaches. The system is evaluated with detailed experimental investigations where higher system throughput, packet delivery ratio (PDR), end-to-end delay and system reliability are corroborated.

Mechanical equipment fault diagnosis based on wireless sensor network data fusion technology

Abstract To save network energy consumption and prolong network life cycle in complex mechanical fault diagnosis, a research method of data fusion routing protocol algorithm based on wireless sensor network (WSN) is proposed. The specific content of the method is as follows: First, the low-energy adaptive clustering hierarchy algorithm is analyzed and discussed. On this basis, the prim route fusion algorithm is proposed to realize the effective utilization of energy and prolong the life of the network. Then, the WSN is abstracted as an undirected graph. From the perspective of saving the energy of the whole network, several current algorithms for building fusion trees are compared. The experimental results show that the prim algorithm consumes energy only after 700 rounds of clustering, while the leach clustering algorithm consumes energy only after 500 rounds. This shows that applying the prim algorithm can reduce the energy consumption of the whole network and prolong the life cycle of the network. However, the algorithm is carried out on the premise of uniform distribution of nodes, and there is a certain gap with the specific application of WSN in mechanical fault diagnosis. In the comparison of node energy consumption, it is found that compared with using the shortest path tree, using the central point of graph algorithm can greatly save the energy consumption of the node and has better performance. Practice has proved that this method can effectively remove redundant data information and solve the problem of unreliable data collected by a single sensor node. It is more suitable for the specific application of WSN in mechanical fault diagnosis.

Distributed filtering algorithm based on local outlier factor under data integrity attacks

Network security is becoming a prominent issue for the development of information technology, and intelligent network attacks pose great challenges to system security due to its strong concealment. The existence of these attacks threatens the operation process of the complicated control system. Motivated by such a security problem, we study the secure distributed filtering algorithm under a kind of complex data integrity attack which can attack in two forms. We design a detection mechanism based on local outlier factor to distinguish the rightness of exchanged data, which determines whether to fuse the estimates by comparing the local density (LD) of the estimation of each sensor. Such a detection mechanism does not need the sensor to transmit redundant data information, thus greatly saving calculation cost and improving transmission efficiency. Meanwhile, we optimize the distributed filtering algorithm and obtain a suboptimal estimation gain. Finally, we demonstrate a numerical example to verify the availability of the filtering algorithm, and explore the influence of detector parameters on the performance of the estimation system.

Clustering Analysis of Voltage Sag Events Based on Waveform Matching

Voltage sags are a serious problem within power supplies, which pose threats to both residential electricity and industrial manufacturing. Since any one sag may be recorded by multiple monitoring devices from different substations, the issue of redundant information in data arises. In this regard, a novel method for voltage sag events based on projection technology, shape dynamic time warping (shapeDTW), and spectral clustering is proposed. The main contributions of this paper may be summarized as follows: (1) We present a new method for extracting the voltage anomaly waveform, which is a fast projection segmentation algorithm (FPSA). The voltage sag waveform is only a part of the voltage anomaly waveform, so the voltage anomaly waveform contains more information. (2) ShapeDTW and spectral clustering are used to match and cluster voltage anomaly waveforms, so as to achieve the normalization of voltage sag events. (3) In practical engineering, the proposed method in the paper can be used to obtain the impact of voltage sags, reduce computational complexity, and ease the workload of the operation and maintenance engineers. Experiments were conducted using voltage sag data from voltage sag events recorded by the 10 kV monitoring points in Beijing, China. The results showed the effectiveness and reliability of our proposed methods.

SSA optimized back propagation neural network model for dam displacement monitoring based on long-term temperature data

Featured with the harmonic sinusoidal function to reflect temperature effects, the hydrostatic-season-time (HST) model is often used to monitor the concrete gravity dam health, but it does not take account of the effects of environment temperatures in real-term and has flaws especially when applied in conditions of significant temperature variations. A model of Sparrow Search Algorithm optimized error Back Propagation neural network (SSA-BP) based on the hydrostatic-temperature-time (HTT) model is proposed in this paper for predicting the concrete gravity dam displacement using the long-term environment temperature variable sets to reflect temperature effects. Successive Projections Algorithm (SPA) is used for the first time for feature selection on long-term temperature variables to further optimize the model (as SPA-SSA-BP). Through a case study with the practical observed data from a reality high concrete gravity dam, the effectiveness of the new model is verified, suggesting that HTT-based SSA-BP models have better performance than HST with the best result obtained when using the 2-year long variable sets. The SSA-BP model has much lower error in predicting the concrete dam displacement than Multiple Linear Regression (MLR). The arithmetic speed and prediction accuracy of the SPA-SSA-BP model is optimized as it can minimize the collinearity among feature variables in the long-term HTT variable sets, bring down the input variable dimension close to the level of HST, and diminish the redundant data information.

Internet of Things Network Topology Discovery Algorithm Based on Wireless Sensors

Remote sensor network is a multijump self-coordinating organization framework shaped by countless energy-restricted miniature sensor hubs sent in the observing region. Be that as it may, the correspondence conventions of customary organization configuration cannot be straightforwardly applied to remote sensor organizations. The improvement of committed correspondence conventions and estimation techniques has turned into a pressing exploration subject in the field of remote sensor organizations. The research purpose of this paper is to research the Internet of Things network topology discovery algorithm based on wireless sensors. This paper analyzes and mines an adaptive neighbor discovery scheduling algorithm with lower latency, acquires the information of potential neighbor nodes based on existing neighbor nodes, discovers potential neighbor nodes by proactively waking up, and studies the information recommendation mechanism between neighbor nodes and compares. The intimacy between neighbor nodes (such as common neighbor rate) is used to selectively receive recommended information from neighbor nodes, thereby filtering redundant data information, lessening hub energy utilization, and accomplishing the motivation behind broadening the organization life cycle. A geography disclosure calculation in light of versatile specialists is proposed to tackle the geography revelation issue under the various leveled geography structure in remote sensor organizations. The calculation joins the qualities of portable specialist innovation and various leveled Internet of Things network geography and simultaneously further develops the relocation methodology of versatile specialists for information assortment, intermittent undertaking assignment, and organization status observing. Research shows that in the investigation of the effect of hub obligation cycle on framework execution, the energy utilization of the DDC-Group calculation is marginally expanded contrasted and the gathering calculation, and the increase is not more than 1%, but compared to the CNR-Group increase more.

The Evaluation Model of Legal Risk of Enterprises’ Foreign Investment Based on Data Mining

In a highly competitive market economy, data and information have become a very important economic resource for enterprises. Because of the huge and redundant data information, companies make full use of the acquired data and build a legal risk evaluation model for corporate foreign investment. This model can effectively use foreign investment companies to conduct risk assessments to ensure that the company can achieve maximum victory. Through the use of data mining technology in the legal risk assessment model of foreign-invested enterprises, the correlation between data can be quickly and efficiently mined, which is conducive to enterprises to obtain known related data mining conditions, to evaluate the results that will appear in the future, and to help enterprises to do Make correct strategic decisions.

Novel Convolutional Neural Network with Variational Information Bottleneck for P300 Detection.

In the area of brain-computer interfaces (BCI), the detection of P300 is a very important technique and has a lot of applications. Although this problem has been studied for decades, it is still a tough problem in electroencephalography (EEG) signal processing owing to its high dimension features and low signal-to-noise ratio (SNR). Recently, neural networks, like conventional neural networks (CNN), has shown excellent performance on many applications. However, standard convolutional neural networks suffer from performance degradation on dealing with noisy data or data with too many redundant information. In this paper, we proposed a novel convolutional neural network with variational information bottleneck for P300 detection. Wiht the CNN architecture and information bottleneck, the proposed network termed P300-VIB-Net could remove the redundant information in data effectively. The experimental results on BCI competition data sets show that P300-VIB-Net achieves cutting-edge character recognition performance. Furthermore, the proposed model is capable of restricting the flow of irrelevant information adaptively in the network from perspective of information theory. The experimental results show that P300-VIB-Net is a promising tool for P300 detection.

Dynamic coherent diffractive imaging using unsupervised identification of spatiotemporal constraints.

Dynamic coherent diffractive imaging (CDI) reveals the fine details of structural, chemical, and biological processes occurring at the nanoscale but imposes strict constraints on the object distribution and illumination. Ptychographic CDI relaxes these constraints by exploiting redundant information in data obtained from overlapping regions of an object, but its time resolution is inherently limited. We have extended ptychographic redundancy into the spatiotemporal domain in dynamic CDI, automatically identifying redundant information in time-series coherent diffraction data obtained from dynamic systems. Simulated synchrotron experiments show that high spatiotemporal resolution is achieved without a priori knowledge of the object or its dynamics.

High Level Design of a Flexible PCA Hardware Accelerator Using a New Block-Streaming Method

Principal Component Analysis (PCA) is a technique for dimensionality reduction that is useful in removing redundant information in data for various applications such as Microwave Imaging (MI) and Hyperspectral Imaging (HI). The computational complexity of PCA has made the hardware acceleration of PCA an active research topic in recent years. Although the hardware design flow can be optimized using High Level Synthesis (HLS) tools, efficient high-performance solutions for complex embedded systems still require careful design. In this paper we propose a flexible PCA hardware accelerator in Field-Programmable Gate Arrays (FPGA) that we designed entirely in HLS. In order to make the internal PCA computations more efficient, a new block-streaming method is also introduced. Several HLS optimization strategies are adopted to create an efficient hardware. The flexibility of our design allows us to use it for different FPGA targets, with flexible input data dimensions, and it also lets us easily switch from a more accurate floating-point implementation to a higher speed fixed-point solution. The results show the efficiency of our design compared to state-of-the-art implementations on GPUs, many-core CPUs, and other FPGA approaches in terms of resource usage, execution time and power consumption.

A study of the internet financial interest rate risk evaluation index system in cloud computing

Cloud computing is a product of computer technologies combined with network technologies and it has been widely applied in China. Experts and scholars in all fields begin to make many studies of cloud computing infrastructure construction and effective resource utilisation. With ITFIN, people can enjoy financial services in dealing with various problems. However, one person can play many identities in the network. This phenomenon posed a severe challenge to ITFIN network security and has largely intensified the risks, including the operational risk, market selection risk and network and information security risk. ITFIN resolves the risks by establishing a reliable, reasonable and effective risk assessment model. We conducted theoretical and empirical analysis, then constructed an assessment model against China's ITFIN risk. The model integrates rough set and particle swarm optimisation support vector machine (PSO-SVM). Finally, the model was used to assess the ITFIN risk in China. The empirical research results indicate that the model can effectively reduce redundant data information with rough set theory. The theory also guarantees a reliable, reasonable and scientific model, enhance the classification effect of the model. The parameters of SVM model obtained by optimising with PSO can effectively avoid local optimum, improve the effect of the classification model. Overall, the model has good generalisation ability and learning ability.

A study of the internet financial interest rate risk evaluation index system in cloud computing

Cloud computing is a product of computer technologies combined with network technologies and it has been widely applied in China. Experts and scholars in all fields begin to make many studies of cloud computing infrastructure construction and effective resource utilisation. With ITFIN, people can enjoy financial services in dealing with various problems. However, one person can play many identities in the network. This phenomenon posed a severe challenge to ITFIN network security and has largely intensified the risks, including the operational risk, market selection risk and network and information security risk. ITFIN resolves the risks by establishing a reliable, reasonable and effective risk assessment model. We conducted theoretical and empirical analysis, then constructed an assessment model against China's ITFIN risk. The model integrates rough set and particle swarm optimisation support vector machine (PSO-SVM). Finally, the model was used to assess the ITFIN risk in China. The empirical research results indicate that the model can effectively reduce redundant data information with rough set theory. The theory also guarantees a reliable, reasonable and scientific model, enhance the classification effect of the model. The parameters of SVM model obtained by optimising with PSO can effectively avoid local optimum, improve the effect of the classification model. Overall, the model has good generalisation ability and learning ability.

Monitoring Batch Processes Using Sparse Parallel Factor Decomposition

Traditional multivariate statistical analysis methods used for batch process monitoring fail to build high-performance monitoring models partly because of the inability to extract sparse and interpretable latent variables from the batch data. To overcome this deficiency, we propose sparse parallel factor (SPARAFAC) decomposition for the sparse modeling of three-way data arrays of batch processes. The SPARAFAC extracts sparse factor matrices from the three-way data array. Sparse factor matrices have the potential advantage of being easier to interpret because they eliminate redundant data information and reveal significant variable correlations. A SPARAFAC-based batch process monitoring method is proposed. Because of the sparsity and interpretability, the SPARAFAC monitoring model is well suitable for fault detection, analysis, and diagnosis. Two fault detection indices are defined based on the SPARAFAC model, and they are capable of detecting the occurrence of process faults. Multilevel contribution plots...

An interpretable classification rule mining algorithm

Obtaining comprehensible classifiers may be as important as achieving high accuracy in many real-life applications such as knowledge discovery tools and decision support systems. This paper introduces an efficient Evolutionary Programming algorithm for solving classification problems by means of very interpretable and comprehensible IF-THEN classification rules. This algorithm, called the Interpretable Classification Rule Mining (ICRM) algorithm, is designed to maximize the comprehensibility of the classifier by minimizing the number of rules and the number of conditions. The evolutionary process is conducted to construct classification rules using only relevant attributes, avoiding noisy and redundant data information. The algorithm is evaluated and compared to nine other well-known classification techniques in 35 varied application domains. Experimental results are validated using several non-parametric statistical tests applied on multiple classification and interpretability metrics. The experiments show that the proposal obtains good results, improving significantly the interpretability measures over the rest of the algorithms, while achieving competitive accuracy. This is a significant advantage over other algorithms as it allows to obtain an accurate and very comprehensible classifier quickly.