Elimination Of Redundant Data Research Articles

Efficient implementation of a wavelet neural network model for short-term traffic flow prediction: Sensitivity analysis

The concept of a smart city is emerging to address significant challenges arising from rapid urbanization, economic growth, and climate change. Innovative technologies can be used as a means to promote sustainable and inclusive urban development. These technolgies include the deployment of the internet of things (IoT), artificial intelligence (AI), energy management, and smart transportation. In a smart city, intelligent transportation systems (ITS) play a vital role in efficient traffic management. This paper explores the use of hybrid artificial intelligence techniques for predicting short-term traffic flow data from M25 motorways in the UK. Since volume traffic flow data are non-stationary, wavelet transform (WT) as a powerful signal analyzer is applied for signal decomposition for the elimination of redundant data from input matrices. The feature selection method based on Gram-Schmidt (GS) is used for the selection of more valuable features. The elimination of redundant data can speed up the learning process and improve the generalisation capability of the prediction models. After a pre-processing stage, a wavelet neural network (WNN) with a simple structure is applied as a powerful prediction tool. Two separate structures are considered for the prediction of weekday and weekend traffic volume data. The experiments explore that the debauchies-4 (db4) wavelet function with 7 decomposition levels leads to the best detection accuracy. Moreover, the range of forecasting, the type of the day, the level of decomposition, and other factors all have an impact on prediction stability. Compared with existing prediction methods, the proposed approach produces lower values of root mean square error (RMSE) and mean absolute percentage error (MAPE) for all step-horizons analyzed. These findings provide valuable implications and insights into the development of an efficient and reliable road condition monitoring system for delivering secure and sustainable transportation services.

Optimized Back Propagation Neural Network Using Quasi-Oppositional Learning-Based African Vulture Optimization Algorithm for Data Fusion in Wireless Sensor Networks.

A Wireless Sensor Network (WSN) is a group of autonomous sensors geographically distributed for environmental monitoring and tracking purposes. Since the sensors in the WSN have limited battery capacity, the energy efficiency is considered a challenging task because of redundant data transmission and inappropriate routing paths. In this research, a Quasi-Oppositional Learning (QOL)-based African Vulture Optimization Algorithm (AVOA), referred to as QAVOA, is proposed for an effective data fusion and cluster-based routing in a WSN. The QAVOA-based Back Propagation Neural Network (BPNN) is developed to optimize the weights and threshold coefficients for removing the redundant information and decreasing the amount of transmitted data over the network. Moreover, the QAVOA-based optimal Cluster Head Node (CHN) selection and route discovery are carried out for performing reliable data transmission. An elimination of redundant data during data fusion and optimum shortest path discovery using the proposed QAVOA-BPNN is used to minimize the energy usage of the nodes, which helps to increase the life expectancy. The QAVOA-BPNN is analyzed by using the energy consumption, life expectancy, throughput, End to End Delay (EED), Packet Delivery Ratio (PDR) and Packet Loss Ratio (PLR). The existing approaches such as Cross-Layer-based Harris-Hawks-Optimization (CL-HHO) and Improved Sparrow Search using Differential Evolution (ISSDE) are used to evaluate the QAVOA-BPNN method. The life expectancy of QAVOA-BPNN for 500 nodes is 4820 rounds, which is high when compared to the CL-HHO and ISSDE.

Physics-assisted recursive method for sample selection from wall-bounded turbulence data

The purpose of this work is to improve the efficiency of sample selection with the recursive method for wall-bounded turbulent flows. In the proposed physics-assisted recursive method, the flow field is divided into several sub-regions along the wall distance. Since the distributions of the flow variables have certain similarity along the wall normal direction, fast clustering of similar data can be achieved, which paves the way for the rapid elimination of redundant data. Then, the recursive method is used for sample selection in each sub-region. The effectiveness of the proposed method is investigated through several cases. The results show that the proposed method has good convergence and grid independence and improves the computational efficiency of the recursive method for sample selection. Since the amount of training data is reduced, the time consumption of model training is decreased. In addition, it is demonstrated that sample selection can also be helpful to achieve more balanced model performance by changing the distribution of training data.

Data usage-based privacy and security issues in mobile app recommendation (MAR): a systematic literature review

PurposeSimilarly, Zhu et al. (2014) and Zhang et al. (2014) stated that addressing privacy concerns with the recommendation process is necessary for the healthy development of app recommendation. Recently, Xiao et al. (2020) mentioned that a lack of effective privacy policy hinders the development of personalized recommendation services. According to the reported work, privacy protection technology methods are too limited for mobile focusing on data encryption, anonymity, disturbance, elimination of redundant data to protect the recommendation process from privacy breaches. So, this situation motivated us to conduct a systematic literature review (SLR) to provide the viewpoint of privacy and security concerns as mentioned in current state-of-the-art in the mobile app recommendation domain.Design/methodology/approachIn this work, the authors have followed Kitchenham guidelines (Kitchenham and Charters, 2007) to devise the SLR process. According to the guidelines, the SLR process has three main phases: (1) define, (2) conduct the search and (3) report the results. Furthermore, the authors used systematic mapping approach as well to ensure the whole process.FindingsBased on the selected studies, the authors proposed three main thematic taxonomies, including architectural style, security and privacy strategies, and user-usage in the mobile app recommendation domain. From the studies' synthesis viewpoint, it is observed that the majority of the research efforts have focused on the movie recommendation field, while the mainly used privacy scheme is homomorphic encryption. Finally, the authors suggested a set of future research dimensions useful for the potential researchers interested to perform the research in the mobile app recommendation domain.Originality/valueThis is an SLR article, based on existing published research, where the authors identified key issues and future directions.

A Hybrid Relational Database Model for Uncertain and Imprecise Information

Recent years, many fuzzy or probabilistic database models have been built for representing and handling imprecise or uncertain information of objects in real-world applications. However, relational database models combining the relevance and strength of both fuzzy set and probability theories have rarely been proposed. This paper introduces a new relational database model, as a hybrid one combining consistently fuzzy set theory and probability theory for modeling and manipulating uncertain and imprecise information, where the uncertainty and imprecision of a relational attribute value are represented by a fuzzy probabilistic triple, the computation and combination of relational attribute values are implemented by using the probabilistic interpretation of binary relations on fuzzy sets, and the elimination of redundant data is dealt with by coalescing e-equivalent tuples. The basic concepts of the classical relational database model are extended in this new model. Then the relational algebraic operations are formally defined accordingly. A set of the properties of the relational algebraic operations is also formulated and proven.

Dynamic Prediction of Powerline Frequency for Wide Area Monitoring and Control

This paper presents a novel data driven framework based on $\epsilon$ -Support Vector Regression to reduce the bandwidth requirement for transmission of phasor measurement unit (PMU) data. This is achieved by judicious elimination of redundant data at the PMU before transmission. Simultaneously, the missing samples are predicted at PDC to ensure faithful identification of impending disturbances in the power system. Due to inherent nonstationary nature of PMU data, the hyperparameters are dynamically recomputed as necessary, thereby maintaining the accuracy of prediction and robustness of the algorithm. Performance of the proposed algorithm is evaluated via large scale simulations using powerline frequency data. A trade-off between prediction quality and runtime of the algorithm is observed, which is addressed by suitable selection of hyperparameters. Compared to the competitive data reduction scheme, the proposed algorithm saves around 60% bandwidth and identifies power system disturbances 73% more accurately.

Optimized compression technology for spatial data network transmission

This paper considers how to reduce the amount of data to improve the user experience from data compression and network transmission technology according to the characteristics of map tile technology. The former mainly reduces the amount of data itself through the elimination of redundant data of PNG files and lossless compression. The latter reduces the amount of data in network transmission through the GZIP compression technology to achieve dynamic compression before the vector block data transmission thus effectively reducing the amount of data in network transmission while saving data transmission time. Experiments and applications show that the above two methods can greatly improve data transmission performance.

A Survey on Data Redundancy Check in a Hybrid Cloud by using Convergent Encryption

Background/Objectives: Data deduplication is the elimination of redundant data within an existing environment. It is mainly used in cloud storage environment for bandwidth optimization and to enhance storage space. This paper focuses on different dedupe techniques with a secure cloud. Methods/Statistical Analysis: Symmetric key encryption is used to prevent unauthorized user from accessing the data. Convergent key encryption algorithm is used to encrypt and maintain data confidentiality. Findings: In the proposed system the authorized data deduplication is achieved with minimal overhead when compared with traditional deduplication. Keywords: Convergent Key, Cloud Storage, Data Confidentiality, Data Deduplication, Hybrid Cloud

Enabling Secure and Dependable Client SideDeduplication in Large Cloud Computing

Since from recent years, lot of corporate and private users outsource their data to cloud storage providers. Cloud storage services use deduplication which means elimination of redundant data. So it reduces the storage space and bandwidth consumption of data storage services, but brings many security and privacy implications. There are some issues in cloud computing, but security and privacy are most important issues which need to be considered for the public cloud environments. A new client-side deduplication scheme has been proposed for securely sharing and storing data via the public cloud. The proposed method has twofold. Firstly, higher confidentiality is being ensured from unauthorized users. That is, every consumer computes a per data key in order to encrypt the data which intends to store in the cloud. So the data owner manages every data access. Secondly, metadata file is being integrated with access rights, so approved user can access the file which is encrypted with his own private key only

Short term load forecasting using a hybrid intelligent method

Due to the regulation of electrical power systems, electricity market players need precise information of electrical energy consumption and generation in order to maximize their benefit based on appropriate decisions. In this paper a new hybrid intelligent method is proposed for short term load forecasting. In this method, load and temperature of previous days are used for prediction of the next hour electrical load consumption. Since electrical load signals are non-stationary, Wavelet Transform (WT) as a powerful signal analyzer is applied for the signal decomposing. For elimination of redundant data from input matrices, the Feature Selection (FS) method based on Gram–Schmidt (GS) is used for selection of more valuable features. The elimination of redundant data can speed up learning process and improve the generalization capability of the prediction scheme. Support Vector Machine (SVM) with simple structure and few tuning parameters is applied as a powerful regression tool. Two separate structures are considered for prediction of weekday and weekend electrical load consumption. Besides, in order to increase the forecasting accuracy, indices are determined for each day. The simulation results reveal that the Coiflet wavelet function with 2 decomposition levels lead to the best detection accuracy. Moreover, 30 dominant features of previous 50days should be used to obtain minimum forecasting error. Comparative results show the priority of the proposed method in aspect of prediction accuracy as compared to some reported algorithms.

Research on the Wireless Sensor Network Data Fusion Technology

Wireless sensor network ( WSN ) is a kind of energy constrained network, by using data fusion technology, the elimination of redundant data, can save energy, prolong the network life purpose. Data fusion in wireless sensor network can realize different protocol layers, Based on the introduction of wireless sensor network and data fusion related knowledge, prove that the arithmetic mean method is effective, and use OPNET software tool for network simulation, finally, analysis results and conclude, verify effects of the arithmetic average fusion algorithm for wireless sensor network.

Structure-free message aggregation and routing in traffic information system (SMART)

In a VANET (Vehicular Ad-Hoc Network) environment, it is highly probable that multiple nodes detect and report common events. Hence, elimination of redundant messages can help in efficient bandwidth usage. Data aggregation has also been identified as another potential tool to address broadcast storm in VANETs. Though many aggregation and in-network filtering schemes have been proposed, few studies have been done to combine the two approaches in a structured fashion in order to reap the full benefits.In this work, we have shown how our proposed SMART protocol utilises the data aggregation tool in conjunction with message fragmentation, to address the prevailing challenge of vehicular networks—efficient bandwidth usage. The data is broken down into fragments and subsequently analysed, thus resulting in the elimination of redundant data than simply suppressing the transmission of duplicate data. The data received at every node is compared, analysed and stored into a well-organised data structure for faster and efficient look-up, thus transforming the network into a multi-level filtering system. Our SMART protocol is an efficient and scalable data aggregation scheme that achieves information dissemination without incurring any overhead of constructing a structure. As a proof of concept, we have shown that the SMART protocol improves efficiency in VANET communication by restricting the transmission of 75–80% redundant data.

Untangling associations between chironomid taxa in Neotropical streams using local and landscape filters

Summary1. Analyses of species association have major implications for selecting indicators for freshwater biomonitoring and conservation, because they allow for the elimination of redundant information and focus on taxa that can be easily handled and identified. These analyses are particularly relevant in the debate about using speciose groups (such as the Chironomidae) as indicators in the tropics, because they require difficult and time‐consuming analysis, and their responses to environmental gradients, including anthropogenic stressors, are poorly known.2. Our objective was to show whether chironomid assemblages in Neotropical streams include clear associations of taxa and, if so, how well these associations could be explained by a set of models containing information from different spatial scales. For this, we formulateda priorimodels that allowed for the influence of local, landscape and spatial factors on chironomid taxon associations (CTA). These models represented biological hypotheses capable of explaining associations between chironomid taxa. For instance, CTA could be best explained by local variables (e.g. pH, conductivity and water temperature) or by processes acting at wider landscape scales (e.g. percentage of forest cover).3. Biological data were taken from 61 streams in Southeastern Brazil, 47 of which were in well‐preserved regions, and 14 of which drained areas severely affected by anthropogenic activities. We adopted a model selection procedure using Akaike’s information criterion to determine the most parsimonious models for explaining CTA.4. Applying Kendall’s coefficient of concordance, seven genera (Tanytarsus/Caladomyia,Ablabesmyia,Parametriocnemus,Pentaneura,Nanocladius,PolypedilumandRheotanytarsus) were identified as associated taxa. The best‐supported model explained 42.6% of the total variance in the abundance of associated taxa. This model combined local and landscape environmental filters and spatial variables (which were derived from eigenfunction analysis). However, the model with local filters and spatial variables also had a good chance of being selected as the best model.5. Standardised partial regression coefficients of local and landscape filters, including spatial variables, derived from model averaging allowed an estimation of which variables were best correlated with the abundance of associated taxa. In general, the abundance of the associated genera tended to be lower in streams characterised by a high percentage of forest cover (landscape scale), lower proportion of muddy substrata and high values of pH and conductivity (local scale).6. Overall, our main result adds to the increasing number of studies that have indicated the importance of local and landscape variables, as well as the spatial relationships among sampling sites, for explaining aquatic insect community patterns in streams. Furthermore, our findings open new possibilities for the elimination of redundant data in the assessment of anthropogenic impacts on tropical streams.

Identification of asthma phenotypes using cluster analysis in the Severe Asthma Research Program.

The Severe Asthma Research Program cohort includes subjects with persistent asthma who have undergone detailed phenotypic characterization. Previous univariate methods compared features of mild, moderate, and severe asthma. To identify novel asthma phenotypes using an unsupervised hierarchical cluster analysis. Reduction of the initial 628 variables to 34 core variables was achieved by elimination of redundant data and transformation of categorical variables into ranked ordinal composite variables. Cluster analysis was performed on 726 subjects. Five groups were identified. Subjects in Cluster 1 (n = 110) have early onset atopic asthma with normal lung function treated with two or fewer controller medications (82%) and minimal health care utilization. Cluster 2 (n = 321) consists of subjects with early-onset atopic asthma and preserved lung function but increased medication requirements (29% on three or more medications) and health care utilization. Cluster 3 (n = 59) is a unique group of mostly older obese women with late-onset nonatopic asthma, moderate reductions in FEV(1), and frequent oral corticosteroid use to manage exacerbations. Subjects in Clusters 4 (n = 120) and 5 (n = 116) have severe airflow obstruction with bronchodilator responsiveness but differ in to their ability to attain normal lung function, age of asthma onset, atopic status, and use of oral corticosteroids. Five distinct clinical phenotypes of asthma have been identified using unsupervised hierarchical cluster analysis. All clusters contain subjects who meet the American Thoracic Society definition of severe asthma, which supports clinical heterogeneity in asthma and the need for new approaches for the classification of disease severity in asthma.

Randomized Protocols for Duplicate Elimination in Peer-to-Peer Storage Systems

Distributed peer-to-peer systems rely on voluntary participation of peers to effectively manage a storage pool. In such systems, data is generally replicated for performance and availability. If the storage associated with replication is not monitored and provisioned, the underlying benefits may not be realized. Resource constraints, performance scalability, and availability present diverse considerations. Availability and performance scalability, in terms of response time, are improved by aggressive replication, whereas resource constraints limit total storage in the network. Identification and elimination of redundant data pose fundamental problems for such systems. In this paper, we present a novel and efficient solution that addresses availability and scalability with respect to management of redundant data. Specifically, we address the problem of duplicate elimination in the context of systems connected over an unstructured peer-to-peer network in which there is no a priori binding between an object and its location. We propose two randomized protocols to solve this problem in a scalable and decentralized fashion that does not compromise the availability requirements of the application. Performance results using both large-scale simulations and a prototype built on PlanetLab demonstrate that our protocols provide high probabilistic guarantees while incurring minimal administrative overheads.

TPDS Information for authors

Distributed peer-to-peer systems rely on voluntary participation of peers to effectively manage a storage pool. In such systems, data is generally replicated for performance and availability. If the storage associated with replication is not monitored and provisioned, the underlying benefits may not be realized. Resource constraints, performance scalability, and availability present diverse considerations. Availability and performance scalability, in terms of response time, are improved by aggressive replication, whereas resource constraints limit total storage in the network. Identification and elimination of redundant data pose fundamental problems for such systems. In this paper, we present a novel and efficient solution that addresses availability and scalability with respect to management of redundant data. Specifically, we address the problem of duplicate elimination in the context of systems connected over an unstructured peer-to-peer network in which there is no a priori binding between an object and its location. We propose two randomized protocols to solve this problem in a scalable and decentralized fashion that does not compromise the availability requirements of the application. Performance results using both large-scale simulations and a prototype built on PlanetLab demonstrate that our protocols provide high probabilistic guarantees while incurring minimal administrative overheads