Redundancy Elimination Research Articles

Stable feature selection based on instance learning, redundancy elimination and efficient subsets fusion

Feature selection is frequently used as a preprocessing step to data mining and is attracting growing attention due to the increasing amounts of data emerging from different domains. The large data dimensionality increases the noise and thus the error of learning algorithms. Filter methods for feature selection are specially very fast and useful for high-dimensional datasets. Existing methods focus on producing feature subsets that improve predictive performance, but they often suffer from instability. Instance-based filters, for example, are considered as one of the most effective methods that rank features based on instances neighborhood. However, as the feature weight fluctuates with the instances, small changes in training data result in a different selected subset of features. By another hand, some other filters generate stable results but lead to a modest predictive performance. The absence of a trade-off between stability and classification accuracy decreases the reliability of the feature selection results. In order to deal with this issue, we propose filter methods that improve stability of feature selection while preserving an optimal predictive accuracy and without increasing the complexity of the feature selection algorithms. The proposed approaches first use the strength of instance learning to identify initial sets of relevant features, and the advantage of aggregation techniques to increase the stability of the final set in a second stage. Two classification algorithms are used to evaluate the predictive performance of our proposed instance-based filters compared to state-of-the-art algorithms. The obtained results show the efficiency of our methods in improving both classification accuracy and feature selection stability for high-dimensional datasets.

Comparative Transcriptome Analysis Combining SMRT- and Illumina-Based RNA-Seq Identifies Potential Candidate Genes Involved in Betalain Biosynthesis in Pitaya Fruit.

To gain more valuable genomic information about betalain biosynthesis, the full-length transcriptome of pitaya pulp from ‘Zihonglong’ (red pulp) and ‘Jinghonglong’ (white pulp) in four fruit developmental stages was analyzed using Single-Molecule Real-Time (SMRT) sequencing corrected by Illumina RNA-sequence (Illumina RNA-Seq). A total of 65,317 and 91,638 genes were identified in ‘Zihonglong’ and ‘Jinghonglong’, respectively. A total of 11,377 and 15,551 genes with more than two isoforms were investigated from ‘Zihonglong’ and ‘Jinghonglong’, respectively. In total, 156,955 genes were acquired after elimination of redundancy, of which, 120,604 genes (79.63%) were annotated, and 30,875 (20.37%) sequences without hits to reference database were probably novel genes in pitaya. A total of 31,169 and 53,024 simple sequence repeats (SSRs) were uncovered from the genes of ‘Zihonglong’ and ‘Jinghonglong’, and 11,650 long non-coding RNAs (lncRNAs) in ‘Zihonglong’ and 11,113 lncRNAs in ‘Jinghonglong’ were obtained herein. qRT-PCR was conducted on ten candidate genes, the expression level of six novel genes were consistent with the Fragments Per Kilobase of transcript per Million mapped reads (FPKM) values. In conclusion, we firstly undertook SMRT sequencing of the full-length transcriptome of pitaya, and the valuable resource that was acquired through this sequencing facilitated the identification of additional betalain-related genes. Notably, a list of novel putative genes related to the synthesis of betalain in pitaya fruits was assembled. This may provide new insights into betalain synthesis in pitaya.

A Deep-Learning-Based Oil-Well-Testing Stage Interpretation Model Integrating Multi-Feature Extraction Methods

The interpretation of well-testing data is a key means of decision-making support for oil and gas field development. However, conventional processing methods have many problems, such as the stochastic nature of the data, feature redundancies, the randomness of the initial weights or thresholds, and fluctuations in the generalization ability with slight changes in the network parameters. These result in a poor ability to characterize data features and a low generalization ability of the interpretation models. We propose a new integrated well-testing interpretation model based on a multi-feature extraction method and deep mutual information classifiers (MFE-DMIC). This model can avoid the low model classification accuracy caused by the simple training structures, lacking of redundancy elimination, and the non-optimal classifier configuration parameters. First, we obtained the initial features according to four classical feature extraction methods. Then, we eliminated feature redundancies using a deep belief network and united the maximum information coefficient method to achieve feature purification. Finally, we calculated the interpretation results using a hybrid particle swarm optimization–support vector machine classification system. We used 572 well-testing field samples, including five working stages, for model training and testing. The results show that the MFE-DMIC model had the highest total stage classification accuracy of 98.18% as well as the least of features (nine) compared with the classical feature extraction and classification methods and their combinations. The proposed model can reduce the efforts of oil analysts and allow accurate labeling of samples to be predicted.

Development and Validation of Enterprise Architecture (EA) Readiness Assessment Model

Enterprise architecture (EA) proves to be a holistic strategy in aligning business and information technology. An increasing number of organizations, especially in the public sector, have adopted EA in order to take advantage of the opportunities that it offers. Through EA, the efficiency of the organization is improved, with the optimization of resources and the elimination of duplication and redundancy. However, the successful establishment of EA relies on the organization’s readiness and ability to adopt EA practices because without proper readiness, the practices will probably fail. EA readiness refers to the assessment of how ready an organization is to adopt and to establish EA practices. EA readiness helps the organization to measure their stage of readiness, to identify any gaps, and then to redesign its strategy in order to adopt EA practices. EA is a merger between business and IT. Thus, the important elements of EA readiness should comprise of people, process, technology, and catalyst enabler. There is a lack of readiness for an assessment model that shapes these four elements towards EA; hence, a clear gap has been identified. Therefore, the objective of this study is to validate the EA Readiness Assessment Model (EARAM) by performing the Delphi technique. The study adopts a three-round Delphi Technique to verify the identified elements and factors and this is followed by developing and validating the proposed model. Results from the Delphi analysis have validated four (4) major elements of EARAM, namely people, process, technology, and catalyst enabler. The number of factors that contributes towards the readiness of the EA establishment is fourteen (14). It is anticipated that this model (EARAM) can help the Malaysian Public Sector (MPS) organization to identify and understand the elements and factors that must be considered when assessing the readiness to practice EA.

Pache: A Packet Management Scheme of Cache in Data Center Networks

The communication traffic among servers is a principal bottleneck for data center networks (DCNs), in which redundant traffic has a significant impact on the performance of DCNs. In this paper, we propose Pache , a distributed and efficient cache scheme to achieve traffic redundancy elimination in DCNs. In Pache, nodes cache packets to reduce bandwidth consumption and gain performance revenue. Each cache node manages the local cache and improves cache efficiency with counting bloom filter, hash table and Trie. Each cache node broadcasts the local cache information by a bloom filter to other servers, and servers have cache information tables (cache-table) to record them, which achieves a cache sharing mechanism. Moreover, Pache employs false positive information table (fp-table) to reduce the false positive rate caused by bloom filters. A server determines an original packet or the corresponding fingerprint packet to send by querying cache-table and fp-table. Cache node placement mechanism is also designed to select which node to cache a packet. For evaluating the performance of Pache, we execute extensive simulations and conduct a case study on Amazon EC2 platform from different aspects. The results show that Pache can eliminate about 40 percent redundant traffic on average, and only increase 10 percent runtime, and it is also a scalable and feasible cache scheme in DCNs.

Redundancy Analysis and Elimination on Access Patterns of the Windows Applications Based on I/O Log Data

In this paper, we analyze I/O log data monitored in the Windows operating system for improving the system performance. Especially, we focus on the I/O operations to the Windows registry. As a result, we identify redundant access patterns of the Windows applications. To find all the possible redundant patterns from the large-scale log data, we propose the redundancy detection algorithm. Then, we propose the two-level redundancy elimination method to remove unnecessary redundant operations. We also present an event-driven method that guarantees that the result of redundancy elimination is equivalent to that of the original program. Through experiments, we show that the proposed redundancy elimination method improves the performance of the original program having redundant access patterns by up to 90.25% for individual access patterns; by 8.93% ~ 26.21% when the multiple programs having combined access patterns are running concurrently.

Securing Hardware Accelerators Using Multi-Key Based Structural Obfuscation

A hardware accelerator is a pivotal component of a system-on-chip (SoC) employed in modern electronic systems. However, the design of hardware accelerators can be infected by inserting malicious logic (hardware Trojan) through reverse engineering (RE) by an adversary. Rising threats of RE and Trojan necessitates the security of hardware accelerator based SoCs. The proposed methodology secures the hardware accelerators using a novel structural obfuscation using key-driven transformation techniques such as key-based loop unrolling, key-based partitioning, key-based redundant operation elimination and key-based tree height transformation, which makes the design unobvious (non-interpretable) to an attacker. The results of the proposed approach on DSP hardware accelerators indicated 2.3x enhancement in strength of obfuscation (at gate level) compared to a recent approach (indicating enhanced security), at nominal design cost.

A Novel HDF‐Based Data Compression and Integration Approach to Support BIM‐GIS Practical Applications

Constructing a fully mapped virtual world is the premise of establishing the digital twin system; thus, the integration of BIM (Building Information Modeling) and GIS (Geographic Information System) constitutes a fundamental part of the mature digital twin. Although quite a few theoretical unified data models of BIM and GIS integration have been presented in precedent research, the practical web‐based applications and software support for real‐world city information modeling are still difficult to achieve. One of the challenges is storage inefficiency. It thence leads to burdens in exchange and analysis, which hugely impede the realization of virtual‐real fusion. To address that issue, in this research, we contribute to exploring an HDF‐ (Hierarchical Data Format‐) based, innovative, and synthesized scheme with three significant technical processes. First, data reorganization trims the original data with efficient redundancy elimination and reordering for IFC‐SPF (IFC STEP Physical File) and XML‐encoded CityGML (City Geography Markup Language). Next, bidirectional transformation methods for BIM and GIS modeling data, images, and analytical data into HDF are proposed. They retain the entities and relationships in modeling data and can further improve the compactness and accessibility by ultimately 60%–80% which saves 497,612 KB from 565,152 KB in the test of ZhongChun transformation substation. Finally, data aggregation enhances the bond of the integrated model and heterogeneous data resources from the transformed HDF files. The case studies show that the approach proposed in this paper could reach high efficiency for practicability of the BIM + GIS integration model. This light‐weight integration method can further improve the front‐end service responsiveness in digital twin applications.

Cell-Based Local Search Heuristics for Guide Path Design of Automated Guided Vehicle Systems With Dynamic Multicommodity Flow

This article discusses the guide path design of automated guided vehicle (AGV) systems for which we propose a model that incorporates a dynamic multicommodity flow with capacity constraints from a pickup to a delivery point. The problem is formulated as the selection of a guide path connecting a given set of pickup/delivery points. A cell-based local search that is based on an original neighborhood technique is developed. In the proposed method, the guide path is effectively optimized based on a local search of cell-based neighborhood search preserving connectivity constraints and the subsequent solution of a dynamic multicommodity flow problem. The performance of the proposed method is strengthened by the redundant elimination of arcs and intensification of search according to the concept of the flow-concentrated cell. The effectiveness of the proposed method is demonstrated by comparing it with a general-purpose solver and recent algorithms for fixed-charge capacitated multicommodity network design problems. A real case study is presented to demonstrate the applicability of the proposed method by using simulation software. Note to Practitioners —Due to the recent growth of online shipping, the design of automated warehousing systems is receiving much attention. Most automated warehousing systems introduce multiple automated guided vehicles (AGVs) for transportation. This article presents an efficient optimization of guide path design considering congestions and dynamic multicommodity flow routing. We develop cell-based local search heuristics for the guide path design problem for AGV systems that can be applied to a large-scale transportation model to solve the problem efficiently. Our major finding is that the guide path design with the dynamic multicommodity flow can significantly reduce the delays caused by congestions and improve the efficiency of the warehousing systems. The effectiveness of the derived guide path design is evaluated by simulation software. The feasibility of the guide path is confirmed in several case studies.

TRANSFORMATION OF THE KYIV MURALS AND MOSAICS AT THE TURN OF THE 1950–1960S AND THEIR IMPACT ON THE MONUMENTAL PAINTING DEVELOPMENT

During twentieth century monumental painting has undergone numerous changes in iconography, stylistics and figurative-plastic language itself. An important milestone was the turn of the '50–'60s. When the artists resorted to active decorative searches and the ways of plane expression.In the 1930–50's, socialist realism became the main pointer for the monumental artist, which was the dominant artistic method at that time. For the most effective use of this method, the artists have been oriented on the realistic and academic principles, filling classical forms with Soviet meaning. Panels become burdensome and suffocated by architectural decoration, the expressiveness of the monumental painting was lost.However, at the end of the '50s monumental painting began to move away from realistic tendencies. A search of new ways of expressions have been proceeded. A search of new ways of expressions proceeded. The impetus for rethinking styling was widely known resolution of November 4, 1955 "About the elimination of redundancies in design and construction. "The document became the peculiar starting point in Soviet architecture as well as monumental painting.According to this resolution aesthetical side had to be solved by organically linking the architectural form to the purpose of the building and established proportion. The task of architects was to work on the creation of standard buildings in simple economic forms.Thus, new opportunities for development in the monumental painting appear. With the formation of new architecture, murals and mosaics are gaining ground for experiments both with materials and figurative-plastic solutions.At the turn of the 60's significant projects were created in Kyiv, among them Kiev River Station (1961), Kiev Bus Station (1961), Shulyavska Subway Station (1963), Metro Restaurant (1963), interior design at The Institute of Plant Physiology and Genetics, NAS of Ukraine (1962), exteriors of the Institute of Plant Protection of the UAAS (1962).

Fast and Secure Packet Scheduling with Packet Level Redundancy Elimination in Dynamic Mobile Networks

Mobile networks are fast and flexible for effective communication, where it only needs a dynamic topology and unstructured network construction. Due to this flexible nature, the resource utilization and energy consumption is high when comparing to the other static networks. In this scenario, packet redundancy is major problem on mobile networks, which increases network traffic and energy. Currently redundancy elimination is performed using redundancy elimination (RE) solution, which affects the end-to-end privacy. To eliminate redundant packet transfers and provide fast and secure packet transfer in the dynamic mobile network, a new middle box framework is proposed in this paper. This framework is named as Sift Hub, which is a centre point to verify the packet redundancy and security violations. This includes four algorithms to perform packet redundancy elimination, packet scheduling and verification. The algorithms are One pass signature generation algorithm (OSGA) for packet security, Predictive encryption technique using enhanced hidden vector encryption algorithm for redundancy detection on encrypted traffic, Packet level data filtering algorithm (PLDF) for eliminating un-authenticated and redundant packets and RAPS-Redundancy aware packet scheduling algorithm for fast data scheduling over dynamic mobile networks. The Shift_Hub supports inter and intra packet level redundancy elimination and secure packet scheduling without affecting the end-to-end privacy. The Shift_Hub is implemented and the performance is evaluated on dynamic mobile network scenario created on NS2 simulator. The results shows, the proposed work gives better performance in finding and eliminating redundant packet in secure manner than the existing works.

Deep reinforcement hashing with redundancy elimination for effective image retrieval

Hashing is one of the most promising techniques in approximate nearest neighbor search due to its time efficiency and low cost in memory. Recently, with the help of deep learning, deep supervised hashing can perform representation learning and compact hash code learning jointly in an end-to-end style, and obtains better retrieval accuracy compared to non-deep methods. However, most deep hashing methods are trained with a pair-wise loss or triplet loss in a mini-batch style, which makes them inefficient at data sampling and cannot preserve the global similarity information. Besides that, many existing methods generate hash codes with redundant or even harmful bits, which is a waste of space and may lower the retrieval accuracy. In this paper, we propose a novel deep reinforcement hashing model with redundancy elimination called Deep Reinforcement De-Redundancy Hashing (DRDH), which can fully exploit large-scale similarity information and eliminate redundant hash bits with deep reinforcement learning. DRDH conducts hash code inference in a block-wise style, and uses Deep Q Network (DQN) to eliminate redundant bits. Very promising results have been achieved on four public datasets, i.e., CIFAR-10, NUS-WIDE, MS-COCO, and Open-Images-V4, which demonstrate that our method can generate highly compact hash codes and yield better retrieval performance than those of state-of-the-art methods.

Elimination of information redundancy of hyperspectral images using the “well-adapted” basis method

The paper considers some issues of eliminating information redundancy of hyperspectral images (HSI). The characteristic properties of the HSI are listed, a brief description of the existing HSI compression methods is given. The possibility of using local, homogeneous “well-adapted” basis functions (LHWABF) to eliminate information redundancy and adaptive compression of the HSI is considered. An algorithm for constructing a LHWABF system for the HSI based on the Chebyshev approximation is proposed. The results of computational experiments, including the use of a graphics processor, are presented. The effectiveness of the proposed method of adaptive compression HSI is shown.

Redundancy elimination for overlapping point clouds based on two-dimensional corresponding point pair constraints between adjacent camera stations in a grating projection rotation measurement system.

A grating projection shape measurement system has been a commonly used method in the field of three-dimensional (3D) reconstruction in recent years, and global point cloud registration is a key step in this method. However, in the registration process, a large amount of low-precision overlapping redundant data (ORD) is generated between adjacent camera stations, which will seriously affect the speed and accuracy of later modeling. Therefore, how to eliminate these low-precision ORD is a major problem to be solved at present. Determining all overlapping 3D point pairs between two adjacent stations and deleting the points with low precision in the point pairs is the key to solving this problem. Therefore, based on an omnidirectional rotation measurement system, combined with the constraint relationships between the projection space and the acquisition space in the global registration process and the stereo-matching method of space conversion, an elimination algorithm for ORD with a two-dimensional (2D) phase constraint and a 2D pixel constraint is proposed. The experimental results show that the proposed algorithm can faster locate overlapping 3D point pairs between adjacent stations, with a higher elimination rate, and the accuracy of the overall point cloud is higher after the redundancy elimination.

Rapid Classification of Wheat Grain Varieties Using Hyperspectral Imaging and Chemometrics

The classification of wheat grain varieties is of great value because its high purity is the yield and quality guarantee. In this study, hyperspectral imaging combined with the chemometric methods was applied to explore and implement the varieties classification of wheat seeds. The hyperspectral images of all the samples covering 874–1734 nm bands were collected. Exploratory analysis was first carried out while using principal component analysis (PCA) and linear discrimination analysis (LDA). Spectral preprocessing methods including standard normal variate (SNV), multiplicative scatter correction (MSC), and wavelet transform (WT) were introduced, and their effects on discriminant models were studied to eliminate the interference of instrumental and environmental factors. PCA loading, successive projections algorithm (SPA), and random frog (RF) were applied to extract feature wavelengths for redundancy elimination owing to the possibility of existing redundant spectral information. Classification models were developed based on full wavelengths and feature wavelengths using LDA, support vector machine (SVM), and extreme learning machine (ELM). This optimal model was finally utilized to generate visualization map to observe the classification performance intuitively. When comparing with other models, ELM based on full wavelengths achieved the best accuracy up to 91.3%. The overall results suggested that hyperspectral imaging was a potential tool for the rapid and accurate identification of wheat varieties, which could be conducted in large-scale seeds classification and quality detection in modern seed industry.

Domain independent redundancy elimination based on flow vectors for static video summarization

Video summarization aims to find a compact representation of input videos. The method finds out interesting parts of the video by discarding the remaining parts of the video. The abstracts thus generated enhances browsing and retrieval of video data. The quality of summaries generated by video summarization algorithms can be improved if the redundant frames in the input video are taken care of before summarization. This paper presents a novel domain-independent method for redundancy elimination from input videos before summarization maintaining keyframes in the original video. The frames of input video are first presampled by selecting two frames in one second. The flow vectors between consecutive frames are computed using SIFT Flow algorithm. The magnitude of flow vectors at each pixel position of the frame are summed up to find the displacement magnitude between the consecutive frames. The redundant frames are filtered out based on local averaging of the displacement values. The evaluation of the method is done using two standard datasets namely VSUMM and OVP. The results demonstrate that an average reduction rate of 97.64% is achieved consistently on videos of all categories. The method also gives superior results compared to other state-of-the-art redundancy elimination methods for video summarization

A survey on data aggregation techniques in IoT sensor networks

There is a growing interest in using wireless sensor technologies in various Internet of things scenarios. Considering the huge growth of smart objects and their applications, the need to collect and analyze their product data are becoming one of the main challenges. Sensor nodes are powered by batteries, efficient operations in term of energy are critical. Toward that end, it is desirable for a sensor node to eliminate redundancies in the received data from the neighboring nodes before transferring the final data to the central station. Data aggregation is one of the influential techniques in elimination of data redundancy and improvement of energy efficiency; also it increases the lifespan of Wireless Sensor Networks. In addition, the efficient data aggregation protocol can reduce network traffic. When a specific objective takes place in a specific area, it might be detected by more than one sensor. Considering the main challenges and aspects of data aggregation in wireless sensor networks, a review on different types of data aggregation techniques and protocols are presented in this paper. The ultimate objective of this study is to make the basic foundations to develop new advanced designs based on data integration techniques and clustering that have been proposed so far. Major techniques of data integration in wireless sensor networks covering ground, underground and underwater sensor networks are presented in this paper and the applications, advantages and disadvantages of using each technique are described.

Multi-path data fusion method based on routing algorithm for wireless sensor networks

ABSTRACT For the purpose of saving the resources of wireless sensor networks, it is necessary to fuse the existing data in wireless sensor networks. When using the current method to fuse data in wireless sensor networks, the fusing time is too long to remove the noise signal in wireless sensor networks effectively. In the process of fusing, data packet loss is easy to occur. There are some problems such as low fusion efficiency, poor denoising performance and high packet loss rate. A multi-path data hierarchical fusion method based on routing algorithm is proposed in this paper. The redundant node decision method based on mesh partitioning is combined with the grid algorithm and EECRS protocol to establish a routing protocol based on redundant node elimination, and the redundant data existing in the redundant nodes of the wireless sensor network is removed. Based on the statistical data of the data, the probability density function is deduced and analyzed, and the noise filtering matrix is constructed to optimize the measurement matrix and remove the noise existing in the wireless sensor network. The multi-path scheduling technology is used to synthesize the pheromone table and the data forwarding flow table, and the ant is used to actively monitor the link QOS real-time information in the path finding process to update the pheromone table and the flow table, and calculate the link idle rate according to the update result. According to the calculation result, the data transmission path is selected, and the multi-path data layer fusion is completed. The experimental results show that the proposed method has high fusion efficiency, good denoising performance and low packet loss rate.

A Predictive Acknowledgement Model for Enhancing Traffic Redundancy Elimination in Cloud

Due to the huge increase in the utilization of cloud storage in recent days, it leads to a massive growth in data traffic from application based servers to applications like smart phones, which not only influences batteries and computational capacities but as well swamp down multi-hopping strategies during data transmission. To resolve this crisis, traffic redundancy elimination (TRE) is an effectual solution, where the chunks to be transmitted will be directly fetched out from the receivers’ cache. Moreover, prevailing solutions cannot be directly applied or it is not appropriate for smart phones owing to its energy overhead ad high computation that is imposed on the applications. In order to overcome this problem, in this investigation, a novel a Predictive Acknowledgement for Eliminating Traffic (PACKET) is proposed which comprises of three significant elements. Initially, every application possess a clone in cloud that are responsible for calculating intensive tasks like detecting redundancy and parsing traffic. Secondly, consider that every cloud user has some specific applications like Facebook to be used in regular day to day life, every clones of cloud has to selectively determine the applications that are most frequently utilized and also reduce the high redundancy ratio. Thirdly, some cloud users always possess certain common applications; the proposed PACKET clusters those clones to co-operatively perform redundancy detection so as to diminish cache resource consumption in cloud. The simulation is carried out in MATLAB environment; the traces of applications are collected from online available data and are utilized for simulation purpose. Experimental outcomes demonstrates that PACKET can attain much higher hit ratio, reduced E2E delay, increased E2E throughput, energy efficiency and effectual bandwidth utilization in contrast to existing approaches. The proposed PACKET shows better and efficient trade-off than prevailing techniques.

ABox Abduction via Forgetting in ALC

Abductive reasoning generates explanatory hypotheses for new observations using prior knowledge. This paper investigates the use of forgetting, also known as uniform interpolation, to perform ABox abduction in description logic (ALC) ontologies. Non-abducibles are specified by a forgetting signature which can contain concept, but not role, symbols. The resulting hypotheses are semantically minimal and consist of a disjunction of ABox axioms. These disjuncts are each independent explanations, and are not redundant with respect to the background ontology or the other disjuncts, representing a form of hypothesis space. The observations and hypotheses handled by the method can contain both atomic or complex ALC concepts, excluding role assertions, and are not restricted to Horn clauses. Two approaches to redundancy elimination are explored in practice: full and approximate. Using a prototype implementation, experiments were performed over a corpus of real world ontologies to investigate the practicality of both approaches across several settings.