Approximate Matching Techniques Research Articles

M3F: A novel multi-session and multi-protocol based malware traffic fingerprinting

In recent years, cyber attacks have become increasingly frequent, which has had a tremendous negative impact on public life and social order. Accurately and quickly finding malware traffic from massive network traffic is one of the keys to defending against network attacks. Traditional detection methods, whether signature-based or machine learning-based, use a packet or a session as the smallest detection unit. Usually, malware creates more than one session while executing malicious functions. Detecting these sessions alone without contextual information is prone to false negatives and false positives.To address these problems, we propose M3F, a Multi-session and Multi-protocol based Malware traffic Fingerprinting that uses multiple related sessions with different protocols as the smallest classification unit. We associate multiple sessions of different protocols together to form several session sequences. For each session in a session sequence, we represent it with a state with three features so that a session sequence can be transformed into a state sequence. For a malware family, we learn a first-order homogeneous Markov chain using its state sequences as its traffic fingerprint (aka M3F). M3Fs reflect the dynamics of malware communication traffic. Meanwhile, the approximate matching technique is used to deal with the evolution of malware, which can improve the recall rate. Armed with M3F, we can locate malware traffic in massive network traffic. We use a large amount of malicious traffic to verify M3F. The experimental results show that M3F has 99.41% precision and 99.54% recall, both outperforming the baseline methods. It does not mark normal traffic as malicious traffic. Additionally, M3F is well interpretable and is the network-level representation of the malware’s behavior.

Genome Subsequences Assembly Using Approximate Matching Techniques in Hadoop

Sequencing DNA will provide valuable insights into several aspects of human life. The major requirement of this domain is for a faster and more accurate sequencing mechanism. The process becomes difficult due to the huge size of DNA. This paper presents an effective genome assembly technique in Hadoop architecture using MapReduce. The fragment assembly is based on initially matching the subsequences and then depending on the matching levels, the final complete matching subsequences are filtered. The consensus alignment and recalibration are performed using Greedy approximate matching techniques. The experimental results show that our approach is more accurate and exhibits better coverage; however, the processing time is found to be high. In future, our contributions will be based on reducing the processing time. Discussions about these techniques are also presented in this paper.

Consistency-driven alternating optimization for multigraph matching: a unified approach.

The problem of graph matching (GM) in general is nondeterministic polynomial-complete and many approximate pairwise matching techniques have been proposed. For a general setting in real applications, it typically requires to find the consistent matching across a batch of graphs. Sequentially performing pairwise matching is prone to error propagation along the pairwise matching sequence, and the sequences generated in different pairwise matching orders can lead to contradictory solutions. Motivated by devising a robust and consistent multiple-GM model, we propose a unified alternating optimization framework for multi-GM. In addition, we define and use two metrics related to graphwise and pairwise consistencies. The former is used to find an appropriate reference graph, which induces a set of basis variables and launches the iteration procedure. The latter defines the order in which the considered graphs in the iterations are manipulated. We show two embodiments under the proposed framework that can cope with the nonfactorized and factorized affinity matrix, respectively. Our multi-GM model has two major characters: 1) the affinity information across multiple graphs are explored in each iteration by fixing part of the matching variables via a consistency-driven mechanism and 2) the framework is flexible to incorporate various existing pairwise GM solvers in an out-of-box fashion, and also can proceed with the output of other multi-GM methods. The experimental results on both synthetic data and real images empirically show that the proposed framework performs competitively with the state-of-the-art.

Network-theoretic classification of parallel computation patterns

Parallel computation in a high-performance computing environment can be characterized by the distributed memory access patterns of the underlying algorithm. During execution, networks of compute nodes exchange messages that indirectly exhibit these access patterns. Identifying the algorithm underlying these observable messages is the problem of latent class analysis over information flows in a computational network. Towards this end, our work applies methods from graph and network theory to classify parallel computations solely from network communication patterns. Pattern classification has applications to several areas including anomaly detection, performance analysis, and automated algorithm replacement. We discuss the difficulties encountered by previous efforts, introduce two new approximate matching techniques, and compare these approaches using massive datasets collected at Lawrence Berkeley National Laboratory.

SEGA: Semiglobal Graph Alignment for Structure-Based Protein Comparison

Comparative analysis is a topic of utmost importance in structural bioinformatics. Recently, a structural counterpart to sequence alignment, called multiple graph alignment, was introduced as a tool for the comparison of protein structures in general and protein binding sites in particular. Using approximate graph matching techniques, this method enables the identification of approximately conserved patterns in functionally related structures. In this paper, we introduce a new method for computing graph alignments motivated by two problems of the original approach, a conceptual and a computational one. First, the existing approach is of limited usefulness for structures that only share common substructures. Second, the goal to find a globally optimal alignment leads to an optimization problem that is computationally intractable. To overcome these disadvantages, we propose a semiglobal approach to graph alignment in analogy to semiglobal sequence alignment that combines the advantages of local and global graph matching.

Image Compression using Approximate Matching and Run Length

Image compression is currently a prominent topic for both military and commercial researchers. Due to rapid growth of digital media and the subsequent need for reduced storage and to transmit the image in an effective manner Image compression is needed. Image compression attempts to reduce the number of bits required to digitally represent an image while maintaining its perceived visual quality. This study concentrates on the lossless compression of image using approximate matching technique and run length encoding. The performance of this method is compared with the available jpeg compression technique over a wide number of images, showing good agreements.

Evolutionary construction of multiple graph alignments for the structural analysis of biomolecules

The concept of multiple graph alignment (MGA) has recently been introduced as a novel method for the structural analysis of biomolecules. Using approximate graph matching techniques, this method enables the robust identification of approximately conserved patterns in biologically related structures. In particular, MGA enables the characterization of functional protein families independent of sequence or fold homology. This article first recalls the concept of MGA and then addresses the problem of computing optimal alignments from an algorithmic point of view. In this regard, a method from the field of evolutionary algorithms is proposed and empirically compared with a hitherto existing heuristic approach. Empirically, it is shown that the former yields significantly better results than the latter, albeit at the cost of an increased runtime.

An efficient graph-based recognizer for hand-drawn symbols

We describe a trainable, multi-stroke symbol recognizer for pen-based user interfaces. The approach is insensitive to orientation, non-uniform scaling, and drawing order. Symbols are represented internally as attributed relational graphs describing both the geometry and topology of the symbols. Symbol definitions are statistical models, which makes the approach robust to variations common in hand-drawn shapes. Symbol recognition requires finding the definition symbol whose attributed relational graph best matches that of the unknown symbol. Much of the power of the approach derives from the particular set of attributes used, and our metrics for measuring similarity between graphs. One challenge addressed in the current work is how to perform the graph matching efficiently. We present five approximate matching techniques: stochastic matching, which is based on stochastic search; error-driven matching, which uses local matching errors to drive the solution to an optimal match; greedy matching, which uses greedy search; hybrid matching, which uses exhaustive search for small problems and stochastic matching for larger ones; and sort matching, which relies on geometric information to accelerate the matching. Finally, we present the results of a user study, and discuss the tradeoffs between the various matching techniques.

Optimal spaced seeds for faster approximate string matching

Filtering is a standard technique for fast approximate string matching in practice. In filtering, a quick first step is used to rule out almost all positions of a text as possible starting positions for a pattern. Typically this step consists of finding the exact matches of small parts of the pattern. In the followup step, a slow method is used to verify or eliminate each remaining position. The running time of such a method depends largely on the quality of the filtering step, as measured by its false positives rate. The quality of such a method depends on the number of true matches that it misses, that is, on its false negative rate. A spaced seed is a recently introduced type of filter pattern that allows gaps (i.e. do not cares) in the small sub-pattern to be searched for. Spaced seeds promise to yield a much lower false positives rate, and thus have been extensively studied, though heretofore only heuristically or statistically. In this paper, we show how to design almost optimal spaced seeds that yield no false negatives.

SAGA: a subgraph matching tool for biological graphs

With the rapid increase in the availability of biological graph datasets, there is a growing need for effective and efficient graph querying methods. Due to the noisy and incomplete characteristics of these datasets, exact graph matching methods have limited use and approximate graph matching methods are required. Unfortunately, existing graph matching methods are too restrictive as they only allow exact or near exact graph matching. This paper presents a novel approximate graph matching technique called SAGA. This technique employs a flexible model for computing graph similarity, which allows for node gaps, node mismatches and graph structural differences. SAGA employs an indexing technique that allows it to efficiently evaluate queries even against large graph datasets. SAGA has been used to query biological pathways and literature datasets, which has revealed interesting similarities between distinct pathways that cannot be found by existing methods. These matches associate seemingly unrelated biological processes, connect studies in different sub-areas of biomedical research and thus pose hypotheses for new discoveries. SAGA is also orders of magnitude faster than existing methods. SAGA can be accessed freely via the web at http://www.eecs.umich.edu/saga. Binaries are also freely available at this website.

Bit-Parallel Witnesses and Their Applications to Approximate String Matching

We present a new bit-parallel technique for approximate string matching. We build on two previous techniques. The first one, BPM (Myers, 1999), searches for a pattern of length m in a text of length n permitting k differences in $O(\lceil m/w \rceil n)$ time, where w is the width of the computer word. The second one, ABNDM (Navarro and Raffinot, 2000), extends a sublinear-time exact algorithm to approximate searching. ABNDM relies on another algorithm, BPA (Wu and Manber, 1992), which makes use of an $O(k \lceil m/w \rceil n)$ time algorithm for its internal workings. BPA is slow but flexible enough to support all operations required by ABNDM. We improve previous ABNDM analyses, showing that it is average-optimal in number of inspected characters, although the overall complexity is higher because of the $O(k \lceil m/w \rceil )$ work done per inspected character. We then show that the faster BPM can be adapted to support all the operations required by ABNDM. This involves extending it to compute edit distance, to search for any pattern suffix, and to detect in advance the impossibility of a later match. The solution to those challenges is based on the concept of a witness, which permits sampling some dynamic programming matrix values to bound, deduce or compute others fast. The resulting algorithm is average-optimal for m ≤ w, assuming the alphabet size is constant. In practice, it performs better than the original ABNDM and is the fastest algorithm for several combinations of m, k and alphabet sizes that are useful, for example, in natural language searching and computational biology. To show that the concept of witnesses can be used in further scenarios, we also improve a recent variant of BPM. The use of witnesses greatly improves the running time of this algorithm too.

Parametric Sequence Alignment with Constraints

Approximate matching techniques based on string alignment are important tools for investigating similarities between strings, such as those representing DNA and protein sequences. We propose a constraint based approach for parametric sequence alignment which allows for more general string alignment queries where the alignment cost can itself be parameterized as a query with some initial constraints. Thus, the costs need not be fixed in a parametric alignment query unlike the case in normal alignment. The basic dynamic programming string edit distance algorithm is generalized to a naive algorithm which uses inequalities to represent the alignment score. The naive algorithm is rather costly and the remainder of the paper develops an improvement which prunes alternatives where it can and approximates the alternatives otherwise. This reduces the number of inequalities significantly and strengthens the constraint representation with equalities. We present some preliminary results using parametric alignment on some general alignment queries.

Structural matching and discovery in document databases

Structural matching and discovery in documents such as SGML and HTML is important for data warehousing [6], version management [7, 11], hypertext authoring, digital libraries [4] and Internet databases. As an example, a user of the World Wide Web may be interested in knowing changes in an HTML document [2, 5, 10]. Such changes can be detected by comparing the old and new version of the document (referred to as structural matching of documents). As another example, in hypertext authoring, a user may wish to find the common portions in the history list of a document or in a database of documents (referred to as structural discovery of documents). In SIGMOD 95 demo sessions, we exhibited a software package, called TreeDiff [13], for comparing two latex documents and showing their differences. Given two documents, the tool represents the documents as ordered labeled trees and finds an optimal sequence of edit operations to transform one document (tree) to the other. An edit operation could be an insert, delete, or change of a node in the trees. The tool is so named because documents are represented and compared using approximate tree matching techniques [9, 12, 14].

Classification and representation of design knowledge for circulating fluidized bed boiler

The knowledge components and the solution heuristics used by expert designers of Circulating Fluidized Bed Boilers (CFBB) have been analysed using a knowledge-based approach. Techniques for knowledge acquisition have been found to be effective for structuring the design process, and formalizing the domain knowledge which had initially been found to be less structured compared to that required for the design of more conventional equipment. It has been shown that design automation of CFBB requires some unique features arising primarily because the equipment is complex and its technology is still evolving. Designers tend to adopt a stepwise refinement procedure consisting of a configuration synthesis step, preliminary design phase and a detailed design phase. Designers often utilise data and experience of previous successful designs. An interesting aspect of the design process is the use of constraints as a means of reducing design complexity and as a substitute for optimisation. The paper also describes the features, structure, knowledge representation and problem-solving aspects of an expert system which has been developed for the preliminary design of a CFB boiler using the methodology and knowledge components of expert designers. A conventional expert-system shell enhanced with supportive paradigms like approximate matching, experience tables, numerical computation and database has been used for the implementation. The expert designer software encodes design knowledge in production rule and experience table form. This provides the capability to assume default values of selected parameters (when the user fails to provide them) obtained from the so-called “experience tables” of previous successful designs by an approximate matching technique. Thus, evolving design experience can be incorporated.

Channel Friction and Slope Effects on Harbor Resonance

An approximate matching technique uses separate solutions in a semi-infinite ocean bounded by a straight coastline, a harbor basin, and a narrow straight entrance channel which connects them. Wave height and velocity are matched at the interfaces. Such an approach allows different solutions to be used in any of the three regions, when the physical description of that region is modified, without changing the solutions used in the remaining regions. In the present work, the entrance channel is first considered to possess a wall/bottom friction although the remaining regions are kept frictionless. Results indicate that the channel effect, which leads to increasing resonance amplification with increasing channel length analogous to the well-known harbor paradox, is counteracted by the increase in frictional effects for a longer channel. The second problem considered modified the entrance channel to have a linear slope from a constant depth ocean to a harbor basin at some other constant depth.