Input String Research Articles

Automatic and lightweight grammar generation for fuzz testing

Blackbox fuzz testing can only test a small portion of code when rigorously checking the well-formedness of input values. To overcome this problem, blackbox fuzz testing is performed using a grammar that delineates the format information of input values. However, it is almost impossible to manually construct a grammar if the input specifications are not known. We propose an alternative technique: the automatic generation of fuzzing grammars using API-level concolic testing. API-level concolic testing collects constraints at the library function level rather than the instruction level. While API-level concolic testing may be less accurate than instruction-level concolic testing, it is highly useful for speedily generating fuzzing grammars that enhance code coverage for real-world programs. To verify the feasibility of the proposed concept, we implemented the system for generating ActiveX control fuzzing grammars, named YMIR. The experiment results showed that the YMIR system was capable of generating fuzzing grammars that can raise branch coverage for ActiveX control using highly-structured input string by 15–50%. In addition, the YMIR system discovered two new vulnerabilities revealed only when input values are well-formed. Automatic fuzzing grammar generation through API-level concolic testing is not restricted to the testing of ActiveX controls; it should also be applicable to other string processing program whose source code is unavailable.

Maximizing Intelligent Use Of Unintelligent Response Handling Devices

While by design unintelligent, relaxed pattern matching devices and automatic mark-up algorithms can nonetheless be used intelligently to evaluate student responses in computer-based drill exercises. One of the most troublesome aspects of answer evaluation is the occurrence of minor orthographic discrepancies in replies. Relaxed pattern matching, the ability to ignore or otherwise neutralize irrelevant details, is essential to dealing with this kind of non-differential variation. To be maximally effective, however, its use requires considerable direct intervention in the answer judging process. Automatic pattern mark-up algorithms seek to alleviate the task of the courseware author by shifting the burden of error recognition onto the language learner. Though very easy to use, based as it is upon totally mechanical displays of formal deviations between input strings and template answers, pattern mark-up returns only the most rudimentary appraisal of mistakes. Despite their intrinsic limitations, combined under flexible programming control, these two response evaluation procedures are capable of satisfactorily overcoming the most nettlesome difficulties of answer judging. By allowing the systematic categorization of student input into fundamental reply types, as determined by relative degree of matching with canonical forms, they also provide the basis of meaningful feedback in tutorial CALL.

On the Bit-Complexity of Lempel--Ziv Compression

One of the most famous and investigated lossless data-compression schemes is the one introduced by Lempel and Ziv about 30 years ago [IEEE Trans. Inform. Theory, 23 (1977), pp. 337--343]. This compression scheme is known as “dictionary-based compressor” and consists of squeezing an input string by replacing some of its substrings with (shorter) codewords which are actually pointers to a dictionary of phrases built as the string is processed. Surprisingly enough, although many fundamental results are nowadays known about the speed and effectiveness of this compression process, “we are not aware of any parsing scheme that achieves optimality$\ldots$under any constraint on the codewords other than being of equal length” [N. Rajpoot and C. Sahinalp, in Handbook of Lossless Data Compression, Academic Press, New York, 2002, p. 159]. Here optimality means to achieve the minimum number of bits in compressing each individual input string without any assumption on its generating source. In this paper we investigate...

The substring inclusion constraint longest common subsequence problem can be solved in quadratic time

In this paper, we study some variants of the Constrained Longest Common Subsequence (CLCS) problem, namely, the substring inclusion CLCS (Substring-IC-CLCS) problem and a generalized version thereof. In the Substring-IC-CLCS problem, we are to find a longest common subsequence (LCS) of two given strings containing a third constraint string (given) as a substring. Previous solution to this problem runs in cubic time, i.e, O(nmk) time, where n,m and k are the length of the 3 input strings. In this paper, we present simple O(nm) time algorithms to solve the Substring-IC-CLCS problem. We also study the Generalized Substring-IC-LCS problem where we are given two strings of length n and m respectively and an ordered list of p strings and the goal is to find an LCS containing each of them as a substring in the order they appear in the list. We present an O(nmp) algorithm for this generalized version of the problem.

Inhibitor Petri net executing an arbitrary given Markov normal algorithm

An inhibitor Petri net with a fixed structure that executes an arbitrary given Markov normal algorithm is constructed. The algorithm and its input string are encoded by nonnegative integer numbers and put into dedicated places of the Petri net, which implements the application of the algorithm productions over the string of symbols. Rules of encoding sequential, branching, and cyclic processes by Petri nets are used. Upon completion of the net operation, the output string is restored (decoded) from the integer form of the representation. Within the paradigm of computations on Petri nets, the constructed net provides the compatibility of systems.

When Maybe is not good enough

Many variations upon the theme of parser combinators have been proposed, too many to list here, but the main idea is simple: A parser for phrases of type α is a function that takes an input string and produces results (x, rest), where x is a value of type α, and rest is the remainder of the input after the phrase with value x has been consumed. The results are often arranged into a list, because this allows a parser to signal failure with the empty list of results, an unambiguous success with one result, or multiple possibilities with a longer ‘list of successes’.

UNWEIGHTED AND WEIGHTED HYPER-MINIMIZATION

Hyper-minimization of deterministic finite automata (DFA) is a recently introduced state reduction technique that allows a finite change in the recognized language. A generalization of this lossy compression method to the weighted setting over semifields is presented, which allows the recognized weighted language to differ for finitely many input strings. First, the structure of hyper-minimal deterministic weighted finite automata is characterized in a similar way as in classical weighted minimization and unweighted hyper-minimization. Second, an efficient hyper-minimization algorithm, which runs in time [Formula: see text], is derived from this characterization. Third, the closure properties of canonical regular languages, which are languages recognized by hyper-minimal DFA, are investigated. Finally, some recent results in the area of hyper-minimization are recalled.

The smoothed complexity of edit distance

We initiate the study of the smoothed complexity of sequence alignment, by proposing a semi-random model of edit distance between two input strings, generated as follows: First, an adversary chooses two binary strings of length d and a longest common subsequence A of them. Then, every character is perturbed independently with probability p , except that A is perturbed in exactly the same way inside the two strings. We design two efficient algorithms that compute the edit distance on smoothed instances up to a constant factor approximation. The first algorithm runs in near-linear time, namely d {1+ϵ} for any fixed ϵ > 0. The second one runs in time sublinear in d , assuming the edit distance is not too small. These approximation and runtime guarantees are significantly better than the bounds that were known for worst-case inputs. Our technical contribution is twofold. First, we rely on finding matches between substrings in the two strings, where two substrings are considered a match if their edit distance is relatively small, a prevailing technique in commonly used heuristics, such as PatternHunter of Ma et al. [2002]. Second, we effectively reduce the smoothed edit distance to a simpler variant of (worst-case) edit distance, namely, edit distance on permutations (a.k.a. Ulam's metric). We are thus able to build on algorithms developed for the Ulam metric, whose much better algorithmic guarantees usually do not carry over to general edit distance.

DNA Secret Writing with Laplace Transform

symmetric key cryptographic system has been proposed and it is termed as DSWLT. This proposed technique is very fast, suitable for encryption of large files. DSWLT consider the plain text (i.e. the input file) as binary string with finite no of bits. The input string converted to DNA nucleotides using DNA coding and then the DNA codes are converted to positive integers. Laplace transform is applied considering these numbers to be the co-efficient of the expansion. To provide multilevel security the resultant coefficients are converted to their binary equivalent and another level of encryption with cumulative XOR is performed and respective MSBs found at every iteration are taken to construct the cipher text. Decryption is performed in the reverse manner. Experimental results are tested, analyzed and a comparison with existing and industrially accepted TDES and AES has been performed.

Rule-restricted Automaton-grammar transducers: Power and Linguistic Applications

This paper introduces the notion of a new transducer as a two-component system, which consists of a nite automaton and a context-free grammar. In essence, while the automaton reads its input string, the grammar produces its output string, and their cooperation is controlled by a set, which restricts the usage of their rules. From a theoretical viewpoint, the present paper discusses the power of this sys- tem working in an ordinary way as well as in a leftmost way. In addition, the paper introduces an appearance checking, which allows us to check whether some symbols are present in the rewritten string, and studies its eect on the power. It achieves the following three main results. First, the system generates and accepts languages dened by matrix grammars and partially blind multi-counter automata, respec- tively. Second, if we place a leftmost restriction on derivation in the context-free grammar, both accepting and generating power of the system is equal to generative power of context-free grammars. Third, the system with appearance checking can accept and generate all recursively enumerable languages. From more pragmatical viewpoint, this paper describes several linguistic applications. A special attention is paid to the Japanese-Czech translation.

On the Complexity of Conclusive Update

This work is concerned with finite identifiability of languages from positive data. We focus on the characterization of finite identifiability [Mukouchi (1992), Lange and Zeugmann (1992)], which uses definite finite tell-tale sets (DFTTs for short), finite subsets of languages which are uniquely characteristic for them. We introduce preset learners, learning functions that explicitly use (collections of) DFTTs, and, in cases where there exist only finitely many DFTTs for each language, strict preset learners which in each case use this whole finite collection. We also introduce the concept of fastest learner, a learner which comes up with the right conjecture on any input string that objectively leaves only the right choice of language. We study the use of minimal DFTTs and their influence on the speed of finite identification. We show that: (a) in the case of finite collections of finite sets—finding a minimal DFTT is polynomial time computable, while finding a minimal-size DFTT is NP-complete; (b) in the general case—finite identifiability, minimal strict preset finite identifiability and fastest finite identifiability are shown to be mutually nonequivalent. In the end we mention the relevance of this work for dynamic epistemic logic.

이진 삼차 재귀 신경망과 유전자 알고리즘을 이용한 문맥-자유 문법의 추론

이 논문은 이진 삼차 재귀 신경망(Binary Third-order Recurrent Neural Networks: BTRNN)에 유전자 알고리즘을 적용하여 문맥-자유 문법을 추론하는 방법을 제안한다. BTRNN은 각 입력심볼에 대응되는 재귀 신경망들의 다층적 구조이고 외부의 스택과 결합된다. BTRNN의 매개변수들은 모두 이진수로 표현되며 상태 전이와 동시에 스택의 한 동작이 실행된다. 염색체로 표현된 BTRNN들에 유전자 알고리즘을 적용하여 긍정과 부정의 입력 패턴들의 문맥-자유 문법을 추론하는 최적의 BTRNN를 얻는다. 이 방법은 기존의 신경망 이용방법보다 적은 학습량과 적은 학습회수로 작거나 같은 상태 수를 갖는 BTRNN을 추론한다. 또한 문법 표현의 염색체 이용방법보다 parsing과정에서 결정적인 상태전이와 스택동작이 실행되므로 입력 패턴에 대한 인식처리 시간복잡도가 우수하다. 문맥-자유 문법의 비단말 심볼의 개수 p, 단말 심볼의 개수 q, 그리고 길이가 k인 문자열이 입력이 될 때, BTRNN의 최대 상태수가 m이라고 하면, BTRNN의 인식처리 병렬처리 시간은 O(k)이고 순차처리 시간은 O(km)이다. We present the method to infer Context-Free Grammars by applying genetic algorithm to the Binary Third-order Recurrent Neural Networks(BTRNN). BTRNN is a multiple-layered architecture of recurrent neural networks, each of which is corresponding to an input symbol, and is combined with external stack. All parameters of BTRNN are represented as binary numbers and each state transition is performed with any stack operation simultaneously. We apply Genetic Algorithm to BTRNN chromosomes and obtain the optimal BTRNN inferring context-free grammar of positive and negative input patterns. This proposed method infers BTRNN, which includes the number of its states equal to or less than those of existing methods of Discrete Recurrent Neural Networks, with less examples and less learning trials. Also BTRNN is superior to the recent method of chromosomes representing grammars at recognition time complexity because of performing deterministic state transitions and stack operations at parsing process. If the number of non-terminals is p, the number of terminals q, the length of an input string k, and the max number of BTRNN states m, the parallel processing time is O(k) and the sequential processing time is O(km).

Phonological short-term memory in conduction aphasia

Background: Within cognitive neuropsychological models conduction aphasia has been conceptualised as a phonological buffer deficit. It may affect the output buffer, the input buffer, or both. The phonological output buffer is a short-term storage, responsible for the short-term maintenance of phonological units until their articulation, as well as for phonological and morphological composition. The phonological input buffer holds input strings until they are identified in the input lexicon. Thus the phonological buffers are closely related to phonological short-term memory (pSTM), and hence it is important to assess pSTM in conduction aphasia. Because the input and output buffers play different roles, impairment in each of them predicts different impairments in the patient's ability to understand certain sentences, to learn new words and names, and to remember and recall lists of words and numbers for short time periods. This research was supported by a research grant from the National Institute for Psychobiology in Israel (Friedmann 2004-5-2b), by the Israel Science Foundation (grant no. 1296/06, Friedmann), and by the ARC Centre of Excellence in Cognition and its Disorders (CCD), Macquarie University. Aims: This study explored in detail pSTM in individuals with conduction aphasia, comparing individuals with input and output deficits, recall and recognition tasks, and stimuli of various types. It also tested pSTM in six age groups of healthy individuals, assessing the effect of age on various types of stimuli. This paper presents a new battery of 10 recall and recognition span tests, designed to assess pSTM in aphasia and to measure spans and effects on spans. Methods & Procedures: The participants were 14 Hebrew-speaking individuals with conduction aphasia, 12 with input or input-output phonological buffer deficit, and 2 with only output deficit, and 296 healthy individuals. Outcomes & Results: The analyses of the spans and effects on pSTM in the 10 tests indicated that all the participants with conduction aphasia had limited pSTM, significantly poorer than that of the control participants, and no semantic STM impairment. They had shorter spans, smaller length and similarity effects, and larger sentential effect than the controls. The individuals with conduction aphasia who had an impairment in the phonological input buffer showed deficit in both the recall and recognition span tasks. The individuals with the output conduction aphasia showed impairment only in the recall tasks. The healthy individuals showed age effect on span tasks involving words, but no effect of age on span tasks of nonwords. Conclusions: pSTM is impaired in conduction aphasia, and different pSTM impairments characterise different types of conduction aphasia. Output conduction aphasia causes difficulties only when verbal output is required, whereas input conduction aphasia also causes a deficit when only recognition is required. This suggests that rehearsal can take place without the phonological output buffer. Age differentially affects pSTM for words and nonwords in healthy adults: whereas the encoding of words changes, the ability to remember nonwords is unchanged.

Efficient retrieval of approximate palindromes in a run-length encoded string

In this paper, we study the palindrome retrieval problem with the input string compressed into run-length encoded form. Given a run-length encoded string rle(T), we show how to preprocess rle(T) to support subsequent queries of the longest palindrome centered at any specified position and having any specified number of mismatches between its arms. We present two algorithms for the problem, both taking time and space polynomial in the compressed string size. Let n denote the number of runs of rle(T) and let k denote the number of mismatches. The first algorithm, devised for small k, identifies the desired palindrome in O(logn+min{k,n}) time with O(nlogn) preprocessing time, while the second algorithm achieves O(log2n) query time, independent of k, after O(n2logn)-time preprocessing.

Factors that affect the accuracy of text-based language identification

The classification accuracy of text-based language identification depends on several factors, including the size of the text fragment to be identified, the amount of training data available, the classification features and algorithm employed, and the similarity of the languages to be identified. To date, no systematic study of these factors and their interactions has been published. We therefore investigate the effects of each of these factors and their relations on the performance of text-based language identification.Our study uses n-gram statistics as features for classification. In particular, we compare support vector machines, Naïve Bayesian and difference-in-frequency classifiers on different amounts of input text and various values of n for different amounts of training data. For a fixed value of n the support vector machines generally outperform the other classifiers, but the simpler classifiers are able to handle larger values of n. The additional computational complexity of training the support vector machine classifier may not be justified in light of importance of using a large value of n, except possibly for small sizes of the input window when limited training data is available.Our training and testing corpora consisted of text from the 11 official languages of South Africa. Within these languages distinct language families can be found. We find that it is much more difficult to discriminate languages within languages families than languages in different families. The overall accuracy on short input strings is low for this reason, but for input strings of 100 characters or more there is only a slight confusion within families and accuracies as high as 99.4% are achieved. For the smallest input strings studied here, which consist of 15 characters, the best accuracy achieved is only 83%, but when languages in different families are grouped together, this corresponds to a usable 95.1% accuracy.The relationship between the amount of training data and the accuracy achieved is found to depend on the window size: for the largest window (300 characters) about 400000 characters are sufficient to achieve close-to-optimal accuracy, whereas improvements in accuracy are found even beyond 1.6 million characters of training data for smaller windows.Our study concludes that the correlation between the factors studied significantly affect classification accuracy; therefore, to assure credible and comparable results, these factors need to be controlled in any text-based language identification task.

Unambiguous finite automata over a unary alphabet

Nondeterministic finite automata (NFA) with at most one accepting computation on every input string are known as unambiguous finite automata (UFA). This paper considers UFAs over a one-letter alphabet, and determines the exact number of states in DFAs needed to represent unary languages recognized by n-state UFAs in terms of a new number-theoretic function g˜. The growth rate of g˜(n), and therefore of the UFA–DFA tradeoff, is estimated as eΘ(nln2n3). The conversion of an n-state unary NFA to a UFA requires UFAs with g(n)+O(n2)=e(1+o(1))nlnn states, where g(n) is the greatest order of a permutation of n elements, known as Landauʼs function. In addition, it is shown that representing the complement of n-state unary UFAs requires UFAs with at least n2−o(1) states in the worst case, while the Kleene star requires up to exactly (n−1)2+1 states.

State Complexity of Finite Tree Width NFAs

We consider nondeterministic finite automata having finite tree width (ftw-NFA) where the computation on any input string has a constant number of branches. We give effective characterizations of ftw-NFAs and a tight worst-case state size bound for determinizing an ftw-NFA $A$ as a function of the tree width and the number of states of $A$. We also introduce a lower bound technique for ftw-NFAs.

A Survey of Results on Stateless Multicounter Automata

A stateless multicounter machine has m-counters operating on a one-way input delimited by left and right end markers. A move of the machine depends only on the symbol under the input head and the sign pattern of the counters. An input string is accepted if, when the input head is started on the left end marker with all counters zero, the machine eventually reaches the configuration where the input head is on the right end marker with all the counters again zero. We bring together a number of results on stateless multicounter automata of various different types: deterministic, nondeterministic, realtime (the input head moves right at every step), or non-realtime. We investigate realtime and non-realtime machines in both deterministic and nondeterministic cases with respect to the number of counters and reversals. In addition to hierarchy results, we also consider closure properties and the connections to stateless multihead automata.

LL conflict resolution using the embedded left LR parser

A method for resolving LL(k) conflicts using small LR(k) parsers (called embedded left LR(k) parsers) is described. An embedded left LR(k) parser is capable of (a) producing the prefix of the left parse of the input string and (b) stopping not on the end-of-file marker but on any string from the set of lookahead strings fixed at the parser generation time. The conditions regarding the termination of the embedded left LR(k) parser if used within LL(k) (and similar) parsers are defined and examined in-depth. It is proved that an LL(k) parser augmented with a set of embedded left LR(k) parsers can parse any deterministic context-free grammar in the same asymptotic time as LR(k) parser. As the embedded left LR(k) parser produces the prefix of the left parse, the LL(k) parser augmented with embedded left LR(k) parsers still produces the left parse and the compiler writer does not need to bother with different parsing strategies during the compiler implementation.

다양한 스마트폰 키패드 환경에서 유사 단어 검색을 위한 수정된 편집 거리 계산 방법

대부분의 스마트폰은 터치패드 기반의 가상 키패드를 사용한다. 가상 키패드는 기기의 화면 크기나 입력 방법의 물리적인 한계로 입력 오류가 자주 발생한다. 이 문제를 해결하기 위해 유사 단어를 찾기 위한 많은 연구가 있었다. 본 논문에서는 편집 거리 방법을 다양한 가상 키패드를 고려하여 수정하는 방법을 제안한다. 제안 방법은 다양한 키패드에서 발생하는 입력 오류를 효과적으로 해결하기 위해, 입력 문자열을 사용자가 실제 누르게 되는 입력열로 변환하고, 가상 키패드의 특성에 따라 편집 비용을 수정하였다. 다양한 키패드에서 실험한 결과 제안 방법이 일반적인 편집 거리 방법을 이용한 것 보다 좋은 성능을 보였다. Most smartphone use virtual keypads based on touch-pad. The virtual keypads often make typographical errors because of the physical limitations of device such as small screen and limited input methods. To resolve this problem, many similar word-finding methods have been studied. In the paper, we propose an edit distance method (a well-known string similarity measure) that is modified to consider various types of virtual keypads. The proposed method effectively covers typographical errors in various keypads by converting an input string into a physical key sequence and by reflecting characteristics of virtual keypads to edit scores. In the experiments with various keypads, the proposed method showed better performances than a typical edit distance method.