Automatic Language Processing Research Articles

The Nature of Referent Resolution in Japanese E-mail

We examine a corpus of naturally occurring data (Japanese e-mail texts) in order to explore the nature of referent resolution in such a corpus. Specifically, our aims are to give a characterization of Japanese e-mail, and to describe reference resolution processes operating in that medium in terms that will make them tractable for automatic language processing. Because the centering approach to modeling discourse phenomena is well defined computationally, we use that as a starting point for our analysis. We begin by describing the nature of Japanese e-mail, characterizing it both quantitatively and qualitatively. We then examine each instance of anaphora in the corpus to determine the process by which its reference can be resolved. Finally, we suggest a computationally tractable description for each type of reference and estimate how successful the application of each description would be. Our analysis gives a specific and empirically validated account of the possibility of processing reference automatically using centering parameters as a foundation.

Automatic language and information processing: rethinking evaluation

System evaluation has mattered since research on automatic language and information processing began. However, the (D)ARPA conferences have raised the stakes substantially in requiring and deliveri...

Automatic language and information processing: rethinking evaluation

System evaluation has mattered since research on automatic language and information processing began. However, the (D)ARPA conferences have raised the stakes substantially in requiring and delivering systematic evaluations and in sustaining these through long term programmes; and it has been claimed that this has both significantly raised task performance, as defined by appropriate effectiveness measures, and promoted relevant engineering development. These controlled laboratory evaluations have made very strong assumptions about the task context. The paper examines these assumptions for six task areas, considers their impact on evaluation and performance results, and argues that for current tasks of interest, e.g. summarising, it is now essential to play down the present narrowly-defined performance measures in order to address the task context, and specifically the role of the human participant in the task, so that new measures, of larger value, can be developed and applied.

On adaptive decision rules and decision parameter adaptation for automatic speech recognition

Recent advances in automatic speech recognition are accomplished by designing a plug-in maximum a posteriori decision rule such that the forms of the acoustic and language model distributions are specified and the parameters of the assumed distributions are estimated from a collection of speech and language training corpora. Maximum-likelihood point estimation is by far the most prevailing training method. However, due to the problems of unknown speech distributions, sparse training data, high spectral and temporal variabilities in speech, and possible mismatch between training and testing conditions, a dynamic training strategy is needed. To cope with the changing speakers and speaking conditions in real operational conditions for high-performance speech recognition, such paradigms incorporate a small amount of speaker and environment specific adaptation data into the training process. Bayesian adaptive learning is an optimal way to combine prior knowledge in an existing collection of general models with a new set of condition-specific adaptation data. In this paper, the mathematical framework for Bayesian adaptation of acoustic and language model parameters is first described. Maximum a posteriori point estimation is then developed for hidden Markov models and a number of useful parameters densities commonly used in automatic speech recognition and natural language processing.

CORPORATE ACTIVITIES IN SPEECH RECOGNITION AND NATURAL LANGUAGE: ANOTHER "NEW SCIENCE"-BASED TECHNOLOGY

We have used data on patents and publications, and from an Internet-based survey, to analyse corporate technological activities in Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) technologies. Two distinct clusters of firms exist: large firms mainly in telecommunications, desktop computing and consumer electronics; and small firms specialising in speech technologies. The small specialised firms depend heavily on nearby universities and public research institutes and, to some extent, on nearby large firms; their relations with the large firms are complementary as well as competitive. Similar patterns can be observed in other, recently emerging, "new science"-based technologies. Integration between ASR and NLP has so far been weak with the two research communities functioning more or less independently, with the former progressing more rapidly than the latter. Having built technological capabilities in ASR and NLP with a small proportion of their corporate technological resources, the large firms have two options depending on the rate of progress in these technologies (especially NLP) in the future. If it is high, substantial investments (including those in complementary technologies) could open up massive market opportunities. If it is low, modest investments will allow the exploitation of niche markets.

Wordplay and Translation

The present bibliography does not intend to cover the whole problem of humour in translation, nor that of ambiguity in automatic language processing and translation; both require at once a broader and a narrower perspective. The selection of entries generally favours more recent material and reflects the compilers’ unfortunate lack of competence in non-European languages. Priority is given to material that combines an attempt at conceptualization with an empirical orientation. Questions of translation and polysemy alike are crucial to philosophically inspired work along deconstructionist lines:for an introduction to thisfield, see Davis (Forthcoming). Coverage of unpublished theses is limited to doctoral dissertations. Articles in Laurian (1986), ‘Meta’ (1989), ‘New Comparison’ (1987), and Delabastita (Forthcoming) are not listed as separate entries.

Commercial applications of speech interface technology: an industry at the threshold.

Speech interface technology, which includes automatic speech recognition, synthetic speech, and natural language processing, is beginning to have a significant impact on business and personal computer use. Today, powerful and inexpensive microprocessors and improved algorithms are driving commercial applications in computer command, consumer, data entry, speech-to-text, telephone, and voice verification. Robust speaker-independent recognition systems for command and navigation in personal computers are now available; telephone-based transaction and database inquiry systems using both speech synthesis and recognition are coming into use. Large-vocabulary speech interface systems for document creation and read-aloud proofing are expanding beyond niche markets. Today's applications represent a small preview of a rich future for speech interface technology that will eventually replace keyboards with microphones and loud-speakers to give easy accessibility to increasingly intelligent machines.

Stochastic modeling in spoken dialogue system design

In this paper, we review the current state of the art in stochastic modeling for spoken dialogue system design. We discuss acoustic modeling of speech units for automatic speech recognition and language modeling of linguistic units for natural language processing. We point out some of the emerging stochastic modeling techniques and show the similarity between language modeling and acoustic modeling. Finally, we address search and decision issues related to the integration of knowledge sources for automatic speech recognition and natural language processing.

Progress in the Application of Natural Language Processing to Information Retrieval Tasks

Techniques of automatic natural language processing have been under development since the earliest computing machines, and in recent years these techniques have proven to be robust, reliable and efficient enough to lead to commercial products in many areas. The applications include machine translation, natural language interfaces and the stylistic analysis of texts but NLP techniques have also been applied to other computing tasks besides these. In this paper we will examine and review recent progress in using the lexical, syntactic, semantic and discourse levels of the language analysis for tasks like automatic and semi-automatic indexing of text, text retrieval, text abstracting and summarisation, thesaurus generation from text corpus and conceptual information retrieval. Our own work on the application of syntactic analysis to the matching and ranking of phrases using structured representations of texts, will be included in the overview. Finally, the prospects for gains in terms of overall retrieval effectiveness or quality will be discussed.

The application of morpho-syntactic language processing to effective phrase matching

The application of automatic natural language processing techniques to the indexing and the retrieval of text information has been a target of information retrieval researchers for some time. Incorporating semantic-level processing of language into retrieval has led to conceptual information retrieval, which is effective but usually restricted in its domain. Using syntactic-level analysis is domain-independent, but has not yet yielded significant improvements in retrieval quality. This paper describes a process whereby a morpho-syntactic analysis of phrases or user queries is used to generate a structured representation of text. A process of matching these structured representations is then described that generates a metric value or score indicating the degree of match between phrases. This scoring can then be used for ranking the phrases. In order to evaluate the effectiveness or quality of the matching and scoring of phrases, some experiments are described that indicate the method to be quite useful. Ultimately the phrasematching technique described here would be used as part of an overall document retrieval strategy, and some future work towards this direction is outlined.

Prospects for intelligent, language‐based information retrieval

Current approaches to text retrieval based on indexing by words or index terms and on retrieving by specifying a Boolean combination of keywords are well known, as are their limitations. Statistical approaches to retrieval, as exemplified in commercial products like STATUS/IQ and Personal Librarian, are slightly better but still have their own weaknesses. Approaches to the indexing and retrieval of text based on techniques of automatic natural language processing (NLP) may soon start to realise their undoubted potential in terms of improving the quality and effectiveness of information retrieval. In this article we will explore what that potential is. We will divide information retrieval functionality into conceptual and traditional information retrieval and we will examine some of the current attempts at using various NLP techniques in both the indexing and retrieval operations.

Extensive Acquisition of Syntactic Knowledge in NLP Systems

Most NLP programs have failed to ‘scale-up’ because of the difficulty of developing large and robust lexical knowledge bases In our view, the development of tools for rapid prototyping and testing of domain-competent grammars has priority over the settlement of existing grammar theories In addition, the extensive acquisition of linguistic knowledge allows for a systematic, field evaluation of existing linguistic theories and demonstrates their adequacy to the purpose of automatic language processing. The important issue of semantic learning has been discussed in other papers (Velardi and Pazienza 1989): In this Paper, the focus is on the acquisition and testing of natural language grammars, to be used at different stages of natural language parsing, such as inflectional morphology, syntactic analysis, and sentence generation. The objectives of this research are: first, to render the activity of writing and customizing grammars better understood and time-con-stramed; secondly, to evaluate via extensive experimentation the practical limitations and advantages of Augmented Context Free Grammars (ACFG) in NLP; thirdly, to achieve uniformity in parsing. That one benefits from the view that developing grammars can be seen as manipulation of strings, without worrying about their content or associated content. The grammar development environment presented in this paper is a meta-analyzer, called META, that produces arbitrary semiotic relations between form and meaning. META is a flexible tool for string manipulation, independent of the morphologic, syntactic, or semantic interpretation of the attribute values synthesized by the production rules

Elements of Mathematical Linguistics

Most of this book (the first 148 pages) is an English version of an introduction to linguistics originally published in Russian in 1969. This material has been expanded into a more standard-sized book for the English-speaking market by tacking on three articles on special topics in linguistics published elsewhere by one or both co-authors; but each of these articles is already available in English translation, and the book stands or falls by the usefulness of its first part. When Gladkij and Mel'~uk wrote their book in the late 1960s, linguistics was little known, even in the USA where most of it had been created not long before. I guess that it was a fairly remarkable achievement on their part to discover and master this exotic discipline (and, in Gladkij 's case, to make a number of original contributions to it), and then to make it known to their fellow-countrymen. However, whether it makes good sense to bring out an English version of the book in the 1980s is a different matter. I must say that I feel the answer is no, for many reasons. In the first place, linguistics has moved on since the 1960s. Gladkij and Mel'~uk's book deals ahnost exclusively with the hierarchy of types of language, defined in terms of types of grammar (unrestricted rewrite systems, context-sensitive grammars, context-free grammars, one-sided linear grammars), with a small amount on the relationship with types of automata, on decidability theorems, and a few other matters. topic of parsing and formalisms with special relevance for automatic language processing, such as the ATN are not mentioned. In 1969, one would not have expected them to be; in the 1980s, these are surely indispensable components even in an introduction to mathematical (as opposed to computat ional) linguistics. Gladkij and Mel'~uk are quite explicit about the fact that they have made no attempt to bring the book up to date: The manuscript of the Russian version of this book was completed in 1967 and we are not in a position to revise it now . . . . Even a mere list of references would be out of the question. Secondly, the book relies heavily on Russian-language examples which are lost on an English reader. Thus, early chapters largely revolve round a largish formal grammar produced by the authors in order to generate the complex range of Russian participles. It must have been a significant virtue of the original book that it demonstrated how the concepts of linguistics could be made to achieve a novel task relating specifically to the readers ' own language, but for a British or American student the result is that ideas many people find difficult at the best of times are rendered wholly opaque. And, finally, Gladkij and Mel'~uk just are not very good at writing for an unknowledgeable audience. They make their formal rules notationally much more complex and exotic-looking than they need be to a fellow mathematician, a trivial matter, but for an unconfident student very unfortunate. On page 25 they use the term monoid without, I think, ever explaining what it means (there is no index, so it is hard to check); on page 81 they use a technical term of their own which is first defined on page 122. fact is that there are now enough English-language books that do the same job as this book and do it much better.

The Analysis of Belief Sentences in the Philosophy of Language

The present stage of development of the philosophy of language is marked by the joining of efforts of philosophers, logicians, and linguists for solution of one of the most difficult problems posed by the progress of scientific knowledge - the problem of modeling the aspect of meaning, of semantics, of natural language. The solution of this problem governs not only the building of a general theory of language, not only scientific explanation of the phenomenon of the understanding of language and solution of numerous problems of practical importance with respect to automatic language processing, but also the defining of the place and function of language in the process of cognition and understanding of the world, a matter that is fundamental to methodology.

The contextological dictionary: Use in programmed language teaching

According to the theory of determinants put forward by the present author and described first in , every meaning (translation) of a polysemantic word differing from the most general one (the general procedure of the translation algorithm) is determined in the context by other words with which it forms combinations, or by some grammatical distinguishers. In other words, it is believed that in most cases the differentiation of meanings is possible on the basis of formal characteristics of the word itself and its context; the cases when such differentiation is impossible are rare enough. The ACD, being a collection of algorithms, is based on certain grammars of English and Russian and presupposes some special features of the general and auxiliary algorithms with which that dictionary works. The grammar of English is defined mainly as follows: -all English words have been divided into grammatical classes (corresponding in general to traditional parts of speech: noun, adjective, verb, adverb, preposition, conjunction, punctuation mark; the last is considered as forming a grammatical class because of the role it plays in syntactic analysis-for example, a comma-and besides, a punctuation mark also receives a translation equivalent). In the automatic language processing, pronouns and numerals are divided according to their functions, into subclasses of nouns and adjectives. Figures form a special subclass.4 In the classes of nouns and adjectives, special distributional semantic subclasses are singled out. These consist of nouns: conjunctive (which, who), pronominal (I, he, it), numerals (one, two), nouns of time and duration, geographical names, periodicals, names of organizations

Computer understanding of natural language

The state of research in automatic natural language processing at Yale University is described. Five programs are discussed: SAM, FRUMP, PAM, TALESPIN, and POLITICS.

Language universalization for improved information management: The necessity for esperanto

Lacking a universal working language, information managers around the world cannot now deal reliably and efficiently with multilingual documentation. Language mismatch paralyzes international cooperative efforts such as multinational bibliographic standardization, linking of collections, and sharing the work of classification and indexing. Knowledge of the same second language by all information managers can open the communication channels needed for worldwide cooperation. Ethnic and ideological rivalries preclude success in this role by any of the conventional languages. The planned language, Esperanto, is the logical choice because of its neutrality, rational structure, clarity and expressive power. Pioneering projects in automatic language processing, not possible in English, are feasible in Esperanto.

ARTIFICIAL INTELLIGENCE—A NEW DISCIPLINE IN THE COMPUTER SCIENCES (A VIEWPOINT)

Some recent work in Artificial Intelligence concerning machine vision, game‐playing and automatic language processing is reviewed and related to Turing's early requirements for an “intelligent machine”. The relationship between computer science and artificial intelligence is discussed from the software angle — programming and programming languages.

1973 student paper award. An alogorithm for generating structural surrogates of english text

AbstractThis paper describes the development and application of an algorithm which generates non‐linear representations of English text. The algorithm uses the results of a syntactic analysis system and a set of rules which prescribe linkages to generate a graph of a sentence. The shape of these graphs corresponds to the syntax of the sentence; the labels correspond to the vocabulary of the sentence and the edge types correspond to case grammar roles. The sentence graphs can then be interconnected at common nodes and analyzed according to common edges. Preliminary experimentation has yielded promising results. It appears that the algorithm produces a representation of English text which could be quite useful in automatic language processing.

The resolution of syntactic ambiguity in automatic language processing

This paper describes how the problem of resolution of syntactic ambiguities is approached in the parser PHRASE, developed for use in experiments in automatic indexing and extracting. PHRASE is a multilevel parser for declarative sentences, in which the syntactic structure is built up in four stages. At each stage ambiguities on that level are resolved to yield but one structure for each sentence. At each level the resultant structure is designed to be useful in its own right, and also to form the basis for the analyses at the next higher level. At the various levels of the parser, ambiguity is resolved by rules on legal juxtaposition of parts of speech, legal combination of phrases, constraints on sentence structure, and semantic word government by common nouns and verbs. The nature of these rules is discussed for all four levels of analysis, and examples of the level 1 and 2 analyses, which have been implemented, are given.