Unrestricted Text Research Articles

Natural language information retrieval

In this paper we describe an information retrieval system in which advanced natural language processing techniques are used to enhance the effectiveness of term-based document retrieval. The backbone of our system is a traditional statistical engine that builds inverted index files from pre-processed documents, and then searches and ranks the documents in response to user queries. Natural language processing is used to (a) preprocess the documents in order to extract content-carrying terms, (b) discover inter-term dependencies and build a conceptual hierarchy specific to the database domain, and (c) process the user's natural language requests into effective search queries. During the course of the Text REtrieval Conferences, TREC-1 and TREC-2, ∗ ∗ See Harman (1993) for a detailed description of TREC. our system has evolved from a scaled-up prototype, originally tested on such collections as CACM-3204 and Cranfield, to its present form, which can be effectively used to process hundreds of millions of words of unrestricted text.

Gauging Similarity with n -Grams: Language-Independent Categorization of Text

A language-independent means of gauging topical similarity in unrestricted text is described. The method combines information derived from n-grams (consecutive sequences of n characters) with a simple vector-space technique that makes sorting, categorization, and retrieval feasible in a large multilingual collection of documents. No prior information about document content or language is required. Context, as it applies to document similarity, can be accommodated by a well-defined procedure. When an existing document is used as an exemplar, the completeness and accuracy with which topically related documents are retrieved is comparable to that of the best existing systems. The results of a formal evaluation are discussed, and examples are given using documents in English and Japanese.

Wrap-Up: a Trainable Discourse Module for Information Extraction

The vast amounts of on-line text now available have ledto renewed interest in information extraction (IE) systems thatanalyze unrestricted text, producing a structured representation ofselected information from the text. This paper presents a novel approachthat uses machine learning to acquire knowledge for some of the higher level IE processing. Wrap-Up is a trainable IE discourse component that makes intersentential inferences and identifies logicalrelations among information extracted from the text. Previous corpus-based approaches were limited to lower level processing such as part-of-speech tagging, lexical disambiguation, and dictionary construction. Wrap-Up is fully trainable, and not onlyautomatically decides what classifiers are needed, but even derives the featureset for each classifier automatically. Performance equals that of a partially trainable discourse module requiring manual customization for each domain.

Phonetic hidden Markov model speech synthesizer

A method and a system for synthesizing speech from unrestricted text, based on the principle of associating a written string of text with a sequence of speech features vectors that most probably model the corresponding speech utterance. The synthesizer is based on the interaction between two different Ergodic Hidden Markov Models: an acoustic model reflecting the constraints on the acoustic arrangement of speech, and a phonetic model interfacing phonemic transcription to the speech features representation.

A learning scheme for information retrieval in hypertext

In proposing a searching strategy well suited to the hypertext environment, we have considered four criteria: (1) the retrieval scheme should be integrated into a large hypertext environment; (2) the retrieval process should be operable with an unrestricted text collection; (3) the processing time should be reasonable; and (4) the system should be capable of learning in order to improve its retrieval effectiveness. To satisfy these four criteria, we have designed and implemented a search strategy for hypertext systems based on an extended Boolean model (the p-norm scheme) and supplemented it with links to improve the ranking of the retrieved items in a sequence most likely to fulfill the intent of the user. These links, representing additional information about document content, are established according to the requests and relevance judgments. Using a fully automatic procedure, our retrieval scheme can be applied to most existing systems. Based on the CACM test collection, which includes 3,204 documents and the CISI corpus (1,460 documents), we have built a hypertext and evaluated our proposed strategy. The retrieval effectiveness of our solution presents encouraging results.

Speech synthesis in telecommunications

A text-to-speech synthesis system that synthesizes speech from unrestricted text is discussed. The text analysis system, which includes text preprocessing, phrasing and intonation, and letter-to-phoneme conversion, is described. The analyzed text is represented by phonetic characters, stress values, minor- and major-phrase markers, and intonational descriptors. The synthesizer uses this information to compute a speech signal in several stages. The duration of the different speech events is computed, and the intonational descriptors are converted to a fundamental frequency contour. Loudness control is also generated. After these prosodic parameters have been computed, the synthesis parameters that describe the different sounds or phonemes are generated. These parameters are converted to speech by a waveform synthesizer. >

Pitch accent in context predicting intonational prominence from text

Explaining speakers' choice of items to emphasize or de-emphasize intonationally has been an important topic in theoretical linguistics, as well as in applications such as speech synthesis, where accent decisions affect naturalness as well as interpretation. Heretofore, most researchers have assumed that detailed syntactic, semantic, and discourse-level information must be available in order for accent assignment to be predicted successfully. However, a series of recent experiments on corpora of recorded (read) speech and spontaneous (elicited) speech suggest that it is indeed possible to model human accent strategies with fair success (80–98% correct) for unrestricted text—with only the tools for automatic text analysis currently available. The algorithm developed from these experiments is currently used to assign pitch accent in the Bell Laboratories Text-to-Speech System.

Prosodic variation for text‐to‐speech synthesis

Appropriate intonational variation is critical for text‐to‐speech synthesis to convey appropriate meaning and improve naturalness. While in the past it has been thought that a full linguistic analysis is prerequisite to the generation of appropriate intonational features, in recent years considerable progress has been made in assigning prosodic variation in text‐to‐speech synthesis for unrestricted text, using simple and currently available techniques for text analysis. Techniques for varying pitch range, prominence, and phrasing based on such text analysis will be discussed. Also discussed will be techniques, such as phrasing and accent assignment procedures. Such automatic training allows quicker development of new prosodic variation procedures and also allows such procedures to be tailored to particular speaking styles or applications.

Current MT research orientation/disorientation

Somers' article brings into focus the state of technology in the subject of MT. In a certain measure, the reflection of Somers “despite nearly 25 years since the ALPAC report...for all the investment...perhaps one could have expected better results” should be applied not only to MT systems, but equally to NLP systems. However, this reflection of Somers should be interpreted and reevaluated in accordance with the distance between the evolution of our knowledge of the mecanisms of translatology and the ambitious objectives of producing FAHQT for more than 80% of sentences in unrestricted texts.

Coherent user interfaces for language-based editing systems

Many kinds of complex documents, including programs, are based on underlying formal languages. Language-based editing systems exploit knowledge of these languages to provide services beyond the scope of traditional text editors. To be effective, these services must use the power of language-based information to broaden the options available to the user, but without revealing complex linguistic and implementation models. Users understand complex documents in terms of many overlapping structures, only some of which are related to linguistic structure. Communications with the user concerning document structures must be based on models of document structure that are natural, convenient and coherent to the user. Pan is a language-based editing and browsing system designed to support development and maintenance of complex software documents. Pan's implementation combines several approaches: unrestricted text editing, language-based browsing and editing, description-driven language definition for incremental analysis and support for multiple languages per session. Pan uses a variety of mechanisms to help users understand and manipulate complex documents effectively, in terms of underlying language when necessary, but always in the framework of a coherent, user-oriented interface. This paper describes that interface, the mechanisms needed to support it, and the complex relationships between interface design and implementation techniques demanded by the goals of the system.

Using multiple knowledge sources for word sense discrimination

This paper addresses the problem of how to identify the intended meaning of individual words in unrestricted texts, without necessarily having access to complete representations of sentences. To discriminate senses, an understander can consider a diversity of information, including syntactic tags, word frequencies, collocations, semantic context, role-related expectations, and syntactic restrictions. However, current approaches make use of only small subsets of this information. Here we will describe how to use the whole range of information. Our discussion will include how the preference cues relate to general lexical and conceptual knowledge and to more specialized knowledge of collocations and contexts. We will describe a method of combining cues on the basis of their individual specificity, rather than a fixed ranking among cue-types. We will also discuss an application of the approach in a system that computes sense tags for arbitrary texts, even when it is unable to determine a single syntactic or semantic representation for some sentences.

Global Text Matching for Information Retrieval

An approach is outlined for the retrieval of natural language texts in response to available search requests and for the recognition of content similarities between text excerpts. The proposed retrieval process is based on flexible text matching procedures carried out in a number of different text environments and is applicable to large text collections covering unrestricted subject matter. For unrestricted text environments this system appears to outperform other currently available methods.

Definiteness and the Processing of Noun Phrases in Natural Discourse

Definiteness is commonly seen as the watershed between those noun phrases (NPs) that introduce new referents and those that refer to referents already familiar. Furthermore, for definite NPs, the anaphoric use is taken to be the paradigm case, while other, so-called first-mention uses are regarded as secondary. The aim of the present paper is to challenge this view, and to argue for a more complex picture of the role of definiteness in the processing of NPs. The paper consists of two parts. The first part presents a corpus-based study of the co-referential properties of definite and indefinite NPs in natural, unrestricted texts. The data bring into light several issues with regard to co-referentiality in unrestricted discourse and the possible referential functions of indefinite and definite NPs. Particular attention is drawn to the fact that the most common function of definite NPs is not anaphoric but different types of first-mention uses. This is the point of departure for the second part of the paper, in which three different approaches to first-mention definities are discussed, and some preliminaries to an alternative model of the processing of first-mention definite NPs are presented.

Evaluating computer-generated domain-oriented vocabularies

It is generally accepted that natural language understanding systems are not now able to deal successfully with unrestricted text, except in very superficial ways. Certainly no current NL system exhibits any significant degree of understanding over arbitrary subject matter. Moreover, there is no convincing reason to believe this situation will change in the near future. Successful systems, therefore, have been restricted to specific applications in particular discourse domains. In those situations where users are expected to provide the domain vocabulary (e.g., TEAM, TQA, etc.) it would be very desirable to provide at least suggestions as to what this vocabulary might be, because a good part of the difficulty in customizing a general system consists of supplying the domain vocabulary and specifying its grammatical properties. This paper discusses some methods for identifying domain vocabulary, as well as techniques for evaluating the quality of the resulting word list.

Adaptive networks as a model for human speech development

Unrestricted English text can be converted to speech through the use of a look-up table, or through a parallel feed-forward network of deterministic processing units. Here, we reproduce the network structure used in NETtalk. Several experiments are carried out to determine which characteristics of the network are responsible for which learning behavior, and how closely that maps human speech development. The network is also trained with different levels of speech complexity and with a second language. The results are shown to be highly dependent on statistical characteristics of the input.

Pronoun Resolution in Unrestricted Text

Quantitative and qualitative studies of referential relations in unrestricted natural text are necessary both for a better theoretical understanding of referential processes, and for the development of empirically well-founded algorithms for anaphora resolution in the framework of natural language processing (NLP) systems. The aim of the study reported in this paper was to provide preliminary empirical data on anaphoric pronouns in Swedish. The relation between the pronoun and its antecedent was studied for 600 pronouns in three different types of unrestricted written Swedish text, and a simple pronoun resolution algorithm was tested on the sample.

A Computer-Based System for Urology and Nephrology

Abstract Modern technology offers new opportunities for the storage and manipulation of hospital information. A computer-based system for urology and nephrology has been designed and developed in this centre. It uses a program that allows the analysis of unrestricted non-codified texts and the operation by hospital staff who have had no more than minimal computer training. The potential of such a system is outlined and a comparison is made with other existing systems. Future prospects and potential lines of development are presented.

A computer-based system for urology and nephrology.

Modern technology offers new opportunities for the storage and manipulation of hospital information. A computer-based system for urology and nephrology has been designed and developed in this centre. It uses a program that allows the analysis of unrestricted non-codified texts and the operation by hospital staff who have had no more than minimal computer training. The potential of such a system is outlined and a comparison is made with other existing systems. Future prospects and potential lines of development are presented.

Hand-held text-to-speech device for the non-vocal disabled

A hand-held, battery-powered synthetic speech aid for the non-vocally disabled has been constructed. The device accepts as its input, largely unrestricted text keyed by the user. This is converted by test-to-speech software, based on 349 letter-to-sound rules and some simple rules of continuity, intonation and stress, to appropriate control signals which drive a single-chip (series formant) speech synthesizer. A number of implementation constraints are imposed by portability; the system has, as far as possible, been designed using CMOS components. To extend the time for which the system will operate between battery charges, power saving facilities are incorporated. Hand-held use implies the need for a one-handed keyboard: a unique integral keyboard is used, designed to minimize the visual search time to locate a letter key. Considerable attention has been paid to rule-search strategies, the handling of ‘exceptions’ which violate the letter-to-sound principle and the resolution of conflicts when more than one rule might apply. The quality and intelligibility of speech from a rule-based system is typically poor, and every effort has been made to improve it. Limits on possible improvement are, however, set by the use of a proprietary single-chip synthesizer and by the minimal nature of a portable system. To facilitate the task of composing messages, a two-line liquid crystal display is provided together with a range of editing functions. The display can also be shown to the message receiver should he/she be deaf, or used for silent communication as an analogue to ‘whispering’. Up to nine particularly useful phrases can be pre-stored and subsequently spoken with just three keypresses. The main message text can be up to 200 characters in length, whereas the pre-stored phrases are limited to 107 characters. Limited clinical trials with nine non-speaking individuals have confirmed the imperfect nature of the speech output, but have otherwise been highly encouraging.

The use of phonotactic constraints to determine word boundaries

Phonotactic constraints limit the permissible word internal consonant sequences in English. In some cases, knowledge of the phoneme sequence uniquely specifies the location of the word boundary, while in other cases, phonological rules based on allowable consonant sequences are not sufficient. For example, the word boundary can be uniquely placed in the sequence /…mgl…/, as in the word pair “some glass,” whereas the word boundary location is ambiguous in the phoneme sequence /…str…/ without further acoustic information. The /…str…/ may have a word boundary in one of three places as in “last rain”, “race trials”, and “may stretch.” Studies were conducted to determine the utility of phonotactic constraints to predict word boundaries. The data bases included the Merriam‐Webster Pocket Dictionary, a phonemically balanced set of sentences, and samples of unrestricted text. Results indicate that: (1) word internal consonant sequences represent a very small subset of all permissible consonant sequences across word boundaries; and (2) acoustic‐phonetic knowledge is needed when the word boundary is ambiguous. Results on the differences between word and syllable boundaries will also be presented. [Work supported by the Office of Naval Research under contract N00014‐82‐k‐0727 and the System Development Foundation.]