Practical Natural Language Processing Research Articles

The role of artificial intelligence in auditing and fraud detection in accounting information systems: moderating role of natural language processing

Purpose This study aims to investigate the moderating role of natural language processing natural language processing (NLP) on the relationship between AI-empowered AIS (data gathering, data analysis, risk assessment, detection, prevention and Investigation) and auditing and fraud detection. Design/methodology/approach Quantitative methodology was adapted through a questionnaire. In total, 221 individuals represented the population of the study, and SPSS was used to screen primary data. The study indicated the acceptance of the hypothesis that “Artificial Intelligence in AIS has a statistically significant influence on auditing and fraud detection,” showing a strong correlation between auditing and fraud detection. The study concluded that NLP moderates the relationship between AI in AIS and auditing and fraud detection. Findings The study’s implications lie in its contribution to the development of theoretical models that explore the complementary attributes of AI and NLP in detecting financial fraud. Research limitations/implications A cross-sectional design is a limitation. Practical implications NLP is a useful tool for developing more efficient methods for detecting fraudulent activities and audit risks. Originality/value The study’s originality stems from its focus on the use of AI-empowered AIS, a relatively new technology that has the potential to significantly impact auditing and fraud detection processes within the accounting field.

Chinese news topic prediction using bidirectional encoder representation from transformers

Nowadays, there are many researches on natural language processing (NLP). Through the research of NLP method, many problems in machine learning field have been solved. However, since the study of Chinese NLP has not developed rapidly until recent years, there is still much to be studied on Chinese NLP. As an excellent pre-training model, whether Bidirectional Encoder Representation from Transformers (BERT) performs well on specific Chinese NLP remains to be studied. Therefore, this paper uses BERT for Chinese NLP, and trains BERT model by collecting news title data to achieve Chinese text classification. Finally, the prediction results are studied by statistical methods. The research shows that BERT method performs well on Chinese NLP and can predict different types of news headlines well. Although it performs differently on different kinds of titles, its performance is satisfactory on the whole, and the prediction results are relatively balanced in different categories. Therefore, BERT can be used as a very practical and efficient NLP method. At the same time, it can also be predicted that it will play a great role in Chinese NLP.

Implementing associative memories by Echo State Network for the applications of natural language processing

This paper presents Echo State Network (ESN) based associative memories and their applications to English words. Among the papers describing ESN applications in general, some papers deal with natural language processing (NLP). Those NLP papers utilize an ESN’s property of temporal signal learning to translate an English sentence into its corresponding predicate logic formula. In practical NLP applications, input sentences often include misspelled or undefined words. To cope with such problems, we constructed a training algorithm to realize an ESN-based associative memory. This algorithm can be used in two ways: auto-associative one when the input and output patterns are the same in each pair of the training data, and hetero-associative one otherwise. In this paper, we firstly show the basic performance of an ESN-based auto-associative memory by applying it to two-dimensional images. We made sure that the performance is improved by averaging the result of multiple runs. Secondly, we describe the performances of ESN-based auto- and hetero-associative memories when being applied to English words. The former is to recall correct English words from incomplete spelling ones, and the latter is to chain two different English words of an input and its corresponding output. The former performance in recalling 26 incomplete words can be improved by averaging the outputs of multiple runs, and the improved success rate of correct words is around 65%. The latter performance in chaining 26 words attains the success rate of 83.4%.

Fine-tuning transformers: Vocabulary transfer

Transformers are responsible for the vast majority of recent advances in natural language processing. The majority of practical natural language processing applications of these models are typically enabled through transfer learning. This paper studies if corpus-specific tokenization used for fine-tuning improves the resulting performance of the model. Through a series of experiments, we demonstrate that such tokenization combined with the initialization and fine-tuning strategy for the vocabulary tokens speeds up the transfer and boosts the performance of the fine-tuned model. We call this aspect of transfer facilitation vocabulary transfer.

Natural Language Processing and Computational Linguistics

Natural Language Processing and Computational Linguistics

Named-Entity Recognition Using Automatic Construction of Training Data From Social Media Messaging Apps

In recent years, social media messaging app data has served as a precious resource to extract useful information, such as critical clues and evidence in legal trials and criminal investigations. Although these data can be of various types, they are mostly in the form of natural language text. Therefore, to extract information from them efficiently, it is essential to research practical natural language processing approaches. This study proposes applying a deep-learning-based named-entity recognition (NER) system as a natural language processing approach for information extraction to these messaging data. In addition, a system for automatically constructing NER training data is presented using the distant supervision method for the training data of deep-learning models. Because social media messaging app data generally include a significant amount of noise, such as typographical and word-spacing errors, a NER system with robustness against these types of noisy data is required to extract information from the messaging data effectively. The results demonstrate that the proposed approach outperforms that of a NER system with manually labeled training data.

Authorship recognition via fluctuation analysis of network topology and word intermittency

Statistical methods have been widely employed in many practical natural language processing applications. More specifically, complex network concepts and methods from dynamical systems theory have been successfully applied to recognize stylistic patterns in written texts. Despite the large number of studies devoted to representing texts with physical models, only a few studies have assessed the relevance of attributes derived from the analysis of stylistic fluctuations. Because fluctuations represent a pivotal factor for characterizing a myriad of real systems, this study focused on the analysis of the properties of stylistic fluctuations in texts via topological analysis of complex networks and intermittency measurements. The results showed that different authors display distinct fluctuation patterns. In particular, it was found that it is possible to identify the authorship of books using the intermittency of specific words. Taken together, the results described here suggest that the patterns found in stylistic fluctuations could be used to analyze other related complex systems. Furthermore, the discovery of novel patterns related to textual stylistic fluctuations indicates that these patterns could be useful to improve the state of the art of many stylistic-based natural language processing tasks.

Multiword expressions: hard going or plain sailing?

Over the past two decades or so, Multi-Word Expressions (MWEs; also called Multi-word Units) have been an increasingly important concern for Computational Linguistics and Natural Language Processing (NLP). The term MWE has been used to refer to various types of linguistic units and expressions, including idioms, noun compounds, phrasal verbs, light verbs and other habitual collocations. However, while there is no universally agreed definition for MWE as yet, most researchers use the term to refer to those frequently occurring phrasal units which are subject to certain level of semantic opaqueness, or non-compositionality. Non-compositional MWEs pose tough challenges for automatic analysis because their interpretation cannot be achieved by directly combining the semantics of their constituents, thereby causing the "pain in the neck of NLP" (Sag et al. 2001). In fact, MWEs have been studied for decades in Phraseology under the term phraseological unit. But in the early 1990s, MWEs started receiving increasing attention in corpus-based computational linguistics and NLP. Early influential work on MWEs includes Smadja (1993), Dagan and Church (1994), Wu (1997), Daille (1995), Wermter and Chen (1997), McEnery et al. (1997), and Michiels and Dufour (1998). These studies address the automatic treatment of MWEs and their applications in practical NLP and information systems. A milestone for MWE

Capturing practical natural language transformations

We study automata for capturing the transformations in practical natural language processing (NLP) systems, especially those that translate between human languages. For several variations of finite-state string and tree transducers, we survey answers to formal questions about their expressiveness, modularity, teachability, and generalization. We conclude that no formal device yet captures everything that is desirable, and we point to future research.

A formal approach to subgrammar extraction for NLP

The problem of subgrammar extraction is precisely defined in the following way: Given a grammar G and a set of words W , find a smallest subgrammar of G that accepts the same set of sentences from W ∗ as G , and for each of them produces the same parse trees. In practical Natural Language Processing applications, the set of words W is obtained from the text unit. There are practical motivations for doing this operation “just-in-time”, i.e. just before processing the text; hence it is required that this operation be done in an automatic and efficient way. After defining the problem in the general framework, we discuss the problem for context-free grammars (CFG), and give an efficient algorithm for it. We prove that finding the smallest subgrammar for HPSGs is an NP-hard problem, and give an efficient algorithm that solves an easier, approximate version of the problem. We also discuss how the algorithm can be efficiently implemented.

Mood and modality: out of theory and into the fray

The topic of mood and modality (MOD) is a difficult aspect of language description because, among other reasons, the inventory of modal meanings is not stable across languages, moods do not map neatly from one language to another, modality may be realised morphologically or by free-standing words, and modality interacts in complex ways with other modules of the grammar, like tense and aspect. Describing MOD is especially difficult if one attempts to develop a unified approach that not only provides cross-linguistic coverage, but is also useful in practical natural language processing systems. This article discusses an approach to MOD that was developed for and implemented in the Boas Knowledge-Elicitation (KE) system. Boas elicits knowledge about any language, L, from an informant who need not be a trained linguist. That knowledge then serves as the static resources for an L-to-English translation system. The KE methodology used throughout Boas is driven by a resident inventory of parameters, value sets, and means of their realisation for a wide range of language phenomena. MOD is one of those parameters, whose values are the inventory of attested and not yet attested moods (e.g. indicative, conditional, imperative), and whose realisations include flective morphology, agglutinating morphology, isolating morphology, words, phrases and constructions. Developing the MOD elicitation procedures for Boas amounted to wedding the extensive theoretical and descriptive research on MOD with practical approaches to guiding an untrained informant through this non-trivial task. We believe that our experience in building the MOD module of Boas offers insights not only into cross-linguistic aspects of MOD that have not previously been detailed in the natural language processing literature, but also into KE methodologies that could be applied more broadly.

On the Corpus Size Needed for Compiling a Comprehensive Computational Lexicon by Automatic Lexical Acquisition

Comprehensive computational lexicons areessential to practical natural languageprocessing (NLP). To compile such computationallexicons by automatically acquiring lexicalinformation, however, we previously requiresufficiently large corpora. This study aims atpredicting the ideal size of suchautomatic-lexical-acquisition oriented corpora,focusing on six specific factors: (1) specificversus general purpose prediction, (2)variation among corpora, (3) base forms versus inflected forms, (4) open class items,(5) homographs, and (6) unknown words.Another important and related issue withregard to predictability has something to dowith data sparseness. Research using theTOTAL Corpus reveals serious datasparseness in this corpus. This, again, pointstowards the importance and necessity ofreducing data sparseness to a satisfactorylevel for the automatic lexical acquisition andreliable corpus predictions. The functions ofpredicting the number of tokens and lemmas in acorpus are based on the piecewisecurve-fitting algorithm. Unfortunately, thepredicted size of a corpus for automaticlexical acquisition is too astronomicalto compile it by using presently existingcompiling strategies. Therefore, we suggest apractical and efficient alternative method. Weare confident that this study will shed newlight on issues such as corpus predictability,compiling strategies and linguisticcomprehensiveness.

Introduction to the Special Issue on Computational Anaphora Resolution

Anaphora accounts for cohesion in texts and is a phenomenon under active study in formal and computational linguistics alike. The correct interpretation of anaphora is vital for natural language processing (NLP). For example, anaphora resolution is a key task in natural language interfaces, machine translation, text summarization, information extraction, question answering, and a number of other NLP applications. After considerable initial research, followed by years of relative silence in the early 1980s, anaphora resolution has attracted the attention of many researchers in the last 10 years and a great deal of successful work on the topic has been carried out. Discourseoriented theories and formalisms such as Discourse Representation Theory and Centering Theory inspired new research on the computational treatment of anaphora. The drive toward corpus-based robust NLP solutions further stimulated interest in alternative and/or data-enriched approaches. Last, but not least, application-driven research in areas such as automatic abstracting and information extraction independently highlighted the importance of anaphora and coreference resolution, boosting research in this area. Much of the earlier work in anaphora resolution heavily exploited domain and linguistic knowledge (Sidner 1979; Carter 1987; Rich and LuperFoy 1988; Carbonell and Brown 1988), which was difficult both to represent and to process, and which required considerable human input. However, the pressing need for the development of robust and inexpensive solutions to meet the demands of practical NLP systems encouraged many researchers to move away from extensive domain and linguistic knowledge and to embark instead upon knowledge-poor anaphora resolution strategies. A number of proposals in the 1990s deliberately limited the extent to which they relied on domain and/or linguistic knowledge and reported promising results in knowledge-poor operational environments (Dagan and Itai 1990, 1991; Lappin and Leass 1994; Nasukawa 1994; Kennedy and Boguraev 1996; Williams, Harvey, and Preston 1996; Baldwin 1997; Mitkov 1996, 1998b). The drive toward knowledge-poor and robust approaches was further motivated by the emergence of cheaper and more reliable corpus-based NLP tools such as partof-speech taggers and shallow parsers, alongside the increasing availability of corpora and other NLP resources (e.g., ontologies). In fact, the availability of corpora, both raw and annotated with coreferential links, provided a strong impetus to anaphora resolu

An Algorithm for Predicting the Relation between Lemmas and Corpus Size

Much research on natural language processing (NLP), computational linguistics and lexicography has relied and depended on linguistic corpora. In recent years, many organizations around the world have been constructing their own large corpora to achieve corpus representativeness and/or linguistic comprehensiveness. However, there is no reliable guideline as to how large machine readable corpus resources should be compiled to develop practical NLP software and/or complete dictionaries for humans and computational use. In order to shed some new light on this issue, we shall reveal the flaws of several previous researches aiming to predict corpus size, especially those using pure regression or curve-fitting methods. To overcome these flaws, we shall contrive a new mathematical tool: a piecewise curve-fitting algorithm, and next, suggest how to determine the tolerance error of the algorithm for good prediction, using a specific corpus. Finally, we shall illustrate experimentally that the algorithm presented is valid, accurate and very reliable. We are confident that this study can contribute to solving some inherent problems of corpus linguistics, such as corpus predictability, compiling methodology, corpus representativeness and linguistic comprehensiveness.

Information extraction: What have we learned?

Practical natural language processing applications typically confront significant difficulties with lexical ambiguity, knowledge‐engineering bottlenecks, and brittle processing techniques that fail to handle unrestricted input in a robust fashion. However, recent research efforts suggest that these problems can be successfully negotiated for an important class of applications. Information extraction systems operate in restricted discourse domains where they summarize the content of input texts in a highly goal‐oriented manner. Machine learning techniques appear to be a promising foundation for high‐quality information extraction performance, requiring only a training corpus of representative input documents that have been annotated by a domain expert. In this paper we will describe innovative research in information extraction, and speculate a bit about the future of information extraction.

A NOVEL AND EFFICIENT METHOD FOR PARSING UNRESTRICTED TEXTS OF QUASI FREE WORD ORDER LANGUAGES

This paper deals with the problems stemming from the parsing of long sentences in quasi free word order languages. Due to the word order freedom of a large category of languages including Greek and the limitations of rule-based grammar parsers in parsing unrestricted texts of such languages, we propose a flexible and effective method for parsing long sentences of such languages that combines heuristic information and pattern-matching techniques in early processing levels. This method is deeply characterized by its simplicity and robustness. Although it has been developed and tested for the Greek language, its theoretical background, implementation algorithm and results are language independent and can be of considerable value for many practical natural language processing (NLP) applications involving parsing of unrestricted texts.

A lexical analyser for Arabic

Practical natural language processing (NLP) systems such as database front-ends, deductive databases or object-oriented databases are at the forefront of research into the next-generation intelligent database systems. The research described in this paper has been aimed at integrating front-end paradigms and rule-based deduction to provide a single powerful framework for database systems in Arabic. The lexicon stores only roots of verbs and uses a program intelligent enough to handle all derived forms automatically. This is significant, as these alone represent 70% of the total dictionary. As part of the discussion of this system, its utility in such NLP applications as parsing and machine translation is examined.