Language Sentences Research Articles

Help Transformer Improve Performance in Automatic Mathematics Word Problem-Solving

Solving Mathematics Word Problem (MWP) is a basic ability of humanity, which can be mastered by most students at a young age. The existing artificial intelligence system is not good enough in numerical questions, like MWPs. The hard part of this problem is translating natural language sentences in MWP into mathematical expressions or equations. In recent researches, the Transformer network, which proved a great success in machine translation, is applied to automatic mathematic word problem-solving. While previous works have only shown the ability of Transformer model in MWP, how multiple factors such as encoding, decoding, and pre-training affect the performance of Transformer model has not received enough attention. The study is the first to examine the role of these factors experimentally. This paper proposes several methods to improve Transformer network performance in MWPs under the basis of previous studies, achieves higher accuracy compared to the previous state of the art. Pre-training on target tasks dataset improves the translation quality of the Transformer model greatly. Different token encoding and search algorithms also benefit prediction accuracy at the expense of more training and testing time.

AAP-MIT: Attentive Atrous Pyramid Network and Memory Incorporated Transformer for Multisentence Video Description.

Generating multi-sentence descriptions for video is considered to be the most complex task in computer vision and natural language understanding due to the intricate nature of video-text data. With the recent advances in deep learning approaches, the multi-sentence video description has achieved an impressive progress. However, learning rich temporal context representation of visual sequences and modelling long-term dependencies of natural language descriptions is still a challenging problem. Towards this goal, we propose an Attentive Atrous Pyramid network and Memory Incorporated Transformer (AAP-MIT) for multi-sentence video description. The proposed AAP-MIT incorporates the effective representation of visual scene by distilling the most informative and discriminative spatio-temporal features of video data at multiple granularities and further generates the highly summarized descriptions. Profoundly, we construct AAP-MIT with three major components: i) a temporal pyramid network, which builds the temporal feature hierarchy at multiple scales by convolving the local features at temporal space, ii) a temporal correlation attention to learn the relations among various temporal video segments, and iii) the memory incorporated transformer, which augments the new memory block in language transformer to generate highly descriptive natural language sentences. Finally, the extensive experiments on ActivityNet Captions and YouCookII datasets demonstrate the substantial superiority of AAP-MIT over the existing approaches.

Multi-level Video Captioning based on Label Classification using Machine Learning Techniques

Video captioning is the heuristic and most essential task in the current world to save time by converting long and highly content-rich videos into simple and readable reports in text form. It is narrating the events happening in videos in natural language sentences. It makes the way to many more interesting tasks by the use of labels, tags, and terms such as video content retrieval, video search, video tagging, etc. Video captioning is currently being attempted by many researchers using some exciting Deep learning techniques. But this approach is to find the best of machine learning for the process of captioning videos in a different way. The novel part of the proposed approach is classifying videos by using the labels existing in video frames that belong to the various categories and producing consecutive Multi-Level captions that describe the entire video in a round-robin way. Informative features are extracted from the video frames such as Gray Level Co-occurrence Matrix (GLCM) features, Hu moments, and Statistical features to provide optimal results. This model is designed with two superior and optimal classifiers such as Support Vector Machine (SVM) and Naive Bayes separately. The models are demonstrated with the prevailing standard dataset Microsoft Research Video Description corpus (MSVD) and evaluated by the benchmark classification metrics such as Accuracy, Precision, Recall, and F1-Score.

Reviews and Prospects of Foreign Researches on Second Language Sentence Processing

三仓出版社主要从事中文国际开源期刊的出版与发行，期刊领域涵盖工程技术、数学与物理、人文社会、信息科学、经济与管理、医药卫生、农业、地球环境等学科，借助互联网思维及现代信息技术，立足国内高校及研究机构的科研需求，力图推动中国学术界知识的交流与传播，可期刊发表、发表论文、快速发表，见刊快。三仓出版社欢迎广大科研学者加入我们，参与审稿，合作办刊，投稿荐稿，与中文期刊一同成长壮大。如果您对期刊有任何的意见或者建议，请联系service@sciscanpub.com.

ARTEMIS: parsing non-peripheral complex sentences in ASD-STE100

The simulation of natural language understanding is one of the main objectives of natural language processing (NLP). Within the different applications designed for this purpose, the ARTEMIS prototype follows the paradigm of unification grammars, and unlike other trending computational resources, it is theoretically grounded in Role and Reference Grammar (RRG). The syntax-to-semantics linking algorithm proposed in this functional grammar lies at the basis of a parsing process that starts with a natural language sentence, extracts its morphosyntactic features and provides a representation of these in terms of the so-called layered structure of the clause (LSC). The Grammar Development Environment (GDE) in ARTEMIS is a major component where feature- based production rules (syntactic, lexical and constructional) are stored and ready to allow the generation of the enhanced layered structure of the clause of natural language expressions. Syntactic rules that account for phrasal constituents and simple sentences have already been described, but it is now turn to focus on the study of non-peripheral complex sentences. In an attempt to validate these syntactic rules and to avoid some of the common problems that may arise in parsing applications, our research will concentrate on the analysis of RRG’s juncture-nexus combinations as found in a controlled natural language (CNL) such as ASD-STE100 (Aerospace and Defence Simplified Technical English).

Automation of the conversion of natural language to formalized node of knowledge record

The paper describes the improvement of the system for automatic question answering based on the knowledge expressed in natural language sentences. The system is implemented using a relational database. This system will be the basis for the development of a web application for obtaining answers to the posed questions. In order for natural language sentences to be entered into a relational database, they must be prepared and formally recorded. The development of a question-answering system is based on the application of the conceptual framework Node of Knowledge (NOK), whose formalized record is suitable for input to a relational database from which answers to questions can be obtained. This paper presents an application for automated conversion of English sentences into a formalized record. The application was tested on one hundred simple English sentences, and the results of the automated conversion were compared with those obtained by processing the same sentences manually. Ključne riječi Node of Knowledge (NOK), Formalized Node of Knowledge (FNOK), Knowledge Representation, Question-Answering (QA) system

Long Short-Term Relation Transformer With Global Gating for Video Captioning.

Video captioning aims to generate a natural language sentence to describe the main content of a video. Since there are multiple objects in videos, taking full exploration of the spatial and temporal relationships among them is crucial for this task. The previous methods wrap the detected objects as input sequences, and leverage vanilla self-attention or graph neural network to reason about visual relations. This cannot make full use of the spatial and temporal nature of a video, and suffers from the problems of redundant connections, over-smoothing, and relation ambiguity. In order to address the above problems, in this paper we construct a long short-term graph (LSTG) that simultaneously captures short-term spatial semantic relations and long-term transformation dependencies. Further, to perform relational reasoning over the LSTG, we design a global gated graph reasoning module (G3RM), which introduces a global gating based on global context to control information propagation between objects and alleviate relation ambiguity. Finally, by introducing G3RM into Transformer instead of self-attention, we propose the long short-term relation transformer (LSRT) to fully mine objects' relations for caption generation. Experiments on MSVD and MSR-VTT datasets show that the LSRT achieves superior performance compared with state-of-the-art methods. The visualization results indicate that our method alleviates problem of over-smoothing and strengthens the ability of relational reasoning.

Machine Translation Systems for English Captions to Hindi Language Using Deep Learning

Machine Translation is the process of translating text from one language to another which helps to reduce the conversation gap among people from different cultural backgrounds. The task performed by the Machine Translation System is to automatically translate between pairs of different natural languages, where Neural Machine Translation System stands out from all because it provides fluent translation along with reasonable translation accuracy. The Convolution Neural Network encoder is used to find patterns in the images and encode it into a vector that is passed to the Long Short Term Memory decoder which finds the caption word-by-word to best describe the image. Upon reaching the end-line token, the entire description of the image in English is generated and that is our output for that particular image. Automatically creating the description of an image in English using any natural language sentences and then translating it using Neural Machine Translation to Hindi is a very challenging task. It requires expertise in both image processing as well as natural language processing. In this paper, the aim is to compare the two Machine Translation Systems: Google Translation System and the proposed Neural Machine Translation System to convert the text obtained from an image in English to Hindi language.

Penanda Klausa Adverbial Dalam Bahasa Angkola

An adverbial clause is a subordinate clause that serves to provide information on the main clause. The presence of an adverbial clause is not a must, but it can help create coherence in a discourse. This study aims to describe the markers of adverbial clauses in Angkola language. The method used is descriptive qualitative markers. Data collections are conducted by speaking, listening, and taking notes. The data in this study are adverbial clauses in Angkola language obtained from native Angkola speakers and also written sources obtained from Angkola language books. Data analysis was carried out by matching the data with the theory contained in the study, namely adverbial clause markers in Angkola language sentences. The results showed that there were five types of adverbial clauses in Angkola language, namely temporal clauses marked by the word dung 'after' and dompak 'when', conditional clauses marked by the word molo 'if', causal clauses (causal clause) which is marked by the word harana 'because', the purpose clause (purposal clause) which is marked by the word anso 'so that/so', and the concession clause (consessive clause) which is marked by the words bope 'although' and aha pe 'what ever'. The use of adverbial clauses can be found at the initial or final position in a sentence.

Analysis of picture book vocabulary and sentence complexity based on the 2015 revised national English curriculum

This study explores the possibility of using English picture books to supplement the English input in primary English education. To this end, a corpus of 107 popular English picture books was constructed and analyzed in the context of the revised national English curriculum of 2015. Vocabulary in the corpus was compared with the basic vocabulary lists of the national curriculum, and the syntactic complexity of sentences in the picture books was compared with the syntactic complexity of the example sentences of elementary-level language forms in the national curriculum. The results show that the vocabulary and syntactic complexity of picture books of ATOS book level 0-2 are generally suitable for elementary school students. Lexical analyses also reveal that picture books contain exclamations, onomatopoeia, colloquialisms, and words that children can relate to their everyday life, which are likely to function as authentic input in primary English education.

INVESTIGATION OF DECODING FILLERS USED IN AN ENGLISH LEARNING TALK SHOW “ENGLISH WITH ALICE”

This research was conducted to analyze decoding fillers in English learning talk show. The researchers made two groups of non-native English speakers to investigate fillers, Group one with little or no abroad experienced speakers and Group two with long term abroad experienced speakers. The researchers used descriptive qualitative as a method that describes an objective situation or certain event based on facts that appear or should be followed by an attempt to draw general conclusion. The procedure is ethnographic research which is ethnographic by qualitative enable researchers to understand the data by the functions of language (words, phrase and sentence) by grouping words that have same functions into categories. The researchers would analyze and conclude the data to be studied. In this research the data to be reviewed is English with Alice which contain fillers. The researchers found one hundred sixty-eight fillers with the exclusion of filler used by the host and both of speakers. After analyzing the data, the researchers found that 2 groups (14 guests) in the talk show English with Alice used different functions of decoding fillers. The functions in Group one are mostly (a) to begin and close a discourse and (c) to hold the floor. Moreover, the functions in Group two are mostly (c) to hold the floor and (g) to express the response.

Implicitness, Logical Form and Arguments

In the paper I suggest that a loose notion of logical form can be a useful tool for the understanding or evaluation of everyday language and the explicit and implicit content of communication. Reconciling ordinary language and logic provides formal guidelines for rational communication, giving strength and order to ordinary communication and content to logical schemas. The starting point of the paper is the idea that the bearers of logical form are not natural language sentences, but what we communicate with them, that is, their content in a particular context. On the basis of that idea, I propose that we can ascribe logical proprieties to what is communicated using ordinary language and suggest a continuum between semantic phenomena such as explicatures and pragmatic communicational strategies such as (particularized) conversational implicatures, which challenges the idea that an implicatum is completely separate from what is said. I believe that this continuum can be best explained by the notion of logical form, taken as a propriety of sentences relative to particular interpretations.

One-directional parsing function for information security in computer networks of unmanned aerial vehicles

The tools of the formal languages and grammars theory are proposed to be used in developing the ways to increase the information security of wireless data transmission systems in computer networks of unmanned aerial vehicles (UAV). Multiple use of production rules for generating sentences of a (regular) language in the processes of inter-level transformations and transmission of PDUs from a true primary information generator through UAV wireless computer network nodes allows to create the pseudo-sources of primary information, which hinders unauthorized access.

Analysis and Comparative Study of POS Tagging Techniques for National (Urdu) Language and other Regional Languages of Pakistan

Defining algorithms and techniques to enable computers to understand human language is the Natural Language Processing (NLP), which is an integral part of speech recognition. Parts of Speech (POS) is considered as one of the well understood problems of Natural Language Processing, in which natural language words and sentence are tagged or assigned grammatical classes, because tagging a single word by human hand is a time consuming and tedious job. To automate the tagging job is the way to automate the lexicons of the text of a language. Many of the languages are enriched with their POS tagging systems. Pakistani regional languages are less developed due to the many reasons and much of the work is needed in POS tagging system. Some of the regional languages have their POS tagging systems but still they need some more attention to refine their system. Some of the languages need to develop from the scratch. Balochi language has no any POS tagging system. This study presents the comparative analysis of POS tagging approaches for the national language (Urdu) and other regional languages of Pakistan. The approaches and their data sets used and their reported results are presented here

Recursion in programs, thought, and language

This article presents a theory of recursion in thinking and language. In the logic of computability, a function maps one or more sets to another, and it can have a recursive definition that is semi-circular, i.e., referring in part to the function itself. Any function that is computable - and many are not - can be computed in an infinite number of distinct programs. Some of these programs are semi-circular too, but they needn't be, because repeated loops of instructions can compute any recursive function. Our theory aims to explain how naive individuals devise informal programs in natural language, and is itself implemented in a computer program that creates programs. Participants in our experiments spontaneously simulate loops of instructions in kinematic mental models. They rely on such loops to compute recursive functions for rearranging the order of cars in trains on a track with a siding. Kolmogorov complexity predicts the relative difficulty of abducing such programs - for easy rearrangements, such as reversing the order of the cars, to difficult ones, such as splitting a train in two and interleaving the two resulting halves (equivalent to a faro shuffle). This rearrangement uses both the siding and part of the track as working memories, shuffling cars between them, and so it relies on the power of a linear-bounded computer. Linguistic evidence implies that this power is more than necessary to compose the meanings of sentences in natural language from those of their grammatical constituents.

Representation of the Use of Sumbawa Language During the Covid 19 Pandemic

This article discussed Representation of the Use of Sumbawa Language During the Covid 19 Pandemic. The purpose of this study is to describe of the sentences mening by using Sumbawa language, which it applied to give socialization about the corona virus 19 toward the Sumbawa societies. The location of this study was conducted in Sumbawa district, West Nusa Tenggara Province, Indonesia. Futhermore, the method of this study used the qualitative descriptive method. Then, the instruments used for collecting data implemented the documentation and photographic techniques. The result of this study classified into two points, the first the meaning of senteces related to the interpretation of Fairclough, found several sentences in the Sumbawa regional language, namely Patis Pang Bale Turit Pemerintah advice which means that stay at home following government advice. According to the government's advice, Tama Ko IGD means entering the Emergency Care Unit (IGD) at the hospital, and Tama Ko IGD means entering the Emergency Room (ER) at the hospital, and Fotomu Nempel Pang Buku Yasin means your photo is stuck in Yasin's book. The second the meaning of the use of the Sumbawa language related to Pearce which classifies it into three categories, namely icon, index and symbol. The iconic element in this study is in the form of a caricature of a medical worker, while the index is a sentence that uses the Sumbawa language 'ba meling nene nene mo!ompa kami ta!' which means 'it's up to you all! We are too tired to work! Furthermore, symbols are represented on images of Personal Protective Equipment (PPE) and images of masks used by medical personnel.

Natural Language Processing and Computational Linguistics

Natural Language Processing and Computational Linguistics

Object-based unawareness: Axioms

This paper provides foundations for a model of unawareness, called object-based unawareness (OBU) structures, that can be used to distinguish between what an agent is unaware of and what she simply does not know. At an informal level, this distinction plays a key role in a number of papers such as Tirole (2009) and Chung & Fortnow (2016). In this paper, we give the model-theoretic description of OBU structures by showing how they assign truth conditions to every sentence of the formal language used. We then prove a model-theoretic sound and completeness theorem, which characterizes OBU structures in terms of a system of axioms. We then verify that agents in OBU structures do not violate any of the introspection axioms that are generally considered to be necessary conditions for a plausible notion of unawareness. Applications are provided in our companion paper.

A prison term prediction model based on fact descriptions by capturing long historical information

The legal judgments are always based on the description of the case, the legal document. However, retrieving and understanding large numbers of relevant legal documents is a time-consuming task for legal workers. The legal judgment prediction (LJP) focus on applying artificial intelligence technology to provide decision support for legal workers. The prison term prediction(PTP) is an important task in LJP which aims to predict the term of penalty utilizing machine learning methods, thus supporting the judgement. Long-Short Term Memory(LSTM) Networks are a special type of Recurrent Neural Networks(RNN) that are capable of handling long term dependencies without being affected by an unstable gradient. Mainstream RNN models such as LSTM and GRU can capture long-distance correlation but training is time-consuming, while traditional CNN can be trained in parallel but pay more attention to local information. Both have shortcomings in case description prediction. This paper proposes a prison term prediction model for legal documents. The model adds causal expansion convolution in general TextCNN to make the model not only limited to the most important keyword segment, but also focus on the text near the key segments and the corresponding logical relationship of this paragraph, thereby improving the predicting effect and the accuracy on the data set. The causal TextCNN in this paper can understand the causal logical relationship in the text, especially the relationship between the legal text and the prison term. Since the model uses all CNN convolutions, compared with traditional sequence models such as GRU and LSTM, it can be trained in parallel to improve the training speed and can handling long term. So causal convolution can make up for the shortcomings of TextCNN and RNN models. In summary, the PTP model based on causality is a good solution to this problem. In addition, the case description is usually longer than traditional natural language sentences and the key information related to the prison term is not limited to local words. Therefore, it is crucial to capture substantially longer memory for LJP domains where a long history is required. In this paper, we propose a Causality CNN-based Prison Term Prediction model based on fact descriptions, in which the Causal TextCNN method is applied to build long effective history sizes (i.e., the ability for the networks to look very far into the past to make a prediction) using a combination of very deep networks (augmented with residual layers) and dilated convolutions. The experimental results on a public data show that the proposed model outperforms several CNN and RNN based baselines.

Children’s query formulation and search result exploration

Our research aims at understanding children’s information search and their use of information search tools during educational pursuits. We conducted an observation study with 50 New Zealand school children between the ages of 9 and 13 years old. In particular, we studied the way that children constructed search queries and interacted with the Google search engine when undertaking a range of educationally appropriate inquiry tasks. As a result of this in situ study, we identified typical query-creation and query-reformulation strategies that children use. The children worked through 250 tasks, and created a total of 550 search queries. 64.4% of the successful queries made were natural language queries compared to only 35.6% keyword queries. Only three children used the related searches feature of the search engine, while 46 children used query suggestions. We gained insights into the information search strategies children use during their educational pursuits. We observed a range of issues that children encountered when interacting with a search engine to create searches as well as to triage and explore information in the search engine results page lists. We found that search tasks posed as questions were more likely to result in query constructions based on natural language questions, while tasks posed as instructions were more likely to result in query constructions using natural language sentences or keywords. Our findings have implications for both educators and search engine designers.