Natural Sentences Research Articles

User opinion-focused abstractive summarization using explainable artificial intelligence

Recent methodologies have achieved good performance in objectively summarizing important information from fact-based datasets such as XSUM and CNN/DM. These methodologies involve abstractive summarization, extracting the core content from an input text and transforming it into natural sentences. Unlike fact-based documents, opinion-based documents require a thorough analysis of sentiment and understanding of the writer’s intention. However, existing models do not explicitly consider these factors. Therefore, in this study, we propose a novel text summarization model that is specifically designed for opinion-based documents. Specifically, we identify the sentiment distribution of the entire document and train the summarization model to focus on major opinions that conform to the intended message while randomly masking minor opinions. Experimental results show that the proposed model outperforms existing summarization models in summarizing opinion-based documents, effectively capturing and highlighting the main opinions in the generated abstractive summaries.

RDMTL: Reverse dictionary model based on multitask learning

A reverse dictionary is a task of finding appropriate words based on given definitions. Existing studies struggle to capture the subtle differences between words with similar definitions. Motivated by humans’ ability to verify their understanding of a word's semantics by making example sentences and recognizing parts of speech, we propose a reverse dictionary model based on multitask learning (RDMTL), which can alleviate the problem. RDMTL comprises a primary task component that extracts different levels of semantic features from definitions and two auxiliary task components that predict part-of-speech tags and generate sentences for target words. Through jointly learning these three tasks, RDMTL can enhance the understanding of definitions and discover subtle differences among words with similar meanings. Moreover, it can generate more accurate and natural example sentences. We evaluate RDMTL on a modified version of the New Oxford dataset and compare its performance with several baseline models. The experimental results show that RDMTL enhances the rank value metric for the reverse dictionary task by 1.02%, the F1 value metric for the part-of-speech classification task by 20.47%, and the SB-4 metric for the sentence generation task by 14.7%. In addition, to analyze the contribution of each component and the impact of multitask learning, we conducted an ablation study. In this study, a new method and perspective for the reverse dictionary task is introduced.

Personality perception in synthetic versus natural speech: The effects of voice quality and prosody

Synthetic speech technology, now approaching human-like naturalness due to advancements in deep learning, has turned to focusing on personality or persona design as a common practice in the industry. The present study aimed to identify speech characteristics affecting personality impressions in synthetic and natural speech. Thirty native Korean speakers participated in a personality rating experiment in which they evaluated natural Korean sentences and their synthetic counterparts in terms of the Big-Five personality model. Acoustic analyses were performed to examine voice quality and prosody, including Intonational Phrase (IP) boundary tones. The results revealed that scores of agreeableness, conscientiousness, and emotional stability increased overall when the voices contained greater aperiodicity in the harmonics (i.e., were likely breathier) and were weaker in energy. The results also demonstrated that different prosodic features affected personality perception in synthetic and natural speech; synthetic speech with a wider F0 range received higher scores on extroversion, openness, and emotional stability. In contrast, the effect of IP boundary tones was most frequently found for female natural speech, which contained a wider range of tones, including multitonals (e.g., LHL%). Our findings suggest that intonation is one of the key factors which can be adjusted to generate synthetic speech with various personalities.

Morphosyntactic analysis of surigaonun: “Martsa Surigao”

This paper argued that Surigaonun morphosyntactic linearity is revealed in the textuality of Ellen Amores’ “Martsa Surigao”. The knowledge of the structure of Surigaonun language plays significant role in language learning, comprehension, formation of phrases, clauses and sentences, and effective communication using the language. Surigaonun language has no established pedagogic grammar, and there is no research conducted yet on the morphology and syntax of Surigaonun language using textuality. This study aimed to describe the morphosyntactic characteristics of Surigaonun language by analyzing the text “Martsa Surigao” to generate linguistic data on the Surigaonun word structure, the very basis for the morphemic structure of the words, which make up the constituents of the sentence. This qualitative research used the discourse analysis. Specifically, it focused on raw data idealization, morphological analysis, and syntactic analysis on immediate constituents and rewrite rules for each syntactic unit. The linear morphologic segmentation of morphemic contents of “Martsa Surigao” reveals 39 lexical phonemes of 13 simple forms and 26 complex forms; 21 grammatical morphemes of 4 prepositions, 9 pronouns, 5 conjunctions, and 3 determiners. The linear structuration of the textuality in idealized syntactic units reveals 12 simple sentences, 1 compound sentence, and 1 complex sentence according to form. Syntactically, Surigaonun sentences always begin with the predicate and end with the subject as their natural sentence patterning. Sentences are introduced by noun predicates, adjective predicates, adverb predicates, prepositional phrase predicates, intransitive verb predicates and transitive verb predicates. Therefore, the text reveals the morphosyntactic segmentation linearity in its textuality.

ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models

Personalizing generative models offers a way to guide image generation with user-provided references. Current personalization methods can invert an object or concept into the textual conditioning space and compose new natural sentences for text-to-image diffusion models. However, representing and editing specific visual attributes such as material, style, and layout remains a challenge, leading to a lack of disentanglement and editability. To address this problem, we propose a novel approach that leverages the step-by-step generation process of diffusion models, which generate images from low to high frequency information, providing a new perspective on representing, generating, and editing images. We develop the Prompt Spectrum Space P*, an expanded textual conditioning space, and a new image representation method called ProSpect. ProSpect represents an image as a collection of inverted textual token embeddings encoded from per-stage prompts, where each prompt corresponds to a specific generation stage (i.e., a group of consecutive steps) of the diffusion model. Experimental results demonstrate that P* and ProSpect offer better disentanglement and controllability compared to existing methods. We apply ProSpect in various personalized attribute-aware image generation applications, such as image-guided or text-driven manipulations of materials, style, and layout, achieving previously unattainable results from a single image input without fine-tuning the diffusion models. Our source code is available at https://github.com/zyxElsa/ProSpect.

Emotion-and-knowledge grounded response generation in an open-domain dialogue setting

The neural-based interactive dialogue system focuses on engaging and retaining humans in long-lasting conversations. This has been explored for a variety of goal-oriented dialogue domains, such as education, health care, entertainment, sports, and politics. To develop an understanding and awareness of social and cultural norms, and to address specific social skills, we need to invent strategies for building interactive systems that take into account the user’s emotions and relevant-facts in a multi-turn conversation. In this paper, we propose a new neural generative model that combines step-wise co-attention with a self-attention-based transformer network along with an emotion classifier to jointly control emotion and knowledge transfer during response generation. Quantitative, qualitative, and human evaluation results on the benchmark Topical Chat and the CMU_DoG dataset show that the proposed models can generate natural and coherent sentences, capturing essential facts with considerable improvement over emotional content.

An eye on semantics: a study on the influence of concreteness and predictability on early fixation durations

ABSTRACT We used eye-tracking during natural reading to study how semantic control and representation mechanisms interact for the successful comprehension of sentences, by manipulating sentence context and single-word meaning. Specifically, we examined whether a word’s semantic characteristic (concreteness) affects first fixation and gaze durations (FFDs and GDs) and whether it interacts with the predictability of a word. We used a linear mixed effects model including several possible psycholinguistic covariates. We found a small but reliable main effect of concreteness and replicated a predictability effect on FFDs, but we found no interaction between the two. The results parallel previous findings of additive effects of predictability (context) and frequency (lexical level) in fixation times. Our findings suggest that the semantics of a word and the context created by the preceding words additively influence early stages of word processing in natural sentence reading.

Emotion effects in second language processing: Evidence from eye movements in natural sentence reading

Abstract There exists insufficient eye-tracking evidence on the differences in emotional word processing between the first language (L1) and second language (L2) readers. This study conducted an eye-tracking experiment to investigate the emotional effects in L2 sentence reading, and to explore the modulation of L2 proficiency and individual emotional states. Adapted from Knickerbocker et al. (2015), the current study recorded eye movements at both early and late processing stages when late Chinese–English bilinguals read emotion-label and neutral target words in natural L2 sentences. Results indicated that L2 readers did not show the facilitation effects of lexical affective connotations during sentence reading, and they even demonstrated processing disadvantages for L2 emotional words. Additionally, the interaction effect between L2 proficiency and emotion was consistently significant for the measure of total reading time in positive words. Measurements of participants’ depressive and anxious states were not robustly correlated with eye movement measures. Our findings supplemented new evidence to existing sparse eye-tracking experiments on L2 emotion processing, and lent support to several theoretical frameworks in the bilingual research field, including the Emotional Contexts of Learning Theory, Lexical Quality Hypothesis and Revised Hierarchical Model.

An quantitative analysis of Punjabi tones

Punjabi has three tones, the falling and the rising tones developed due to the historically lost voiced aspirated consonants and word-final glottal fricative /ɦ/, and the default tone occurred elsewhere. While there has been some experimental research on Punjabi tones, there have been limitations due few stimuli and speakers. The main aim of the current study was to provide a phonetic examination of the fundamental frequency (f0) patterns across a large number of Punjabi words produced by multiple speakers. The experiment was conducted online using 24 native speakers (9F, 15M) of Indian Punjabi. The list of stimuli contained 66 monosyllabic words (default = 18, falling = 15, rising = 33) with either /a/ or /ə/. The speakers were recorded producing the stimuli in carrier and natural sentence environments. Linear mixed effects analyses were conducted on six f0 measures: the onset, mid, and offset of the vowel, the f0 range (MaxF0-MinF0), and the f0 trajectory in the first (MidF0-BegF0) and final (EndF0-MidF0) halves of the vowel. The results confirmed three distinct pitch curves for the three tones. Each of the three tones were distinguished from one another for the onset f0 and the EndF0-MidF0 measures. Vowel quality influenced tone realization. Between-speaker and word variation was observed.

교양영어수업에서 챗GPT 활용에 대한 학습자의 인식 및 태도 연구

The purpose of this paper aims to examine students’ perceptions and attitude on chatGPT and provide some suggestions about them. To that end, an online survey was conducted on 134 college students enrolled in an university in Chungcheong-namdo and the collected data were analyzed using descriptive statistics. As a result of this study, students used chatGPT the most to check and modify English grammar. After that, it appeared in the order of natural context, information collection, sentence structure, words, ideas, and translation. Second, overall, students showed high satisfaction on the use of chatGPT. Using chatGPT has increased the students’ usefulness on English learning and confidence in writing English presentation script. Based on the results of this study, instructors should include activities through the use of chatGPT in the curriculum to enhance students' various language experiences and present guidelines to become self-directed learners who control and control their own learning.

Fixation-related fMRI analysis reveals the neural basis of natural reading of unspaced and spaced Chinese sentences.

Although there are many eye-movement studies focusing on natural sentence reading and functional magnetic resonance imaging research on reading with serial visual presentation paradigms, there is a scarcity of investigations into the neural mechanism of natural sentence reading. The present study recruited 33 adults to read unspaced and spaced Chinese sentences with the eye tracking and functional magnetic resonance imaging data recorded simultaneously. By using fixation-related functional magnetic resonance imaging analysis, this study showed that natural reading of Chinese sentences produced activations in ventral visual, dorsal attention, and semantic brain regions, which were modulated by the properties of words such as word length and word frequency. The multivoxel pattern analysis showed that the activity pattern in the left middle temporal gyrus could significantly predict the visual layout categories (i.e. unspaced vs. spaced conditions). Dynamic causal modeling analysis showed that there were bidirectional brain connections between the left middle temporal gyrus and the left inferior occipital cortex in the unspaced Chinese sentence reading but not in the spaced reading. These results provide a neural mechanism for the natural reading of Chinese sentences from the perspective of word segmentation.

Aspect-Category based Sentiment Analysis with Unified Sequence-To-Sequence Transfer Transformers

In recent years, Aspect-based sentiment analysis (ABSA) has received increasing attention from the scientific community for Vietnamese language. However, most previous studies solved various subtasks in ABSA based on machine learning, deep learning and transformer-based architectures in a classification way. Recently, the release of pre-trained sequence-to-sequence brings a new approach to address the ABSA as a text generation problem for Vietnamese ABSA tasks. In this paper, we formulate the Aspect-category based sentiment analysis task as the conditional text generation task and investigate different unified generative transformer-based models. To represent the labels in a natural sentence, we apply a simple statistical method and observation of the commenting style. We conduct experiments on two benchmark datasets. As a result, our model achieved new state-of-the-art results with the micro F1-score of 75.53% and 86.60% for the two datasets with different levels for the restaurant domain. In addition, our experimental results achieved the best score for the smartphone domain with the macro F1-score of 81.10%.

Body Perception From Connected Speech: Speaker Height Discrimination from Natural Sentences and Sine-Wave Replicas with and without Pitch.

In addition to language, the human voice carries information about the physical characteristics of speakers, including their body size (height and weight). The fundamental speaking frequency, perceived as voice pitch, and the formant frequencies, or resonators of the vocal tract, are the acoustic speech parameters that have been most intensely studied for perceiving a speaker's body size. In this study, we created sine-wave (SW) replicas of connected speech (sentences) uttered by 20 male and 20 female speakers, consisting of three time-varying sinusoidal waves matching the frequency pattern of the first three formants of each sentence. These stimuli only provide information about the formant frequencies of a speech signal. We also created a new experimental condition by adding a sinusoidal replica of the voice pitch of each sentence. Results obtained from a binary discrimination task revealed that (a) our SW replicas provided sufficient useful information to accurately judge the speakers' body height at an above chance level; (b) adding the sinusoidal replica about the voice pitch did not significantly increase accuracy; and (c) stimuli from female speakers were more informative for body height detection and allowed higher perceptual accuracy, due to a stronger correlation between formant frequencies and actual body height than stimuli from male speakers.

MALip: Modal Amplification Lipreading based on reconstructed audio features

Lipreading is a type of human–computer interaction based on visual information. From the perspective of pronunciation principle, lip is only one of the vocal tract organs, and it is difficult to express the complete pronunciation process. It is very challenging to recognize speech contents solely through lip movements. As the shape of the vocal tract determines the final sound during the pronunciation process, we propose to improve the accuracy of lipreading by concurrently exploiting the lip features and reconstructed vocal tract features, and we call our method Modal Amplification Lipreading (MALip). We extend the U-Net model to learn the vocal tract features from Mel-Spectrogram features extracted from audio. Our model aims to reduce the computation complexity while ensuring the extended vocal tract features to have a good quality. We also introduce techniques to ensure the vocal tract features to be effective without being compromised by interference from noise or invalid audio. In addition, to facilitate the study with the incorporation of vocal tract features, we recorded a maximum sentence-level Chinese dataset ICSLR in an experimental environment, and verify the effectiveness of reconstructed audio features in improving the lip recognition accuracy for the first time. Through extensive experiments on ICSLR and publicly natural sentence dataset CMLR, we demonstrate the effectiveness of our MALip method compared with state-of-the-art counterparts.

Image Caption Generator Using Attention Based Neural Networks

Abstract: Image caption generation is a method used to create sentences that describe the scene depicted in a given image. The process includes identifying objects within the image, carrying out various operations, and identifying the most important features of the image. Once the system has identified this information, it generates the most relevant and concise description of the image, which is both grammatically and semantically correct. With the progress in deep-learning techniques, algorithms are able to generate text in the form of natural sentences that can effectively describe an image. However, replicating the natural human ability to comprehend image content and produce descriptive text is a difficult task for machines. The uses of image captioning are vast and of great significance, as it involves creating succinct captions utilizing a variety of techniques such as Natural Language Processing (NLP), Computer Vision(CV), and Deep Learning (DL) techniques. The current study presents a system that employs an attention mechanism, in addition to an encoder and a decoder, to generate captions. It utilizes a pretrained CNN, Inception V3, to extract features from the image and a RNN, GRU, to produce a relevant caption. The attention mechanism used in this model is Bahdanau attention, and the Flickr-8Kdataset is utilized for training the model. The results demonstrate the model's capability to understand images and generate text in a reasonable manner.

Implementation of Simple and Efficient Picture Caption Generator

Image captioning or picture captioning has become one of the most widely used technologies in applications that generate and provide captions for specific photographs. All these things are done with the help of deep neural networks. It identifies the specific objects in an image and their attributes and relationships. The purpose of this research is to find different things in a photograph, figure out their relationships, and write captions. The proposed system is implemented on dataset Flickr8k along with python. The input images are pre-processed and then features from images are extracted using CNN. To translate the features and objects extracted by CNN to a natural sentence in English LSTM is utilized in the implementation. Different types of images are tested with the proposed system. The results are presented with the generated image captions. The results presented shows the accuracy of the system. The presented method has potentials for such applications where image captioning is essential.

On the Viability of Diagrams and Drawings as System Requirements

Requirement expression media is an overlooked consideration in requirements innovation. The path to including media other than natural language requirement sentences begins with determining the viability of media to be used as requirement expressions in systems engineering. This research uses the INCOSE characteristics for individual requirements and sets of requirements as a basis for an assessment of the sufficiency of model diagrams and engineering drawings to express requirements. A critical review of high-impact sources is used to develop characteristic lists for natural language requirement sentences and for model diagrams and engineering drawings. A comparative analysis of the characteristic lists for requirement sentences, model diagrams, and engineering drawings shows that each have the same fundamental characteristics and the differences are based on the ability of the media to represent abstraction. The contribution of this research is the establishment of model diagrams and engineering drawings as viable alternatives to natural language sentences for system engineering requirements expressions.

Retrieval Augmented Convolutional Encoder-decoder Networks for Video Captioning

Video captioning has been an emerging research topic in computer vision, which aims to generate a natural sentence to correctly reflect the visual content of a video. The well-established way of doing so is to rely on encoder-decoder paradigm by learning to encode the input video and decode the variable-length output sentence in a sequence-to-sequence manner. Nevertheless, these approaches often fail to produce complex and descriptive sentences as natural as those from human being, since the models are incapable of memorizing all visual contents and syntactic structures in the human-annotated video-sentence pairs. In this article, we uniquely introduce a Retrieval Augmentation Mechanism (RAM) that enables the explicit reference to existing video-sentence pairs within any encoder-decoder captioning model. Specifically, for each query video, a video-sentence retrieval model is first utilized to fetch semantically relevant sentences from the training sentence pool, coupled with the corresponding training videos. RAM then writes the relevant video-sentence pairs into memory and reads the memorized visual contents/syntactic structures in video-sentence pairs from memory to facilitate the word prediction at each timestep. Furthermore, we present Retrieval Augmented Convolutional Encoder-Decoder Network (R-ConvED), which novelly integrates RAM into convolutional encoder-decoder structure to boost video captioning. Extensive experiments on MSVD, MSR-VTT, Activity Net Captions, and VATEX datasets validate the superiority of our proposals and demonstrate quantitatively compelling results.

Generating Natural Language From Logic Expressions With Structural Representation

Incorporating logic reasoning with deep neural networks (DNNs) is an important challenge in machine learning. In this article, we study the problem of converting logical expressions into natural language. In particular, given a sequential logic expression, the goal is to generate its corresponding natural sentence. Since the information in a logic expression often has a hierarchical structure, a sequence-to-sequence baseline struggles to capture the full dependencies between words, and hence it often generates incorrect sentences. To alleviate this problem, we propose a model to convert Structural Logic Expressions into Natural Language (SLEtoNL). SLEtoNL converts sequential logic expressions into structural representation and leverages structural encoders to capture the dependencies between nodes. The quantitative and qualitative analyses demonstrate that our proposed method outperforms the seq2seq model, which is based on the sequential representation, and outperforms strong pretrained language models ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , T5, BART, GPT3) with a large margin (28.6 in BLEU3) in out-of-distribution evaluation. The data and code will be available on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com</uri> .

Controllable Text Generation Using Semantic Control Grammar

Controllable text generation is the primary technique for controlling specific attributes such as topic, keywords and obtaining augmented data. This work proposes a novel controllable text generation framework to improve the controllability of generation models. First, we introduce semantic control grammar, a controllable input format to generate sentences that satisfy the constraints. Second, we adopt a generation and rerank method to obtain semantically reranked controlled sentences. Extensive experiments and analyses are conducted on benchmark, natural language understanding, data-to-text generation, and text classification datasets. Through automatic evaluations, we show that our method leads to improvement over strong baselines. The results show that our model can control sentence and word attributes and semantically generate natural sentences. Furthermore, our techniques improve the generation quality of the model.