Story Generation Research Articles

Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation

Abstract Storytelling is an integral part of human experience and plays a crucial role in social interactions. Thus, Automatic Story Evaluation (ASE) and Generation (ASG) could benefit society in multiple ways, but they are challenging tasks which require high-level human abilities such as creativity, reasoning, and deep understanding. Meanwhile, Large Language Models (LLMs) now achieve state-of-the-art performance on many NLP tasks. In this paper, we study whether LLMs can be used as substitutes for human annotators for ASE. We perform an extensive analysis of the correlations between LLM ratings, other automatic measures, and human annotations, and we explore the influence of prompting on the results and the explainability of LLM behaviour. Most notably, we find that LLMs outperform current automatic measures for system-level evaluation but still struggle at providing satisfactory explanations for their answers.

Comparing Deaf/Hard-of-Hearing Children's Oral Narratives Using Movies and Static Books.

Clinicians utilize various methods for narrative sampling, including oral assessments like story generation and retelling, often aided by visual aids. Assessing language skills in deaf/hard of hearing (DHH) children requires careful narrative technique selection. This comparative observational study investigates the narrative outcomes of story generation and retelling tasks in 21 DHH children, using both book and movie contexts. Most microstructural elements (except for the mean length of utterance) were similar across the book and movie conditions. Differences in word choice, such as the use of action verbs, were evident. Macrostructural differences between book and movie conditions were insignificant regarding story grammar elements. However, movies, being visually engaging, potentially enhance the inclusion of characters, settings, plot, and actions. This research illuminates narrative assessment considerations, emphasizing technology's role in enhancing options for assessment for DHH children.

A comparison of narrative abilities in Malay school-age typically developing children and children with developmental language disorder

ABSTRACT Research on narrative skills in the Malay language is scarce for both typically developing (TD) and children with Developmental Language Disorders (DLD). This study examines the differences in narrative abilities of Malay-speaking school-age children with and without DLD. Fifteen Malay-speaking TD children and 15 children with DLD told four stories elicited using the Multilingual Assessment Instrument for Narrative (MAIN). Group comparisons were conducted in narrative production (story retelling and story generation) as well as in comprehension. We also examined the differences in both groups’ story complexity (e.g. the combination of goals (G), attempts (A) and outcomes (O)). General microstructure features such as the total number of words (TNW), total number of sentences (TNS) and total number of different words (NDW) were also investigated. TD children outperformed children with DLD in both narrative production and comprehension performances. TD children constructed a higher combination of G, A, and O components than children with DLD. Children with DLD frequently produced G, A, and O in isolation in comparison to TD children. In language productivity and lexical complexity, TD children had a significantly higher frequency in TNW and NDW than children with DLD, resulting in longer narratives and higher uses of different words than children with DLD. Distinct differences are found in the narrative profile of TD children and children with DLD. The results are relatively consistent with the findings of similar studies that utilised and adapted the MAIN in other languages. Clinical implications and recommendations are discussed.

Automatic user story generation: a comprehensive systematic literature review

Automatic user story generation: a comprehensive systematic literature review

Intérêt d’une intervention de soutien langagier basée sur la narration et menée par l’enseignante auprès d’enfants de grande section de maternelle avec difficultés langagières.

Context: Narrative intervention enhances narrative abilities by addressing macrostructural elements within storytelling tasks for instance and can be an effective way of supporting the language development of preschool-aged children. Moreover, language support in educational settings plays a preventive role in addressing potential difficulties in language skills due to a lack of linguistic stimulation. Objectives: This study assesses the implementation fidelity of a tier 2 intervention with a pre-school teacher, measures the effectiveness of this intervention with children with language difficulties, and determines whether these children can catch up the level of narrative skills of the rest of the class, following the intervention Methods: Children with language difficulties were selected by the master student based on an inclusion test determining their linguistic profile, which was administered to all pupils in the class. Narrative skills were assessed for the entire class in a pre-test, considering elements of macrostructure and microstructure elicited during narrative generation and retelling activities. A Rapid Automa-tized Naming (RAN) control task was also included to evaluate the specificity of the intervention. The teacher was trained by the master student for the intervention, according to profession-al development principles. Subsequently, the intervention, lasting 45 minutes once a week for 6 weeks, was delivered by the teacher to five children identified as having language difficulties, aged 5;5 to 5;10. The intervention aimed to improve the number of macrostructural elements in a retelling activity through the use of icons and sequential picture stories. Finally, the RAN task and narrative skills (retelling and narrative generation) of the entire class were reassessed in a post-test. Results: At the end of the intervention, macro- and microstructural improvements were observed in the story generation activity in the experimental group. Moreover, following the teacher's intervention, the experimental group did not differ anymore from the rest of the class in retelling (when considering the trained story) nor in story generation. However, data analysis revealed weaknesses in the implementation of the intervention, both in content and dosage. Conclusion: The results contribute to the existing research supporting the effectiveness of group narrative intervention. Furthermore, they suggest the possibility of replicating a tier 2 indirect narrative intervention conducted by the teacher, by enhancing the professional development framework.

Beyond Event-Centric Narratives: Advancing Arabic Story Generation with Large Language Models and Beam Search

In the domain of automated story generation, the intricacies of the Arabic language pose distinct challenges. This study introduces a novel methodology that moves away from conventional event-driven narrative frameworks, emphasizing the restructuring of narrative constructs through sophisticated language models. Utilizing mBERT, our approach begins by extracting key story entities. Subsequently, XLM-RoBERTa and a BERT-based linguistic evaluation model are employed to direct beam search algorithms in the replacement of these entities. Further refinement is achieved through Low-Rank Adaptation (LoRA), which fine-tunes the extensive 3 billion-parameter BLOOMZ model specifically for generating Arabic narratives. Our methodology underwent thorough testing and validation, involving individual assessments of each submodel. The ROCStories dataset provided the training ground for our story entity extractor and new entity generator, and was also used in the fine-tuning of the BLOOMZ model. Additionally, the Arabic ComVE dataset was employed to train our commonsense evaluation model. Our extensive analyses yield crucial insights into the efficacy of our approach. The story entity extractor demonstrated robust performance with an F-score of 96.62%. Our commonsense evaluator reported an accuracy of 84.3%, surpassing the previous best by 3.1%. The innovative beam search strategy effectively produced entities that were linguistically and semantically superior to those generated using baseline models. Further subjective evaluations affirm our methodology’s capability to generate high-quality Arabic stories characterized by linguistic fluency and logical coherence.

Examining Adolescent Language Performance in Discourse Production Across Four Elicitation Tasks.

Comprehensive spoken language assessment should include the evaluation of language use in naturalistic contexts. Discourse elicitation and analysis provides the opportunity for such an evaluation to occur. In this article, our overall aim was to describe adolescents' language performance on four elicitation tasks and determine if there are task-related differences across the elicitation tasks. Forty-four typically developing adolescents with ages ranging from 12;2 to 17;11 (years;months; M = 15;2; 21 boys and 23 girls) participated in the study. They completed four spoken discourse tasks: (a) story generation using a wordless picture book, (b) fable retell, (c) six personal narratives in response to emotion-based prompts, and (d) monologic response to two stories that contained a moral dilemma. Responses were transcribed and analyzed for four language performance measures tapping into language productivity, syntactic complexity, lexical diversity, and verbal facility. Despite individual variability in performance, mean scores were close to median scores for most measures, suggesting a symmetrical distribution. As expected, all four language performance measures were significantly different across the four elicitation tasks. The personal narrative task elicited the longest samples, with the highest verbal fluency. In contrast, both lexical diversity and syntactic complexity were the strongest in response to the fable retell and the moral dilemma tasks. This investigation provides speech-language pathologists with an overview of how task-related factors may impact adolescent language performance. These findings may be used to support their clinical decision-making processes in choosing a suitable discourse task when conducting a comprehensive spoken language assessment. Three hypothetical case examples are used to illustrate the decision-making process. https://doi.org/10.23641/asha.25761768.

A cross-attention augmented model for event-triggered context-aware story generation

Despite recent advancements, existing story generation systems continue to encounter difficulties in effectively incorporating contextual and event features, which greatly influence the quality of generated narratives. To tackle these challenges, we introduce a novel neural generation model, EtriCA, that enhances the relevance and coherence of generated stories by employing a cross-attention mechanism to map context features onto event sequences through residual mapping. This feature capturing mechanism enables our model to exploit logical relationships between events more effectively during the story generation process. To further enhance our proposed model, we employ a post-training framework for knowledge enhancement (KeEtriCA) on a large-scale book corpus. This allows EtriCA to adapt to a wider range of data samples. This results in approximately 5% improvement in automatic metrics and over 10% improvement in human evaluation. We conduct extensive experiments, including comparisons with state-of-the-art (SOTA) baseline models, to evaluate the performance of our framework on story generation. The experimental results, encompassing both automated metrics and human assessments, demonstrate the superiority of our model over existing state-of-the-art baselines. These results underscore the effectiveness of our model in leveraging context and event features to improve the quality of generated narratives.

Narration with Graphics: An AI Generated Story Teller

Storytelling has always been a powerful tool to convey information, connect with people, and make an impact. The aim of our project is to generate story by using the concept of AI and Machine learning .AI generated storytelling has apotential to transform a variety of industries, for marketing and education to entertainment and beyond. By utilizing the capabilities of ML and NLP, stories generated by artificial intelligence can be tailored to meet the specific need of each industry like personalized narratives, interactive and dynamic storytelling, multimodal storytelling, cross domain storytelling. The introduction of GEN-AI can lead to both opportunities and potential challenges among different creator communities which required collaboration from both academic and industry. This topic gathers and analyse vast amount of data from various sources including literature, movies, and social media we are using natural language processing which understand and process human language enabling them to communicate with the user effectively. To use the facial recognition technology, we are using DALL-E, OpenCV which uses model like SVM, decision tree, and knn etc. The objective of the research is to investigate the power of AI software and automating the process of story generation. This exploration involves utilizing various AI models, such as natural language generation (NLG) algorithms, deep learning architectures, and reinforcement learning frameworks, to create compelling and coherent stories across different genres and themes. We're showcasing an innovative system that empowers users to effortlessly create short stories accompanied by relevant images with minimal input. Our final project is accessible through a user-friendly web page interface, allowing individuals to craft their narratives seamlessly. Key Words: Storytelling, Artificial intelligence, Natural language processing, Ai system, Narratives, Automatic, DALL -E, OpenCV.

Speech Sounds Production, Narrative Skills, and Verbal Memory of Children with 22q11.2 Microdeletion.

22q11.2 deletion syndrome (22q11.2DS), the most frequent microdeletion syndrome in humans, is related to a high risk of developing neurodevelopmental disorders. About 95% of patients with 22q11.2DS have speech and language impairments. Global articulation, story generation, and verbal memory tests were applied to compare articulatory characteristics of speech sounds, spontaneous language abilities, and immediate verbal memory between four groups of Serbian-speaking children: patients with 22q11.2DS, children with clinical presentation of 22q11.2DS that do not have the microdeletion, children with non-syndromic congenital heart defects, and their peers with typical speech-sound development. The obtained results showed that children with this microdeletion have impaired articulation skills and expressive language abilities. However, we did not observe weaker receptive language skills and immediate verbal memory compared to healthy controls. Children with 22q11.2DS should be considered a risk category for the development of speech-sound pathology and expressive language abilities. Since speech intelligibility is an instrument of cognition and adequate peer socialization, and language impairment in school-aged children with 22q11DS might be an indicator of increased risk for later psychotic symptoms, patients with 22q11.2 microdeletion should be included in a program of early stimulation of speech-language development immediately after diagnosis is established.

Linguistic Disfluencies in Korean-English Bilingual Children in the Narrative Generation and Retell Task

Objectives: The purpose of this study was to examine the frequency and types of linguistic disfluencies that are produced by typically developing bilingual Korean-English (KE) speaking children and to investigate whether their disfluent speech is mediated by language produced and/or language exposure duration.Methods: Nineteen bilingual KE speaking children (12 boys, 7 girls) and twenty-one monolingual Korean speaking children (8 boys, 13 girls) aged 6-9 participated in this study. Narratives were elicited using story generation and retell tasks in each language. The frequency and types of linguistic disfluencies (fillers, repetitions, revisions, and pauses) were analyzed.Results: The bilingual group showed a significantly higher rate of total disfluency, fillers, and repetitions than the monolingual group. There was no significant difference in the rates of disfluency between narrative task types. In terms of the language produced, the bilingual group produced more revisions and more disfluencies in Korean than in English during the story generation task. In the retell task, there was a significant difference in the total disfluency rate, fillers, and revisions depending on the language the child was speaking. A significant negative correlation between English language exposure duration and the total disfluency rate was found.Conclusion: Bilingual KE speaking children showed higher disfluency rates than monolingual peers in story generation and retell tasks. Results showed that the disfluency can be influenced by the grammatical structure of the language being spoken and the levels of language experience. Results suggest the exposure duration to second language, language proficiency, and morphological distinctions between two languages need to be considered for assessment and intervention.

Movie Review Sentiment Analysis and AI-Story Generation Web Application

Abstract: This paper presents a web application designed to enhance the interaction between users and multimedia content through two innovative platforms: "MOVIE REVIEW SENTIMENT ANALYSIS" and "AI-STORY GENERATION." The first platform leverages the Fastai library and LSTM networks to perform sentiment analysis on user-submitted movie reviews, determining their polarity as positive or negative. This analysis directly influences the movie's aggregate rating, with the system capable of processing extensive textual input with high accuracy. Ratings are dynamically updated and stored in JSON format. The second platform, AI-STORY GENERATION, introduces a creative avenue for generating novel stories based on user prompts. It utilizes Natural Language Processing (NLP) techniques and a Generative Pretrained Transformer (GPT-2 Large) model, trained on tokenized movie scripts, to produce coherent and engaging narratives. The Flask web framework supports the application's backend, providing a robust and scalable foundation for user interaction and content delivery. This dual-platform approach not only demonstrates the practical application of advanced machine learning and NLP techniques in enhancing digital entertainment experiences but also showcases the potential for AI-driven tools to foster creative storytelling. The integration of sentiment analysis and story generation within a single application illustrates a novel use case of AI in the context of web-based entertainment, potentially paving the way for more personalized and interactive user experiences

Diagnostic Accuracy of Grammaticality and Utterance Length in Bilingual Children.

The purpose of this study was to investigate the diagnostic accuracy of two measures derived from spontaneous language samples, mean length of utterance in words (MLUw) and percentage of grammatical utterances (PGU), in identifying developmental language disorder (DLD) in Spanish-English bilingual children. We examined two approaches: best language and total language. The participants in this study included 74 Spanish-English bilingual children with (n = 36) and without (n = 38) DLD. Language samples were elicited through a story retell and story generation task using Frog wordless picture books in English and Spanish. Stories were transcribed and coded using the Systematic Analysis of Language Samples (Miller & Iglesias, 2020) to extract MLUw and PGU in both languages. Logistic regression analyses suggested that a model that included PGU, MLUw, and age achieved the best diagnostic accuracy in predicting group membership. Both approaches, best language and total language, had fair diagnostic accuracy. In combination, PGU and MLUw seem to be useful diagnostic tools to differentiate bilingual children with and without DLD. Clinical implications and usability are discussed.

Automated Narrative Craft: Exploring Machine Learning in Story Generation.

Automated Narrative Craft: Exploring Machine Learning in Story Generation.

Narrative Analysis in English Language Teachers’ Professional Identity Research: A Review

Amidst the escalating emergence of generic methodological articles on narrative inquiry, this article critically reviews diff erent narrative analysis approaches and their application in English language teachers’ professional identity research. For this purpose, we reviewed currently available methodological books and articles on narrative inquiry in general and narrative analysis in particular. Additionally, we reviewed twenty purposively selected empirical articles published since 2015 thatemployed narrative inquiry to explore English teachers’ professional identities. A review of methodological articles revealed that the narrative analysis and analysis of narrative dichotomy are blurred as a certain level of interpretation occurs during the co-construction of stories, transcribing, translating, restorying, and fi nally, the consumption of the reports by the readers. Consequently, many researchers have considered paradigmatic analysis as the first step of narrative analysis. In contrast, research practices showed that the interpretation of stories begins only after story generation. Besides, amongst diff erent narrative analysis approaches, positioning analysis underscores how tellers represent themselves through narratives, whereas the small story approach of Barkhuizen accentuates analyzing content and context; however, narrative forms and linguistic features and their motives are rarely investigated. Though the small stories approach of Georgakopoulou (2007) advocates explicitly for considering ways of telling (linguistic features and communicative how), it is rarely evident in practice. This article, therefore, identifies a need for a comprehensive narrative analysis approach that considers what (content), where and when (context), how (form), and who and why (agency and discourse) of the story.

Vocabulary Instruction Embedded in Narrative Intervention: A Repeated Acquisition Design Study With First Graders at Risk of Language-Based Reading Difficulty.

The purpose of the current study was to investigate the effect of vocabulary instruction embedded in the narrative intervention on the immediate and retained definitional knowledge of taught words for first graders at risk for language-related reading difficulties. We employed a repeated acquisition design with innovative quality features and supplemental statistics with 11 treatment students and three control students. In the context of the school's multitiered system of supports, treatment students received 30-min small group interventions, 4 days a week for 12 weeks. Intervention involved story retelling and personal story generation lessons, both of which emphasized the learning and practicing of target vocabulary words in each story. Pre- and postprobes of the taught definitions were conducted every week. According to visual analysis conventions of single-case research, there was a consistent pattern of improvement from pre- to postprobes for all treatment participants, but for none of the control participants. Retention was also consistently observed, when measured at Week 13. Supplemental statistics confirmed that large effects were associated with the intervention. Vocabulary instruction embedded in narrative intervention led to meaningful acquisition and retention of taught vocabulary for students at risk of language-based reading difficulty.

EXplainable AI with GPT4 for story analysis and generation: A novel framework for diachronic sentiment analysis

eXplainable AI with GPT4 for story analysis and generation: A novel framework for diachronic sentiment analysis

Summarizing Web Archive Corpora via Social Media Storytelling by Automatically Selecting and Visualizing Exemplars

People often create themed collections to make sense of an ever-increasing number of archived web pages. Some of these collections contain hundreds of thousands of documents. Thousands of collections exist, many covering the same topic. Few collections include standardized metadata. This scale makes understanding a collection an expensive proposition. Our Dark and Stormy Archives (DSA) five-process model implements a novel summarization method to help users understand a collection by combining web archives and social media storytelling. The five processes of the DSA model are: select exemplars, generate story metadata, generate document metadata, visualize the story, and distribute the story. Selecting exemplars produces a set of k documents from the N documents in the collection, where k < < N , thus reducing the number of documents visitors need to review to understand a collection. Generating story and document metadata selects images, titles, descriptions, and other content from these exemplars. Visualizing the story ties this metadata together in a format the visitor can consume. Without distributing the story, it is not shared for others to consume. We present a research study demonstrating that our algorithmic primitives can be combined to select relevant exemplars that are otherwise undiscoverable using a conventional search engine and query generation methods. Having demonstrated improved methods for selecting exemplars, we visualize the story. Previous work established that the social card is the best format for visitors to consume surrogates. The social card combines metadata fields, including the document’s title, a brief description, and a striking image. Social cards are commonly found on social media platforms. We discovered that these platforms perform poorly for mementos and rely on web page authors to supply the necessary values for these metadata fields. With web archives, we often encounter archived web pages that predate the existence of this metadata. To generate this missing metadata and ensure that storytelling is available for these documents, we apply machine learning to generate the images needed for social cards with a Precision@1 of 0.8314. We also provide the length values needed for executing automatic summarization algorithms to generate document descriptions. Applying these concepts helps us create the visualizations needed to fulfill the final processes of story generation. We close this work with examples and applications of this technology.

CLICK: Integrating Causal Inference and Commonsense Knowledge Incorporation for Counterfactual Story Generation

Counterfactual reasoning explores what could have happened if the circumstances were different from what actually occurred. As a crucial subtask, counterfactual story generation integrates counterfactual reasoning into the generative narrative chain, which requires the model to preserve minimal edits and ensure narrative consistency. Previous work prioritizes conflict detection as a first step, and then replaces conflicting content with appropriate words. However, these methods mainly face two challenging issues: (a) the causal relationship between story event sequences is not fully utilized in the conflict detection stage, leading to inaccurate conflict detection, and (b) the absence of proper planning in the content rewriting stage results in a lack of narrative consistency in the generated story ending. In this paper, we propose a novel counterfactual generation framework called CLICK based on causal inference in event sequences and commonsense knowledge incorporation. To address the first issue, we utilize the correlation between adjacent events in the story ending to iteratively calculate the contents from the original ending affected by the condition. The content with the original condition is then effectively prevented from carrying over into the new story ending, thereby avoiding causal conflict with the counterfactual conditions. Considering the second issue, we incorporate structural commonsense knowledge about counterfactual conditions, equipping the framework with comprehensive background information on the potential occurrence of counterfactual conditional events. Through leveraging a rich hierarchical data structure, CLICK gains the ability to establish a more coherent and plausible narrative trajectory for subsequent storytelling. Experimental results show that our model outperforms previous unsupervised state-of-the-art methods and achieves gains of 2.65 in BLEU, 4.42 in ENTScore, and 3.84 in HMean on the TIMETRAVEL dataset.

There and Back Again: Extracting Formal Domains for Controllable Neurosymbolic Story Authoring

Story generators using language models offer the automatic production of highly fluent narrative content, but they are hard to control and understand, seizing creative tasks that many authors wish to perform themselves. On the other hand, planning-based story generators are highly controllable and easily understood but require story domains that must be laboriously crafted; further, they lack the capacity for fluent language generation. In this paper, we explore hybrid approaches that aim to bridge the gap between language models and narrative planners. First, we demonstrate that language models can be used to author narrative planning domains from natural language stories with minimal human intervention. Second, we explore the reverse, demonstrating that we can use logical story domains and plans to produce stories that respect the narrative commitments of the planner. In doing so, we aim to build a foundation for human-centric authoring tools that facilitate novel creative experiences.