Language Research Articles

Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition

Multimodal intent recognition aims to leverage diverse modalities such as expressions, body movements and tone of speech to comprehend user's intent, constituting a critical task for understanding human language and behavior in real-world multimodal scenarios. Nevertheless, the majority of existing methods ignore potential correlations among different modalities and own limitations in effectively learning semantic features from nonverbal modalities. In this paper, we introduce a token-level contrastive learning method with modality-aware prompting (TCL-MAP) to address the above challenges. To establish an optimal multimodal semantic environment for text modality, we develop a modality-aware prompting module (MAP), which effectively aligns and fuses features from text, video and audio modalities with similarity-based modality alignment and cross-modality attention mechanism. Based on the modality-aware prompt and ground truth labels, the proposed token-level contrastive learning framework (TCL) constructs augmented samples and employs NT-Xent loss on the label token. Specifically, TCL capitalizes on the optimal textual semantic insights derived from intent labels to guide the learning processes of other modalities in return. Extensive experiments show that our method achieves remarkable improvements compared to state-of-the-art methods. Additionally, ablation analyses demonstrate the superiority of the modality-aware prompt over the handcrafted prompt, which holds substantial significance for multimodal prompt learning. The codes are released at https://github.com/thuiar/TCL-MAP.

Intelligent Voice Mail System

As technology is advancing, people are coming closer to digital life and digital communication. In this modern period, there are many ways to communicate with others through the internet. Common mode of communication among people through internet is Email. A lot of Business and personal information is exchanged over emails in today's time. Some people cannot use these technologies because either they are illiterate or cannot see the screen. Using this system entitled Voice-based Bare Email for Visually Impaired, it will make the email system very easily accessible to visually challenged people and also help society. This application uses text-to-speech and speech-to-text converters so that visually challenged people can operate the system easily. This system reduces the complexity of remembering the characters or information regarding keyboard shortcuts. Every function will be based on simple voice commands so that those people can easily make use of the technology. It also includes various expert systems, natural language processing, machine vision, and speech recognition. To understand and analyze human language such as English by extracting metadata from keywords, emotions, relations, and concepts in Natural Language Processing

Communication Is Key

The relationship between language and communication has engaged researchers for more than a century. Studies have compared human language with animal communication systems, discussing to what degree the complex features and mechanisms that constitute human language have parallels and precursors in animal communication systems. This article discusses the characteristic design features of human language and concludes that language is a uniquely human feature, even if some aspects can also be identified in animal communication systems. The discussion of experiments aiming towards teaching human language to animals leads to the observation that there is a difference between understanding a language and using it to communicate. Human language, however, is effective as a means in interspecies relations pertaining to human-animal communication.

Abstract 4931: The Cancer Proteome Atlas: Expanded datasets and empowered with an AI-driven chatbot

Abstract Reverse-phase protein arrays (RPPAs) represent a powerful functional proteomic approach to elucidate cancer-related molecular mechanisms and develop novel cancer therapies. To facilitate community-based investigation of the large-scale protein expression data generated by this platform, we have developed a user-friendly, open-access bioinformatic resource, The Cancer Proteome Atlas (TCPA, http://tcpaportal.org), which contains several applications. The first one focuses on RPPA data of patient tumors, which contains >8,000 samples of 32 cancer types from The Cancer Genome Atlas (TCGA) and other independent patient cohorts. The second application focuses on the RPPA data of >1,500 cancer cell lines with publicly available, high-quality DNA, RNA and drug screening data. The third one focuses on perturbed RPPA profiles of >14,000 samples given drug treatments. We recently expanded our RPPA protein set from 200 to 500 proteins Importantly, this highly selected set is enriched in therapeutic targets and biomarkers and covers key oncogenic pathways such as PI3K/AKT, RAS/MAPK, Src/FAK, TGFβ/SMAD, JAK/STAT, epithelial-mesenchymal transition, DNA damage repair, Hippo, Notch, cell cycle, apoptosis, and immune-oncology. Using this upgraded RPPA platform, we generated an harmonized, expanded RPPA dataset of 8000 TCGA patient samples and 1000 CCLE cell line samples. To further address the informatic challenges of analyzing such large, diverse datasets, in addition to the GUI interfaces, we recently developed a chatbot through which users can analyze the RPPA data through human nature languages and obtain the results and related analytic reports without a learning curve. Such a chatbot empowers a broad research community to explore high-quality RPPA datasets and generate testable hypotheses in an effective and intuitive manner, representing the direction of next-generation data analytics. Citation Format: Han Liang, Wei Liu, Jun Li, Yitao Tang. The Cancer Proteome Atlas: Expanded datasets and empowered with an AI-driven chatbot [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4931.

Neural evidence for referential understanding of object words in dogs

Using words to refer to objects in the environment is a core feature of the human language faculty. Referential understanding assumes the formation of mental representations of these words.1,2 Such understanding of object words has not yet been demonstrated as a general capacity in any non-human species,3 despite multiple behavior-based case reports.4,5,6,7,8,9,10 In human event-related potential (ERP) studies, object word knowledge is typically tested using the semantic violation paradigm, where words are presented either with their referent (match) or another object (mismatch).11,12 Such mismatch elicits an N400 effect, a well-established neural correlate of semantic processing.12,13 Reports of preverbal infant N400 evoked by semantic violations14 assert the use of this paradigm to probe mental representations of object words in nonverbal populations. Here, measuring dogs' (Canis familiaris) ERPs to objects primed with matching or mismatching object words, we found a mismatch effect at a frontal electrode, with a latency (206-606ms) comparable to the human N400. A greater difference for words that dogs knew better, according to owner reports, further supported a semantic interpretation of this effect. Semantic expectations emerged irrespective of vocabulary size, demonstrating the prevalence of referential understanding in dogs. These results provide the first neural evidence for object word knowledge in a non-human animal. VIDEO ABSTRACT.

A fuzzy description logic based IoT framework: Formal verification and end user programming.

The Internet of Things (IoT) has become one of the most popular technologies in recent years. Advances in computing capabilities, hardware accessibility, and wireless connectivity make possible communication between people, processes, and devices for all kinds of applications and industries. However, the deployment of this technology is confined almost entirely to tech companies, leaving end users with only access to specific functionalities. This paper presents a framework that allows users with no technical knowledge to build their own IoT applications according to their needs. To this end, a framework consisting of two building blocks is presented. A friendly interface block lets users tell the system what to do using simple operating rules such as "if the temperature is cold, turn on the heater." On the other hand, a fuzzy logic reasoner block built by experts translates the ambiguity of human language to specific actions to the actuators, such as "call the police." The proposed system can also detect and inform the user if the inserted rules have inconsistencies in real time. Moreover, a formal model is introduced, based on fuzzy description logic, for the consistency of IoT systems. Finally, this paper presents various experiments using a fuzzy logic reasoner to show the viability of the proposed framework using a smart-home IoT security system as an example.

Capturing the full potential of Maltese language learning through ChatGPT

Chat Generative Pretrained Transformer (ChatGPT) is a state-of-the-art artificial intelligence (AI) language model developed by OpenAI. It employs advanced deep learning algorithms to generate text that mimics human language. Although ChatGPT has gained immense popularity in recent times, its significance is expected to rise further in shaping the future of technology and AI. This study investigates the potential of ChatGPT as a tool for enhancing the learning of Maltese for international adult students. The study involved 41 participants who embarked on a journey of exploration, employing ChatGPT for two weeks to support their Maltese learning. The study evaluated the effectiveness of ChatGPT through surveys and focus group discussions, which revealed that while ChatGPT was convenient and accessible, its ability to provide accurate responses to students' questions and support Maltese grammar, vocabulary, and conversational practice was limited. The participants expressed frustration with the ChatGPT's limitations in understanding and responding to Maltese language. The study emphasizes the need to fully unleash ChatGPT's potential by improving its training on Maltese language and collaborating with Maltese language and AI experts to better support Maltese language learning. The findings have important implications for the development of ChatGPT for less widely spoken languages like Maltese.

Functional features of the mirror neuron system during action observation and execution in patients with anxiety and depressive symptoms

The perception and comprehension of non-verbal information and body language in humans and higher primates are realized by the mirror neuron system (MNS). Anxiety and depressive symptoms may change social perception, which could manifest as functional changes in the MNS. In this paper, using the inverse electroencephalography (EEG) problem and rhythm suppression, we investigated spatial and frequency distortions of the MNS in 24 patients exhibiting depressive and anxiety symptoms and 23 controls. EEG was recorded during four motor tasks: action observation (where participants observed a hand gesture performed by a demonstrator), imagination, execution, and joint execution (simultaneous execution with the demonstrator). Mu suppression was employed across a wide frequency and spatial range to assess the level of MNS activity, while the sLORETA method was employed to localize the activity sources. The results indicate that the patients demonstrated task-selective mu suppression mainly during observation and joint execution in the frontal, central, and occipital areas of the cortex across a wide frequency range. In contrast, the controls demonstrated clear and pronounced mu rhythm suppression in the central regions of the brain in the upper-frequency range (10.5 – 13 Hz) during all mirroring tasks. These results suggest that patients with anxiety and depressive symptoms engage additional neural resources to complete social tasks, particularly involving auxiliary neural networks located in the frontal associative arrays and visual cortex.

Simulation modeling of the system of managing the psychophysiological state of a person based on non-verbal extralinguistic characteristics

Today, when artificial intelligence systems are being created, the time of constant dia-logue between humans and computer, there is a great demand for intelligent control systems and human-machine systems. These systems are very complex and require analysis and re-search. The publication is devoted to the simulation computer modeling of control of the psy-chological and physical state of a person. The basis of the human-computer dialogue system is the speech-to-text conversion sys-tem. The use of this system is the basis for the creation of computer language control systems. Unlike systems that work on the principle of identifying a single user response to a request and a proposal or a question from a password database, it is necessary to expand the lan-guage interface of the computer. It is necessary for the computer to accept commands from the operator and execute them only if the voice and its timbre match those registered in the database. Such a system will allow you to differentiate access to the computer and ensure se-curity for the user, preserve his privacy or save his personal data. After analyzing the speech activity of the operator, it became clear that it was necessary to develop a system of command control and «speech-to-text» conversion in the fused speech stream. This task is solved by comparing the words of the commands and is part of the problem of automatic recognition and understanding of the operator's human language. On the basis of such systems, a computer language control system can be created. The analysis has shown that unlike text-dependent identification systems, systems with a dialog method implement not only a one-time operator response to a query or question from the password database, but also the extension of its full-fledged language interface. The computer receives commands from the operator and executes them only if the voice matches the one registered in the database. This analysis is effective in the context of real-life speech activity of the operator. The literature review showed that for the development of sys-tems of command control and "speech-to-text" transformation in a stream of fused speech, it is necessary to use "mechanisms" of automatic recognition and understanding of the opera-tor's language.

Borrowing and the Conditions of its Acceptability in French

Linguistics is a vast field of study that is concerned with the study of human language. It is subdivided into several disciplines, among which we should mention lexicology, which deals with the study of the lexicon in its meaning, frequency, structure and evolution. The lexicon can be understood and studied under several aspects such as the study of the phenomenon of borrowing from other languages, the process of their integration into the nomenclature of the host language as well as the linguistic policies put in place to in the face of the interrupted influx of foreign lexies. Certainly the loan meets the needs of the evolution of society to name new extralinguistic realities, however their? massive presence in the lexical structure of a language can constitute a real danger to its very existence. This is the reason why language policies have emerged like (the Toubon Law of 1994). The context must be understood to examine this phenomenon

Acquiring a language vs. inducing a grammar

Standard computational models of language acquisition treat acquiring a language as a process of inducing a set of string-generating rules from a collection of linguistic data assumed to be generated by these very rules. In this paper I give theoretical and empirical arguments that such a model is radically unlike what a human language learner must do to acquire their native language. Most centrally, I argue that such models presuppose that linguistic data is directly a product of a grammar, ignoring the myriad non-grammatical systems involved in the use of language. The significance of these non-target systems in shaping the linguistic data children are exposed to undermines any simple reverse inference from linguistic data to grammatical competence.

Call combination in African forest elephants Loxodonta cyclotis.

Syntax, the combination of meaning-devoid phonemes into meaningful words, which in turn are combined in structurally and semantically complex sentences, is fundamental to the unlimited expressiveness of human languages. Studying the functions of call combinations in non-human species provides insights into the evolution of such syntactic capabilities. Here, we investigated the combination of high amplitude broadband calls with low frequency rumble vocalizations in a highly social species, the African forest elephant Loxodonta cyclotis. Rumbles play an integral role in coordinating social interactions by transmitting socially relevant information, including individual identity. By contrast, broadband calls, such as roars, are thought to function as signals of distress and urgency as they are typically produced in situations of high emotional intensity. Functional changes associated with the combination of these calls remain little understood. We found that call combinations were produced by all age-sex classes but were most prevalent in immature individuals. We found that rumbles used singularly occurred in all five investigated social contexts, whereas single broadband calls were restricted to two resource-related contexts. Call combinations also occurred in all five contexts, suggesting an increase in the functional use of broadband calls when combined with rumbles, analogous to the generativity brought about through syntax in human speech. Moreover, combining calls appeared to lead to functional shifts towards high-stake contexts. Call combinations were more likely in competition contexts compared to single rumbles, and more likely in separation contexts compared to single broadband calls. We suggest that call combination in forest elephants may aide to reduce message ambiguity in high-stake situation by simultaneously communicating distress and individual identity, which may be critical to secure access to resources, reduce the risk of injury and to reunite with or recruit the support of the family group.

Using protein language models for protein interaction hot spot prediction with limited data.

Protein language models, inspired by the success of large language models in deciphering human language, have emerged as powerful tools for unraveling the intricate code of life inscribed within protein sequences. They have gained significant attention for their promising applications across various areas, including the sequence-based prediction of secondary and tertiary protein structure, the discovery of new functional protein sequences/folds, and the assessment of mutational impact on protein fitness. However, their utility in learning to predict protein residue properties based on scant datasets, such as protein-protein interaction (PPI)-hotspots whose mutations significantly impair PPIs, remained unclear. Here, we explore the feasibility of using protein language-learned representations as features for machine learning to predict PPI-hotspots using a dataset containing 414 experimentally confirmed PPI-hotspots and 504 PPI-nonhot spots. Our findings showcase the capacity of unsupervised learning with protein language models in capturing critical functional attributes of protein residues derived from the evolutionary information encoded within amino acid sequences. We show that methods relying on protein language models can compete with methods employing sequence and structure-based features to predict PPI-hotspots from the free protein structure. We observed an optimal number of features for model precision, suggesting a balance between information and overfitting. This study underscores the potential of transformer-based protein language models to extract critical knowledge from sparse datasets, exemplified here by the challenging realm of predicting PPI-hotspots. These models offer a cost-effective and time-efficient alternative to traditional experimental methods for predicting certain residue properties. However, the challenge of explaining why specific features are important for determining certain residue properties remains.

Enhancing Natural Language Processing (NLP) through VIKOR Method: A Comprehensive Approach for Improved Computational Linguistics

Natural Language Processing (NLP) is a revolutionary discipline that resides at the crossroads of computer science and linguistics, with its primary emphasis on the interaction between computers and human language. With roots in artificial intelligence, NLP seeks to equip machines with the ability to comprehend, interpret, and respond to natural language, enabling more intuitive and meaningful communication between humans and computers. This multidisciplinary domain leverages advanced algorithms and models to tackle a range of linguistic challenges, from language translation and sentiment analysis to speech recognition. Recent breakthroughs, particularly in deep learning and neural networks, have propelled NLP to new heights, with applications spanning diverse sectors such as healthcare, finance, and education. As NLP continues to advance, its potential impact on enhancing human-machine interaction and information processing is increasingly evident, promising a future where technology seamlessly integrates with our natural modes of communication. The significance of Natural Language Processing (NLP) in research lies in its capacity to revolutionize human-computer interaction. NLP empowers machines to understand and generate human language, enabling advancements in sentiment analysis, language translation, and conversational AI. This transformative capability has profound implications across various industries, including healthcare, finance, and education. NLP research not only enhances the efficiency of information retrieval and processing but also holds the key to developing more intuitive and user-friendly technologies. As the frontier of NLP research expands, its potential to bridge the communication gap between humans and machines continues to shape the future of technology and information access. VIKOR functions as a multi-criteria decision-making technique that determines the best option by comparing it to the ideal option. The ranking procedure comprises determining distances concerning the optimal solution. VIKOR uses linear normalization to get the best possible outcomes. This technique was first presented by Opricovic in 1998 and was intended to optimize multi-attribute complex systems. Its main focus is on ranking lists that allow for flexibility in the strategy weight interval and incorporate compromise alternatives to obtain desired results. Alternative taken as Open AI GPT-4, Google BERT, Microsoft Azure Text Analytics, IBM Watson Natural Language Understanding, spaCy, NLTK (Natural Language Toolkit), Amazon Comprehend. Evaluation Parameter taken as Sentiment Analysis (F1 Score), NER Performance, Language Support, Processing Speed (Response Time) (ms), Translation Accuracy (BLEU Score), Data Privacy. The result it is seen that Amazon Comprehend is got the first rank where as is the IBM Watson Natural Language Understanding is having the lowest rank.

Spatial–temporal hypergraph based on dual-stage attention network for multi-view data lightweight action recognition

For the problems of irrelevant frames and high model complexity in action recognition, we propose a Spatial–Temporal Hypergraph based on Dual-Stage Attention Network (STHG-DAN) for multi-view data lightweight action recognition. It includes two stages: Temporal Attention Mechanism based on Trainable Threshold (TAM-TT) and Hypergraph Convolution based on Dynamic Spatial–Temporal Attention Mechanism (HG-DSTAM). In the first stage, TAM-TT uses a learning threshold to extract keyframes from multi-view videos, with the multi-view data serving as a guarantee for providing more comprehensive information subsequently; In the second stage, HG-DSTAM divides the human joints into three parts: trunk, hand and leg to build spatial–temporal hypergraphs, extracts high-order features from spatial–temporal hypergraphs constructed of multi-view human body joints, inputs them into the dynamic spatial–temporal attention mechanism, and learns the intra frame correlation of multi-view data between the joint features of body parts, which can obtain the significant areas of action; We use multi-scale convolution operation and depth separable network, which can realize efficient action recognition with a few trainable parameters. We experiment on the NTU-RGB+D, NTU-RGB+D 120 and the imitating traffic police gesture dataset. The performance and accuracy of the model are better than the existing algorithms, effectively improving the machine and human body language interaction cognitive ability.

The investigation and development of chatbot related to NBA players

At present, the utilization and development of large language models have been relatively mature, but there are still some areas that have not yet been involved, nba player information retrieval is one of them, this paper will be on through the development and design of large language models, so that it becomes a chatbot to facilitate user retrieval. This project uses mata's llama2-13b model to act as a chatbot, the model can play a great performance in the case of minimizing the memory, and can well meet the user's needs, by imitating the database of the big language model to build a database on nba player information, so that the nba player information is filled into the model for the model to learn, and through the model itself to continuously learn and Through the model's own continuous learning and imitation of human language, the model can be made to output as concise a language as possible to describe the personal information of the nba players, and then meet the user's needs. In this project, the model successfully reads the database and outputs the information that the user wants to get in easy-to-understand language, so that the user can get the personal information of the NBA players more conveniently and intuitively, and the correctness rate reaches the expectation, which makes the study successful.

PEMANFAATAN LARGE LANGUAGE MODELS CHATGPT-4 UNTUK KONTROL DRONE: STUDI KASUS DJI TELLO

This study explores the potential of using ChatGPT-4, a Large Language Model (LLM) from OpenAI, for creating flight trajectories for drones. The primary focus is on the DJI Tello drone, which is controlled through Python code to execute various flying commands. Unlike traditional approaches that involve direct programming in Python, this research utilizes ChatGPT-4 to automatically generate Python computer programs capable of instructing the drone. Results indicate ChatGPT-4's intriguing capability to produce the necessary code for flying the drone according to given commands. This suggests that LLMs like ChatGPT-4 can be employed to determine drone flight trajectories using human language, facilitating ease of use compared to traditional computer programming languages.

Artificial Intelligence Methods in Natural Language Processing: A Comprehensive Review

The rapid evolution of Artificial Intelligence (AI) since its inception in the mid-20th century has significantly influenced the field of Natural Language Processing (NLP), transforming it from a rule-based system to a dynamic and adaptive model capable of understanding the complexities of human language. This paper aims to offer a comprehensive review of the various applications and methodologies of AI in NLP, serving as a detailed guide for future research and practical applications. In the early sections, the paper elucidates the indispensable role of AI in NLP, highlighting its transition from symbolic reasoning to a focus on machine learning and deep learning, and its extensive applications in sectors such as healthcare, transportation, and finance. It emphasizes the symbiotic relationship between AI and NLP, facilitated by platforms like AllenNLP, which aid in the development of advanced language understanding models. Further, the paper explores specific AI techniques employed in NLP, including machine learning, Naive Bayes, and Support Vector Machines, and identifies pressing challenges and avenues for future research. It delves into the applications of AI in NLP, showcasing its transformative potential in tasks such as machine translation, facilitated by deep learning methods, and the development of chatbots and virtual assistants that have revolutionized human-technology interaction. The paper also highlights other fields impacted by AI techniques, including text summarization, sentiment analysis, and named entity recognition, emphasizing the efficiency and accuracy brought about by the integration of AI in these areas. In conclusion, the paper summarizes the remarkable advancements and persistent challenges in NLP, such as language ambiguity and contextual understanding, and underscores the need for diverse and representative labeled data for training. Looking forward, it identifies promising research avenues including Explainable AI, Few-shot and Zero-shot Learning, and the integration of NLP with other data modalities, aiming for a holistic understanding of multimodal data. The paper calls for enhanced robustness and security in NLP systems, especially in sensitive applications like content moderation and fake news detection, to foster trust and reliability in AI technologies. It advocates for continual learning in NLP models to adapt over time without losing previously acquired knowledge, paving the way for a future where AI and NLP work synergistically to understand and generate human language more effectively and efficiently.

Study on the Impact of Utilizing ChatGPT and Other AI Tools for Feedback in EAP Writing Classrooms on the Discursive Writing Performance of English Major Students

This study aims to delve into the impact of utilizing ChatGPT and other AI tools on the discourse writing performance of English major students, as well as to explore their potential value and future prospects in the field of English education. With the continuous development and application of artificial intelligence technology, AI tools have gradually demonstrated their unique advantages and potential in language learning and writing instruction. However, for English major students, how to effectively utilize these technological tools to enhance their discourse writing abilities remains a subject of considerable interest. In the field of language learning, AI technology has made significant strides in vocabulary acquisition, grammar correction, and discourse generation. ChatGPT, as a natural language generation model, possesses strong language comprehension and generation capabilities and is widely used in dialogue generation, text generation, and other areas. In English education, ChatGPT can provide personalized writing guidance and feedback to students, helping them clarify their thoughts and improve their expression skills. Additionally, other AI tools such as speech recognition technology and text-to-speech technology offer more possibilities for students' English learning and writing. However, despite the numerous advantages demonstrated by AI tools in discourse writing instruction for English major students, their application also faces challenges and limitations. For instance, AI tools still have limitations in understanding academic language and specialized terminology, and they cannot fully replace human professional judgment and language application abilities. Furthermore, the widespread adoption and use of AI tools require addressing technical challenges and adaptation issues in teaching practice to ensure their effective application in education. In conclusion, this study will comprehensively explore the actual effects and potential value of AI tools in discourse writing instruction for English major students through a combination of literature review and empirical research methods, providing new insights and guidance for teaching practice and future research endeavors.

Information Extraction and Knowledge Map Construction based on Natural Language Processing

As a key branch of artificial intelligence, Natural Language Processing (NLP) focuses on making machines understand and generate human language. This paper introduces the basic tasks of NLP, such as lexical analysis, syntactic analysis and semantic understanding, and discusses the cutting-edge technologies such as word embedding. In the aspect of information extraction, this paper deeply discusses the methods of named entity recognition, relationship extraction and event extraction, and points out the challenges in dealing with complex texts. Finally, the paper focuses on the construction of knowledge map, expounds the complete process from data collection to entity identification, relationship extraction, graph construction and query, and emphasizes the core position of graph query in the application of knowledge map. On the whole, this paper provides a comprehensive perspective for understanding NLP, information extraction and knowledge map construction, and points out the importance and future development direction of these technologies in intelligent systems.