Natural Language Research Articles

Generation of black-box adversarial attacks using many independent objective-based algorithm for testing the robustness of deep neural networks

Deep neural networks (DNNs) have become increasingly ubiquitous in our daily lives, finding applications in areas such as image recognition, voice recognition, and natural language processing. However, a growing concern revolves around the vulnerability of DNNs to adversarial examples—malicious inputs that can compromise the safety and accuracy of their outcomes. Existing studies predominantly fall into two categories: white-box and black-box techniques. White-box techniques require detailed internal information about the model, often focusing on gradient-based methods. In contrast, black-box techniques, which emulate real-world scenarios more closely, only rely on input–output knowledge. This study focuses specifically on black-box strategies, with key contributions from Single-Objective Variant of Differential Evolution (Pixel-SOO) and Multi-Objective Variant of Differential Evolution (Pixel-MOO) algorithms. While these approaches show promise, they suffer from drawbacks like long execution times and being unable to generate instances for adversarial purposes. To address these challenges, we introduce a novel archive-based Many Independent Objective (MIO) algorithm for using the first time in this context. Our proposed algorithm identifies the most vulnerable image pixels through the MIO algorithm, enabling efficient label flip attacks with minimized attempts. Furthermore, we balance exploration and exploitation by incorporating an adaptive parameter mechanism. The effectiveness of the proposed algorithm is assessed by employing VGG (VGG16 and VGG19) and ResNet (ResNet50, ResNet101, and ResNet152) architectures, both of which are convolutional neural network (CNN) models. The success criterion for the algorithms is to minimize the number of pixel changes while achieving high flip rates with a minimal number of requests to the network. A comprehensive analysis of the experimental results reveals that our algorithm consistently outperforms Pixel-SOO and Pixel-MOO, exhibiting an average speedup of three times compared to Pixel-SOO and seven times compared to Pixel-MOO. In most runs, adversarial attacks are generated with fewer pixel changes than Pixel-MOO and Pixel-SOO. In addition, our findings are openly accessible on GitHub to ensure transparency and reproducibility and encourage future research efforts.

The rise of and demand for identity‐oriented media coverage

AbstractWhile some assert that social identities have become more salient in American media coverage, existing evidence is largely anecdotal. An increased emphasis on social identities has important political implications, including for polarization and representation. We first document the rising salience of different social identities using natural language processing tools to analyze all tweets from 19 media outlets (2008–2021) alongside 553,078 URLs shared on Facebook. We then examine one potential mechanism: Outlets may highlight meaningful social identities—race/ethnicity, gender, religion, or partisanship—to attract readers through various social and psychological pathways. We find that identity cues are associated with increases in some forms of engagement on social media. To probe causality, we analyze 3,828 randomized headline experiments conducted via Upworthy. Headlines mentioning racial/ethnic identities generated more engagement than headlines that did not, with suggestive evidence for other identities. Identity‐oriented media coverage is growing and rooted partly in audience demand.

Word Forms Reflect Trade-Offs Between Speaker Effort and Robust Listener Recognition.

How do cognitive pressures shape the lexicons of natural languages? Here, we reframe George Kingsley Zipf's proposed "law of abbreviation" within a more general framework that relates it to cognitive pressures that affect speakers and listeners. In this new framework, speakers' drive to reduce effort (Zipf's proposal) is counteracted by the need for low-frequency words to have word forms that are sufficiently distinctive to allow for accurate recognition by listeners. To support this framework, we replicate and extend recent work using the prevalence of subword phonemic sequences (phonotactic probability) to measure speakers' production effort in place of Zipf's measure of length. Across languages and corpora, phonotactic probability is more strongly correlated with word frequency than word length. We also show this measure of ease of speech production (phonotactic probability) is strongly correlated with a measure of perceptual difficulty that indexes the degree of competition from alternative interpretations in word recognition. This is consistent with the claim that there must be trade-offs between these two factors, and is inconsistent with a recent proposal that phonotactic probability facilitates both perception and production. To our knowledge, this is the first work to offer an explanation why long, phonotactically improbable word forms remain in the lexicons of naturallanguages.

Blowing against the Wind? a narrative approach to central Bank foreign exchange intervention

Most countries in the world use foreign exchange intervention, but measuring the success of the policy is difficult. By using a narrative approach, I identify interventions when the central bank manages to influence the exchange rate based on pure luck. I separate them from interventions when the central bank actually impacted the exchange rate. Because intervention records are daily aggregates, an intervention might appear to have changed the direction of the exchange rate, when it is more likely to have been caused by market news. This analysis allows to have a better understanding of how successful central bank operations really are. I use new daily data on Bank of England interventions in the 1980 s and 1990 s. Some studies find that interventions work in up to 80 % of cases. Yet, by accounting for intraday market moving news, I find in adverse conditions, the Bank of England managed to influence the exchange rate only in 8 % of cases. I use natural language processing to confirm the validity of the narrative approach. Using lasso and a VAR analysis, I investigate what makes the Bank of England intervene. I find that only movement on the Deutschmark and not US dollar exchange rate made the Bank intervene. Also, I find that interest rate hikes were mostly a tool for currency management and accompanied by large reserve sales.

The status of the AI medical industry in China: A database and statistical analysis

BackgroundArtificial intelligence (AI) technology has developed rapidly in recent years, leading to exponential growth in the AI medical industry. However, a comprehensive investigation of approved medical devices in China is needed. MethodsWe utilized a web crawler to collect data on all medical devices from the China National Medical Products Administration website since 2018. Through natural language processing techniques and manual analysis, we identified all medical devices developed by artificial intelligence medical devices (AIMD) companies and conducted a statistical analysis. ResultsSince 2018, the number of AI-related medical devices approved in China has significantly increased. Most devices (79%) were classified as Class II with moderate risk, whereas 21% were classified as Class III with high risk. Most devices (74.2%) were categorized as medical device software, and the most common application was medical image processing (63.2%). In terms of target body areas, devices related to the heart accounted for the highest proportion (12.8%), followed by those related to the lungs (11.3%) and brain (6.7%). ConclusionThis study establishes a comprehensive database of medical devices developed by AIMD companies in China, enabling the public to gain a coherent understanding of their current development status.

Curriculum meta-learning for zero-shot cross-lingual transfer

Learning with small data is the challenge facing low-resource domains with insufficient data to perform the target tasks. Among the state-of-the-art approachs in deep learning and transfer learning, the approach based on meta-learning stands out for its ability to balance the bias from high-resource to the low-resource domains. To crack the aforementioned problem, we proposed a framework utilizing meta-learning with zero-shot cross-lingual natural language inferences as a use case. Additionally, to explore how the sensitivity of the meta-gradient impacts learning performance, we applied two curriculum strategies: cross-review evaluation and annealing arrangement. The experiments confirm the outperforming of our framework on a benchmark dataset.

Immigration Narrative and Home Prices

This study characterizes the relationship between U.S. National Home Prices and Immigration Narrative as portrayed on TV news. Integrating Narrative Economics and Natural Language Processing (NLP) sentiment analysis, I analyze a large dataset of 1.96 million TV news transcripts spanning July 2009 to December 2023 to capture the sentiment of the Immigration Narrative. Immigration Narrative Sentiment and U.S. Home Prices are associated. One standard deviation orthogonalized shock to the sentiment Granger-causes a statistically significant and economically meaningful increase in the Case–Shiller U.S. National Home Price. The cumulative increase is equivalent to about 26 percent of the average monthly change during the sample period. Moreover, the effect of a shock to the Immigration Narrative Sentiment on Home Prices is permanent, suggesting that the Immigration Narrative contains fundamental information about Home Prices not captured by standard economic variables. Conversely, there is no evidence that Home Price variation affects Immigration Narrative.

Parts-of-speech tagger for Sindhi language using deep neural network architecture

Language is a fundamental medium for human communication, encompassing spoken and written forms, each governed by grammatical rules. Sindhi, one of the oldest languages, is characterized by its rich morphology and grammatical structure. Part-of-speech (POS) tagging, a crucial process in natural language processing, involves assigning grammatical tags to words. This research presents a novel approach to POS tagging for Sindhi text using deep learning techniques. We developed a POS tagger employing Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models, with LSTM demonstrating superior effectiveness. This study represents the first application of these deep learning methods for POS tagging in Sindhi. Utilizing fastText, we trained 79,959 Sindhi word vectors, derived from a corpus compiled from diverse sources including Sindhi books, stories, and poetry. The corpus comprises 1,459 sentences and 10,584 unique words, split into 80% for training and 20% for validation. Our results indicate that the LSTM model achieved an accuracy of 85.80%, outperforming the GRU model, which achieved 80.77%, by a margin of 5%. This work's novelty lies in the application of deep learning techniques to enhance POS tagging accuracy in the Sindhi language corpus.

Relevance of RPA in Banking & CRM

This journal explores the relevance of Artificial Intelligence (AI) in the field of banking and Customer Relationship Management (CRM). It delves into how AI technologies are reshaping traditional banking operations and enhancing customer experiences and optimizing CRM strategies. In banking, AI is being used to improve customer services, strengthen security, and provide personalized financial services. AI technologies, such as machine learning, natural language processing, and predictive analytics, have revolutionized various aspects of banking operations. These technologies delegates’ banks to automate routine tasks enhance fraud detection capabilities, personalized customer experiences. The study also highlights the challenges and opportunities associated with AI adoption in CRM strategies, so here we are using Robotic Process Automation (RPA) concepts to automate repetitive and rule-based tasks in banking and CRM processes. RPA is reshaping business processes, workforce dynamics, and strategic decision making. Additionally, it discusses key considerations for successful RPA implementations. Ultimately, this paper highlights the importance of understanding and harnessing the power of RPA to drive innovation and competitiveness in the digital age.

A spatial scene reconstruction framework in emergency response scenario

Rapid and accurate acquisition and analysis of information is crucial for emergency management, but traditional methods have limitations such as incomplete information acquisition and slow processing speed. The natural language oriented spatial scene reconstruction method provides a new solution for emergency management, but existing generative models have limited understanding of spatial relationships and lack high-quality training samples. To address these issues, this paper proposes a novel spatial scene reconstruction framework. Specifically, the BERT based spatial information knowledge graph extraction method is used to encode the input text, label and classify the encoded text, identify spatial objects and relationships in the text, and accurately extract spatial information. Additionally, a large number of manual experiments were conducted to explore quantitative biases in human spatial cognition, and based on the obtained biases, a greedy resolution method based on cost functions was used to fine tune the layout of conflicting spatial objects and solve the conflicting spatial information in the spatial information knowledge graph. Finally, use graph convolutional neural networks to obtain scene knowledge graph embeddings that consider spatial constraints. In addition, a high-quality training sample set of “text-scene-knowledge graph” was constructed.

Development of a Natural Language Processing Algorithm for the Detection of Spinal Metastasis Based on MRI Reports

BackgroundMetastasis to the spinal column is a common complication of malignancy, potentially causing pain and neurologic injury. An automated system to identify and refer patients with spinal metastases can help overcome barriers to timely treatment. We describe the training, optimization and validation of a natural language processing algorithm to identify the presence of vertebral metastasis and metastatic epidural cord compression (MECC) from radiology reports of spinal MRIs. MethodsReports from patients with spine MRI studies performed between January 1, 2008 and April 14, 2019 were reviewed by a team of radiologists to assess for the presence of cancer and generate a labeled dataset for model training. Using regular expression, impression sections were extracted from the reports and converted to all lower-case letters with all non-alphabetic characters removed. The reports were then tokenized and vectorized using the doc2vec algorithm. These were then used to train a neural network to predict the likelihood of spinal tumor or MECC. For each report, the model provided a number from 0 to 1 corresponding to its impression. We then obtained 111 MRI reports from outside the test set, 92 manually labeled negative and 19 with MECC to test the model's performance. Results37,579 radiology reports were reviewed. 36,676 were labeled negative, and 903 with MECC. We chose a cutoff of 0.02 as a positive result to optimize for a low false negative rate. At this threshold we found a 100% sensitivity rate with a low false positive rate of 2.2%. ConclusionThe NLP model described predicts the presence of spinal tumor and MECC in spine MRI reports with high accuracy. We plan to implement the algorithm into our EMR to allow for faster referral of these patients to appropriate specialists, allowing for reduced morbidity and increased survival.

Analysis of High-Performance Parallel Computing using TOPSIS Method

High-performance parallel computing involves the simultaneous execution of multiple tasks or processes, orchestrated to achieve improved computational speed and efficiency. This approach leverages the power of parallelism, exploiting both multi-core CPUs and GPUs, distributed computing clusters, and specialized hardware accelerators. The fundamental idea is to divide a task into smaller sub-tasks that can be executed concurrently, thereby reducing processing time and enhancing overall performance. High-performance parallel computing is a transformative approach that enables us to tackle computationally intensive tasks efficiently. This abstract highlight its significance in contemporary computing and sets the stage for further exploration of the intricacies and innovations within this dynamic field. Researchers and practitioners continue to push the boundaries of what is achievable, making high-performance parallel computing a cornerstone of modern computational science and technology. High-performance parallel computing research is of paramount significance due to its transformative impact across diverse fields. It empowers scientists to tackle complex problems that were once computationally intractable, unlocking new frontiers in scientific discovery. It drives innovation in engineering and design, optimizing product development and manufacturing processes across industries. In healthcare, it accelerates genomics research and drug discovery, offering hope for improved medical treatments. Financial institutions rely on it for data analysis and risk assessment, shaping the global economy. Weather forecasting and environmental modelling are enhanced, aiding disaster preparedness and conservation efforts. In the digital age, parallel computing underpins artificial intelligence, enabling advancements in natural language processing and machine learning. Furthermore, it has vital applications in national security, space exploration, and materials science. In essence, high-performance parallel computing research serves as the backbone of technological progress, fostering innovation, efficiency, and problem-solving across a wide spectrum of disciplines, ultimately shaping the future of our world. TOPSIS, this method involves evaluating the geometric distance between each alternative solution and two reference solutions: the positive ideal solution and the negative ideal solution. The underlying principle of TOPSIS assumes that the criteria being assessed are of an ascending nature, where larger values represent better performance. To account for disparate dimensions or scales among the criteria, normalization is often employed within the TOPSIS framework. From the result Scatter- free imaging is got the first rank and object-scatter imaging is having the lowest rank.

Research on the Training and Application Methods of a Lightweight Agricultural Domain-Specific Large Language Model Supporting Mandarin Chinese and Uyghur

In the field of Natural Language Processing (NLP), the lack of support for minority languages, especially Uyghur, the scarcity of Uyghur language corpora in the agricultural domain, and the lightweight nature of large language models remain prominent issues. This study proposes a method for constructing a bilingual (Uyghur and Chinese) lightweight specialized large language model for the agricultural domain. By utilizing a mixed training approach of Uyghur and Chinese, we extracted Chinese corpus text from agricultural-themed books in PDF format using OCR (Optical Character Recognition) technology, converted the Chinese text corpus into a Uyghur corpus using a rapid translation API, and constructed a bilingual mixed vocabulary. We applied the parameterized Transformer model algorithm to train the model for the agricultural domain in both Chinese and Uyghur. Furthermore, we introduced a context detection and fail-safe mechanism for the generated text. The constructed model possesses the ability to support bilingual reasoning in Uyghur and Chinese in the agricultural domain, with higher accuracy and a smaller size that requires less hardware. It (our work) addresses issues such as the scarcity of Uyghur corpora in the agricultural domain, mixed word segmentation and word vector modeling in Uyghur for widespread agricultural languages, model lightweighting and deployment, and the fragmentation of non-relevant texts during knowledge extraction from small-scale corpora. The lightweight design of the model reduces hardware requirements, facilitating deployment in resource-constrained environments. This advancement promotes agricultural intelligence, aids in the development of specific applications and minority languages (such as agriculture and Uyghur), and contributes to rural revitalization.

The speech neuroprosthesis.

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore naturalcommunication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinicalepilepsy monitoringand subsequently in individuals with paralysis as part ofearlyfeasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapidandsubstantial progress indecoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

Attenuation and distortion components of age-related hearing loss: Contributions to recognizing temporal-envelope filtered speech in modulated noise.

Older adults with hearing loss may experience difficulty recognizing speech in noise due to factors related to attenuation (e.g., reduced audibility and sensation levels, SLs) and distortion (e.g., reduced temporal fine structure, TFS, processing). Furthermore, speech recognition may improve when the amplitude modulation spectrum of the speech and masker are non-overlapping. The current study investigated this by filtering the amplitude modulation spectrum into different modulation rates for speech and speech-modulated noise. The modulation depth of the noise was manipulated to vary the SL of speech glimpses. Younger adults with normal hearing and older adults with normal or impaired hearing listened to natural speech or speech vocoded to degrade TFS cues. Control groups of younger adults were tested on all conditions with spectrally shaped speech and threshold matching noise, which reduced audibility to match that of the older hearing-impaired group. All groups benefitted from increased masker modulation depth and preservation of syllabic-rate speech modulations. Older adults with hearing loss had reduced speech recognition across all conditions. This was explained by factors related to attenuation, due to reduced SLs, and distortion, due to reduced TFS processing, which resulted in poorer auditory processing of speech cues during the dips of the masker.

서비스형 버추얼 휴먼의 감성적 디자인 분석 및 UX평가

As the digitization of museums accelerates, service-oriented virtual humans are leading the museum industry's entry into the metaverse. However, compared to other countries, the application of service-oriented virtual humans in Chinese museums is still in its early stages. This study explores ways to enhance the user experience of service-oriented virtual humans in museums through emotional design and UX evaluation. Firstly, the study identifies emotional design elements and UX evaluation factors through prior research. Secondly, it discusses seven elements—visual, auditory, storytelling, interaction, pleasure, satisfaction, and convenience—across the three levels of emotional design(instinctive, behavioral, and reflective) using representative cases from Chinese museums. Thirdly, based on prior research, the study establishes UX evaluation criteria and selects expert interview subjects, identifying key factors affecting user experience: meaningful, pleasurable, convenient, usable, reliable, and interaction. Finally, the study synthesizes insights from case analysis and interview results. From an emotional design perspective, virtual humans are modeled after real people and incorporate traditional Chinese cultural elements in their appearance, using natural dialogue and emotional expression. They explain the background and history of artifacts and enhance participation through diverse interaction methods. Rich knowledge and engaging interactions attract visitor attention and enhance enjoyment. In terms of UX evaluation, diverse interactions increase immersion, while natural language and visual/auditory elements stimulate curiosity and engagement. Concise information delivery and voice interaction improve convenience. Functions such as inquiry response and route planning meet individual needs. Natural voice interaction and data security enhance trust, while improved voice recognition accuracy and facial expression consistency enhance user immersion and interaction experience.

Dyadic negation in natural language

AbstractIn order to provide for a linguistically and cognitively sound theory of negation, we argue for the introduction of a dyadic negation predicate lack and a force dynamic account of affirmation and negation in general.

Elevating Performance Through AI-Driven Mock Interviews

Abstract: This paper proposes an innovative AI-based mock interview platform designed to enhance interview preparedness by assessing candidates across three key dimensions: emotions, confidence, and knowledge. Utilizing deep learning convolutional neural networks, the system analyzes facial expressions to gauge emotional responses, while speech recognition and natural language processing evaluate the candidate's confidence levels. Additionally, semantic analysis and keyword mapping assess the candidate's knowledge by comparing responses with relevant online resources. This comprehensive approach aims to reduce pre-interview anxiety, boost confidence, and refine interview skills, providing a more effective preparation tool compared to traditional methods.

Arabic Chatbots Challenges and Solutions: A Systematic Literature Review

Since the beginning of Natural Language Processing, researchers have been highly interested in the idea of equipping machines with the ability to think, understand, and communicate in a human-like manner. However, despite the significant progress in this field, particularly in English, Arabic research remains in its early stages of development. This study presents a Systematic Literature Review on challenges faced in Arabic chatbot development and the proposed solutions. Utilizing the search terms "ARABIC," "CHATBOT," "CHALLENGES," and "SOLUTION," (including synonyms) we systematically surveyed studies published between 2000 and 2023 from Scopus, Science Direct, Web of Science, PubMed, SpringerLink, IEEE Xplore, ACM, Ebesco, and ICI. Moreover, besides Google Scholar and ResearchGate, we employed a manual snowballing technique to discover supplementary relevant research by examining the references of all chosen primary studies. The included studies were assessed for eligibility based on the quality assessment checklist we developed. Out of 3,891 studies, only 64 were deemed eligible. Many challenges were identified, including the scarcity of well-structured datasets. To overcome this (n=35) studies manually collected and preprocessed data. Additionally, Arabic language complexity led (n=53) researchers to adopt pre-scripted rules approaches, followed by generative approaches (n=10), and hybrid approach (n=1). Furthermore, (n=27) studies employed human-based evaluation metrics to assess the chatbot performance. while, (n=11) studies haven’t used any metrics. Based on conducted research, a critical research priority is providing Arabic with high-quality resources, such as an Arabic dataset that includes dialectal variations and incorporates empathy, lexicon corpora, and also a word normalization library. These resources will enable the chatbot to interact more naturally and humanely. Additionally, hybrid approaches have shown promising results, particularly in low-resource languages, such as Arabic. Therefore, more focus should be dedicated into implementing hybrid approaches in chatbot development. Furthermore, evaluating the chatbot performance is still an open domain for further research and contribution, highlighting the need for innovative standardized evaluation methods.

Innovative Machine Learning Solutions: Investigating Algorithms and Applications

This comprehensive review paper provides a thorough examination of the ever-evolving landscape of machine learning (ML), spanning from its historical origins to contemporary applications, challenges, and future prospects. It begins by elucidating the foundational concepts of machine learning, encompassing the diverse types, key terminology, and the intricate pipeline involved in the machine learning process. Delving into its historical perspective, the paper chronicles the development of machine learning, tracing its roots in artificial intelligence and highlighting key milestones and influential researchers who have shaped its trajectory. The core of this review explores an array of machine learning algorithms and techniques, spanning regression, classification, clustering, dimensionality reduction, deep learning, and ensemble methods. These algorithms are contextualized with real-world applications, ranging from healthcare to finance, natural language processing (NLP), computer vision, recommender systems, and robotics. Each application domain is buttressed with illustrative examples and case studies. In recognizing the challenges and open problems that confront machine learning, the review delves into issues pertaining to data quality, model interpretability, bias, ethics, generalization, and overfitting. Moreover, it identifies pressing research questions and areas where advancements are needed. Recent trends and developments in machine learning, including transfer learning, explainable AI, federated learning, reinforcement learning, and ethical considerations, are also highlighted to provide a glimpse into the evolving landscape. The paper culminates in a thought-provoking discussion on the future of machine learning, its potential societal impact, and its transformative role across industries. In summary, this review amalgamates key findings and insights, offering a comprehensive view of machine learning's multifaceted journey, from its inception to its promising future.