Natural Language Processing Models Research Articles

Evolution and advancements in deep learning models for Natural Language Processing

This paper provides a comprehensive review of the evolution and advancements in deep learning models for Natural Language Processing (NLP). It explores the transition from statistical models to neural networks, highlighting the paradigm shift towards data-driven methodologies and the implications for NLP tasks. The emergence of neural network architectures, such as Recurrent Neural Networks (RNNs) and transformer-based models like BERT and GPT, has revolutionized language understanding and generation. Furthermore, the integration of deep learning in traditional NLP tasks, such as part-of-speech tagging and named entity recognition, has led to significant improvements in accuracy and efficiency. The paper also discusses the quantitative analysis of deep learning models, including performance metrics, computational efficiency, and mathematical modeling of language tasks. Case studies and applications, including sentiment analysis, machine translation, and automated content generation, exemplify the transformative impact of deep learning in NLP.

Developing and testing a framework for coding general practitioners’ free-text diagnoses in electronic medical records - a reliability study for generating training data in natural language processing

BackgroundDiagnoses entered by general practitioners into electronic medical records have great potential for research and practice, but unfortunately, diagnoses are often in uncoded format, making them of little use. Natural language processing (NLP) could assist in coding free-text diagnoses, but NLP models require local training data to unlock their potential. The aim of this study was to develop a framework of research-relevant diagnostic codes, to test the framework using free-text diagnoses from a Swiss primary care database and to generate training data for NLP modelling.MethodsThe framework of diagnostic codes was developed based on input from local stakeholders and consideration of epidemiological data. After pre-testing, the framework contained 105 diagnostic codes, which were then applied by two raters who independently coded randomly drawn lines of free text (LoFT) from diagnosis lists extracted from the electronic medical records of 3000 patients of 27 general practitioners. Coding frequency and mean occurrence rates (n and %) and inter-rater reliability (IRR) of coding were calculated using Cohen’s kappa (Κ).ResultsThe sample consisted of 26,980 LoFT and in 56.3% no code could be assigned because it was not a specific diagnosis. The most common diagnostic codes were, ‘dorsopathies’ (3.9%, a code covering all types of back problems, including non-specific lower back pain, scoliosis, and others) and ‘other diseases of the circulatory system’ (3.1%). Raters were in almost perfect agreement (Κ ≥ 0.81) for 69 of the 105 diagnostic codes, and 28 codes showed a substantial agreement (K between 0.61 and 0.80). Both high coding frequency and almost perfect agreement were found in 37 codes, including codes that are particularly difficult to identify from components of the electronic medical record, such as musculoskeletal conditions, cancer or tobacco use.ConclusionThe coding framework was characterised by a subset of very frequent and highly reliable diagnostic codes, which will be the most valuable targets for training NLP models for automated disease classification based on free-text diagnoses from Swiss general practice.

A Reference Paper Collection System Using Web Scraping

Collecting reference papers from the Internet is one of the most important activities for progressing research and writing papers about their results. Unfortunately, the current process using Google Scholar may not be efficient, since a lot of paper files cannot be accessed directly by the user. Even if they are accessible, their effectiveness needs to be checked manually. In this paper, we propose a reference paper collection system using web scraping to automate paper collections from websites. This system can collect or monitor data from the Internet, which is considered as the environment, using Selenium, a popular web scraping software, as the sensor; this examines the similarity against the search target by comparing the keywords using the Bert model. The Bert model is a deep learning model for natural language processing (NLP) that can understand context by analyzing the relationships between words in a sentence bidirectionally. The Python Flask is adopted at the web application server, where Angular is used for data presentations. For the evaluation, we measured the performance, investigated the accuracy, and asked members of our laboratory to use the proposed method and provide their feedback. Their results confirm the method’s effectiveness.

Leveraging natural language processing models to automate speech-intelligibility scoring

ABSTRACT Assessment of speech intelligibility in noise is critical for measuring the impact of age-related hearing loss. However, quantifying intelligibility often requires a human to manually process responses provided by a participant or patient to obtain a speech-intelligibility score – typically the proportion of correctly heard words. This manual process can be time-consuming and thus costly. The current study investigates whether state-of-the-art Natural Language Processing (NLP) models from Google and OpenAI could be used to calculate speech-intelligibility scores as an alternative to human scoring. It was specifically tested whether NLP models capture common speech-in-noise perception phenomena in younger and older adults (N = 144) listening to speech masked by modulated or unmodulated babble noise. The results show that NLP speech-intelligibility scores closely matched intelligibility scores from a human scorer (r ∼0.95). The main difference is, on average, ∼2% underestimation of NLP intelligibility scores relative to human intelligibility scores for moderate to high signal-to-noise ratios. This underestimation results from participants making minor errors related to misspellings, gender, or tense, to which NLP models are sensitive, but human scorers typically correct prior to scoring. Critically, NLP models capture the known age-related reduction in intelligibility and the age-related reduction in the benefit from a modulated relative to an unmodulated masker. OpenAI’s ADA2 appears to perform the best out of the tested NLP models, showing no difference in the speech-in-noise phenomena compared to human scoring. The current study suggests that modern NLP models can be used to score speech-intelligibility data.

Racial Differences in Stigmatizing and Positive Language in Emergency Medicine Notes.

Language used by providers in medical documentation may reveal evidence of race-related implicit bias. We aimed to use natural language processing (NLP) to examine if prevalence of stigmatizing language in emergency medicine (EM) encounter notes differs across patient race/ethnicity. In a retrospective cohort of EM encounters, NLP techniques identified stigmatizing and positive themes. Logistic regression models analyzed the association of race/ethnicity and themes within notes. Outcomes were the presence (or absence) of 7 different themes: 5 stigmatizing (difficult, non-compliant, skepticism, substance abuse/seeking, and financial difficulty) and 2 positive (compliment and compliant). The sample included notes from 26,363 unique patients. NH Black patient notes were less likely to contain difficult (odds ratio (OR) 0.80, 95% confidence interval (CI), 0.73-0.88), skepticism (OR 0.87, 95% CI, 0.79-0.96), and substance abuse/seeking (OR 0.62, 95% CI, 0.56-0.70) compared to NH White patient notes but more likely to contain non-compliant (OR 1.26, 95% CI, 1.17-1.36) and financial difficulty (OR 1.14, 95% CI, 1.04-1.25). Hispanic patient notes were less likely to contain difficult (OR 0.68, 95% CI, 0.58-0.80) and substance abuse/seeking (OR 0.78, 95% CI, 0.66-0.93). NH NA/AI patient notes had twice the odds as NH White patient notes to contain a stigmatizing theme (OR 2.02, 95% CI, 1.64-2.49). Using an NLP model to analyze themes in EM notes across racial groups, we identified several inequities in the usage of positive and stigmatizing language. Interventions to minimize race-related implicit bias should be undertaken.

From large language models to small logic programs: building global explanations from disagreeing local post-hoc explainers

The expressive power and effectiveness of large language models (LLMs) is going to increasingly push intelligent agents towards sub-symbolic models for natural language processing (NLP) tasks in human–agent interaction. However, LLMs are characterised by a performance vs. transparency trade-off that hinders their applicability to such sensitive scenarios. This is the main reason behind many approaches focusing on local post-hoc explanations, recently proposed by the XAI community in the NLP realm. However, to the best of our knowledge, a thorough comparison among available explainability techniques is currently missing, as well as approaches for constructing global post-hoc explanations leveraging the local information. This is why we propose a novel framework for comparing state-of-the-art local post-hoc explanation mechanisms and for extracting logic programs surrogating LLMs. Our experiments—over a wide variety of text classification tasks—show how most local post-hoc explainers are loosely correlated, highlighting substantial discrepancies in their results. By relying on the proposed novel framework, we also show how it is possible to extract faithful and efficient global explanations for the original LLM over multiple tasks, enabling explainable and resource-friendly AI techniques.

Surveillance of Health Care-Associated Violence Using Natural Language Processing.

Patient and family violent outbursts toward staff, caregivers, or through self-harm, have increased during the ongoing behavioral health crisis. These health care-associated violence (HAV) episodes are likely under-reported. We sought to assess the feasibility of using nursing notes to identify under-reported HAV episodes. We extracted nursing notes across inpatient units at 2 hospitals for 2019: a pediatric tertiary care center and a community-based hospital. We used a workflow for narrative data processing using a natural language processing (NLP) assisted manual review process performed by domain experts (a nurse and a physician). We trained the NLP models on the tertiary care center data and validated it on the community hospital data. Finally, we applied these surveillance methods to real-time data for 2022 to assess reporting completeness of new cases. We used 70 981 notes from the tertiary care center for model building and internal validation and 19 332 notes from the community hospital for external validation. The final community hospital model sensitivity was 96.8% (95% CI 90.6% to 100%) and a specificity of 47.1% (39.6% to 54.6%) compared with manual review. We identified 31 HAV episodes in July to December 2022, of which 26 were reportable in accordance with the hospital internal criteria. Only 7 of 26 cases were reported by employees using the self-reporting system, all of which were identified by our surveillance process. NLP-assisted review is a feasible method for surveillance of under-reported HAV episodes, with implementation and usability that can be achieved even at a low information technology-resourced hospital setting.

Towards digital-twin-enabled facility management: the natural language processing model for managing facilities in buildings

ABSTRACT Facility management is an essential part of the building which encompasses multiple service areas and processes to ensuring optimum functionality of the facility. Digital twin (DT) has been increasingly introduced to the FM domain to facilitate FM tasks such as decision-making. Despite the advancements in DT-enabled FM, there still exists a problem in the disconnections between DT academics and FM practitioners. The emergence of knowledge management can act as a bridge between academics and FM practitioners. To mine knowledge in the DT-enabled FM industry for a better communication between different stakeholders and timely decisions in the case of emergency, this study proposes a natural language processing (NLP)-based approach. A detailed classification of FM-related entities is first presented, which divides the FM into five categories. The NLP model is used to extract entities and knowledge from a large volume of text data. The results indicate that an NLP-based approach can accurately extract FM-related knowledge. The findings of this study not only provide a basis for automatic DT-enabled FM-related knowledge extraction but also support the downstream tasks such as knowledge graph construction and FM-related emergency decision-making.

Reduced speech coherence in psychosis-related social media forum posts

The extraction of linguistic markers from social media posts, which are indicative of the onset and course of mental disorders, offers great potential for mental healthcare. In the present study, we extracted over one million posts from the popular social media platform Reddit to analyze speech coherence, which reflects formal thought disorder and is a characteristic feature of schizophrenia and associated psychotic disorders. Natural language processing (NLP) models were used to perform an automated quantification of speech coherence. We could demonstrate that users who are active on forums geared towards disorders with a higher degree of psychotic symptoms tend to show a lower level of coherence. The lowest coherence scores were found in users of forums on dissociative identity disorder, schizophrenia, and bipolar disorder. In contrast, a relatively high level of coherence was detected in users of forums related to obsessive–compulsive disorder, anxiety, and depression. Users of forums on posttraumatic stress disorder, autism, and attention-deficit hyperactivity disorder exhibited medium-level coherence. Our findings provide promising first evidence for the possible utility of NLP-based coherence analyses for the early detection and prevention of psychosis on the basis of posts gathered from publicly available social media data. This opens new avenues for large-scale prevention programs aimed at high-risk populations.

Obtaining Insights from a Natural Language Processing model for naming academic programs in higher education

El Procesamiento del Lenguaje Natural (NLP) es esencial en la Inteligencia Artificial para la interacción entre computadoras y humanos. En este artículo, se explora el uso del NLP y técnicas de visualización en la creación de un modelo para la obtención de hallazgos (insights) en el diseño de nuevos programas académicos. La metodología cuantitativa utiliza técnicas como tokenización y TextRank, y con el apoyo del Scattertext Plot, discrimina categorías sobre programas de doctorado en tecnologías de información. Esta metodología, ampliamente usada y validada por expertos, destaca particularidades, conocimientos generales, y tendencias emergentes en los programas comparados. Los resultados identifican cuatro cuadrantes esenciales para la toma de decisiones en el diseño de programas, y presentan las necesidades actuales y futuras en la denominación de programas académicos. En conclusión, el estudio resalta la importancia del NLP en el diseño académico adaptado a tendencias contemporáneas y proporciona una herramienta robusta para diseñadores de programas.

PowerNovo: de novo peptide sequencing via tandem mass spectrometry using an ensemble of transformer and BERT models

The primary objective of analyzing the data obtained in a mass spectrometry-based proteomic experiment is peptide and protein identification, or correct assignment of the tandem mass spectrum to one amino acid sequence. Comparison of empirical fragment spectra with the theoretical predicted one or matching with the collected spectra library are commonly accepted strategies of proteins identification and defining of their amino acid sequences. Although these approaches are widely used and are appreciably efficient for the well-characterized model organisms or measured proteins, they cannot detect novel peptide sequences that have not been previously annotated or are rare. This study presents PowerNovo tool for de novo sequencing of proteins using tandem mass spectra acquired in a variety of types of mass analyzers and different fragmentation techniques. PowerNovo involves an ensemble of models for peptide sequencing: model for detecting regularities in tandem mass spectra, precursors, and fragment ions and a natural language processing model, which has a function of peptide sequence quality assessment and helps with reconstruction of noisy sequences. The results of testing showed that the performance of PowerNovo is comparable and even better than widely utilized PointNovo, DeepNovo, Casanovo, and Novor packages. Also, PowerNovo provides complete cycle of processing (pipeline) of mass spectrometry data and, along with predicting the peptide sequence, involves the peptide assembly and protein inference blocks.

The changing landscape of text mining: a review of approaches for ecology and evolution.

In ecology and evolutionary biology, the synthesis and modelling of data from published literature are commonly used to generate insights and test theories across systems. However, the tasks of searching, screening, and extracting data from literature are often arduous. Researchers may manually process hundreds to thousands of articles for systematic reviews, meta-analyses, and compiling synthetic datasets. As relevant articles expand to tens or hundreds of thousands, computer-based approaches can increase the efficiency, transparency and reproducibility of literature-based research. Methods available for text mining are rapidly changing owing to developments in machine learning-based language models. We review the growing landscape of approaches, mapping them onto three broad paradigms (frequency-based approaches, traditional Natural Language Processing and deep learning-based language models). This serves as an entry point to learn foundational and cutting-edge concepts, vocabularies, and methods to foster integration of these tools into ecological and evolutionary research. We cover approaches for modelling ecological texts, generating training data, developing custom models and interacting with large language models and discuss challenges and possible solutions to implementing these methods in ecology and evolution.

Impact Analysis of using Natural Language Processing and Large Language Model on Automated Correction of Systems Engineering Requirements

AbstractThe increasing complexity of Electronic Control Units (ECUs) in the Automotive Industry due to the integration of more sophisticated vehicle features has made the Systems Engineering (SE) application a necessity to define and implement efficient solutions. In this context, requirements emerge as a critical part of the communication between cross‐functional teams. Thus, the more complex systems become, the more requirements are needed to define them. However, lack of information, misalignment and ambiguity on requirements impact the entire development process, resulting in issues later, harder to be fixed. Some studies are being applied to evaluate techniques using Natural Language Processing (NLP) and how it can replace extensive peer reviews, identifying weaknesses in requirements earlier in the process, avoiding wasted time and large financial losses. Normally, NLP is combined with templates such as Easy Approach Requirements to Syntax (EARS), or other rule‐based techniques such as INCOSE's requirements writing best practice rules to define metrics and assess the compliance of the requirements syntax automatically. The focus of this work is to enhance the use of requirements syntax assessment algorithm by combining NLP techniques with Large Language Models (LLMs) to provide automatically corrected requirements.

Assessing Laterality Errors in Radiology: Comparing Generative Artificial Intelligence and Natural Language Processing

PurposeWe compared the performance of generative artificial intelligence (AI) (Augmented Transformer Assisted Radiology Intelligence [ATARI, Microsoft Nuance, Microsoft Corporation, Redmond, Washington]) and natural language processing (NLP) tools for identifying laterality errors in radiology reports and images. MethodsWe used an NLP-based (mPower, Microsoft Nuance) tool to identify radiology reports flagged for laterality errors in its Quality Assurance Dashboard. The NLP model detects and highlights laterality mismatches in radiology reports. From an initial pool of 1,124 radiology reports flagged by the NLP for laterality errors, we selected and evaluated 898 reports that encompassed radiography, CT, MRI, and ultrasound modalities to ensure comprehensive coverage. A radiologist reviewed each radiology report to assess if the flagged laterality errors were present (reporting error—true-positive) or absent (NLP error—false-positive). Next, we applied ATARI to 237 radiology reports and images with consecutive NLP true-positive (118 reports) and false-positive (119 reports) laterality errors. We estimated accuracy of NLP and generative AI tools to identify overall and modality-wise laterality errors. ResultsAmong the 898 NLP-flagged laterality errors, 64% (574 of 898) had NLP errors and 36% (324 of 898) were reporting errors. The text query ATARI feature correctly identified the absence of laterality mismatch (NLP false-positives) with a 97.4% accuracy (115 of 118 reports; 95% confidence interval [CI] = 96.5%-98.3%). Combined vision and text query resulted in 98.3% accuracy (116 of 118 reports or images; 95% CI = 97.6%-99.0%), and query alone had a 98.3% accuracy (116 of 118 images; 95% CI = 97.6%-99.0%). ConclusionThe generative AI-empowered ATARI prototype outperformed the assessed NLP tool for determining true and false laterality errors in radiology reports while enabling an image-based laterality determination. Underlying errors in ATARI text query in complex radiology reports emphasize the need for further improvement in the technology.

A Systematic Review of Attention Models in Natural Language Processing

The role of attention models in neural networks has drawn much focus among scholars in recent years mainly because of its efficiency and versatility. Attention models are important for many Natural Language Processing applications like question answering, semantic analysis, sentiment analysis and machine translation performing significantly better than other usual approaches. These mechanisms improve neural networks capability in interpretation and tackle issues like performance decay, which affects the Recurrent Neural Networks by concentrating on significant data inputs and context. A stream of research has explored the integration of attention mechanisms to other fields, namely computer vision and graph analysis improving tasks like object detection, image captioning or node representation learning. However, for more detailed research of modern achievements and various domains and model architecture structures there is still the necessity to use the more profound survey-based investigation. This research article provides a review of previous work done on attention-based model in NLP. It also outlines the problems and provides relevant solutions using different methodologies. Future directions of the research are provided for the improvement of the attention models for NLP, discussing potential issues in terms of model performance and its capacity, as well as its interpretability. This review is intended to help further research to more universal attention-based solutions in the domain of NLP.

Reshaping medical education: Performance of ChatGPT on a PES medical examination.

We are currently experiencing a third digital revolution driven by artificial intelligence (AI), and the emergence of new chat generative pre-trained transformer (ChatGPT) represents a significant technological advancement with profound implications for global society, especially in the field of education. The aim of this study was to see how well ChatGPT performed on medical school exams and to highlight how it might change medical education and practice. Recently, OpenAI's ChatGPT (OpenAI, San Francisco; GPT-4 May 24 Version) was put to the test against a significant Polish medical specialization licensing exam (PES), and the results are in. The version of ChatGPT-4 used in this study was the most up-to-date model at the time of publication (GPT-4). ChatGPT answered questions from June 28, 2023, to June 30, 2023. ChatGPT demonstrates notable advancements in natural language processing models on the tasks of medical question answering. In June 2023, the performance of ChatGPT was assessed based on its ability to answer a set of 120 questions, where it achieved a correct response rate of 67.1%, accurately responding to 80 questions. ChatGPT may be used as an assistance tool in medical education. While ChatGPT can serve as a valuable tool in medical education, it cannot fully replace human expertise and knowledge due to its inherent limitations.

Assessing citation integrity in biomedical publications: corpus annotation and NLP models.

Citations have a fundamental role in scholarly communication and assessment. Citation accuracy and transparency is crucial for the integrity of scientific evidence. In this work, we focus on quotation errors, errors in citation content that can distort the scientific evidence and that are hard to detect for humans. We construct a corpus and propose natural language processing (NLP) methods to identify such errors in biomedical publications. We manually annotated 100 highly-cited biomedical publications (reference articles) and citations to them. The annotation involved labeling citation context in the citing article, relevant evidence sentences in the reference article, and the accuracy of the citation. A total of 3063 citation instances were annotated (39.18% with accuracy errors). For NLP, we combined a sentence retriever with a fine-tuned claim verification model to label citations as ACCURATE, NOT_ACCURATE, or IRRELEVANT. We also explored few-shot in-context learning with generative large language models. The best performing model-which uses citation sentences as citation context, the BM25 model with MonoT5 reranker for retrieving top-20 sentences, and a fine-tuned MultiVerS model for accuracy label classification-yielded 0.59 micro-F1 and 0.52 macro-F1 score. GPT-4 in-context learning performed better in identifying accurate citations, but it lagged for erroneous citations (0.65 micro-F1, 0.45 macro-F1). Citation quotation errors are often subtle, and it is currently challenging for NLP models to identify erroneous citations. With further improvements, the models could serve to improve citation quality and accuracy. We make the corpus and the best-performing NLP model publicly available at https://github.com/ScienceNLP-Lab/Citation-Integrity/.

Reshaping free-text radiology notes into structured reports with generative question answering transformers

BackgroundRadiology reports are typically written in a free-text format, making clinical information difficult to extract and use. Recently, the adoption of structured reporting (SR) has been recommended by various medical societies thanks to the advantages it offers, e.g. standardization, completeness, and information retrieval. We propose a pipeline to extract information from Italian free-text radiology reports that fits with the items of the reference SR registry proposed by a national society of interventional and medical radiology, focusing on CT staging of patients with lymphoma. MethodsOur work aims to leverage the potential of Natural Language Processing and Transformer-based models to deal with automatic SR registry filling. With the availability of 174 Italian radiology reports, we investigate a rule-free generative Question Answering approach based on the Italian-specific version of T5: IT5. To address information content discrepancies, we focus on the six most frequently filled items in the annotations made on the reports: three categorical (multichoice), one free-text (free-text), and two continuous numerical (factual). In the preprocessing phase, we encode also information that is not supposed to be entered. Two strategies (batch-truncation and ex-post combination) are implemented to comply with the IT5 context length limitations. Performance is evaluated in terms of strict accuracy, f1, and format accuracy, and compared with the widely used GPT-3.5 Large Language Model. Unlike multichoice and factual, free-text answers do not have 1-to-1 correspondence with their reference annotations. For this reason, we collect human-expert feedback on the similarity between medical annotations and generated free-text answers, using a 5-point Likert scale questionnaire (evaluating the criteria of correctness and completeness). ResultsThe combination of fine-tuning and batch splitting allows IT5 ex-post combination to achieve notable results in terms of information extraction of different types of structured data, performing on par with GPT-3.5. Human-based assessment scores of free-text answers show a high correlation with the AI performance metrics f1 (Spearman's correlation coefficients>0.5, p-values<0.001) for both IT5 ex-post combination and GPT-3.5. The latter is better at generating plausible human-like statements, even if it systematically provides answers even when they are not supposed to be given. ConclusionsIn our experimental setting, a fine-tuned Transformer-based model with a modest number of parameters (i.e., IT5, 220 M) performs well as a clinical information extraction system for automatic SR registry filling task. It can extract information from more than one place in the report, elaborating it in a manner that complies with the response specifications provided by the SR registry (for multichoice and factual items), or that closely approximates the work of a human-expert (free-text items); with the ability to discern when an answer is supposed to be given or not to a user query.

Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews

BackgroundPRISMA-based literature reviews require meticulous scrutiny of extensive textual data by multiple reviewers, which is associated with considerable human effort. ObjectiveTo evaluate feasibility and reliability of using GPT-4 API as a complementary reviewer in systematic literature reviews based on the PRISMA framework. MethodologyA systematic literature review on the role of natural language processing and Large Language Models (LLMs) in automatic patient-trial matching was conducted using human reviewers and an AI-based reviewer (GPT-4 API). A RAG methodology with LangChain integration was used to process full-text articles. Agreement levels between two human reviewers and GPT-4 API for abstract screening and between a single reviewer and GPT-4 API for full-text parameter extraction were evaluated. ResultsAn almost perfect GPT–human reviewer agreement in the abstract screening process (Cohen’s kappa > 0.9) and a lower agreement in the full-text parameter extraction were observed. ConclusionAs GPT-4 has performed on a par with human reviewers in abstract screening, we conclude that GPT-4 has an exciting potential of being used as a main screening tool for systematic literature reviews, replacing at least one of the human reviewers.

A Spatial Semantic Feature Extraction Method for Urban Functional Zones Based on POIs

Accurately extracting semantic features of urban functional zones is crucial for understanding urban functional zone types and urban functional spatial structures. Points of interest provide comprehensive information for extracting the semantic features of urban functional zones. Many researchers have used topic models of natural language processing to extract the semantic features of urban functional zones from points of interest, but topic models cannot consider the spatial features of points of interest, which leads to the extracted semantic features of urban functional zones being incomplete. To consider the spatial features of points of interest when extracting semantic features of urban functional zones, this paper improves the Latent Dirichlet Allocation topic model and proposes a spatial semantic feature extraction method for urban functional zones based on points of interest. In the proposed method, an assumption (that points of interest belonging to the same semantic feature are spatially correlated) is introduced into the generation process of urban functional zones, and then, Gibbs sampling is combined to carry out the parameter inference process. We apply the proposed method to a simulated dataset and the point of interest dataset for Chaoyang District, Beijing, and compare the semantic features extracted by the proposed method with those extracted by the Latent Dirichlet Allocation. The results show that the proposed method sufficiently considers the spatial features of points of interest and has a higher capability of extracting the semantic features of urban functional zones than the Latent Dirichlet Allocation.