Part-of-speech Tagging Research Articles

Ga-RBP: A Rule-based Parser for the Syntactic Analysis of the Ga Language of Ghana

This research presents a Parts of Speech (POS) corpus and a static rule-based technique, which we refer to as Ga Rule-based Parser (Ga-RBP), for the syntactic analysis and the parsing of sentences for the Ga language of Ghana. The technique is developed to parse sentences by utilising the POS tagged corpus; the corpus was developed by manually tagging the Ga words with their corresponding POS tags following a standard Ga-English dictionary and custom Tagset for the language. The syntax rules were computationally defined using production rules, which establish how a word should follow the other in the right sequence to form a correct grammatical statement based on their POS. The model generally analyses the sentence structure of the language to assert its syntactic state for correctness or otherwise.

Part-of-speech tagger for Bodo language using deep learning approach

Abstract Language processing systems such as part-of-speech (POS) tagging, named entity recognition, machine translation, speech recognition, and language modeling have been well-studied in high-resource languages. Nevertheless, research on these systems for several low-resource languages, including Bodo, Mizo, Nagamese, and others, is either yet to commence or is in its nascent stages. Language model (LM) plays a vital role in the downstream tasks of modern natural language processing. Extensive studies are carried out on LMs for high-resource languages. However, these low-resource languages are still underreprese. In this study, we first present BodoBERT, an LM for the Bodo language. To the best of our knowledge, this work is the first such effort to develop an LM for Bodo. Second, we present an ensemble deep learning-based POS tagging model for Bodo. The POS tagging model is based on combinations of BiLSTM with conditional random field and stacked embedding of BodoBERT with BytePairEmbeddings. We cover several LMs in the experiment to see how well they work in POS tagging tasks. The best-performing model achieves an F1 score of 0.8041. A comparative experiment was also conducted on Assamese POS taggers, considering that the language is spoken in the same region as Bodo.

Comparative Analysis of Deep Learning Models for Part of Speech Tagging in the Malay Language

Despite the widespread use of Malay, under-resourced languages like Malay face challenges in Natural Language Processing (NLP), particularly in Part-of-Speech (POS) tagging. The scarcity of annotated corpora poses a primary obstacle to POS tagging in Malay. This study aims to enhance the effectiveness and reliability of POS tagging models explicitly tailored for under-resourced languages within the field of NLP, focusing on Malay. Existing models, which rely on Conditional Random Fields and Hidden Markov Models, exhibit limitations, underscoring the need for more robust approaches. The research conducts a comparative analysis of various deep-learning models with different encoders for POS tagging in Malay sentences. The experimental analysis demonstrates that the Bidirectional Long Short-Term Memory (Bi-LSTM) model, leveraging a pre-trained Bidirectional Encoder Representations from Transformers (BERT) embedding model, achieves exceptional accuracy, precision, recall, and F1 scores in predicting tags. Notably, the BERT + Bi-LSTM model, boasting an accuracy of 98.82%, outperforms other models, showcasing superior performance across all evaluated metrics. Additionally, this combined model effectively handles known and unknown words, yielding highly accurate POS tagging results for Malay sentences. Doi: 10.28991/HIJ-2024-05-02-04 Full Text: PDF

Comparison of Sentences using POS Tagging Tool under Subjective Examination

The Question answering system is used to generate the correct result that is asked by humans in natural language. In an online examination system, most of the work has been done but still, problems occur in preprocessing i.e. Part Of Speech (POS). POS tagger is used to properly tag each word in the sentences. In this paper, we used two datasets i.e. TREC DATA and data collected from the student. We apply the POS tagger to both datasets and compare the result. For generating the POS tagger we used NLTK and spaCy libraries for comparison. We observed that using those libraries the same word has a different tag. Using both tools, we computed the difference between the words and assigned the count to the POS tagging on that result, we calculate the accuracy of both libraries. The result shows that the spaCy library is best for POS tagging because it generates more correct results as compared to NLTK.

Keep me PoS-ted: experimenting with Part-of-Speech prediction on Old Babylonian letters

Abstract Within this paper we will account for a cooperation between Ghent University based Assyriologists and computational linguists that has set up a pilot study to analyse the language used in Old Babylonian (OB) letters using Natural Language Processing (NLP) techniques. OB letters make up an interesting dataset because (1) they form an invaluable source for everyday vernacular language, and (2) more than 5000 have been recovered, many of which are accessible in transliteration and translation through the series Altbabylonische Briefe and the Cuneiform Digital Library Initiative. Based on a first batch of letters from OB Sippar, later extended by other Akkadian letters, we aim to develop machine learning approaches to perform semi-automatic text analysis and annotation of the letters. We will here present a Part-of-Speech (PoS) tag prediction model using machine learning. The input data is Akkadian in transliteration and the best performing model is a fine-tuned Multilingual BERT Transformer with Word embeddings (weighted avg F1: 90.19 %). When compared to the benchmark attempt of PoS tagging on a larger Akkadian corpus (97.67 %), it leaves room for improvement. However, analysing the results shows us that multilingual word embeddings improve the model performance and with an enlargement of the corpus targeting certain classes, we could considerably better the macro average F1 scores.

Analyzing fuzzy semantics of reviews for multi-criteria recommendations

Hotel reviews play a vital role in tourism recommender system. They should be analyzed effectively to enhance the accuracy of recommendations which can be generated either from crisp ratings on a fixed scale or real sentiments of reviews. But crisp ratings cannot represent the actual feelings of reviewers. Existing tourism recommender systems mostly recommend hotels on the basis of vague and sparse ratings resulting in inaccurate recommendations or preferences for online users. This paper presents a semantic approach to analyze the online reviews being crawled from tripadvisor.in. It discovers the underlying fuzzy semantics of reviews with respect to the multiple criteria of hotels rather than using the crisp ratings. The crawled reviews are preprocessed via data cleaning such as stopword and punctuation removal, tokenization, lemmatization, pos tagging to understand the semantics efficiently. Nouns representing frequent features of hotels are extracted from pre-processed reviews which are further used to identify opinion phrases. Fuzzy weights are derived from normalized frequency of frequent nouns and combined with sentiment score of all the synonyms of adjectives in the identified opinion phrases. This results in fuzzy semantics which form an ideal representation of reviews for a multi-criteria tourism recommender system. The proposed work is implemented in python by crawling the recent reviews of Jaipur hotels from TripAdvisor and analyzing their semantics. The resultant fuzzy semantics form a manually tagged dataset of reviews tagged with sentiments of identified aspects, respectively. Experimental results show improved sentiment score while considering all the synonyms of adjectives. The results are further used to fine-tune BERT models to form encodings for a query-based recommender system. The proposed approach can help tourism and hospitality service providers to take advantage of such sentiment analysis to examine the negative comments or unpleasant experiences of tourists and making appropriate improvements. Moreover, it will help online users to get better recommendations while planning their trips.

APPLICATION OF HYBRID APPROACH FOR WOLAITA LANGUAGE PART OF SPEECH TAGGING

The main purpose of this study is to develop part-of-speech tagger for Wolaita Language using hybrid approach. Part of speech tagger is one of the subtasks in NLP application which is important for other Natural Language Processing (NLP) applications, like parser, machine translator, speech recognizer and search engines. PoST is a process of tagging a corresponding part of speech tag for a word that tag defines how the word is used in a sentence. The PoST for Wolaita language is not enough yet to be used as one vital module in other natural language processing applications. In this study, the development of PoS tagger using hybrid approach that combines HMM and rule based approaches was conducted for Wolaita language. In general HMM model need large data to increase the performance and the rule based model learner rule based on the language features. The HMM tagger, tags the words based on the optimal path for a given sequence of words and transformation based learning (TBL) is a rule based approaches that learns rule directly from the training corpus without expert knowledge. The developed hybrid approach of Wolaita language PoS tagger uses HMM tagger as initial annotators and rule based tagger as a corrector based on fixed threshold values. For implementation and experiment purpose the researcher used python programming and NLTK. For training and testing the models, 1256 sentences or 15,268 words are collected from three different categories (Bible, Social media in Wolaita language (Wogetta FM 96.6 ) and Wolaita language department) and annotated data manually. For tagging purpose 26 PoS tag are identified. From entire corpus, 90% for training and the remaining of entire corpus for testing purpose. The performance of the taggers, are tested by using different experiments. After experiment the researcher found that the performance of HMM, rule based and hybrid taggers shows 88.14%, 93.19% and 94.82% respectively. Generally, hybrid approach showed the better performance to assigning part of speech tag for Wolaita language.

Incorporating knowledge for joint Chinese word segmentation and part-of-speech tagging with SynSemGCN

PurposeRecent trends have shown the integration of Chinese word segmentation (CWS) and part-of-speech (POS) tagging to enhance syntactic and semantic parsing. However, the potential utility of hierarchical and structural information in these tasks remains underexplored. This study aims to leverage multiple external knowledge sources (e.g. syntactic and semantic features, lexicons) through various modules for the joint task.Design/methodology/approachWe introduce a novel learning framework for the joint CWS and POS tagging task, utilizing graph convolutional networks (GCNs) to encode syntactic structure and semantic features. The framework also incorporates a pre-defined lexicon through a lexicon attention module. We evaluate our model on a range of public corpora, including CTB5, PKU and UD, the novel ZX dataset and the comprehensive CTB9 dataset.FindingsExperimental results on these benchmark corpora demonstrate the effectiveness of our model in improving the performance of the joint task. Notably, we find that syntax information significantly enhances performance, while lexicon information helps mitigate the issue of out-of-vocabulary (OOV) words.Originality/valueThis study introduces a comprehensive approach to the joint CWS and POS tagging task by combining multiple features. Moreover, the proposed framework offers potential adaptability to other sequence labeling tasks, such as named entity recognition (NER).

Multi Task Learning Based Shallow Parsing for Indian Languages

Shallow Parsing is an important step for many Natural Language Processing tasks. Although shallow parsing has a rich history for resource rich languages, it is not the case for most Indian languages. Shallow Parsing consists of POS Tagging and Chunking. Our study focuses on developing shallow parsers for Indian languages. As part of shallow parsing we included morph analysis as well. For the study, we first consolidated available shallow parsing corpora for 7 Indian Languages (Hindi, Kannada, Bangla, Malayalam, Marathi, Urdu, Telugu) for which treebanks are publicly available. We then trained models to achieve state of the art performance for shallow parsing in these languages for multiple domains. Since analyzing the performance of model predictions at sentence level is more realistic, we report the performance of these shallow parsers not only at the token level, but also at the sentence level. We also present machine learning techniques for multitask shallow parsing. Our experiments show that fine-tuned contextual embedding with multi-task learning improves the performance of multiple as well as individual shallow parsing tasks across different domains. We show the transfer learning capability of these models by creating shallow parsers (only with POS and Chunk) for Gujarati, Odia, and Punjabi for which no treebanks are available. As a part of this work, we will be releasing the Indian Languages Shallow Linguistic (ILSL) benchmarks for 10 Indian languages including both the major language families Indo-Aryan and Dravidian as common building blocks that can be used to evaluate and understand various linguistic phenomena found in Indian languages and how well newer approaches can tackle them.

Named Entity Recognition Using Conditional Random Fields for Flood Detection In Gerbang Kertosusila Based Twitter Data

The national strategic area Gerbang Kertosusila East Java should be aware of floods. One of the existing efforts is to place flood sensors at several flood-prone points. However, that way is constrained by the need for more equipment to handle the many needy areas. So it is necessary to develop technology for the dissemination of flood information. Dissemination of flood information was quickly obtained from social media Twitter. One way is to use Twitter's text data source for a Named Entity Recognition model to help detect flood events and their locations. The Named Entity Recognition (NER) model was constructed using the Conditional Random Fields (CRFs) method to achieve research objectives. This research adds slang word handling at the preprocessing stage to improve model performance and the use of the BIO format in the labeling process and POS Tagging in the Feature Extraction process. Evaluation results with five Kfolds, 80% training data, and 20% test data show that the NER CRFs model performs excellently with a Precision of 0.981, Recall of 0.926, and f-measure of 0.950 so that these results can help the community and government regarding the information on the distribution of floods.

Advancements in Neural Machine Translation: Methodological Innovations and Empirical Insights for Cross-Linguistic Discourse Preservation

Abstract: Natural Language Processing (NLP), in the form recognizable today, really began to take hold in the 1980s, when machine learning helped propel it to soaring heights. However, due to a lack of processing power, machine learning, and to an extent, NLP, started slowing down in innovation and ideas and had almost ground to a relative halt, until the last decade, when a sudden increase in both productivity and interest in the machine learning helped increase the amount of knowledge in the space itself. This review provides several different case studies using different methodologies. The first paper was a deep analysis on how researchers were able to use Tesseract and Google Vision in tandem with automatic data mining methods to enrich the Cherokee language database in order to preserve it from extinction. The second paper takes a query translation-based approach toward translating English to Indian languages and utilizes a Multilingual Cross-Language Information Retrieval (MLCIR) system with tools such as Part of Speech Tagger (POST), Stop-Word, and Porter Stemmer. The third paper presents CoVe, which transfers knowledge from machine translation to improve performance on NLP tasks like sentiment analysis and question answering by using contextualized word vectors along with word embeddings, achieving new state-of-the-art results on some datasets. The fourth paper aims to translate English to Pakistan Sign Language (PSL) and also uses POST and goes through dependency analysis, sentence classification, and PSL using PLS trees. The fifth paper uses a Multilingual Neural Machine Translation (NMT) system for LowResource languages and incorporates two main models: a recurrent NMT and a transformer NMT. The sixth paper analyzes how a fine-tuned transformer model seems to work better than transformer models trained from scratch on high-resource languages, while vice-versa seems to occur for low-resource languages. The seventh paper adds to this by talking about how multilingual translation seems to work better than a back-translation model. Given the diverse array of approaches that could be used, we aim to identify the most efficient and correct methodology for future researchers to use in their work, based on the papers in this literature review.

Towards automatic question generation using pre-trained model in academic field for Bahasa Indonesia

AbstractExam evaluations are essential to assessing students’ knowledge and progress in a subject or course. To meet learning objectives and assess student performance, questions must be themed. Automatic Question Generation (AQG) is our novel approach to this problem. A comprehensive process for autonomously generating Bahasa Indonesia text questions is shown. This paper suggests using a decoder to generate text from deep learning models’ tokens. The suggested technique pre-processes Vectorized Corpus, Token IDs, and Features Tensor. The tensors are embedded to increase each token, and attention is masked to separate padding tokens from context-containing tokens. An encoder processes the encoded tokens and attention masks to create a contextual understanding memory that the decoder uses to generate text. Our work uses the Sequence-to-Sequence Learning architecture of BiGRU, BiLSTM, Transformer, BERT, BART, and GPT. Implementing these models optimizes computational resources while extensively exploring the research issue. The model uses context sentences as input and question sentences as output, incorporating linguistic elements like response placement, POS tags, answer masking, and named entities (NE) to improve comprehension and linguistic ability. Our approach includes two innovative models: IndoBERTFormer, which combines a BERT encoder with a Transformer decoder, and IndoBARTFormer, which decodes vectors like BERT. IndoTransGPT uses the Transformer as an encoder to improve understanding, extending the GPT model’s adaptability.

A framework for decision making to purchase the best product using feature-based opinions

As more people express their thoughts on products on various online shopping platforms, the feelings expressed in these opinions are becoming a significant source of information for marketers and buyers. These opinions have a big impact on consumers’ decision to buy the best quality product. When there are too many features or a small number of records to analyze, the decision-making process gets difficult. A recent stream of study has used the conventional quantitative star score ratings and textual content reviews in this context. In this research, a decision-making framework is proposed that relies on feature-based opinions to analyze the textual content of reviews and classify buyer’s opinions, thereby assisting consumers in making long-term purchases. The framework is proposed in this paper for product purchase decision making based on feature-based opinions and deep learning. Framework consists of four components: i) Pre-processing, ii) Feature extraction, iii) Feature-based opinion classification, and iv) Decision-making. Web scraping is used to obtain the dataset of Smartphone reviews, which is subsequently clean and pre-processed using tokenization and POS tagging. From the tagged dataset, noun labeled words are retrieved, and then the probable product’s features are extracted. These feature-based sentences or reviews are processed using a word embedding to generate review vectors that identify contextual information. These word vectors are used to construct hidden vectors at the word and sentence levels using a hierarchical attention method. With respect to each feature, reviews are divided into five classes: extremely positive, positive, extremely negative, negative, and neutral. The proposed method may readily detect a customer’s opinion on the quality of a product based on a certain attribute, which is beneficial in making a purchase choice.

Evolution of Natural Language Processing: A Review

Over the years, Natural Language Processing (NLP) has evolved dramatically, moving from early rule based systems to the current era dominated by advanced deep learning models. An overview of the significant turning points and patterns that have influenced the development of NLP is given in this study. In the early days of Natural Language Processing (NLP), rule based methods were the main focus. Linguists would manually create rules to analyze and comprehend human language. Although these systems were somewhat successful, they were unable to handle the complexity and unpredictability of spoken language. A major change was brought about by the introduction of probabilistic models and machine learning techniques with the emergence of statistical approaches. The development of methods like n gram models and hidden Markov models during this time allowed computers to handle linguistic patterns. Large scale linguistic resources like word embeddings and annotated corpora started to appear, which further accelerated the development of NLP. Machine learning algorithms have led to notable advancements in tasks such as machine translation, named entity recognition, and part of speech tagging. NLP has seen a revolution in recent years thanks to deep learning, which uses neural networks to learn intricate language representations. Sequential dependencies in language can now be better understood because of models like Long Short Term Memory Networks (LSTMs) and Recurrent Neural Networks (RNNs). The addition of attention mechanisms, as demonstrated by Transformer and other models, improves the model's ability to manage long range dependencies and perform better on a variety of NLP tasks. In the future, NLP will develop in ways that go beyond performance measurements, exploring interpretability, ethical issues, and the incorporation of multimodal data. As the area develops, it becomes increasingly important to eliminate biases, ensure ethical AI deployment, and improve user centric experiences. This introduction lays the groundwork for an in depth examination of the development of NLP, highlighting significant turning points, difficulties, and potential future directions in this vibrant and quickly developing subject.

English Vocabulary Learning Aid System Using Digital Twin Wasserstein Generative Adversarial Network Optimized With Jelly Fish Optimization Algorithm

ABSTRACT Natural Language Processing (NLP) is a technology that permits computers to recognize human languages. Words are the fundamental unit of analysis in deep-level grammatical and semantic analysis. The main goal of NLP is typically word segmentation. Since the machine learning techniques cannot be directly applied to the practical issue of significant structural disparities between various data modalities in a multi-modal context. In this paper, English Vocabulary Learning Aid System Using Digital Twin Wasserstein Generative Adversarial Network Optimized using Jelly Fish Optimization Algorithm is proposed. The problematic of multiple modal data heterogeneity is handled by the feature extraction of Parameterized Local Maximum Synchro squeezing Transform and extract the features such as Phonetic features, sentence length, word embedding’s, part of speech tags, word frequencies, N-grams. Then, the Digital twin Wasserstein generative adversarial network classifies the English vocabulary to easy words, intermediate words, and difficult words. The performance of the proposed EVLS-DtwinWGAN-NLP approach attains 3.101%, 7.12%, 7.73% higher accuracy, 24.13%, 13.04%, 29.51% lower computation Time and 2.292%, 5.365%, 1.551% higher AUC compared with existing methods like Feature extraction and analysis of natural language processing for deep learning English language (EVLS-BiLSTM-NLP), State of art for semantic analysis of natural language processing (EVLS-SA-NLP) respectively.

Deep Learning Model for Tamil Part-of-Speech Tagging

Abstract Part-of-Speech (POS) tagging is one of the popular Natural Language Processing (NLP) tasks. It is also considered to be a preliminary task for other NLP applications such as speech recognition, machine translation, and sentiment analysis. A few works have been published on POS tagging for the Tamil language. However, the performance of the POS tagger with unknown words is not explored in the literature. The appearance of unknown words is a frequently occurring problem in POS tagging and makes it a challenging task. In this paper, we propose a deep learning-based POS tagger for Tamil using Bi-directional Long Short Term Memory (BLSTM). The performance of the POS tagger was evaluated using known and unknown words. The POS tagger with regular word-level embeddings produces 99.83 and 92.46% accuracies for all known and 63.21% unknown words. It clearly shows that the accuracy decreases when the number of unknown words increases. To improve the performance of the POS tagger with unknown words, the proposed BLSTM model that uses word-level, character-level and pre-trained word embeddings. Test results of this model show a 2.57% improvement for 63.21% of unknown words, with an accuracy of 95.03%.

Enabling Natural Language Processing and AI Research in Low-Resource Languages: Development and Description of an Assamese UPoS Tagged Dataset

This paper describes in detail the Universal Parts of Speech (UPoS) tagged dataset for the Assamese language. PoS tagged dataset in a language is crucial for experimenting and creating resources for various Natural Language Processing (NLP) and AI research. With the growing usage of Universal Dependency standards, tagged dataset with Universal PoS tags are becoming very much essential for contemporary experiments in the NLP community. NLP research in Assamese, and Indo-Aryan language, is relatively new, and the language is considered a Low Resource language. The dataset of UPoS tagged Assamese text is created with an aim of contributing towards enriching this low resource language for NLP and AI tasks. The size of the dataset is 283506 tokens of Assamese vocabulary, against total 20280 sentences, tagged with 17 standard UPoS tags of core lexical categories. The raw data are taken from an open-source corpus originally tagged with BIS tagset. The original size of 453457 tokens against 29504 sentences, after subjected to data filtering, was reduced to this clean resource of 283506 tokens. Lexical categories mapping is done with linguistic expertise, from BIS to UPoS tagsets. Mapped pattern was used for a first-level conversion of BIS tags to UPoS tags. Linguistic validation is also performed with linguistic experts and inter annotator agreement/disagreements were recorded. Second level validation resulted in deciding on the agreement, producing the final version of the dataset. This Assamese UPoS tagged dataset is the first of its kind with UPoS annotations and shall serve a wider Assamese NLP research community for model training using Machine Learning/Deep Learning Techniques.

BioBBC: a multi-feature model that enhances the detection of biomedical entities

The rapid increase in biomedical publications necessitates efficient systems to automatically handle Biomedical Named Entity Recognition (BioNER) tasks in unstructured text. However, accurately detecting biomedical entities is quite challenging due to the complexity of their names and the frequent use of abbreviations. In this paper, we propose BioBBC, a deep learning (DL) model that utilizes multi-feature embeddings and is constructed based on the BERT-BiLSTM-CRF to address the BioNER task. BioBBC consists of three main layers; an embedding layer, a Long Short-Term Memory (Bi-LSTM) layer, and a Conditional Random Fields (CRF) layer. BioBBC takes sentences from the biomedical domain as input and identifies the biomedical entities mentioned within the text. The embedding layer generates enriched contextual representation vectors of the input by learning the text through four types of embeddings: part-of-speech tags (POS tags) embedding, char-level embedding, BERT embedding, and data-specific embedding. The BiLSTM layer produces additional syntactic and semantic feature representations. Finally, the CRF layer identifies the best possible tag sequence for the input sentence. Our model is well-constructed and well-optimized for detecting different types of biomedical entities. Based on experimental results, our model outperformed state-of-the-art (SOTA) models with significant improvements based on six benchmark BioNER datasets.

CoreNLP dependency parsing and pattern identification for enhanced opinion mining in aspect-based sentiment analysis

Aspect-Based Sentiment Analysis (ABSA) aims to identify the sentiment expressed towards a specific feature or aspect of a given text. Although certain ABSA techniques employ syntactic information to capture the connection between the opinion target and the sentiment word, they often do not incorporate data processing techniques such as dependency parsing, which can be beneficial in accurately capturing the sentiment expressed towards the opinion target. In this paper a method for ABSA that employs both syntax and semantic information and incorporates dependency parsing(Semantic-Syntactic Dependency Parsing (SSDP) Method) with Core Natural Language Processing (CoreNLP) which is a natural language processing library for processing the input text and identifying patterns effectively (according to the CoreNLP relations and part of speech tagging(POS)) to extract the critical relations that accurately reflect the sentiment conveyed regarding the opinion target is proposed. The results show that the proposed pattern captured approximately 75% of the data, and the rest were classified via Long Short-Term Memory (LSTM) based on semantic information. We illustrated the efficacy of SSDP, through experiments on the SemEval14,Semval15 and Semval16 datasets, which include two datasets (laptops and restaurants) carefully categorized by a human annotator into categories of positive, negative, or neutral attitudes. The experimental results reveal that SSDP is superior to the other state-of-the-art ABSA approaches, that use syntax information but do not utilize data processing techniques. Additionally, we highlight the limitations of ABSA methods that do not incorporate syntax information and the potential improvements that can be made through the use of data processing.

Exploring part of speech (pos) tag sequences in a large-scale learner corpus of L2 English: a developmental perspective

This research explores the pos tag sequences that shape the transition from upper intermediate (B2 cefr) to near-native proficiency (C2 cefr) in a corpus of essays ( n=32,410) from the Cambridge Learner Corpus. Gilquin (2018) and others have shown that pos tag sequences offer a holistic approach to extracting the most commonly used patterns without a starting point of an a priori set of words and word sequences. Using corpus linguistics informed by usage-based theories of language learning, this paper examines the frequency and distribution of 4-slot pos-tag sequences in L2 English writing, drawing on the taxonomy of pattern grammar ( Francis et al., 1996 , 1998 ; and Hunston and Francis, 2000 ). Findings point to the presence of both core and emergent pos-tag sequences in learner language in the two proficiency levels analysed. These sequences point to the presence of dynamic language restructuring processes as learners become more proficient and re-evaluate their understanding of frequency and distribution in English. This paper shows evidence of how language competence increases with proficiency. The research offers new evidence in our understanding of the development of L2 writing in efl contexts.