Transformation Language Research Articles

Maximizing Penetration Testing Success with Effective Reconnaissance Techniques Using ChatGPT

Background/Objective: The study investigates the integration of ChatGPT, a generative pretrained transformer language model into the reconnaissance phase of penetration testing. The research aims to enhance the efficiency and depth of information gathering during critical security assessments offering potential improvements to traditional approaches.
 Research Problem: The research study addresses the challenge of optimizing the reconnaissance phase in penetration testing. It seeks to provide a solution by exploring the capabilities of ChatGPT in extracting valuable data, such as various aspects of the digital footprint or infrastructure of a system or an organization. The scope of the research relies in demonstrating how ChatGPT can contribute to the planning phase of penetration testing, guiding the selection of tactics, tools, and techniques for identifying and mitigating potential risks that could be used to assist with securing Internet accessible assets of a system or an organization.
 Methodology: The research adopts a case study methodology to assess the effectiveness of ChatGPT in reconnaissance. Tailored questions are formulated to extract specific information relevant to penetration testing. The study highlights the importance of prompt engineering emphasizing the need for carefully constructed questions to ensure usable results.
 Results: The research showcases the ability of ChatGPT to provide diverse and insightful reconnaissance information. The extracted data includes IP address ranges, domain names, vendor technologies, SSL/TLS ciphers, and network protocols. The information gathering improves efficiency of the reconnaissance phase aiding penetration testers in planning subsequent phases of the assessment.
 Discussion: The research study extends to the broader field of cybersecurity where artificial intelligence language models can play a valuable role in enhancing the success of reconnaissance in penetration testing. The research suggests that integrating ChatGPT into penetration testing can bring about positive changes in the efficiency and depth of information obtained during reconnaissance.
 Conclusion: The results of the study determine that incorporating ChatGPT in the reconnaissance phase significantly benefits penetration testers by offering valuable insights and streamlining subsequent assessment planning. The results affirm ChatGPT as a pivotal tool in maximizing success in penetration testing, contributing to ongoing advancements in cybersecurity practices.

Traceability and reuse mechanisms, the most important properties of model transformation languages

ContextDedicated model transformation languages are claimed to provide many benefits over the use of general purpose languages for developing model transformations. However, the actual advantages and disadvantages associated with the use of model transformation languages are poorly understood empirically. There is little knowledge and even less empirical assessment about what advantages and disadvantages hold in which cases and where they originate from. In a prior interview study, we elicited expert opinions on what advantages result from what factors surrounding model transformation languages as well as a number of moderating factors that moderate the influence.ObjectiveWe aim to quantitatively asses the interview results to confirm or reject the influences and moderation effects posed by different factors. We further intend to gain insights into how valuable different factors are to the discussion so that future studies can draw on these data for designing targeted and relevant studies.MethodWe gather data on the factors and quality attributes using an online survey. To analyse the data and examine the hypothesised influences and moderations, we use universal structure modelling based on a structural equation model. Universal structure modelling produces significance values and path coefficients for each hypothesised and modelled interdependence between factors and quality attributes that can be used to confirm or reject correlation and to weigh the strength of influence present.ResultsWe analyzed 113 responses. The results show that the MTL capabilities Tracing and Reuse Mechanisms are most important overall. Though the observed effects were generally 10 times lower than anticipated. Furthermore, we found that moderation effects need to be individually assessed for each influence on a quality attribute. The moderation effects of a single moderating variable vary significantly for each influence, with the strongest effects being 1000 times higher than the weakest.ConclusionThe empirical assessment of MTLs is a complex topic that cannot be solved by looking at a single stand-alone factor. Our results provide clear indication that evaluation should consider transformations of different sizes and use-cases that go beyond mapping one elements attributes to another. Language development on the other hand should focus on providing practical, transformation specific reuse mechanisms that allow MTLs to excel in areas such as maintainability and productivity compared to GPLs.

Spoken service language for customer well-being in a transformative service context: residential aged care

PurposeSpoken service language is critical for service experiences and human welfare in many service settings. However, little is known about how spoken service language can enhance customer well-being in transformative service contexts. This paper explores spoken service language and well-being for customers experiencing vulnerability in a transformative service context, informed by an empirical account of the human welfare service of residential aged care.Design/methodology/approachSituated within transformative service research (TSR), this study was guided by a theoretical framework of service language and adopts a strengths-based approach to customer experiences of vulnerability. A qualitative multiple case study methodology was applied to explore carers’ perspectives on spoken service language and well-being from three residential aged care homes in Australia.FindingsThe findings demonstrate five spoken service language practices and four principles of spoken service language for well-being that co-create customer well-being and support the alleviation of customer experiences of vulnerability. Conceptualised as transformative spoken service language, the spoken service language practices and principles collectively recognise, support and leverage residents’ capabilities and uplift customer well-being, by enacting a process of mattering highly salient to transformative service contexts.Originality/valueThis study is the first to conceptualise how employee spoken service language can be used to support customer well-being and enhance transformative value for customers experiencing vulnerability to align with the goals of TSR. Practically, the study advocates for a greater awareness and more considered use of transformative spoken service language in human welfare and other transformative service contexts.

SRBerta—A Transformer Language Model for Serbian Cyrillic Legal Texts

Language is a unique ability of human beings. Although relatively simple for humans, the ability to understand human language is a highly complex task for machines. For a machine to learn a particular language, it must understand not only the words and rules used in a particular language, but also the context of sentences and the meaning that words take on in a particular context. In the experimental development we present in this paper, the goal was the development of the language model SRBerta—a language model designed to understand the formal language of Serbian legal documents. SRBerta is the first of its kind since it has been trained using Cyrillic legal texts contained within a dataset created specifically for this purpose. The main goal of SRBerta network development was to understand the formal language of Serbian legislation. The training process was carried out using minimal resources (single NVIDIA Quadro RTX 5000 GPU) and performed in two phases—base model training and fine-tuning. We will present the structure of the model, the structure of the training datasets, the training process, and the evaluation results. Further, we will explain the accuracy metric used in our case and demonstrate that SRBerta achieves a high level of accuracy for the task of masked language modeling in Serbian Cyrillic legal texts. Finally, SRBerta model and training datasets are publicly available for scientific and commercial purposes.

ULDNA: integrating unsupervised multi-source language models with LSTM-attention network for high-accuracy protein-DNA binding site prediction.

Efficient and accurate recognition of protein-DNA interactions is vital for understanding the molecular mechanisms of related biological processes and further guiding drug discovery. Although the current experimental protocols are the most precise way to determine protein-DNA binding sites, they tend to be labor-intensive and time-consuming. There is an immediate need to design efficient computational approaches for predicting DNA-binding sites. Here, we proposed ULDNA, a new deep-learning model, to deduce DNA-binding sites from protein sequences. This model leverages an LSTM-attention architecture, embedded with three unsupervised language models that are pre-trained on large-scale sequences from multiple database sources. To prove its effectiveness, ULDNA was tested on 229 protein chains with experimental annotation of DNA-binding sites. Results from computational experiments revealed that ULDNA significantly improves the accuracy of DNA-binding site prediction in comparison with 17 state-of-the-art methods. In-depth data analyses showed that the major strength of ULDNA stems from employing three transformer language models. Specifically, these language models capture complementary feature embeddings with evolution diversity, in which the complex DNA-binding patterns are buried. Meanwhile, the specially crafted LSTM-attention network effectively decodes evolution diversity-based embeddings as DNA-binding results at the residue level. Our findings demonstrated a new pipeline for predicting DNA-binding sites on a large scale with high accuracy from protein sequence alone.

Multimodal emotion recognition based on the fusion of vision, EEG, ECG, and EMG signals

This paper presents a novel approach for emotion recognition (ER) based on Electroencephalogram (EEG), Electromyogram (EMG), Electrocardiogram (ECG), and computer vision. The proposed system includes two different models for physiological signals and facial expressions deployed in a real-time embedded system. A custom dataset for EEG, ECG, EMG, and facial expression was collected from 10 participants using an Affective Video Response System. Time, frequency, and wavelet domain-specific features were extracted and optimized, based on their Visualizations from Exploratory Data Analysis (EDA) and Principal Component Analysis (PCA). Local Binary Patterns (LBP), Local Ternary Patterns (LTP), Histogram of Oriented Gradients (HOG), and Gabor descriptors were used for differentiating facial emotions. Classification models, namely decision tree, random forest, and optimized variants thereof, were trained using these features. The optimized Random Forest model achieved an accuracy of 84%, while the optimized Decision Tree achieved 76% for the physiological signal-based model. The facial emotion recognition (FER) model attained an accuracy of 84.6%, 74.3%, 67%, and 64.5% using K-Nearest Neighbors (KNN), Random Forest, Decision Tree, and XGBoost, respectively. Performance metrics, including Area Under Curve (AUC), F1 score, and Receiver Operating Characteristic Curve (ROC), were computed to evaluate the models. The outcome of both results, i.e., the fusion of bio-signals and facial emotion analysis, is given to a voting classifier to get the final emotion. A comprehensive report is generated using the Generative Pretrained Transformer (GPT) language model based on the resultant emotion, achieving an accuracy of 87.5%. The model was implemented and deployed on a Jetson Nano. The results show its relevance to ER. It has applications in enhancing prosthetic systems and other medical fields such as psychological therapy, rehabilitation, assisting individuals with neurological disorders, mental health monitoring, and biometric security.

ArMT-TNN: Enhancing natural language understanding performance through hard parameter multitask learning in Arabic

Multitask learning (MTL) is a machine learning paradigm where a single model is trained to perform several tasks simultaneously. Despite the considerable amount of research on MTL, the majority of it has been centered around English language, while other language such as Arabic have not received as much attention. Most existing Arabic NLP techniques concentrate on single or multitask learning, sharing just a limited number of tasks, between two or three tasks. To address this gap, we present ArMT-TNN, an Arabic Multi-Task Learning using Transformer Neural Network, designed for Arabic natural language understanding (ANLU) tasks. Our approach involves sharing learned information between eight ANLU tasks, allowing for a single model to solve all of them. We achieve this by fine-tuning all tasks simultaneously and using multiple pre-trained Bidirectional Transformer language models, like BERT, that are specifically designed for Arabic language processing. Additionally, we explore the effectiveness of various Arabic language models (LMs) that have been pre-trained on different types of Arabic text, such as Modern Standard Arabic (MSA) and Arabic dialects. Our approach demonstrated outstanding performance compared to all current models on four test sets within the ALUE benchmark, namely MQ2Q, OOLD, SVREG, and SEC, by margins of 3.9%, 3.8%, 10.1%, and 3.7%, respectively. Nonetheless, our approach did not perform as well on the remaining tasks due to the negative transfer of knowledge. This finding highlights the importance of carefully selecting tasks when constructing a benchmark. Our experiments also show that LMs which were pretrained on text types that differ from the text type used for finetuned tasks can still perform well.

Mining drug-target interactions from biomedical literature using chemical and gene descriptions-based ensemble transformer model.

Drug-target interactions (DTIs) play a pivotal role in drug discovery, as it aims to identify potential drug targets and elucidate their mechanism of action. In recent years, the application of natural language processing (NLP), particularly when combined with pre-trained language models, has gained considerable momentum in the biomedical domain, with the potential to mine vast amounts of texts to facilitate the efficient extraction of DTIs from the literature. In this article, we approach the task of DTIs as an entity-relationship extraction problem, utilizing different pre-trained transformer language models, such as BERT, to extract DTIs. Our results indicate that an ensemble approach, by combining gene descriptions from the Entrez Gene database with chemical descriptions from the Comparative Toxicogenomics Database (CTD), is critical for achieving optimal performance. The proposed model achieves an F1 score of 80.6 on the hidden DrugProt test set, which is the top-ranked performance among all the submitted models in the official evaluation. Furthermore, we conduct a comparative analysis to evaluate the effectiveness of various gene textual descriptions sourced from Entrez Gene and UniProt databases to gain insights into their impact on the performance. Our findings highlight the potential of NLP-based text mining using gene and chemical descriptions to improve drug-target extraction tasks. Datasets utilized in this study are accessible at https://dtis.drugtargetcommons.org/.

DL-SPhos: Prediction of serine phosphorylation sites using transformer language model

Serine phosphorylation plays a pivotal role in the pathogenesis of various cellular processes and diseases. Roughly 81% of human diseases have links to phosphorylation, and an overwhelming 86.4% of protein phosphorylation takes place at serine residues. In eukaryotes, over a quarter of proteins undergo phosphorylation, with more than half implicated in numerous disorders, notably cancer and reproductive system diseases. This study primarily focuses on serine-phosphorylation-driven pathogenesis and the critical role of conserved motif identification. While numerous techniques exist for predicting serine phosphorylation sites, traditional wet lab experiments are resource-intensive. Our paper introduces a cutting-edge deep learning tool for predicting S phosphorylation sites, integrating explainable AI for motif identification, a transformer language model, and deep neural network components. We trained our model on protein sequences from UniProt, validated it against the dbPTM benchmark dataset, and employed the PTMD dataset to explore motifs related to mammalian disorders. Our results highlight that our model surpasses other deep learning predictors by a significant 3%. Furthermore, we utilized the local interpretable model-agnostic explanations (LIME) approach to shed light on the predictions, emphasizing the amino acid residues crucial for S phosphorylation. Notably, our model also outperformed competitors in kinase-specific serine phosphorylation prediction on benchmark datasets.

Graph rewriting primitives for semantic graph databases sanitization

Due to the rapid proliferation of data online, an important quantity of private or sensitive informations is being stored as linked data in graph databases (e.g., represented as RDF). For such databases to be shared without jeopardizing privacy, they must first undergo a process known as database sanitization. During this process, databases are transformed following graph transformations that are usually described informally or through ad-hoc processes. However, a more thourough formalization of these transformations would aid in analysing the sanitization process, ensuring its correctness, and demonstrating the resulting privacy guarantees. This paper is an effort toward bridging the gap between the rigorous graph rewriting approaches and graph sanitization. We propose a graph transformation language to serve as a basis for constructing various sanitization mechanisms. This language relies on a set of elementary transformation operators formalized using a generic algebraic graph rewriting approach. Our language takes into account semantic and supports the equivalent of WHERE and EXCEPT clauses. As a proof of concept, we use these operators to implement two mechanisms from the literature, one generic (Local Differential Privacy) and one specifically introduced for semantic graph databases (sensitive attribute masking through anatomization). We propose an open-sourced tool implementing the elementary operators and the privacy mechanisms we derive from them relying on the Attributed Graph Grammar System (AGG) and its java API, providing a concrete tool implementing formal graph rewriting mechanisms to sanitize semantic graph databases. We present experimental results on this implementation regarding both proposed schemes and discuss its efficiency and scalability.

Part of speech distributions for Grimm versus artificially generated fairy tales

ChatGPT is a chatbot tool that relies on GPT3 and later OpenAI transformer language models to generate responses to user prompts. In this study, we sought to investigate the statistical differences between naturally generated and artificially generated text due to the dramatic increase in quality of natural language generation from large language models, popularized by ChatGPT. To constrain our problem, we considered fairy tales as these texts have existed for centuries. To explore statistical differences, we focused on the distribution of words according to their parts of speech (POS), elements that characterize words based on their grammatical function. We generated a novel corpus of 101 fairy tales “authored” by ChatGPT. We compared this against 209 fairy tales written by the Grimm Brothers and made available freely online. Our hypothesis was that the distributions of POS for Grimm fairy tales and ChatGPT fairy tales would be different and that the POS distributions will vary among Grimm fairy tales more than among ChatGPT fairy tales. We performed appropriate preprocessing and computed total variation distances for individual fairy tales within and between authorship conditions. We found out that in fact, the distribution of POS in ChatGPT fairy tales is significantly different from the distribution of POS in Grimm fairy tales.

Generative Pre-Trained Transformer-Based Reinforcement Learning for Testing Web Application Firewalls

Web Application Firewalls (WAFs) are widely deployed to protect key web applications against multiple security threats, so it is important to test WAFs regularly to prevent attackers from bypassing them easily. Machine-learning-based black-box WAF testing is gaining more attention, though existing learning-based approaches have strict requirements on the source and scale of payload data and suffer from the local optimum problem, limiting their effectiveness and practical application. We propose GPTFuzzer, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">practical</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">effective</i> generation-based approach to test WAFs by generating attack payloads token-by-token. Specifically, we fine-tune a Generative Pre-trained Transformer language model with reinforcement learning to make GPTFuzzer have the least restrictions on payload data and thus more applicable in practice, and we use reward modeling and KL-divergence penalty to improve the effectiveness of our approach and mitigate the local optimum issue. We implement GPTFuzzer and evaluate it on two well-known open-source WAFs against three kinds of common attacks. Experimental results show that GPTFuzzer significantly outperforms state-of-the-art approaches, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> ML-Driven and RAT, finding up to 7.8× (3.2× on average) more bypassing payloads within 1,250,000 requests, or finding out all bypassing payloads using up to 8.1× (3.3× on average) fewer requests.

On the Mathematical Relationship Between Contextual Probability and N400 Amplitude.

Accounts of human language comprehension propose different mathematical relationships between the contextual probability of a word and how difficult it is to process, including linear, logarithmic, and super-logarithmic ones. However, the empirical evidence favoring any of these over the others is mixed, appearing to vary depending on the index of processing difficulty used and the approach taken to calculate contextual probability. To help disentangle these results, we focus on the mathematical relationship between corpus-derived contextual probability and the N400, a neural index of processing difficulty. Specifically, we use 37 contemporary transformer language models to calculate the contextual probability of stimuli from 6 experimental studies of the N400, and test whether N400 amplitude is best predicted by a linear, logarithmic, super-logarithmic, or sub-logarithmic transformation of the probabilities calculated using these language models, as well as combinations of these transformed metrics. We replicate the finding that on some datasets, a combination of linearly and logarithmically-transformed probability can predict N400 amplitude better than either metric alone. In addition, we find that overall, the best single predictor of N400 amplitude is sub-logarithmically-transformed probability, which for almost all language models and datasets explains all the variance in N400 amplitude otherwise explained by the linear and logarithmic transformations. This is a novel finding that is not predicted by any current theoretical accounts, and thus one that we argue is likely to play an important role in increasing our understanding of how the statistical regularities of language impact language comprehension.

ConceptGCN: Knowledge concept recommendation in MOOCs based on knowledge graph convolutional networks and SBERT

Massive Open Online Courses (MOOCs) have gained popularity in the technology-enhanced learning (TEL) domain. To enhance the learning experience in MOOCs, educational recommender systems (ERSs) can play a crucial role by suggesting courses or learning materials that align with students' knowledge states. Thereby, understanding a student's learning needs and predicting knowledge concepts that the student might be interested in are important to provide effective recommendations. Inspired by the superior ability of knowledge graphs (KGs) in modeling the heterogeneous data in MOOCs and Graph Neural Networks (GNNs) in learning on graph-structured data, few works focusing on GNN-based recommendation of knowledge concepts in MOOCs have emerged recently. However, existing approaches in this domain have limitations mainly related to complexity, semantics, and transparency. To address these limitations, in this paper we propose ConceptGCN, an end-to-end framework that combines KGs, Graph Convolutional Networks (GCNs), and pre-trained transformer language model encoders (SBERT) to provide personalized and transparent recommendations of knowledge concepts in the MOOC platform CourseMapper. We conducted extensive offline experiments and an online user study (N=31), demonstrating the benefits of the ConceptGCN-based recommendation approach, in terms of several important user-centric aspects including accuracy, novelty, diversity, usefulness, overall satisfaction, use intentions, and reading intention. In particular, our results indicate that, if SBERT is used for the initial embeddings of items in the KG, a self-connection operation and a semantic similarity-based score function in the aggregation operation of GCN are not necessarily needed.

The consideration of semantic gap between design and code

In project management on creation of program application, specialists from different subject areas are involved, who include their contributions, for instance, UI/UX designers who create mock-ups of the future application or developers who write the code according to the prototype. The design conception may go beyond the possibilities of interpreting it from a technical point of view of implementation. The realization of such idea could not to be able to collect on only one defined program platform or language, and accordingly the problem is appeared. To eliminate semantic gap between the designers’ concepts and opportunity of program developers in technical affordance, released methodology, Model Driven Architecture (MDA), which is, on the one hand, a concept for implementation of software, on the other hand a standard. In paper, considerate the MDA and its transformation levels with determine a pragmatical semantics of mapping, reasons of chosen a class diagram as model of transformation and Java language for code generation.

Epidemic effects in the diffusion of emerging digital technologies: evidence from artificial intelligence adoption

The properties of emerging, digital, general-purpose technologies make it hard to observe their adoption by firms and identify the salient determinants of adoption. However, these aspects are critical since the patterns related to early-stage diffusion establish path-dependencies which have implications for the distribution of the technological opportunities and socio-economic returns linked to these technologies. We focus on the case of artificial intelligence (AI) and train a transformer language model to identify firm-level AI adoption using textual data from over 1.1 million websites and constructing a hyperlink network that includes >380,000 firms in Germany, Austria, and Switzerland. We use these data to expand and test epidemic models of inter-firm technology diffusion by integrating the concepts of social capital and network embeddedness. We find that AI adoption is related to three epidemic effect mechanisms: 1) Indirect co-location in industrial and regional hot-spots associated to production of AI knowledge; 2) Direct exposure to sources transmitting deep AI knowledge; 3) Relational embeddedness in the AI knowledge network. The pattern of adoption identified is highly clustered and features a rather closed system of AI adopters which is likely to hinder its broader diffusion. This has implications for policy which should facilitate diffusion beyond localized clusters of expertise. Our findings also point to the need to employ a systemic perspective to investigate the relation between AI adoption and firm performance to identify whether appropriation of the benefits of AI depends on network position and social capital.

Code Swarm: A Code Generation Tool based on the Automatic Derivation of Transformation Rule Set

Automatic generation of software code from system design models remains an actively explored research area for the past several years. A number of tools are currently available to facilitate and automate the task of generating code from software models. To the best of our knowledge, existing software tools rely on an explicitly defined transformation rule set to perform the model-to-code transformation process. In this paper, we introduce a novel tool named Code Swarm, abbreviated as CodS, that automatically generates implementation code from system design models by utilizing a swarm-based approach. Specifically, CodS is capable of generating Java code from the class and state models of the software system by making use of the previously solved model-to-code transformation examples. Our tool enables the designers to specify behavioural actions in the input models using the Action Specification Language (ASL). We use an industrial case study of the Elevator Control System (ECS) to perform the experimental validation of our tool. Our results indicate that the code generated by CodS is correct and consistent with the input design models. CodS performs the process of automatic code generation without taking the explicit transformation rule set or languages metamodels’ information as input, which distinguishes it from all the existing automatic code generation tools.

Transformer and Natural language processing; A recent development.

The convergence of transformers and natural language processing (NLP) represents a watershed moment in the realm of artificial intelligence. Recent NLP advancements have been profoundly shaped by the emergence of transformers, a class of deep learning models renowned for their remarkable proficiency in comprehending, generating, and manipulating human language. This abstract offers a succinct exploration of the symbiotic relationship between transformers and NLP, emphasizing their central role in propelling recent process.&#x0D; The introduction of transformers heralded a paradigm shift in the realm of NLP, primarily owing to their innovative self-attention mechanism, which empowers them to adeptly capture intricate contextual associations within textual data. Distinguished by their multi-head attention layers and feed-forward networks, the architecture of transformers has ushered in a new era in NLP. Models such as BERT,&#x0D; GPT-3, and their offshoots have not only redefined but also set the gold standard for a wide array of NLP tasks.

POS-BERT: Point cloud one-stage BERT pre-training

Recently, the pre-training paradigm combining Transformer and masked language modeling in BERT has achieved tremendous success not only in NLP, but also in images and point clouds. However, directly extending BERT from NLP to point clouds requires first training a discrete Variational AutoEncoder (dVAE) as the tokenizer, which results in a complex two-stage process, as in Point-BERT. Inspired by BERT and MoCo, we propose POS-BERT, a one-stage BERT pre-training method for point clouds. Specifically, we use the masked patch modeling (MPM) task to perform point cloud pre-training, which aims to recover masked patch information under the supervision of a tokenizer’s output. Unlike Point-BERT, whose tokenizer is extra-trained and frozen, we propose a momentum tokenizer which is dynamically updated during training the Transformer. Furthermore, in order to better learn high-level semantic representation, we integrate contrastive learning into the proposed framework to maximize the class token consistency between augmented point cloud pairs. Experiments show that POS-BERT achieves the state-of-the-art performance on linear SVM classification of ModelNet40 with fixed feature extractors, and it exceeds Point-BERT by 3.5%. In addition, POS-BERT has significantly improved many downstream tasks, including fine-tuned classification, few-shot classification and part segmentation. The code and trained models will be released on https://github.com/fukexue/POS-BERT.git.

MetaQA: Enhancing human-centered data search using Generative Pre-trained Transformer (GPT) language model and artificial intelligence.

Accessing and utilizing geospatial data from various sources is essential for developing scientific research to address complex scientific and societal challenges that require interdisciplinary knowledge. The traditional keyword-based geosearch approach is insufficient due to the uncertainty inherent within spatial information and how it is presented in the data-sharing platform. For instance, the Gulf of Mexico Coastal Ocean Observing System (GCOOS) data search platform stores geoinformation and metadata in a complex tabular. Users can search for data by entering keywords or selecting data from a drop-down manual from the user interface. However, the search results provide limited information about the data product, where detailed descriptions, potential use, and relationship with other data products are still missing. Language models (LMs) have demonstrated great potential in tasks like question answering, sentiment analysis, text classification, and machine translation. However, they struggle when dealing with metadata represented in tabular format. To overcome these challenges, we developed Meta Question Answering System (MetaQA), a novel spatial data search model. MetaQA integrates end-to-end AI models with a generative pre-trained transformer (GPT) to enhance geosearch services. Using GCOOS metadata as a case study, we tested the effectiveness of MetaQA. The results revealed that MetaQA outperforms state-of-the-art question-answering models in handling tabular metadata, underlining its potential for user-inspired geosearch services.