Query Response Research Articles

Adapting OGC's SensorThings API and data model to support data management and sharing for environmental sensors

Software is critical in managing environmental sensor data. The Open Geospatial Consortium (OGC) developed the “OGC SensorThings API” (STA) standard to address variability across sensors, observed variables, platforms, and protocols, facilitating development of sensing and Internet of Things applications. This paper details a Python/Django implementation of the STA application programming interface (API) and a PostgreSQL/Timescale implementation of the STA data model, enhancing availability of robust software for management and sharing of environmental sensor data. STA offers a RESTful interface with JSON data encoding, aligning with modern development patterns and facilitating interoperability. Integration of metadata from the Observations Data Model ensures data can be adequately described and interpreted. STA's flexibility allows lightweight query responses or comprehensive metadata inclusion, and a complementary data management API enhances use of STA within multi-user systems. Open-source code and deployment instructions in GitHub enable standalone or cloud deployments, enhancing accessibility and usability for researchers and practitioners.

Transforming academic assessment: The metaverse‐backed Web 3 secure exam system

AbstractMetaverse—a three‐dimensional computational environment—combines physical and virtual reality to enable social relationships and immersive experiences by mimicking real‐world scenarios. Metaverse is considered the third wave of the internet revolution (exploiting Web 3.0), leveraging upcoming technologies such as extended reality and artificial intelligence shaping a new era of human–‐machine interactions. In recent years, increased research and development on educational technologies (EduTech) based on blockchain technology has seen substantial growth of metaverse‐based solutions within the higher education context. This research aims to synergize blockchain technology and metaverse environments to conduct online exams (metaExam) in a trustworthy, reliable, and secure way. The synergy between blockchain and the metaverse brings various benefits, such as improved security, cost effectiveness, and increased efficiency in the online examination process. One of the central features of the proposed solution metaExam is to leverage cryptographic protocols via blockchain to control data access, making verification faster and protecting against misuse. Exam scores and grades are stored on a blockchain ledger using a digital signature method to enhance security. We validated the proposed solution by testing a prototype on the Ethereum platform using the Sepolia Testnet network using Microsoft Windows environment. Evaluation results indicate (i) query response time (10–50 ms), (ii) and query execution performance (CPU utilization between 1%–5%) offering computationally feasible solution. This research contributes by integrating blockchain and metaverse technologies to offer a solution metaExam that can offer improved security and immersive user experience for exam management. The proposed solution and its validation can provide insights into transforming online exams, offering a fresh perspective on addressing concerns about exam grade authenticity and verifying academic credentials in EduTech.

Subsampling Suffices for Adaptive Data Analysis

Ensuring that analyses performed on a dataset are representative of the entire population is one of the central problems in statistics. Most classical techniques assume that the dataset is independent of the analyst’s query and break down in the common setting where a dataset is reused for multiple, adaptively chosen, queries. This problem of adaptive data analysis was formalized in the seminal works of Dwork et al. (STOC, 2015) and Hardt and Ullman (FOCS, 2014). We identify a remarkably simple set of assumptions under which the queries will continue to be representative even when chosen adaptively: The only requirements are that each query takes as input a random subsample and outputs few bits. This result shows that the noise inherent in subsampling is sufficient to guarantee that query responses generalize. The simplicity of this subsampling-based framework allows it to model a variety of real-world scenarios not covered by prior work. In addition to its simplicity, we demonstrate the utility of this framework by designing mechanisms for two foundational tasks, statistical queries and median finding. In particular, our mechanism for answering the broadly applicable class of statistical queries is both extremely simple and state of the art in many parameter regimes.

A conceptual data modeling framework with four levels of abstraction for environmental information

Environmental data generated by observation infrastructures and models is widely heterogeneous in both structure and semantics. The design and implementation of an ad hoc data model for each new dataset is costly and creates barriers for data integration. On the other hand, designing a single data model that supports any kind of environmental data has shown to be a complex task, and the resulting tools do not provide the required efficiency. In this paper, a new data modeling framework is proposed that enables the reuse of generic structures among different application domains and specific applications. The framework considers four levels of abstraction for the data models. Levels 1 and 2 provide general data model structures for environmental data, based on those defined by the Observations and Measurements (O&M) standard of the Open Geospatial Consortium (OGC). Level 3 incorporates generic data models for different application areas, whereas specific application models are designed at Level 4, reusing structures of the previous levels. Various use cases were implemented to illustrate the capabilities of the framework. A performance evaluation using six datasets of three different use cases has shown that the query response times achieved over the structures of Level 4 are very good compared to both ad hoc models and to a direct implementation of O&M in a Sensor Observation Service (SOS) tool. A qualitative evaluation shows that the framework fulfills a collection of general requirements not supported by any other existing solution.

Data Lakehouse: A survey and experimental study

Efficient big data management is a dire necessity to manage the exponential growth in data generated by digital information systems to produce usable knowledge. Structured databases, data lakes, and warehouses have each provided a solution with varying degrees of success. However, a new and superior solution, the data Lakehouse, has emerged to extract actionable insights from unstructured data ingested from distributed sources. By combining the strengths of data warehouses and data lakes, the data Lakehouse can process and merge data quickly while ingesting and storing high-speed unstructured data with post-storage transformation and analytics capabilities. The Lakehouse architecture offers the necessary features for optimal functionality and has gained significant attention in the big data management research community. In this paper, we compare data lake, warehouse, and lakehouse systems, highlight their strengths and shortcomings, identify the desired features to handle the evolving challenges in big data management and analysis and propose an advanced data Lakehouse architecture. We also demonstrate the performance of three state-of-the-art data management systems namely HDFS data lake, Hive data warehouse, and Delta lakehouse in managing data for analytical query responses through an experimental study.

Adaptive and Scalable Database Management with Machine Learning Integration: A PostgreSQL Case Study

The increasing complexity of managing modern database systems, particularly in terms of optimizing query performance for large datasets, presents significant challenges that traditional methods often fail to address. This paper proposes a comprehensive framework for integrating advanced machine learning (ML) models within the architecture of a database management system (DBMS), with a specific focus on PostgreSQL. Our approach leverages a combination of supervised and unsupervised learning techniques to predict query execution times, optimize performance, and dynamically manage workloads. Unlike existing solutions that address specific optimization tasks in isolation, our framework provides a unified platform that supports real-time model inference and automatic database configuration adjustments based on workload patterns. A key contribution of our work is the integration of ML capabilities directly into the DBMS engine, enabling seamless interaction between the ML models and the query optimization process. This integration allows for the automatic retraining of models and dynamic workload management, resulting in substantial improvements in both query response times and overall system throughput. Our evaluations using the Transaction Processing Performance Council Decision Support (TPC-DS) benchmark dataset at scale factors of 100 GB, 1 TB, and 10 TB demonstrate a reduction of up to 42% in query execution times and a 74% improvement in throughput compared with traditional approaches. Additionally, we address challenges such as potential conflicts in tuning recommendations and the performance overhead associated with ML integration, providing insights for future research directions. This study is motivated by the need for autonomous tuning mechanisms to manage large-scale, heterogeneous workloads while answering key research questions, such as the following: (1) How can machine learning models be integrated into a DBMS to improve query optimization and workload management? (2) What performance improvements can be achieved through dynamic configuration tuning based on real-time workload patterns? Our results suggest that the proposed framework significantly reduces the need for manual database administration while effectively adapting to evolving workloads, offering a robust solution for modern large-scale data environments.

Secured Public Health data Management Framework using the Internet of Things in a Smart City Environment

The Internet of Things (IoT) is being embraced and impacts various aspects of lives. Integrating IoT technology in the healthcare sector paves the way for continuous remote monitoring of individuals' health status. The healthcare (HC) sector has traditionally produced substantial data. Due to the implementation of the IoT, this quantity has significantly increased. Ensuring proper HC storage is imperative to derive meaningful insights from the data. This study presents a public health (PH) framework that utilizes cloudlets and IoT technology. The primary objective of this PH architecture is to facilitate the retrieval of real-time data by using cloudlets. The research provides and executes an HC data management system to store vast HC data and handle consumer queries to retrieve HC information. The efficacy of the proposed model has been evaluated in terms of data transmission time, energy usage, query response time, and data packet loss. The proposed model outperforms existing PH systems by comparing the assessment findings of the proposed approach with the performance of typical PH systems.

SRank: Guiding schema selection in NoSQL document stores

The rise of big data has led to a greater need for applications to change their schema frequently. NoSQL databases provide flexibility in organizing data and offer multiple choices for structuring and storing similar information. While schema flexibility speeds up initial development, choosing schemas wisely is crucial, as they significantly impact performance, affecting data redundancy, navigation cost, data access cost, and maintainability. This paper emphasizes the importance of schema design in NoSQL document stores. It proposes a model to analyze and evaluate different schema alternatives and suggest the best schema out of various schema alternatives. The model is divided into four phases. The model inputs the Entity-Relationship (ER) model and workload queries. In the Transformation Phase, the schema alternatives are initially developed for each ER model, and subsequently, a schema graph is generated for each alternative. Concurrently, workload queries undergo conversion into query graphs. In the Schema Evaluation phase, the Schema Rank (SRank) is calculated for each schema alternative using query metrics derived from the query graphs and path coverage generated from the schema graphs. Finally, in the Output phase, the schema with the highest SRank is recommended as the most suitable choice for the application. The paper includes a case study of a Hotel Reservation System (HRS) to demonstrate the application of the proposed model. It comprehensively evaluates various schema alternatives based on query response time, storage efficiency, scalability, throughput, and latency. The paper validates the SRank computation for schema selection in NoSQL databases through an extensive experimental study. The alignment of SRank values with each schema's performance metrics underscores this ranking system's effectiveness. The SRank simplifies the schema selection process, assisting users in making informed decisions by reducing the time, cost, and effort of identifying the optimal schema for NoSQL document stores.

Generative artificial intelligence application enhancement in educational activities

In recent years, there has been an active introduction of artificial intelligence in education (AIEd). Recently, one of the most popular AI tools — ChatGPT — is an example of a generative AI technologies, which create new content (different types of text, image, audio, video) in response to a user’s request. On the example of ChatGPT, the article considers the possibilities of using AI in various spheres of educational activity and the ways of increasing its efficiency with the help of generative technologies. To structure the ways of interaction with ChatGPT during its integration into the educational process, the COST model, which describes the receipt and exchange of information in the learning process, is used. In order to utilize the capabilities of ChatGPT more fully and to compensate for its shortcomings, it is necessary to formulate appropriate queries, for example, using a concept tree. In addition, it is necessary to formulate queries in such a way that the concept under investigation is concretized as much as possible. As an example, the article presents fragments of the developed concept tree representing the composition of educational activity and interrelations between its elements, which can be the basis for making queries to a chatbot. The quality of the query response is determined by the completeness of the input information. The results of the research presented in the article will help to get more accurate and relevant answers to queries in the practical application of ChatGPT in the daily work of an educational institution by students and teachers as well as other stakeholders who are involved in the organization of the learning process.

Design and optimization of multidimensional data models for enhanced OLAP query performance and data analysis

This paper explores the design and optimization of multidimensional data models to enhance the query performance and data analysis capabilities of OLAP (Online Analytical Processing) systems. It delves into three prominent dimensional modeling techniques: Star Schema, Snowflake Schema, and Galaxy Schema, analyzing their impact on query complexity, data redundancy, storage requirements, and ease of maintenance. Additionally, it examines three aggregation strategiesPre-Aggregation, Dynamic Aggregation, and Hybrid Aggregationfocusing on their effectiveness in balancing query response time, storage efficiency, flexibility, and computational cost. The study further investigates performance optimization techniques, including query optimization, partitioning, and materialized views, providing case studies and experimental data to illustrate their benefits and challenges. The findings underscore the importance of tailored optimization strategies in OLAP systems to meet varying business needs and query patterns, highlighting the trade-offs between performance gains, storage requirements, and implementation complexity

ChatGPT and Its Role in Academic Libraries: A Discussion

The paper aims to discuss ChatGPT and its role in academic libraries. It identifies the key areas where ChatGPT can be implemented in libraries to improve various services and the overall work experience of academics, along with the risk factors associated with ChatGPT. Research finds that ChatGPT can be aided with many library services such as collection development management, virtual reference services, digital information service and library discovery, research writing and publishing, research performance analysis and bibliometric services. It can be helpful for libraries to meet patron requirements, streamline the research process, and increase the efficiency of the library and staff. It reduces the workload of library staff to a certain extent rather than replacing the human librarians as various threats are associated with the ChatGPT, such as incorrect query responses, data protection, misuse, privacy, security, inaccessibility, limited technology comprehension and other ethical issues. It also discusses the limitations and challenges libraries may face while implementing the technology to impart various services. The importance of this study lies in the fact that it acknowledges the reality of ChatGPT and its limitations and how the library will be ready to cope with and deal with the technology in the future.

Data quality and timeliness analysis for post-vaccination adverse event cases reported through healthcare data exchange to FDA BEST pilot platform.

This study is part of the U.S. Food and Drug Administration (FDA)'s Biologics Effectiveness and Safety (BEST) initiative, which aims to improve the FDA's postmarket surveillance capabilities by using real-world data (RWD). In the United States, using RWD for postmarket surveillance has been hindered by the inability to exchange clinical data between healthcare providers and public health organizations in an interoperable format. However, the Office of the National Coordinator for Health Information Technology (ONC) has recently enacted regulation requiring all healthcare providers to support seamless access, exchange, and use of electronic health information through the interoperable HL7 Fast Healthcare Interoperability Resources (FHIR) standard. To leverage the recent ONC changes, BEST designed a pilot platform to query and receive the clinical information necessary to analyze suspected AEs. This study assessed the feasibility of using the RWD received through the data exchange of FHIR resources to study post-vaccination AE cases by evaluating the data volume, query response time, and data quality. The study used RWD from 283 post-vaccination AE cases, which were received through the platform. We used descriptive statistics to report results and apply 322 data quality tests based on a data quality framework for EHR. The volume analysis indicated the average clinical resources for a post-vaccination AE case was 983.9 for the median partner. The query response time analysis indicated that cases could be received by the platform at a median of 3 min and 30 s. The quality analysis indicated that most of the data elements and conformance requirements useful for postmarket surveillance were met. This study describes the platform's data volume, data query response time, and data quality results from the queried postvaccination adverse event cases and identified updates to current standards to close data quality gaps.

Efficient Heuristic Management of Cloud Data Lakes for Enhanced Big Data Handling in Cloud Computing Environments

Efficient control of cloud data lakes is important for improving big data dealing in cloud computing settings. Cloud data lakes serve as stores for vast amounts of organized, semi-structured, and unstructured data from varied sources, offering scaling and freedom in data storage and processing. However, the sheer amount and range of data offer difficulties in improving storage, access, and analysis processes. This study suggests a heuristic method to handle these issues by improving the order and recovery of data within cloud data lakes. The heuristic method focuses on automatically handling data splitting, storage, and caching strategies based on data access trends and task factors. By leveraging heuristic principles, such as adaptable splitting and predictive caching, the suggested method aims to reduce delay and improve total query speed. Key components of the heuristic management include adaptable splitting methods that actively change data partition sizes based on access frequency and data distribution patterns. This flexible method ensures that frequently accessed data is easily available, while less accessed data is efficiently saved to improve storage usage. Furthermore, predictive storing methods expect future data access patterns using machine learning models or previous data trends. By strategically saving relevant datasets in memory or high-speed storage, the system reduces delay for repeated questions, thereby improving real-time data processing capabilities. The success of the suggested heuristic management method is tested through models and benchmarks using real big data tasks. Comparative research against standard static splitting and caching methods shows significant changes in query response times and resource utilization efficiency.

ChatGPT-3.5 Versus Google Bard: Which Large Language Model Responds Best to Commonly Asked Pregnancy Questions?

Large language models (LLM) have been widely used to provide information in many fields, including obstetrics and gynecology. Which model performs best in providing answers to commonly asked pregnancy questions is unknown. A qualitative analysis of Chat Generative Pre-Training Transformer Version 3.5 (ChatGPT-3.5) (OpenAI, Inc., San Francisco, California, United States) and Bard, recently renamedGoogle Gemini (Google LLC, Mountain View, California, United States), was performed in August of 2023. Each LLM was queried on 12 commonly asked pregnancy questions and asked for their references. Review and grading of the responses and references for both LLMs were performed by the co-authors individually and then as a group to formulate a consensus. Query responses were graded as "acceptable" or "not acceptable" based on correctness and completeness in comparison to American College of Obstetricians and Gynecologists (ACOG) publications, PubMed-indexed evidence, and clinical experience. References were classified as "verified," "broken," "irrelevant," "non-existent," and "no references." Grades of "acceptable" were given to 58% of ChatGPT-3.5 responses (seven out of 12) and 83% of Bard responses (10 out of 12). In regard to references, ChatGPT-3.5 had reference issues in 100% of its references, and Bard had discrepancies in 8% of its references (one out of 12). When comparing ChatGPT-3.5 responses between May 2023 and August 2023, a change in "acceptable" responses was noted: 50%versus 58%, respectively. Bard answered more questions correctly than ChatGPT-3.5 when queried on a small sample of commonly asked pregnancy questions. ChatGPT-3.5 performed poorly in terms of reference verification. The overall performance of ChatGPT-3.5 remained stable over time, with approximately one-half of responses being "acceptable" in both May and August of 2023. Both LLMs need further evaluation and vetting before being accepted as accurate and reliable sources of information for pregnant women.

Operator parallel optimization strategy for distributed databases

With the continuous development of network technology, the scale of data has shown explosive growth, which leads gradually to replacing traditional single machine databases with distributed databases. Distributed databases solve large-scale data storage problems through collaborative work among nodes, but due to increased communication costs between nodes, its query efficiency is not as good as a standalone database. In a distributed architecture, the data of storage nodes is only used as redundancy for multiple backups, providing data recovery in case of system failure, and it is not utilized to improve query efficiency. In response to the above issues, this article proposes an operator parallel optimization strategy for distributed databases. By splitting key physical operators, the split sub requests are evenly distributed to multiple nodes in the storage layer, which are processed in parallel by multiple nodes, thereby reducing query response time. The above strategy has been applied on distributed database CBase, and experiments have shown that the parallel optimization strategy proposed in this paper can significantly shorten SQL request query time and improve system resource utilization.

Conditional heavy hitter monitoring and application of heterogeneous graph streams based on sketches

Graph streams offer valuable insights into real network evolution in the era of big data. Heavy hitters and conditional heavy hitters focus on identifying entities with significant network traffic, revealing network data characteristics. In this paper, we present two methods, BLSketch and MHLSketch, for identifying heavy hitters and conditional heavy hitters in heterogeneous graph streams. BLSketch uses the bidirectional mapper, Bimap, for efficient query response times while retaining candidate information. MHLSketch uses min heap to preserve a minimal set of candidates to minimize storage space, excelling in space efficiency. The two methods can be incorporated with any sketch based graph summarization method, showing excellent adaptability. Experimental results on four real-world heterogeneous datasets of varying scales with the largest dataset containing millions of edges demonstrate our methods significantly improve the quality of query answering, outperforming state-of-the-art methods, with precision close to 1 and average relative error close to 0. We further combine our methods with density clustering to introduce the SDC model, which efficiently identifies hot regions and key users in Location-Based Social Networks (LBSN). In experiments, SDC outperforms traditional clustering by reducing up to 93.1% of cold Points of Interest (POIs) and shows strong potential for practical applications.

ChatGPT Compared With Google Bard: Which Large Language Model Responds Best to Commonly Asked Pregnancy Questions? [ID 2683501

INTRODUCTION: Large language models (LLMs) have proven themselves in a variety of settings and shown the ability to provide information on a variety of topics in obstetrics and gynecology. Which platform performs best in regard to its ability to respond to commonly asked pregnancy questions is unknown. METHODS: A qualitative analysis of ChatGPT and Google Bard was performed in August of 2023. We queried each LLM on 12 commonly asked pregnancy questions and asked for their references. Query responses were graded as “acceptable” or “not acceptable” based on correctness and completeness in comparison to American College of Obstetricians and Gynecologists (ACOG) publications, PubMed-indexed evidence, and clinical experience. References were classified as “verified,” “broken,” “irrelevant,” “non-existent,” and “no references.” Review and grading of the responses and references for both LLMs were performed by the co-authors individually and then as a group to formulate a consensus. RESULTS: A grade of acceptable was given to 58% of ChatGPT’s responses (7 out of 12) and 83% of Google Bard’s responses (10 out of 12). A grade of not acceptable was assigned to 42% of ChatGPT’s responses (2 were incomplete and 3 were incorrect) and 17% of Google Bard’s responses (2 incomplete). In regard to references, ChatGPT had reference issues in 100% of its references (12 out of 12). No references were provided for 10 answers, one answer had a non-existent reference, and another answer had two broken references and two verified references. Google Bard had discrepancies in 8.3% of its references (1 out of 12): 11 answers had verified references, and 1 answer had a broken reference. CONCLUSION: Google Bard has superior response performance regarding content and references when queried on commonly asked pregnancy questions. Both LLMs must be carefully evaluated and vetted prior to being accepted as accurate and reliable for this purpose.

Fast and exact fixed-radius neighbor search based on sorting.

Fixed-radius near neighbor search is a fundamental data operation that retrieves all data points within a user-specified distance to a query point. There are efficient algorithms that can provide fast approximate query responses, but they often have a very compute-intensive indexing phase and require careful parameter tuning. Therefore, exact brute force and tree-based search methods are still widely used. Here we propose a new fixed-radius near neighbor search method, called SNN, that significantly improves over brute force and tree-based methods in terms of index and query time, provably returns exact results, and requires no parameter tuning. SNN exploits a sorting of the data points by their first principal component to prune the query search space. Further speedup is gained from an efficient implementation using high-level basic linear algebra subprograms (BLAS). We provide theoretical analysis of our method and demonstrate its practical performance when used stand-alone and when applied within the DBSCAN clustering algorithm.

Assessing the appropriateness and completeness of ChatGPT-4's AI-generated responses for queries related to diabetic retinopathy.

To evaluate the appropriateness of responses generated by an online chat-based artificial intelligence (AI) model for diabetic retinopathy (DR) related questions. Cross-sectional study. A set of 20 questions framed from the patient's perspective addressing DR-related queries, such as the definition of disease, symptoms, prevention methods, treatment options, diagnostic methods, visual impact, and complications, were formulated for input into ChatGPT-4. Peer-reviewed, literature-based answers were collected from popular search engines for the selected questions and three retinal experts reviewed the responses. An inter-human agreement was analyzed for consensus expert responses and also between experts. The answers generated by the AI model were compared with those provided by the experts. The experts rated the response generated by ChatGPT-4 on a scale of 0-5 for appropriateness and completeness. The answers provided by ChatGPT-4 were appropriate and complete for most of the DR-related questions. The response to questions on the adverse effects of laser photocoagulation therapy and compliance to treatment was not perfectly complete. The average rating given by the three retina expert evaluators was 4.84 for appropriateness and 4.38 for completeness of answers provided by the AI model. This corresponds to an overall 96.8% agreement among the experts for appropriateness and 87.6% for completeness regarding AI-generated answers. ChatGPT-4 exhibits a high level of accuracy in generating appropriate responses for a range of questions in DR. However, there is a need to improvise the model to generate complete answers for certain DR-related topics.

GraphQL response data volume prediction based on Code2Vec and AutoML

PurposeGraphQL is a new Open API specification that allows clients to send queries and obtain data flexibly according to their needs. However, a high-complexity GraphQL query may lead to an excessive data volume of the query result, which causes problems such as resource overload of the API server. Therefore, this paper aims to address this issue by predicting the response data volume of a GraphQL query statement.Design/methodology/approachThis paper proposes a GraphQL response data volume prediction approach based on Code2Vec and AutoML. First, a GraphQL query statement is transformed into a path collection of an abstract syntax tree based on the idea of Code2Vec, and then the query is aggregated into a vector with the fixed length. Finally, the response result data volume is predicted by a fully connected neural network. To further improve the prediction accuracy, the prediction results of embedded features are combined with the field features and summary features of the query statement to predict the final response data volume by the AutoML model.FindingsExperiments on two public GraphQL API data sets, GitHub and Yelp, show that the accuracy of the proposed approach is 15.85% and 50.31% higher than existing GraphQL response volume prediction approaches based on machine learning techniques, respectively.Originality/valueThis paper proposes an approach that combines Code2Vec and AutoML for GraphQL query response data volume prediction with higher accuracy.