Vector Database Research Articles

DEVELOPMENT OF A QUESTION ANSWERING CHATBOT FOR BLOCKCHAIN DOMAIN

Large Language Models (LLMs), such as ChatGPT, have transformed the field of natural language processing with their capacity for language comprehension and generation of human-like, fluent responses for many downstream tasks. Despite their impressive capabilities, they often fall short in domain-specific and knowledge-intensive domains due to a lack of access to relevant data. Moreover, most state-of-art LLMs lack transparency as they are often accessible only through APIs. Furthermore, their application in critical real-world scenarios is hindered by their proclivity to produce hallucinated information and inability to leverage external knowledge sources. To address these limitations, we propose an innovative system that enhances LLMs by integrating them with an external knowledge management module. The system allows LLMs to utilize data stored in vector databases, providing them with relevant information for their responses. Additionally, it enables them to retrieve information from the Internet, further broadening their knowledge base. The research approach circumvents the need to retrain LLMs, which can be a resource-intensive process. Instead, it focuses on making more efficient use of existing models. Preliminary results indicate that the system holds promise for improving the performance of LLMs in domain-specific and knowledge-intensive tasks. By equipping LLMs with real-time access to external data, it is possible to harness their language generation capabilities more effectively, without the need to continually strive for larger models.

Comparing molecular representations, e-nose signals, and other featurization, for learning to smell aroma molecules.

Recent research has attempted to predict our perception of odorants using Machine Learning models. The featurization of the olfactory stimuli usually represents the odorants using molecular structure parameters, molecular fingerprints, mass spectra, or e-nose signals. However, the impact of the choice of featurization on predictive performance remains poorly reported in direct comparative studies. This paper experiments with different sensory features for several olfactory perception tasks. We investigate the multilabel classification of aroma molecules in odor descriptors. We investigate single-label classification not only in fine-grained odor descriptors ('orange', 'waxy', etc.), but also in odor descriptor groups. We created a database of odor vectors for 114 aroma molecules to conduct our experiments using a QCM (Quartz Crystal Microbalance) type smell sensor module (Aroma Coder®V2 Set). We compare these smell features with different baseline features to evaluate the cluster composition, considering the frequencies of the top odor descriptors carried by the aroma molecules. Experimental results suggest a statistically significant better performance of the QCM type smell sensor module compared with other baseline features with F1 evaluation metric.

Statistical Considerations When Measuring Absolute Reliability And Variability Of Vector Data In Sport Performance

Vector-based data in sport performance include a magnitude and direction. Statistically speaking, they are interval in nature as they may be positive or negative. The coefficient of variation (CV) is a commonly reported measure of variability, but its use with vector data is questionable and may be contraindicated. Limits of agreement (LOA) and standard error of measurement (SEM) may be better alternatives for vector data such as acceleration. The purpose of this study was to demonstrate the issues with quantifying variability of vector data, while also evaluating the utility of commonly used measures. Acceleration data at three intervals from 0 to 27.4 m (0 to 90 ft) were calculated from publicly available sprint performance data from 310 athletes participating in the 2018 and 2019 Major League Baseball seasons. CV, LOA, and SEM were calculated to evaluate inter-season variability. Variability of the first two intervals was acceptable for all measures, but the final interval was unclear as the CV was quite large (50.78%), while the LOA and SEM were only slightly larger than the other interval values. The final interval includes both positive and negative acceleration, contraindicating the usage of the CV. LOA and SEM are more useful for vector data, showing that the final interval was more variable between trials than the others, but not to the extent portrayed by the CV. The CV likely should not be used with vector data unless it is known that the data does not cross zero. LOA and SEM are appealing alternatives for the CV and should be considered since they work with positive and negative data.

Integrity assured multi-functional multi-application secure data aggregation in wireless sensor networks (IAMFMA-SDA)

Industrial revolutions and demand of novel applications drive the development of sensors which offer continuous monitoring of remote hostile areas by collecting accurate measurement of physical phenomena. Data aggregation is considered as one of the significant energy-saving mechanism of resource constraint Wireless Sensor Networks (WSNs) which reduces bandwidth consumption by eliminating redundant data. Novel applications demand WSN to provide information about the monitoring region in multiple aspects in large scale. To meet this requirement, different kinds of sensors of different parameters are deployed in the same region which in turn demands the aggregator node to integrate diverse data in a smooth and secure manner. Novelty in applications also requires Base station (BS) to apply multiple statistical functions. Hence, we propose to develop a novel secure cost-efficient data aggregation scheme based on asymmetric privacy homomorphism to aggregate data of multiple parameters and facilitate the BS to compute multiple functions in one round of data collection by providing elaborated view of monitoring region. To meet the claim of large scale WSN which requires dynamic change in size, vector-based data collection method is adopted in our proposed scheme. The security aspect is strengthened by allowing BS to verify the authenticity of source node and validity of data received. The performance of the system is analyzed in terms of computation and communication overhead using the mathematical model and simulation results.

Morphotectonic and morphostructural investigation in northwestern Minas Gerais State, Brazil: a lineament mapping assessment

This study aimed to investigate whether linear natural features are associated with structural or tectonic control of landforms and current drainage patterns in northwestern Minas Gerais State (Southeastern Brazil). To answer this question, more than 1,700 lineaments were manually mapped in a broader area encompassing the Paracatu and Urucuia River catchments. One satellite image scene (Operational Land Imager/Landsat-8), three different digital elevation models (Shuttle Radar Topography Mission, Advanced Spaceborne Thermal Emission and Reflection Radiometer-Global digital elevation model, and Advanced Land Observing Satellite-Phased Array L-band Synthetic Aperture Radar), and digitised analogical maps (geologic and topographic maps) were used to draw linear geometric features representing the lineament population. The resulting vector databases were integrated in a Geographic Information System for quantitative and spatial analyses, which resulted in two main trending lineament sets: NNW-SSE and NE-SW. Subsequent field assessments revealed that numerous lineaments from the first set were correlated to the formerly Upper Precambrian reverse and strike-slip faults. The second set were correlated to NE-SW normal faults derived from the Mesozoic-Cenozoic tectonic event following the breakup of Gondwana. These datasets contribute to a better understanding of the regional-, sub-regional-, and catchment-scale features underlying the structural control of the drainage network arrangement and pattern, which present long straight channels and are interrupted by anomalous and prominent bends that strictly fit major structural directions.

Comparison of different spatial temperature data sources and resolutions for use in understanding intra-urban heat variation

In this study, we investigate the compatibility of specific vulnerability indicators and heat exposure data and the suitability of spatial temperature-related data at a range of resolutions, to represent spatial temperature variations within cities using data from Atlanta, Georgia. For this purpose, we include various types of known and theoretically based vulnerability indicators such as specific street-level landscape features and urban form metrics, population-based and zone-based variables as predictors, and different measures of temperature, including air temperature (as vector-based data), land surface temperature (at resolution ranges from 30 m to 305 m), and mean radiant temperature (at resolution ranges from 1 m to 39 m) as dependent variables. Using regression analysis, we examine how different sets of predictors and spatial resolutions can explain spatial heat variation. Our findings suggest that the lower resolution of land surface temperature data, up to 152 m, and mean radiant temperature data, up to 15 m, may still satisfactorily represent spatial urban temperature variation caused by landscape elements. The results of this study have important implications for heat-related policies and planning by providing insights into the appropriate sets of data and relevant resolution of temperature measurements for representing spatial urban heat variations.

Application of Data Collected by Endpoint Detection and Response Systems for Implementation of a Network Security System based on Zero Trust Principles and the EigenTrust Algorithm

Traditionally, security systems for enterprises have implicit access based on strong cryptography, authentication and key sharing, wherein access control is based on Role Based Access Control (RBAC), in which roles such as manager, accountant and so on provide a way of deciding a subject's authority. However, years of post-attack analysis on enterprise networks has shown that a majority of times, security breaches occur intentionally or accidently due to implicitly trusted people of an enterprise itself. Zero Trust Architecture works on the principle of never granting trust implicitly, but rather continuously evaluating the trust parameters for each resource access request and has a strict, but not rigid, set of protocols for access control of a subject to resources. Endpoint Detection and Response (EDR) systems are tools that collect a large number of attributes in and around machines within an enterprise network to have close visibility into sophisticated intrusion. In our work, we seek to deploy EDR systems and build trust algorithms using tactical provenance analysis, threshold cryptography and reputation management to continuously record data, evaluate trust of a subject, and simultaneously analyze them against a database of known threat vectors to provide conditional access control. However, EDR tools generate a high volume of data that leads to false alarms, misdetections and correspondingly a high backlog of tasks that makes it infeasible, which is addressed using tactical provenance analysis and information theory.

SkyWords: An automatic keyword extraction system based on the skyline operator and semantic similarity

This study presents a hybrid keyword extraction method called SkyWords. It implements a novel supervised step based on the skyline operator and the majority voting principle for high-quality candidate keyword selection and an unsupervised step based on contextual semantics for the candidate keyword ranking. To achieve this, firstly, we build a feature vector database using the features of known keywords and then apply the skyline operator to retrieve the dominating feature vectors. To select the candidate keywords of a document, we compare each word of the document against the dominating feature vectors and assume the words that are as good as the majority of the dominating feature vectors as candidate keywords. To obtain the final set of keywords, we rank the candidate keywords based on their semantic similarity to the document based on their vector representation using the MPNet sentence transformer. We conducted experiments on six benchmark scholarly datasets to evaluate the performance of SkyWords and compared the results against eleven baseline keyword extraction systems. The experimental results show that the proposed novel keyword selection algorithm reduced the number of candidate keywords by several folds. Moreover, SkyWords achieved statistically significant improvements over the baseline methods in precision, recall, and F1 score. Compared to the baseline regarding ranking-based metrics, SkyWords achieved the highest mean average precision score for all datasets and the highest mean reciprocal rank score for all datasets but one. Furthermore, SkyWords extracted more relevant keywords than the baseline methods.

Establishing semantic relatedness through ratings, reaction times, and semantic vectors: A database in Polish.

This study presents a Polish semantic priming dataset and semantic similarity ratings for word pairs obtained with native Polish speakers, as well as a range of semantic spaces. The word pairs include strongly related, weakly related, and semantically unrelated word pairs. The rating study (Experiment 1) confirmed that the three conditions differed in semantic relatedness. The semantic priming lexical decision study with a carefully matched subset of the stimuli (Experiment 2), revealed strong semantic priming effects for strongly related word pairs, whereas weakly related word pairs showed a smaller but still significant priming effect relative to semantically unrelated word pairs. The datasets of both experiments and those of SimLex-999 for Polish were then used in a robust semantic model selection from existing and newly trained semantic spaces. This database of semantic vectors, semantic relatedness ratings, and behavioral data collected for all word pairs enable future researchers to benchmark new vectors against this dataset. Furthermore, the new vectors are made freely available for researchers. Although similar semantically strongly and weakly related word pairs are available in other languages, this is the first freely available database for Polish, that combines measures of semantic distance and human data.

Analiza zmian zagospodarowania osiedla Kozanów we Wrocławiu w latach 1934–2020 na podstawie wybranych źródeł kartograficznych

The purpose of the work was to conduct the analysis covering the area of Kozanów neighborhood before and after the occurrence of the commonly named “Millennium flood” in 1997. The scope of the work included the analysis of four time states presented on old maps from 1934, 1979 and 1997 and a fragment of the OpenStreetMap database (OSM) current for 2020. The research included quantitative analyses, consisting of comparing all feature classes among the analyzed time states, as well as qualitative analyses, relating to the detailed characteristics of the maps and the OSM database. In order to ensure the comparability of all types of land use, old maps were georeferenced, and then the data contained on the maps were vectorized, which allowed to obtain a universal vector database for all tested time states. This enabled the design of maps of the Kozanów neighborhood for individual years and maps of changes presenting quantitative changes by feature classes. The output maps show an increase in the area of buildings and length of roads as well as a decrease of green areas and wastelands. The results of the work allowed us to draw conclusions regarding the severe negative effects of the flood that occurred in this neighborhood and the evaluation of the use of the OSM database in such analyses.

Results of Field and Laboratory Studies of Carriers and Vectors of Natural-Focal Infections on the Territory of the Republic of Armenia

The purpose of the study was to assess the current epizootic potential of the Transcaucasian high-mountain and Pre-Araks low-mountain natural plague foci on the territory of the Republic of Armenia using GIS technologies. Materials and methods. We used the data from an epizootiological survey, records of the abundance and species composition, spatial distribution of rodents and ectoparasites in the plague-enzootic territories of the Republic of Armenia in 2021. Results and discussion. Based on the results of the research, an electronic database of carriers and vectors of pathogens of natural-focal zoonotic infections in the plague-enzootic territories of the Republic of Armenia has been created. Applying GIS technologies, an assessment of the spatial distribution of carriers and vectors of plague has been made and areas of circulation of tularemia and leptospirosis pathogens identified. The results obtained serve as the basis for improving the efficiency of planning and carrying out preventive measures aimed at ensuring the epidemiological welfare as regards natural-focal infectious diseases in the territory of the Republic of Armenia.

Attack Identification for Cyber-Physical Security in Dynamic Games under Cognitive Hierarchy

This paper considers the problem of identifying the profiles and capabilities of attackers injecting adversarial inputs to a cyber-physical system. The system in question interacts with attackers of different levels of intelligence, each employing different feedback controllers against the system. Principles of behavioral game theory – specifically the concept of level-k thinking – is employed to construct a database of potential attack vectors. By observing the state trajectories under sequential interactions with different adversaries, the defender adaptively estimates both the number and profiles of the different attack signals using an online deterministic annealing approach. This information is used to dynamically estimate the level of intelligence of the attackers. Simulation results showcase the efficacy of the proposed method.

ORTHO AI : The Dawn Of A New Era: Artificial Intelligence In Orthopaedics

ORTHO AI : The Dawn Of A New Era: Artificial Intelligence In Orthopaedics

Biometric technology for personal identification and authentication: fingerprint

This paper presents an efficient fingerprint verification algorithm that improves matching accuracy. Algorithms for image processing for biometric technologies of identification and authentication of a person using fingerprints are considered. They are essential tools for improving performance across a wide range of industries. The recognition system is designed for storing data, further processing it, identifying and displaying fingerprints. In forensics, modern image processing algorithms are used and provide a significant improvement in work results. Fingerprint images are degraded and damaged due to changes in skin and print conditions. Thus, image enhancement techniques are used before feature points are detected and fine details are extracted. Templates are generated based on feature vector extraction and databases are created for verification and identification of fingerprint images. The method of experimental research is carried out, the process of processing the identification results is considered. To solve this problem, a recognition system was proposed, which is designed to store data, further process it, identify and display fingerprints. They are used in places of high secrecy as a kind of password access key based on scanning and verifying fingerprints with the FVC2002 database. The results obtained show that the proposed algorithm is more efficient and reliable.

Urban Heat Island Mapping Based on a Local Climate Zone Classification : a Case Study in Strasbourg City, France

The Urban Heat Island (UHI) effect is characterised by higher temperatures in cities than in rural surroundings. This phenomenon leads to increased health risks in urban dwellers, particularly in the context of global climate change. It is essential to consider its spatial variability to propose efficient UHI mitigation strategies. The Local Climate Zone (LCZ) scheme is a climate-based classification that can accurately capture UHI intensities according to the urban area characteristics. In this study, an LCZ classification has been established for the city of Strasbourg by using a vector-based method that relies on a large vector database composed of land cover and cadastral parcels data. LCZ polygons are digitized from cadastral maps, then the different LCZ parameters are calculated for each of them. Six of the ten LCZ parameters proposed in the literature have been obtained. New criteria have been added to improve the classification, i.e. a vegetation parameter (VgSF) and a compactness index (CI). A given LCZ class is then assigned to each polygon using a trapezoidal fuzzy logic model, which is completed by a decision tree. The acquired final LCZ map shows that the developed vector-based method permits obtaining relevant LCZ classification. The LCZ parameters values are subsequently used to determine a multiple linear regression (MLR) aiming to get a UHI intensity for each LCZ polygon. The resulting UHI map of the Eurométropole de Strasbourg (EMS) accurately illustrates the strong spatial heterogeneity of the phenomenon.

Place Recognition with Memorable and Stable Cues for Loop Closure of Visual SLAM Systems

Visual Place Recognition (VPR) is a fundamental yet challenging task in Visual Simultaneous Localization and Mapping (V-SLAM) problems. The VPR works as a subsystem of the V-SLAM. VPR is the task of retrieving images upon revisiting the same place in different conditions. The problem is even more difficult for agricultural and all-terrain autonomous mobile robots that work in different scenarios and weather conditions. Over the last few years, many state-of-the-art methods have been proposed to solve the limitations of existing VPR techniques. VPR using bag-of-words obtained from local features works well for a large-scale image retrieval problem. However, the aggregation of local features arbitrarily produces a large bag-of-words vector database, limits the capability of efficient feature learning, and aggregation and querying of candidate images. Moreover, aggregating arbitrary features is inefficient as not all local features equally contribute to long-term place recognition tasks. Therefore, a novel VPR architecture is proposed suitable for efficient place recognition with semantically meaningful local features and their 3D geometrical verifications. The proposed end-to-end architecture is fueled by a deep neural network, a bag-of-words database, and 3D geometrical verification for place recognition. This method is aware of meaningful and informative features of images for better scene understanding. Later, 3D geometrical information from the corresponding meaningful features is computed and utilised for verifying correct place recognition. The proposed method is tested on four well-known public datasets, and Micro Aerial Vehicle (MAV) recorded dataset for experimental validation from Victoria Park, Adelaide, Australia. The extensive experimental results considering standard evaluation metrics for VPR show that the proposed method produces superior performance than the available state-of-the-art methods.

Effects of Transport Corridor Advancement on Agglomeration and Industrial Relocation – Dallas Fort Worth (US) case study

Abstract Cities serve as hubs for various activities that necessitate comprehensive transportation connectivity. This study examines the decadal urban agglomeration patterns from 2001 to 2020 and critically assesses the relationship between freeway developments, industrial relocation, and population density in the DFW (Dallas Fort Worth) metropolitan area. Landsat satellite imageries, US census, and open-source GIS datasets have been utilized in the study. The Maximum Likelihood Classification (MLC) algorithm helped generate the vector database, using which Land Use/Land Cover (LULC) variations were assessed. The calculated overall accuracies of the classified images for 2001, 2011, and 2020 were 93.12%, 91.87%, and 93.12%, respectively. Eventually, buffer generation techniques and summary statistics helped detect potential boom hotspots. Our results indicate that the highway advancement project lures industries, leading to population migration. The LULC variations suggest that the increase in highway infrastructure resulted in a surge in built-up and a decrease in open spaces in District-3 of DFW. From our study, we find that 79.16% of old industries are located near old freeways, while 78.84% of new industries are located near new freeways. Further, our industrial area to road area comparison clearly shows that industrial relocation was driven by transportation advancements over time. Our results also confirm that this relocation of industries fostered a massive population influx during the following decades.

Near duplicate detection of images with area and proposed pixel‐based feature extraction

SummaryEfficient and accurate near‐duplicate recognition is the trendy research area. Identification of invalid near‐duplicate images offers a wide range of applications, including digital picture forensics, web‐scale retrieval, and, social media analysis. This article intends to introduce a novel near duplicate detection model of images that consists of two stages such as (i) feature extraction and (ii) similarity computation. Originally, the image database is subjected to extracting the features, in which the area‐based features and pixel‐based features are extracted. Here, the area‐based feature extraction includes the contrast context histogram (CCH‐descriptors) and improved weighted bag of visual word (w‐BovW) features; the pixel‐based feature extraction includes the texture features like the proposed local vector pattern. Once the query image is given as the input, it is subjected to the feature extraction stage. Then, the feature vector database and the extracted features of query images are evaluated under similarity computation via improved Jaccard similarity evaluation. Thus, the near duplicate detection of the image is obtained in an effective manner.

Manu

With the development of learning-based embedding models, embedding vectors are widely used for analyzing and searching unstructured data. As vector collections exceed billion-scale, fully managed and horizontally scalable vector databases are necessary. In the past three years, through interaction with our 1200+ industry users, we have sketched a vision for the features that next-generation vector databases should have, which include long-term evolvability, tunable consistency, good elasticity, and high performance. We present Manu, a cloud native vector database that implements these features. It is difficult to integrate all these features if we follow traditional DBMS design rules. As most vector data applications do not require complex data models and strong data consistency, our design philosophy is to relax the data model and consistency constraints in exchange for the aforementioned features. Specifically, Manu firstly exposes the write-ahead log (WAL) and binlog as backbone services. Secondly, write components are designed as log publishers while all read-only analytic and search components are designed as independent subscribers to the log services. Finally, we utilize multi-version concurrency control (MVCC) and a delta consistency model to simplify the communication and cooperation among the system components. These designs achieve a low coupling among the system components, which is essential for elasticity and evolution. We also extensively optimize Manu for performance and usability with hardware-aware implementations and support for complex search semantics. Manu has been used for many applications, including, but not limited to, recommendation, multimedia, language, medicine and security. We evaluated Manu in three typical application scenarios to demonstrate its efficiency, elasticity, and scalability.

A geometrical approach to matching raster & vector databases of buildings

ABSTRACT This study outlines a new approach to matching raster and vector datasets using 3D geometries . The method includes several procedures based on the similarity of selected pixels of an image to 3D polygons. The method provides four results of matching for each pixel with respect to the polygons The results of the application to a set of buildings included in a DSM and polygons digitised from a city model have demonstrated the viability of the method for matching features. The results have shown that the method is efficient for conflating tasks such as integrating data, updating and change detection.