Database Tuples Research Articles

Research on blind reversible database watermarking algorithm based on dual embedding strategy

Abstract Massive databases encounter security such as data theft, illegal copying, and copyright infringement during creating, transmitting, and sharing of big data. The reversible data watermarking technology can effectively solve these problems, which can extract the watermark information accurately and recover the original carrier data without any distortion. However, most existing methods extract watermark information non-blindly and cannot effectively achieve a balance between watermark embedding capacity and data distortion. This paper proposes a blind reversible database watermarking method based on dual embedding, which combines histogram shifting and distortion-free watermarking methods to achieve an adaptive selection of histogram bins, blind extraction of watermark information, and carrier data recovery. The proposed method preprocesses the database tuples by scrambling them and constructs a prediction error histogram using first-layer tuples in square prediction within each group. The watermark information is embedded through adaptive selection and expansion of histogram bins, while the distortion-free watermarking method is used in another layer to assist in the recovery of original carrier data. The experimental results show that the proposed method can achieve an embedding capacity of more than three times the capacity of existing methods. It can also achieve blind watermark extraction and outperform some other state-of-the-art methods.

Database Repairing with Soft Functional Dependencies

A common interpretation of soft constraints penalizes the database for every violation of every constraint, where the penalty is the cost (weight) of the constraint. A computational challenge is that of finding an optimal subset: a collection of database tuples that minimizes the total penalty when each tuple has a cost of being excluded. When the constraints are strict (i.e., have an infinite cost), this subset is a “cardinality repair” of an inconsistent database; in soft interpretations, this subset corresponds to a “most probable world” of a probabilistic database, a “most likely intention” of a probabilistic unclean database, and so on. Within the class of functional dependencies, the complexity of finding a cardinality repair is thoroughly understood. Yet, very little is known about the complexity of finding an optimal subset for the more general soft semantics. The work described in this manuscript makes significant progress in that direction. In addition to general insights about the hardness and approximability of the problem, we present algorithms for two special cases (and some generalizations thereof): a single functional dependency, and a bipartite matching. The latter is the problem of finding an optimal “almost matching” of a bipartite graph where a penalty is paid for every lost edge and every violation of monogamy. For these special cases, we also investigate the complexity of additional computational tasks that arise when the soft constraints are used as a means to represent a probabilistic database in the case of a probabilistic unclean database.

BopSkyline: Boosting privacy-preserving skyline query service in the cloud

With the widespread adoption of cloud computing, there has been great popularity of storing and querying databases in the cloud. However, such service outsourcing also entails critical data privacy concerns, as the cloud providers are generally not in the same trust domain as the data owners/users and could even suffer from data breaches. In this paper, different from most existing works that propose security designs for keyword search, we focus on secure realizations of advanced skyline query processing, which plays an important role in multi-criteria decision support applications. We propose BopSkyline, a new system framework for privacy-preserving skyline query service in cloud computing. BopSkyline is designed to not only ensure the confidentiality of outsourced databases, skyline queries, and query results, but also conceal data patterns (like the dominance relationships among database tuples) and search access patterns that may indirectly lead to data leakages. Notably, through a delicate synergy of key ideas on secure database shuffling and differentially private database padding, BopSkyline achieves a significant performance boost over the state-of-the-art. Extensive experiments demonstrate that compared with the state-of-the-art prior work, BopSkyline is up to 4.7× better in query latency and achieves up to 99.38% cost savings in communication.

How Large Language Models Will Disrupt Data Management

Large language models (LLMs), such as GPT-4, are revolutionizing software's ability to understand, process, and synthesize language. The authors of this paper believe that this advance in technology is significant enough to prompt introspection in the data management community, similar to previous technological disruptions such as the advents of the world wide web, cloud computing, and statistical machine learning. We argue that the disruptive influence that LLMs will have on data management will come from two angles. (1) A number of hard database problems, namely, entity resolution, schema matching, data discovery, and query synthesis, hit a ceiling of automation because the system does not fully understand the semantics of the underlying data. Based on large training corpora of natural language, structured data, and code, LLMs have an unprecedented ability to ground database tuples, schemas, and queries in real-world concepts. We will provide examples of how LLMs may completely change our approaches to these problems. (2) LLMs blur the line between predictive models and information retrieval systems with their ability to answer questions. We will present examples showing how large databases and information retrieval systems have complementary functionality.

A Graph-Based Blocking Approach for Entity Matching Using Contrastively Learned Embeddings

Data integration is considered a crucial task in the entity matching process. In this process, redundant and cunning entries must be identified and eliminated to improve the data quality. To archive this, a comparison between all entities is performed. However, this has quadratic computational complexity. To avoid this, `blocking' limits comparisons to probable matches. This paper presents a k-nearest neighbor graph-based blocking approach utilizing state-of-the-art context-aware sentence embeddings from pre-trained transformers. Our approach maps each database tuple to a node and generates a graph where nodes are connected by edges if they are related. We then invoke unsupervised community detection techniques over this graph and treat blocking as a graph clustering problem. Our work is motivated by the scarcity of training data for entity matching in real-world scenarios and the limited scalability of blocking schemes in the presence of proliferating data. Additionally, we investigate the impact of contrastively trained embeddings on the above system and test its capabilities on four data sets exhibiting more than 6 million comparisons. We show that our block processing times on the target benchmarks vary owing to the efficient data structure of the k-nearest neighbor graph. Our results also show that our method achieves better performance in terms of F1 score when compared to current deep learning-based blocking solutions.

DrTM+B: Replication-Driven Live Reconfiguration for Fast and General Distributed Transaction Processing

Recent in-memory database systems leverage advanced hardware features like RDMA to provide transaction processing at millions of transactions per second. Distributed transaction processing systems can scale to even higher rates, especially for partitionable workloads. Unfortunately, it is challenging to sustain such high rates during live reconfiguration of partitions. In this article, we observe that state-of-the-art approaches would cause notable performance disruption under fast transaction processing. To this end, this article presents DrTM+B, a live reconfiguration approach that seamlessly repartitions data with little performance disruption to running transactions. DrTM+B uses a pre-copy-based mechanism to avoid excessive data transfer by leveraging common properties in recent transactional systems. DrTM+B's reconfiguration plans reduce data movement by preferring existing data replicas, while copying data from multiple replicas asynchronously and in parallel. It further reuses the log forwarding mechanism in primary-backup replication to seamlessly track and forward dirty database tuples and avoids iterative copying costs. To commit a reconfiguration plan in a transactional-safe way, DrTM+B designs a cooperative commit protocol for synchronization of data and state among replicas. To boost the performance during data migration, DrTM+B combines the pre-copy and post-copy schemes to propose a hybrid copy scheme. The live reconfiguration approach can also coexist with fault-tolerance mechanisms of primary-backup replication to provide high availability. Evaluation on a working system based on DrTM+R with 3-way replication using typical OLTP workloads like TPC-C and SmallBank shows that DrTM+B incurs only very small performance degradation during live reconfiguration and provides high availability. Both the reconfiguration time and the downtime are also minimal.

Process Approach and Construction of the Database for Non-Core Asset Management in Credit Organizations

A method for building end-to-end management accounting in a division of the Bank’s subdevision specializing in working with non-core assets is proposed. Has been proposed the process approach, an algorithm for building a database for the formation of key performance and control indicators. Has been described the key stages of the department's work, the attribute composition of entities (set) arriving, enriched and transmitted at each stage of the department's work. By modeling the process has been built a role model, access and editing rights for employees. Data sources (reference books) for optimization and unification of the process of filling the database (tuple) are proposed. A method of accessing the database in the Power Query Microsoft Excel add-in is proposed, which allows you to collect data from files of all basic data types, process and refine the received data. In the interactive programming environment Jupyter Notebook, mathematical and financial models for data analysis (logistic regression, decision tree and discounted cash flow method) were built based on data in order to predict costs, the timing of asset exposure and make a decision on the optimal cost of putting property on the Bank's balance sheet and selling price. Based on ready-made libraries (matpotlib, seaborn, plotly), options for data visualization for management are proposed. Using the example of the Bank's division, the author describes the positive effects and opportunities that open up to the management of different levels in solving day-to-day tasks and planning the activities of the division. A technical task was proposed for the development of a showcase for the sale of non-core assets on the Bank's website as an environment for the accumulation of external data for making flexible management decisions.

Query Games in Databases

Database tuples can be seen as players in the game of jointly realizing the answer to a query. Some tuples may contribute more than others to the outcome, which can be a binary value in the case of a Boolean query, a number for a numerical aggregate query, and so on. To quantify the contributions of tuples, we use the Shapley value that was introduced in cooperative game theory and has found applications in a plethora of domains. Specifically, the Shapley value of an individual tuple quantifies its contribution to the query. We investigate the applicability of the Shapley value in this setting, as well as the computational aspects of its calculation in terms of complexity, algorithms, and approximation.

A Robust and Reversible Watermarking Algorithm for a Relational Database Based on Continuous Columns in Histogram

Due to the discreteness of integer data, there are a large number of gaps and continuous columns in the histogram based on integer data. Aiming at the characteristics, this paper presents a robust and reversible watermarking algorithm for a relational database based on continuous columns in histogram. Firstly, it groups the database tuples according to the watermark length and the grouping key. Secondly, it calculates the prediction errors and uses the absolute values of the prediction errors to construct the histogram. Thirdly, it traverses the histogram to find all the continuous columns and in turn, computes the sum of the height of each continuous column and selects the group of continuous columns that has the largest sum as the positions to embed the watermarks. FCTD (Forest cover type data set) is utilized for experimental verification. A large amount of experimental data shows that the method is effective and robust. Not only does the data distortion caused by shifting histogram columns not exist, but the robustness of the watermark is also greatly improved.

IDR Privacy Protection Based on Database Digital Watermarking

Background: In smart grid, a flexible demand response management mechanism is used to achieve the purpose of stabilizing the power grid, optimizing the power market, and rationally allocating resources. There are two types of demand response management in the demand response management mechanism: Price-based Demand Response (PDR) and Incentive-based Demand Response (IDR). Methods: The paper studied the problem of privacy protection in IDR, and proposed a method based on database digital watermark to protect user privacy. Segment the time, and then embed watermarks in the user’s consumption data of each time segment. At the end of each billing period, extract the watermarks from the data of each segment time, and then send the total consumption data of the user of this billing period to the power supply company. The power supply company only knows the total consumption data of the user, the company does not have any information regarding the users consumption data which can prevent them from snooping the user privacy. The proposed digital watermarking algorithm is based on K-Means clustering and wavelet transform, the K-Means algorithm is used to cluster the database tuple data, and then wavelet transform is carried out on the available attribute values within the clusters, and the watermark is embedded in the transformed attribute values. Results: The experimental results show that the proposed method is more robust when the database is under subset deletion attacks, subset substitution attacks and subset addition attacks. Besides, the computational cost is very low. Conclusion: The proposed digital watermark algorithm can embed the watermarks more decently and overcome the burden of watermark embedding caused by statistical feature control. Besides, the proposed method can protect the user privacy better than the other two methods.

An experimental survey of regret minimization query and variants: bridging the best worlds between top-k query and skyline query

When faced with a database containing millions of tuples, a user may be only interested in a (typically much) smaller representative subset. Recently, a query called the regret minimization query was proposed toward this purpose to create such a subset for users. Specifically, this query finds a set of tuples that minimizes the user regret (measured by how far the user’s favorite tuple in the selected set is from his/her favorite tuple in the whole database). The regret minimization query was shown to be very useful in bridging the best worlds between two existing well-known queries, top-k queries and skyline queries: Like top-k queries, the total number of tuples returned in this new query is controllable, and like skyline queries, this new query does not require a user to specify any preference function. Thus, it has attracted a lot of attention from researchers in the database community. Various methods were proposed for regret minimization. However, despite the abundant research effort, there is no systematic comparison among the existing methods. This paper surveys this interesting and evolving research topic by broadly reviewing and comparing the state-of-the-art methods for regret minimization. Moreover, we study different variants of the regret minimization query that has garnered considerable attention in recent years and present some interesting problems that have not yet been addressed in the literature. We implemented 12 state-of-the-art methods published in top-tier venues such as SIGMOD and VLDB from 2010 to 2018 for obtaining regret minimization sets and give an experimental comparison under various parameter settings on both synthetic and real datasets. Our evaluation shows that the optimal choice of methods for regret minimization depends on the application demands. This paper provides an empirical guideline for making such a decision.

A reversible database watermarking method with low distortion

In this paper, a low distortion reversible database watermarking method based on histogram gap is proposed in view of the large gap in histogram of database integer data. By using the method, the tolerance of the attribute column containing all integer data is firstly calculated and the prediction error is obtained according to the tolerance. Then according to the watermark bits to be embedded, the database tuples will be randomly grouped and the histogram can be constructed by using the prediction error. Finally, the histogram correction rule is used to find the histogram peak bin, the number of consecutive non-zero prediction errors on the left and right sides of the peak is obtained, and the histogram shift is performed on the side with a smaller number of non-zero prediction errors, and then the watermark embedding will be realized. The results of the experiments based on the published dataset of FCTD (Forest Cover Type Dataset) show that compared with the existing GAHSW which also considers distortion, the proposed method significantly reduces the number of histogram column shift while embedding the watermarks, greatly reduces the changes to the carrier data, and effectively reduces the database's data distortion caused by watermark embedding.

An Ameliorated Methodology for Ranking the Tuple

Many users searching databases through the web in various domains like vehicles, real estate, etc. One of the predicament, we observed in this task is ranking the results retrieved from a database for a user query. The contemporary methods are addressing this problem by sorting the database values. A common line in these methods is that ranking is done in a user and query-independent manner i.e. there is no correspondence between different users queries. We proposed a query and user-dependent approach for ranking query results in relational databases, which is articulating the opinion expressed in query by different users. The designed and developed methodology shows that the ranking of the data in database is constituted depending on the various user opinions i.e. access of tuple through users query expression. The design is based on counting the access of database tuple by users query(ies)

PrefDB: Supporting Preferences as First-Class Citizens in Relational Databases

In this paper, we argue that preference-aware query processing needs to be pushed closer to the DBMS. We introduce a preference-aware relational data model that extends database tuples with preferences and an extended algebra that captures the essence of processing queries with preferences. Based on a set of algebraic properties and a cost model that we propose, we provide several query optimization strategies for extended query plans. Further, we describe a query execution algorithm that blends preference evaluation with query execution, while making effective use of the native query engine. We have implemented our framework and methods in a prototype system, PrefDB . PrefDB allows transparent and efficient evaluation of preferential queries on top of a relational DBMS. Our extensive experimental evaluation on two real-world datasets demonstrates the feasibility and advantages of our framework.

Indeterministic Handling of Uncertain Decisions in Deduplication

In current research and practice, deduplication is usually considered as a deterministic approach in which database tuples are either declared to be duplicates or not. In ambiguous situations, however, it is often not completely clear-cut, which tuples represent the same real-world entity. In deterministic approaches, many realistic possibilities may be ignored, which in turn can lead to false decisions. In this article, we present an indeterministic approach for deduplication by using a probabilistic target model including techniques for proper probabilistic interpretation of similarity matching results. Thus, instead of deciding for one of the most likely situations, all realistic situations are modeled in the resultant data. This approach minimizes the negative impact of false decisions. Moreover, the deduplication process becomes almost fully automatic and human effort can be largely reduced. To increase applicability, we introduce several semi-indeterministic methods that heuristically reduce the set of indeterministically handled decisions in several meaningful ways. We also describe a full-indeterministic method for theoretical and presentational reasons.

Matching dependencies: semantics and query answering

Matching dependencies (MDs) are used to declaratively specify the identification (or matching) of certain attribute values in pairs of database tuples when some similarity conditions on other values are satisfied. Their enforcement can be seen as a natural generalization of entity resolution. In what we call the pure case of MD enforcement, an arbitrary value from the underlying data domain can be used for the value in common that is used for a matching. However, the overall number of changes of attribute values is expected to be kept to a minimum. We investigate this case in terms of semantics and the properties of data cleaning through the enforcement of MDs. We characterize the intended clean instances, and also the clean answers to queries, as those that are invariant under the cleaning process. The complexity of computing clean instances and clean query answering is investigated. Tractable and intractable cases depending on the MDs are identified and characterized.

Finding Top-k Answers in Keyword Search over Relational Databases Using Tuple Units

Existing studies on keyword search over relational databases usually find Steiner trees composed of connected database tuples as answers. They on-the-fly identify Steiner trees by discovering rich structural relationships between database tuples, and neglect the fact that such structural relationships can be precomputed and indexed. Recently, tuple units are proposed to improve search efficiency by indexing structural relationships, and existing methods identify a single tuple unit to answer keyword queries. However, in many cases, multiple tuple units should be integrated to answer a keyword query. Thus, these methods will involve false negatives. To address this problem, in this paper, we study how to integrate multiple related tuple units to effectively answer keyword queries. To achieve a high performance, we devise two novel indexes, single-keyword-based structure-aware index and keyword-pair-based structure-aware index, and incorporate structural relationships between different tuple units into the indexes. We use the indexes to efficiently identify the answers of integrated tuple units. We develop new ranking techniques and algorithms to progressively find the top-k answers. We have implemented our method in real database systems, and the experimental results show that our approach achieves high search efficiency and result quality, and outperforms state-of-the-art methods significantly.

Heuristic algorithm for interpretation of multi-valued attributes in similarity-based fuzzy relational databases

In this work, we are presenting implementation details and extended scalability tests of the heuristic algorithm, which we had used in the past [1,2] to discover knowledge from multi-valued data entries stored in similarity-based fuzzy relational databases. The multi-valued symbolic descriptors, characterizing individual attributes of database records, are commonly used in similarity-based fuzzy databases to reflect uncertainty about the recorded observation. In this paper, we present an algorithm, which we developed to precisely interpret such non-atomic values and to transfer the fuzzy database tuples to the forms acceptable for many regular (i.e. atomic values based) data mining algorithms.

Privacy-Preserving Tuple Matching in Distributed Databases

We address the problems of privacy-preserving duplicate tuple matching (PPDTM) and privacy-preserving threshold attributes matching (PPTAM) in the scenario of a horizontally partitioned database among N parties, where each party holds a private share of the database's tuples and all tuples have the same set of attributes. In PPDTM, each party determines whether its tuples have any duplicate on other parties' private databases. In PPTAM, each party determines whether all attribute values of each tuple appear at least a threshold number of times in the attribute unions. We propose protocols for the two problems using additive homomorphic cryptosystem based on the subgroup membership assumption, e.g., Paillier's and ElGamal's schemes. By analysis on the total numbers of modular exponentiations, modular multiplications and communication bits, with a reduced computation cost which dominates the total cost, by trading off communication cost, our PPDTM protocol for the semihonest model is superior to the solution derivable from existing techniques in total cost. Our PPTAM protocol is superior in both computation and communication costs. The efficiency improvements are achieved mainly by using random numbers instead of random polynomials as existing techniques for perturbation, without causing successful attacks by polynomial interpolations. We also give detailed constructions on the required zero-knowledge proofs and extend our two protocols to the malicious model, which were previously unknown.

Path oracles for spatial networks

The advent of location-based services has led to an increased demand for performing operations on spatial networks in real time. The challenge lies in being able to cast operations on spatial networks in terms of relational operators so that they can be performed in the context of a database. A linear-sized construct termed a path oracle is introduced that compactly encodes the n 2 shortest paths between every pair of vertices in a spatial network having n vertices thereby reducing each of the paths to a single tuple in a relational database and enables finding shortest paths by repeated application of a single SQL SELECT operator. The construction of the path oracle is based on the observed coherence between the spatial positions of both source and destination vertices and the shortest paths between them which facilitates the aggregation of source and destination vertices into groups that share common vertices or edges on the shortest paths between them. With the aid of the Well-Separated Pair (WSP) technique, which has been applied to spatial networks using the network distance measure, a path oracle is proposed that takes O ( s d n ) space, where s is empirically estimated to be around 12 for road networks, but that can retrieve an intermediate link in a shortest path in O (log n ) time using a B-tree. An additional construct termed the path-distance oracle of size O ( n · max( s d , 1/ε d )) (empirically ( n · max(12 2 , 2.5/ε 2 ))) is proposed that can retrieve an intermediate vertex as well as an ε-approximation of the network distances in O (log n ) time using a B-tree. Experimental results indicate that the proposed oracles are linear in n which means that they are scalable and can enable complicated query processing scenarios on massive spatial network datasets.