Truth Discovery Methods Research Articles

Truth discovery for mobile workers in edge-assisted mobile crowdsensing

The proliferation of mobile phones has led to the rise of mobile crowdsensing systems. However, many of these systems rely on the deep cloud, which can be complex and challenging to scale. To improve the performance of crowdsensing at the edge cloud, truth-discovery methods are commonly employed. These methods typically involve updating either the truth or the weight associated with a user’s task. While some edge cloud-based crowdsensing systems exist, they do not provide incentives to users based on their experience. In this report, we present a new approach to truth discovery and incentive-giving that considers both the user’s experience and the accuracy of their submitted data. Our modified truth-discovery algorithm updates both the weight and truth concurrently, with greater incentives offered to users who have completed more tasks and whose submitted data is close to the estimated truth. We have run simulations to show how well our suggested strategy works to enhance the incentive system for experienced users.

ANALYSIS OF HOW AND WHY STYLISTIC LANGUAGE IMPACTS ADVERTISING WITH A JOURNALISTIC APPROACH

Objective and Background: The Internet of Things (IoT) devices store a great volume of data. Therefore, with the advancements in technology, some new protection guidelines have to be developed to preserve data privacy and collection. The present study aimed to determine a secure protective framework using the BTD structure in a specified zone, and provide the idea of hiding the sensory data of an operator device to preserve data privacy without losing the data integrity. Methodology: Two simple and enhanced methods were used. The BTD framework and estimated ground truth were used in both modes. The enhanced method allows the users to use a random weight (variance) when combining their sensory data with the estimated ground truth provided by the agent. We ran the BTD framework in the CloudSim (network simulation framework). Each simulation is run for 500 virtual minutes with 12 iterations. The target zone is set to 50×50 meters. The test results were collected on real-world data traces from sensory systems. In addition to the BTD structure, the Voting method has been also used, and to do so, majority voting and CRH truth discovery method (Conflict Resolution on Heterogeneous Data) which does not take any measures to break the security of the sensor during the process is used. Findings: The comparative results show a 0.70-0.71 error level between the BTD and other functions, at least. As long as there is at least one reliable source, the BTD will have an error level of 0. Simulations show that BTD has a worst-case error of 0.05 and a weighted variance of +/- 5%. Based on previous frameworks, the BTD is improved by three requirements of a population identified in the IoT: Preservation of privacy for the device user, data integrity for the data collection group, and low-cost computation on the user device.

Hypergraph-based Truth Discovery for Sparse Data in Mobile Crowdsensing

Mobile crowdsensing leverages the power of a vast group of participants to collect sensory data, thus presenting an economical solution for data collection. However, due to the variability among participants, the quality of sensory data varies significantly, making it crucial to extract truthful information from sensory data of differing quality. Additionally, given the fixed time and monetary costs for the participants, they typically only perform a subset of tasks. As a result, the datasets collected in real-world scenarios are usually sparse. Current truth discovery methods struggle to adapt to datasets with varying sparsity, especially when dealing with sparse datasets. In this article, we propose an adaptive Hypergraph-based EM truth discovery method, HGEM. The HGEM algorithm leverages the topological characteristics of hypergraphs to model sparse datasets, thereby improving its performance in evaluating the reliability of participants and the true value of the event to be observed. Experiments based on simulated and real-world scenarios demonstrate that HGEM consistently achieves higher predictive accuracy.

DWSP-MT: Discovery of workers sensing preferences to match tasks for improving data collection quality in MCS

Mobile CrowdSensing (MCS) is a new sensing paradigm that leverages the widespread use of mobile devices to collect data quickly and enable various key applications in the Internet of Things (IoT). Collecting high-quality data is essential for MCS to design and provide a high-quality service. However, it is difficult for the platform to detect dishonest workers who submit fake data. Therefore, MCS applications face a significant challenge. Although there are a number of studies on truth discovery, these studies mainly failed to consider the sensing preferences of workers, resulting in poor data quality enhancement. To overcome these issues, a Discovery of Workers Sensing Preferences to Match Tasks (DWSP-MT) scheme is proposed to improve the quality of MCS data collection. The DWSP-MT scheme primarily includes three components: 1) Unmanned Aerial Vehicles (UAVs) are employed to sense data as ground truth data and an unsupervised anomaly detection algorithm is adopted to identify the data truth of tasks submitted by unidentified workers. 2) The data truth of unidentified workers in historical tasks is evaluated to calculate their sensing preferences, which are used to identify trusted workers. 3) To improve the quality of data collection, a task assignment model based on the sensing preferences of trusted workers is proposed for the first time, which can ensure that tasks are assigned to their areas of most excellent expertise. At last, two open-access datasets were used to conduct experiments comparing the DWSP-MT scheme with existing truth discovery methods. The outcomes of these experiments indicate that our approach outperforms the state-of-the-art research and improves F1-score by 65.74 %, improves accuracy by 36.62 %, and reduces data bias by 60.61 %

Crowdsourcing Truth Inference Based on Label Confidence Clustering

Truth inference can help solve some difficult problems of data integration in crowdsourcing. Crowdsourced workers are not experts and their labeling ability varies greatly; therefore, in practical applications, it is difficult to determine whether the labels collected from a crowdsourcing platform are correct. This article proposes a novel algorithm called truth inference based on label confidence clustering (TILCC) to improve the quality of integrated labels for the single-choice classification problem in crowdsourcing labeling tasks. We obtain the label confidence via worker reliability, which is calculated from multiple noise labels using a truth discovery method, and then we generate the clustering features and use the K-means algorithm to cluster all the tasks into K different clusters. Each cluster corresponds to a specific class, and the tasks in the cluster are assigned a label. Compared with the performances of six state-of-the-art methods, MV, ZenCrowd, PM, CATD, GLAD, and GTIC, on 12 randomly selected real-world datasets, the performance of our algorithm showed many advantages: no need to set complex parameters, faster running speed, and significantly higher accuracy.

Incentive Mechanism Design for Truth Discovery in Crowdsourcing With Copiers

Crowdsourcing has become an effective tool to utilize human intelligence to perform tasks that are challenging for machines. Many truth discovery methods and incentive mechanisms for crowdsourcing have been proposed. However, most of them cannot deal with the crowdsourcing with copiers, who copy a part (or all) of data from other workers. This article aims at designing crowdsourcing incentive mechanism for truth discovery of textual answers with copiers. We formulate the problem of maximizing the social welfare such that all tasks can be completed with the least confidence for truth discovery and design an three-stage incentive mechanism. In contextual embedding and clustering stage, we construct and cluster the content vector representations of textual crowdsourced answers at the semantic level. In truth discovery stage, we estimate the truth for each task based on the dependence and accuracy of workers. In reverse auction stage, we design a greedy algorithm to select the winners and determine the payment. Through both rigorous theoretical analysis and extensive simulations, we demonstrate that the proposed mechanisms achieve computational efficiency, individual rationality, truthfulness, and guaranteed approximation. Moreover, our truth discovery methods show prominent advantage in terms of precision when there are copiers in the crowdsourcing systems.

Identifikasi Ekspresi Idiomatik Menggunakan Distributional Semantic Based Approach dan Truth Discovery

Idiomatic expressions are phrases that consist of a sequence of two or more words that have a meaning that cannot be predicted from the meaning of the individual words that compose it. Idiomatic expressions exist in almost all languages ??but are difficult to extract because there is no algorithm that can precisely decipher the structure of idiomatic expressions, so most rule-based machine translation systems generally translate idiomatic expressions by translating word for word their constituents, but the translation results do not produce the true meaning of the idiomatic expression. Based on this problem, the author tries to do research on the identification of the use of idiomatic expressions in Indonesian sentences. First, the author conducts the sentence classification process using BERT to find out whether the sentence contains idiomatic expressions or not. Furthermore, idiomatic expressions are identified based on distributional semantic based approach and then validated automatically using the Truth Discovery method. From the research conducted, the identification of idiomatic expressions in Indonesian sentences using Distributional Semantic Based Approach and Truth Discovery obtained an accuracy of 0.82; precision 1.0; recall 0.64 and f1-score 0.78.

Reputation-Based Truth Discovery With Long-Term Quality of Source in Internet of Things

Although the Internet of Things (IoT) devices have been widely used for data collection in various applications, the observed data of an object from each IoT device (i.e., source) may vary from the ground truth due to the different qualities of IoT devices and sensing environments. Truth discovery has become a promising technology to extract the truth among multiple conflicting pieces of data from different sources. Existing methods usually assume the quality of source (source reliability) is unknown a priori and will be estimated as the weight for calculating the truth during each truth discovery task. However, in a long-term data observation scenario, the quality of source can be accumulated and utilized in the future truth discovery process. Aiming to take the long-term quality of source into consideration, in this paper, we propose a reputation-based truth discovery method to derive the truth from the conflicting data. Specifically, we propose a generalized formulation with linear constraint for the truth discovery problem, which can cope with different regulations on the source reliability. Instead of directly using weight as the source reliability, we also define the reliability of a source by its contribution to the loss function. Then, we propose a novel reputation model to quantify the newly defined source reliability, which will be accumulated as the long-term source quality. Finally, we propose a reputation-based truth discovery model, where initial weights are assigned based on source reputations. Experiments conducted on real weather conditions and GPS datasets demonstrate that our reputation-based truth discovery can reduce the number of iterations during truth discovery and achieve high accuracy.

Disguised as Privacy: Data Poisoning Attacks against Differentially Private Crowdsensing Systems

Although crowdsensing has emerged as a popular information collection paradigm, its security and privacy vulnerabilities have come to the forefront in recent years. However, one big limitation of previous research is that the security domain and the privacy domain are typically considered separately. Therefore, it is unclear whether the defense methods in the privacy domain will have unexpected impact on the security domain. To bridge this gap, in this paper, we propose a novel Disguise-based Data Poisoning Attack (DDPA) against the differentially private crowdsensing systems empowered with the truth discovery method. Specifically, we propose a novel stealth strategy, i.e., disguising the malicious behavior as privacy behavior, to avoid being detected by truth discovery methods. With this stealth strategy, the shortcoming of failing to maximize the attack effectiveness is avoided naturally through structuring a bi-level optimization problem, which can be solved with the alternating optimization algorithm. Moreover, we show that the differentially private crowdsensing systems are vulnerable to data poisoning attacks, and enhancing the level of privacy will bring more serious security threats. Finally, the evaluation results on the real-world dataset Emotion and the synthetic dataset SynData demonstrate that DDPA can not only achieve maximum utility damage but also remain undetected.

Probabilistic model for truth discovery with mean and median check framework

In the era of big data, information can be collected from various sources. Unfortunately, information provided by multiple sources on the same entity is inevitably conflicting. Due to the ubiquitous existence of data conflicts, truth discovery has recently attracted considerable attention. Several truth discovery methods focus on providing a point estimate for the truth of each entity and exhibit completely different performances on the same input dataset. Therefore, an appropriate truth discovery method should be adopted to fit the unknown source reliability distributions. To address this, we approach truth discovery from another perspective. We theoretically verify that if the absolute distance between the mean and median value is large, then there must be incorrect claims with large errors in the input dataset. Accordingly, we propose a mean and median check (MMC) framework for truth detection, error claim removal, and iteration-stopping criteria. The experiments demonstrate that MMC can effectively remove incorrect claims provided by unreliable sources. Furthermore, the performance of state-of-the-art truth discovery methods can be significantly improved if MMC is used for input data preprocessing.

Enhanced privacy preserving and truth discovery method for 5G and beyond vehicle crowd sensing systems

As 5G networks are being deployed the key constraints of 5G networks have been highlighted, which encourages the probing research of 5G and beyond networks as the next-generation solutions. Especially for vehicular networks which require a network topology with very low latency and high reliability. 5G Vehicular networks offer higher reliability, lower latency, and secure and efficient communication services but these advancements come at the cost of new security and privacy concerns. Especially in the case of a high number of connected vehicles which leads to the concept of Vehicle Crowd Sensing System (VCS) in 5G enabled Vehicular Ad hoc networks (VANETS). Hence, due to this, the number of connected vehicle nodes is also soaring drastically which is directly proportional to the expansion of the Vehicle Crowd Sensing System (VCS). These vehicles are equipped with the latest gadgetries and a large number of sensors, which works simultaneously for processing at end-user utility applications e.g. navigation, predictions, and traffic monitoring. Therefore, in this scenario, all vehicle nodes upload their sole respective data to the cloud for computing which arise the need for reliability and privacy. Thus, Privacy-Preserving Truth Discovery (PPTD) is one such scheme that distills reliable information with privacy from raw signals acquired from different sources. Currently, PPTD brings substantial overhead in terms of consumed data, computation at the Vehicle level and requires each vehicle to remain online till the completion of uploading at the cloud server. So, in this paper, we explain a newly proposed novel Efficient Privacy-Preserving Truth Discovery (EPTD) entailing double masking for VCS and allowing Vehicles to go offline anytime during upload processing while ensuring privacy and accuracy at the same time. Simulation results have shown that the proposed work has outperformed all the previously existing methods.

A Secure Truth Discovery for Data Aggregation in Mobile Crowd Sensing

With the rapid development of portable mobile devices, mobile crowd sensing systems (MCS) have been widely studied. However, the sensing data provided by participants in MCS applications is always unreliable, which affects the service quality of the system, and the truth discovery technology can effectively obtain true values from the data provided by multiple users. At the same time, privacy leaks also restrict users’ enthusiasm for participating in the MCS. Based on this, our paper proposes a secure truth discovery for data aggregation in crowd sensing systems, STDDA, which iteratively calculates user weights and true values to obtain real object data. In order to protect the privacy of data, STDDA divides users into several clusters, and users in the clusters ensure the privacy of data by adding secret random numbers to the perceived data. At the same time, the cluster head node uses the secure sum protocol to obtain the aggregation result of the sense data and uploads it to the server so that the server cannot obtain the sense data and weight of individual users, further ensuring the privacy of the user’s sense data and weight. In addition, using the truth discovery method, STDDA provides corresponding processing mechanisms for users’ dynamic joining and exiting, which enhances the robustness of the system. Experimental results show that STDDA has the characteristics of high accuracy, low communication, and high security.

Truth Discovery by Claim and Source Embedding

Information gathered from multiple sources on the Web often exhibits conflicts. This phenomenon motivates the need of truth discovery, which aims to automatically find the true claim among multiple conflicting claims. Existing truth discovery methods are mainly based on iterative updates, optimization or probabilistic models. Although these methods have shown their own effectiveness, they have a common limitation. These methods do not model relationships between each pair of source and target such that they do not well capture the underlying interactions in the data. In this paper, we propose a new model for truth discovery, learning the representations of sources and claims automatically from the interactions between sources and targets. Our model first constructs a heterogenous network including source-claim, source-source and truth-claim relationships. It then embeds the network into a low dimensional space such that trustworthy sources and true claims are close. In this way, truth discovery can be conveniently performed in the embedding space. Moreover, our model can be implemented in both semi-supervised and un-supervised manners to deal with the label scarcity problem in practical truth discovery. Experiments on three real-world datasets demonstrate that our model outperforms existing state-of-the-art methods for truth discovery.

From Appearance to Essence

Truth discovery has been widely studied in recent years as a fundamental means for resolving the conflicts in multi-source data. Although many truth discovery methods have been proposed based on different considerations and intuitions, investigations show that no single method consistently outperforms the others. To select the right truth discovery method for a specific application scenario, it becomes essential to evaluate and compare the performance of different methods. A drawback of current research efforts is that they commonly assume the availability of certain ground truth for the evaluation of methods. However, the ground truth may be very limited or even impossible to obtain, rendering the evaluation biased. In this article, we present CompTruthHyp , a generic approach for comparing the performance of truth discovery methods without using ground truth. In particular, our approach calculates the probability of observations in a dataset based on the output of different methods. The probability is then ranked to reflect the performance of these methods. We review and compare 12 representative truth discovery methods and consider both single-valued and multi-valued objects. The empirical studies on both real-world and synthetic datasets demonstrate the effectiveness of our approach for comparing truth discovery methods.

Constrained Truth Discovery

To aggregate useful information among diversified sources, a hotspot research topic called truth discovery has emerged in recent years. Existing truth discovery methods attempt to infer the true attribute values for the entities by identifying and trusting reliable data sources. However, all these methods neglect the relations among different entities, which play important roles in truth discovery task. When reliable data sources cannot provide sufficient information of entities, the true attribute values of these entities can still be inferred by propagating trustworthy information from related entities. Motivated by this, in this paper, we introduce the constrained truth discovery problem. We incorporate denial constraints, a universally quantified first-order logic formalism, into the process of truth discovery. We formulate it as a constrained optimization problem and analyze its hardness. To address the problem, we propose algorithms to partition the entities into disjoint groups, and generate arithmetic constraints for each disjoint group separately. Then, the true attribute values of the entities in each disjoint group are derived by minimizing the objective function under the corresponding arithmetic constraints. Experimental results on both real-world and synthetic datasets demonstrate that the proposed approach achieves good performance even with very few constraints and reliable sources.

Using Social Network Information in Community-Based Bayesian Truth Discovery

We investigate the problem of truth discovery based on opinions from multiple agents (who may be unreliable or biased) that form a social network. We consider the case where agents' reliabilities or biases are correlated if they belong to the same community, which defines a group of agents with similar opinions regarding a particular event. An agent can belong to different communities for different events, and these communities are unknown a priori. We incorporate knowledge of the agents' social network in our truth discovery framework and develop Laplace variational inference methods to estimate agents' reliabilities, communities, and the event states. We also develop a stochastic variational inference method to scale our model to large social networks. Simulations and experiments on real data suggest that when observations are sparse, our proposed methods perform better than several other inference methods, including majority voting, TruthFinder, AccuSim, the Confidence-Aware Truth Discovery method, the Bayesian classifier combination (BCC) method, and the community BCC method.

Quality-aware online task assignment mechanisms using latent topic model

Crowdsourcing has been proven to be a useful tool for the tasks which are hard for computers. Unfortunately, workers with uneven expertise are likely to provide low-quality or even deliberately wrong data. A reliability model that precisely describes workers' performance on the tasks can benefit the development of both task assignment mechanism and truth discovery method. However, existing methods cannot model workers' fine-grained reliability levels accurately. In this paper, we consider dividing tasks into clusters (i.e., topics) based on workers' behaviors and propose a novel latent topic model to describe the topic structure and workers' topical-level expertise. Then, we develop two online task assignment mechanisms that dynamically assign each incoming worker a set of tasks where he can achieve the Maximum Expected Gain (MEG) or Maximum Expected and Potential Gain (MEPG). The experimental results demonstrate that our methods can significantly decrease the number of task assignments and achieve higher accuracy and macro-averaging F1-score than the state-of-the-art approaches.

Truth finding by reliability estimation on inconsistent entities for heterogeneous data sets

An important task in big data integration is to derive accurate data records from noisy and conflicting values collected from multiple sources. Most existing truth finding methods assume that the reliability is consistent on the whole data set, ignoring the fact that different attributes, objects and object groups may have different reliabilities even wrt the same source. These reliability differences are caused by the hardness differences in obtaining attribute values, non-uniform updates to objects and the differences in group privileges. This paper addresses the problem how to compute truths by effectively estimating the reliabilities of attributes, objects and object groups in a multi-source heterogeneous data environment. We first propose an optimization framework TFAR, its implementation and Lagrangian duality solution for Truth Finding by Attribute Reliability estimation. We then present a Bayesian probabilistic graphical model TFOR and an inference algorithm applying Collapsed Gibbs Sampling for Truth Finding by Object Reliability estimation. Finally we give an optimization framework TFGR and its implementation for Truth Finding by Group Reliability estimation. All these models lead to a more accurate estimation of the respective attribute, object and object group reliabilities, which in turn can achieve a better accuracy in inferring the truths. Experimental results on both real data and synthetic data show that our methods have better performance than the state-of-art truth discovery methods.

On Scalable and Robust Truth Discovery in Big Data Social Media Sensing Applications

Identifying trustworthy information in the presence of noisy data contributed by numerous unvetted sources from online social media (e.g., Twitter, Facebook, and Instagram) has been a crucial task in the era of big data. This task, referred to as truth discovery, targets at identifying the reliability of the sources and the truthfulness of claims they make without knowing either a priori. In this work, we identified three important challenges that have not been well addressed in the current truth discovery literature. The first one is “misinformation spread” where a significant number of sources are contributing to false claims, making the identification of truthful claims difficult. For example, on Twitter, rumors, scams, and influence bots are common examples of sources colluding, either intentionally or unintentionally, to spread misinformation and obscure the truth. The second challenge is “data sparsity” or the “long-tail phenomenon” where a majority of sources only contribute a small number of claims, providing insufficient evidence to determine those sources’ trustworthiness. For example, in the Twitter datasets that we collected during real-world events, more than 90 percent of sources only contributed to a single claim. Third, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. In this paper, we develop a Scalable and Robust Truth Discovery (SRTD) scheme to address the above three challenges. In particular, the SRTD scheme jointly quantifies both the reliability of sources and the credibility of claims using a principled approach. We further develop a distributed framework to implement the proposed truth discovery scheme using Work Queue in an HTCondor system. The evaluation results on three real-world datasets show that the SRTD scheme significantly outperforms the state-of-the-art truth discovery methods in terms of both effectiveness and efficiency.

PatternFinder: Pattern discovery for truth discovery

Truth discovery methods infer truths from multiple sources. These methods usually resolve conflicts based on the information on the entity level. However, due to the existence of incompleteness and the difficulty in entity matching, the information on the individual entity is often insufficient. This motivates pattern discovery, which aims to mine useful patterns across entities from a global perspective. In this paper, we introduce pattern discovery for truth discovery and formulate it as an optimization problem. To solve such a problem, we propose an algorithm called PatternFinder that jointly and iteratively learns the variables. Additionally, we also propose an optimized grouping strategy to enhance its efficiency. Experimental results on simulated and real-world datasets demonstrate the advantage of the proposed methods, which outperform the state-of-the-art baselines in terms of both effectiveness and efficiency.