Absence Of Ground Truth Research Articles

Constructing representative group networks from tractography: lessons from a dynamical approach

Human group connectome analysis relies on combining individual connectome data to construct a single representative network which can be used to describe brain organisation and identify differences between subject groups. Existing methods adopt different strategies to select the network structural features to be retained or optimised at group level. In the absence of ground truth, however, it is unclear which structural features are the most suitable and how to evaluate the consequences on the group network of applying any given strategy. In this investigation, we consider the impact of defining a connectome as representative if it can recapitulate not just the structure of the individual networks in the cohort tested but also their dynamical behaviour, which we measured using a model of coupled oscillators. We applied the widely used approach of consensus thresholding to a dataset of individual structural connectomes from a healthy adult cohort to construct group networks for a range of thresholds and then identified the most dynamically representative group connectome as that having the least deviation from the individual connectomes given a dynamical measure of the system. We found that our dynamically representative network recaptured aspects of structure for which it did not specifically optimise, with no significant difference to other group connectomes constructed via methods which did optimise for those metrics. Additionally, these other group connectomes were either as dynamically representative as our chosen network or less so. While we suggest that dynamics should be at least one of the criteria for representativeness, given that the brain has evolved under the pressure of carrying out specific functions, our results suggest that the question persists as to which of these criteria are valid and testable.

DC2T: Disentanglement-Guided Consolidation and Consistency Training for Semi-Supervised Cross-Site Continual Segmentation.

Continual Learning (CL) is recognized to be a storage-efficient and privacy-protecting approach for learning from sequentially-arriving medical sites. However, most existing CL methods assume that each site is fully labeled, which is impractical due to budget and expertise constraint. This paper studies the Semi-Supervised Continual Learning (SSCL) that adopts partially-labeled sites arriving over time, with each site delivering only limited labeled data while the majority remains unlabeled. In this regard, it is challenging to effectively utilize unlabeled data under dynamic cross-site domain gaps, leading to intractable model forgetting on such unlabeled data. To address this problem, we introduce a novel Disentanglement-guided Consolidation and Consistency Training (DC2T) framework, which roots in an Online Semi-Supervised representation Disentanglement (OSSD) perspective to excavate content representations of partially labeled data from sites arriving over time. Moreover, these content representations are required to be consolidated for site-invariance and calibrated for style-robustness, in order to alleviate forgetting even in the absence of ground truth. Specifically, for the invariance on previous sites, we retain historical content representations when learning on a new site, via a Content-inspired Parameter Consolidation (CPC) method that prevents altering the model parameters crucial for content preservation. For the robustness against style variation, we develop a Style-induced Consistency Training (SCT) scheme that enforces segmentation consistency over style-related perturbations to recalibrate content encoding. We extensively evaluate our method on fundus and cardiac image segmentation, indicating the advantage over existing SSCL methods for alleviating forgetting on unlabeled data.

An optimal Bayesian strategy for comparing Wiener–Hunt deconvolution models in the absence of ground truth

Abstract This paper considers the quantitative comparison of several alternative models to perform deconvolution in situations where there is no ground truth data available. With applications to very large data sets in mind, we focus on linear deconvolution models based on a Wiener filter. Although comparatively simple, such models are widely prevalent in large scale setting such as high-resolution image restoration because they provide an excellent trade-off between accuracy and computational effort. However, in order to deliver accurate solutions, the models need to be properly calibrated in order to capture the covariance structure of the unknown quantity of interest and of the measurement error. This calibration often requires onerous controlled experiments and extensive expert supervision, as well as regular recalibration procedures. This paper adopts an unsupervised Bayesian statistical approach to model assessment that allows comparing alternative models by using only the observed data, without the need for ground truth data or controlled experiments. Accordingly, the models are quantitatively compared based on their posterior probabilities given the data, which are derived from the marginal likelihoods or evidences of the models. The computation of these evidences is highly non-trivial and this paper consider three different strategies to address this difficulty—a Chib approach, Laplace approximations, and a truncated harmonic expectation—all of which efficiently implemented by using a Gibbs sampling algorithm specialised for this class of models. In addition to enabling unsupervised model selection, the output of the Gibbs sampler can also be used to automatically estimate unknown model parameters such as the variance of the measurement error and the power of the unknown quantity of interest. The proposed strategies are demonstrated on a range of image deconvolution problems, where they are used to compare different modelling choices for the instrument’s point spread function and covariance matrices for the unknown image and for the measurement error.

AI analysis of super-resolution microscopy: Biological discovery in the absence of ground truth.

Super-resolution microscopy, or nanoscopy, enables the use of fluorescent-based molecular localization tools to study molecular structure at the nanoscale level in the intact cell, bridging the mesoscale gap to classical structural biology methodologies. Analysis of super-resolution data by artificial intelligence (AI), such as machine learning, offers tremendous potential for the discovery of new biology, that, by definition, is not known and lacks ground truth. Herein, we describe the application of weakly supervised paradigms to super-resolution microscopy and its potential to enable the accelerated exploration of the nanoscale architecture of subcellular macromolecules and organelles.

Development and Application of a Virtual Imaging Trial Framework for Longitudinal Quantification of Emphysema in CT.

Pulmonary emphysema is a progressive lung disease that requires accurate evaluation for optimal management. This task, possible using quantitative CT, is particularly challenging as scanner and patient attributes change over time, negatively impacting the CT-derived quantitative measures. Efforts to minimize such variations have been limited by the absence of ground truth in clinical data, thus necessitating reliance on clinical surrogates, which may not have one-to-one correspondence to CT-based findings. This study aimed to develop the first suite of human models with emphysema at multiple time points, enabling longitudinal assessment of disease progression with access to ground truth. A total of 14 virtual subjects were modeled across three time points. Each human model was virtually imaged using a validated imaging simulator (DukeSim), modeling an energy-integrating CT scanner. The models were scanned at two dose levels and reconstructed with two reconstruction kernels, slice thicknesses, and pixel sizes. The developed longitudinal models were further utilized to demonstrate utility in algorithm testing and development. Two previously developed image processing algorithms (CT-HARMONICA, EmphysemaSeg) were evaluated. The results demonstrated the efficacy of both algorithms in improving the accuracy and precision of longitudinal quantifications, from 6.1±6.3% to 1.1±1.1% and 1.6±2.2% across years 0-5. Further investigation in EmphysemaSeg identified that baseline emphysema severity, defined as >5% emphysema at year 0, contributed to its reduced performance. This finding highlights the value of virtual imaging trials in enhancing the explainability of algorithms. Overall, the developed longitudinal human models enabled ground-truth based assessment of image processing algorithms for lung quantifications.

Post-clustering Prioritization Framework for Autonomous Decision Making in the Absence of Ground Truth via Hypothetical Probing

A generic prioritization framework is introduced for addressing the problem of automated prioritization of region of interest or target selection. The framework is based on the assumption that clustering of preliminary data for preidentified regions or targets of interest within an operational area has already occurred, i.e., post-classification, and that the clustering quality can be expressed as an energy/objective function. Region or target of interest prioritization then means to rank regions or targets of interest according to their probability of changing the energy/objective function value upon subsequent hypothetical probing as opposed to actually conducted reexamination, i.e., thorough follow-up or in-situ measurements. The mathematical formalism for calculating these probabilities to contribute to this change of the energy/objective function value is introduced and validated through numerical simulations. Moreover, these probabilities can also be understood as a confidence-check of the classification, i.e., the pre-clustering of the preliminary data. The operation of the prioritization framework is independent of the algorithm used to pre-cluster the preliminary data, and supports autonomous decision-making. It is widely applicable across many scientific disciplines and areas, ranging from the microscopic to the macroscopic scale. Due to its ability to help maximize scientific return while optimizing resource utilization, it is particularly relevant in the context of resource-constrained autonomous robotic planetary exploration as it may extend the Remaining Useful Lifetime (RUL) – a key aspect in Prognostics and Health Management (PHM) – of space missions. On a more general, PHM-relevant level, the prioritization framework may provide an additional mechanism of identifying and correcting the maintenance status of system components to assist predictive maintenance or condition-based maintenance.

Electrographic flow mapping of persistent atrial fibrillation: intra- and inter-procedure reproducibility in the absence of 'ground truth'.

Validating mapping systems that identify atrial fibrillation (AF) sources (focal/rotational activity) is confounded by the absence of ground truth. A key concern of prior mapping technologies is spatiotemporal instability, manifesting as poor map reproducibility. Electrographic flow (EGF) employs a novel algorithm that visualizes atrial electrical wavefront propagation to identify putative AF sources. We analysed both intra- (3 min) and inter- (>3 months) procedure EGF map reproducibility. In 23 persistent AF patients, after pulmonary vein isolation (PVI), EGF maps were generated from 3 serial 1 min recordings using a 64-electrode basket mapping catheter (triplets) at right and left atrial locations. Source prevalence from map triplets was compared between recordings. Per protocol, 12 patients returned for 3-month remapping (1 non-inducible): index procedure post-PVI EGF maps were compared with initial EGF remapping at 3-month redo. Intra-procedure reproducibility: analysing 224 map triplets (111 right atrium, 113 left atrium) revealed a high degree of map consistency with minimal min-to-min shifts: 97 triplets (43%), exact match of leading sources on all 3 maps; 95 triplets (42%), leading source within 1 electrode space on 2 of 3 maps; and 32 triplets (14%), chaotic leading source pattern. Average deviation in source prevalence over 60 s was low (6.4%). Inter-procedure reproducibility: spatiotemporal stability of EGF mapping >3 months was seen in 16 of 18 (89%) sources mapped in 12 patients with (re)inducible AF. Electrographic flow mapping generates reproducible intra- and inter-procedural maps, providing rationale for randomized clinical trials targeting these putative AF sources.

Edge Video Analytics With Adaptive Information Gathering: A Deep Reinforcement Learning Approach

With growing popularity of enormous public safety and transportation infrastructure cameras, there are increasing demands for automatic mobile video analytics. The emerging multi-access edge computing (MEC) technology has been recently applied to improve the accuracy-latency tradeoff of mobile video analytics. In this paper, we study an MEC-enabled multi-device video analytics system and formulate the problem as a Markov decision process (MDP) to meet two practical challenges: i) the absence of ground truth in real-time and ii) the content-varying degradation-accuracy relation. In particular, we aim to design an online joint frame degradation and bandwidth allocation algorithm with the time-varying function and limited feedback from each device. Thanks to the MDP formulation and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -step return technique, the long-term goal offers adaptive information gathering and thus improves the average accuracy and latency. For sample efficiency, we decompose the MDP problem into discrete degradation adaptation subproblems and continuous bandwidth allocation subproblems. Based on the decomposition, we propose a deep reinforcement learning (DRL) based framework, referred to as DBAG, to solve the decomposed subproblems. DBAG integrates model-based optimization and model-free DRL to solve the MDP problem with a discrete-continuous hybrid action space. Under various network setups and public datasets, DBAG greatly improves the accuracy-latency tradeoff.

Fuzzy Cluster-based Group-wise Point Set Registration with Quality Assessment.

This article studies group-wise point set registration and makes the following contributions: "FuzzyGReg", which is a new fuzzy cluster-based method to register multiple point sets jointly, and "FuzzyQA", which is the associated quality assessment to check registration accuracy automatically. Given a group of point sets, FuzzyGReg creates a model of fuzzy clusters and equally treats all the point sets as the elements of the fuzzy clusters. Then, the group-wise registration is turned into a fuzzy clustering problem. To resolve this problem, FuzzyGReg applies a fuzzy clustering algorithm to identify the parameters of the fuzzy clusters while jointly transforming all the point sets to achieve an alignment. Next, based on the identified fuzzy clusters, FuzzyQA calculates the spatial properties of the transformed point sets and then checks the alignment accuracy by comparing the similarity degrees of the spatial properties of the point sets. When a local misalignment is detected, a local re-alignment is performed to improve accuracy. The proposed method is cost-efficient and convenient to be implemented. In addition, it provides reliable quality assessments in the absence of ground truth and user intervention. In the experiments, different point sets are used to test the proposed method and make comparisons with state-of-the-art registration techniques. The experimental results demonstrate the effectiveness of our method. The code is available at https://gitsvn-nt.oru.se/qianfang.liao/FuzzyGRegWithQA.

Unsupervised Pansharpening Method Using Residual Network With Spatial Texture Attention

Recently, deep learning has become one of the most popular tools for pansharpening, many relevant methods have been investigated and reflected great performance. However, a non-negligible problem is the absence of ground-truth (GT). A common solution is using degraded images as training input and the original images are employed as GT. The learned mapping between low resolution (LR) and high resolution (HR) is simulated, is not real, which may cause spectral distortion or insufficient spatial texture enhancement of fused images. In order to address the drawback, a novel unsupervised attention pansharpening net (UAP-Net) is proposed. The proposed UAP-Net mainly contains two major components: 1) the deep residual network (DRN) and 2) spatial texture attention block (STAB). The DRN aims to extract spectral features and spatial details features from low-resolution multi-spectral (LRMS) and panchromatic (PAN), and to fuse those features to make them more representative. The designed STAB adopts the high-frequency component of corresponding input PAN as the weight to enhance the spatial details of the residual block output features. Moreover, a new loss function including two spatial losses and two spectral losses are established. The losses are calculated in the spatial domain and the frequency domain, respectively. Experiments on Gaofen-2 and Worldview-2 remote sensing data demonstrate that the proposed UAP-Net could fuse PAN and LRMS images effectively without the help of high-resolution multi-spectral (HRMS). The proposed framework is fully general and can be used for many multisource remote sensing image fusion, and achieves optimal performance in terms of both the subjective visual effect and the quantitative evaluation.

A Proximal Markov Chain Monte Carlo Method for Bayesian Inference in Imaging Inverse Problems: When Langevin Meets Moreau

Modern imaging methods rely strongly on Bayesian inference techniques to solve challenging imaging problems. Currently, the predominant Bayesian computational approach is convex optimization, which scales very efficiently to high-dimensional image models and delivers accurate point estimation results. However, in order to perform more complex analyses, for example, image uncertainty quantification or model selection, it is often necessary to use more computationally intensive Bayesian computation techniques such as Markov chain Monte Carlo methods. This paper presents a new and highly efficient Markov chain Monte Carlo methodology to perform Bayesian computation for high-dimensional models that are log-concave and nonsmooth, a class of models that is central in imaging sciences. The methodology is based on a regularized unadjusted Langevin algorithm that exploits tools from convex analysis, namely, Moreau--Yosida envelopes and proximal operators, to construct Markov chains with favorable convergence properties. In addition to scaling efficiently to high dimensions, the method can be applied in a straightforward manner to models that are currently solved using proximal optimization algorithms. We provide a detailed theoretical analysis of the proposed methodology, including asymptotic and nonasymptotic convergence results with easily verifiable conditions, and explicit bounds on the convergence rates. The proposed methodology is demonstrated with five experiments related to image deconvolution and tomographic reconstruction with total-variation and $\ell_1$ priors, where we conduct a range of challenging Bayesian analyses related to uncertainty quantification, hypothesis testing, and model selection in the absence of ground truth.

Map-Matching Error Identification in the Absence of Ground Truth

Map-matching of trajectory data has widespread applications in vehicle tracking, traffic flow analysis, route planning, and intelligent transportation systems. Map-matching algorithms snap a set of trajectory points observed by a satellite navigation system to the most likely route segments of a map. However, due to the unavoidable errors in the recorded trajectory points and the incomplete map data, map-matching algorithms may match points to incorrect segments, leading to map-matching errors. Identification of these map-matching errors in the absence of ground truth can only be achieved by visual inspection and reasoning. Thus, the identification of map-matching errors without ground truth is a time-consuming and mundane task. Although research has focused on improving map-matching algorithms, to our knowledge no attempts have been made to automatically classify and identify the residual map-matching errors. In this work, we propose the first method to automatically identify map-matching errors in the absence of ground truth, i.e., only using the recorded trajectory points and the map-matched route. We have evaluated our method on a public dataset and observed an average accuracy of 91% in automatically identifying map-matching errors, thus helping analysts to significantly reduce manual effort for map-matching quality assurance.

An objective evaluation method for head motion estimation in PET—Motion corrected centroid-of-distribution

Head motion presents a continuing problem in brain PET studies. A wealth of motion correction (MC) algorithms had been proposed in the past, including both hardware-based methods and data-driven methods. However, in most real brain PET studies, in the absence of ground truth or gold standard of motion information, it is challenging to objectively evaluate MC quality. For MC evaluation, image-domain metrics, e.g., standardized uptake value (SUV) change before and after MC are commonly used, but this measure lacks objectivity because 1) other factors, e.g., attenuation correction, scatter correction and parameters used in the reconstruction, will confound MC effectiveness; 2) SUV only reflects final image quality, and it cannot precisely inform when an MC method performed well or poorly during the scan time period; 3) SUV is tracer-dependent and head motion may cause increases or decreases in SUV for different tracers, so evaluating MC effectiveness is complicated. Here, we present a new algorithm, i.e., motion corrected centroid-of-distribution (MCCOD) to perform objective quality control for measured or estimated rigid motion information. MCCOD is a three-dimensional surrogate trace of the center of tracer distribution after performing rigid MC using the existing motion information. MCCOD is used to inform whether the motion information is accurate, using the PET raw data only, i.e., without PET image reconstruction, where inaccurate motion information typically leads to abrupt changes in the MCCOD trace. MCCOD was validated using simulation studies and was tested on real studies acquired from both time-of-flight (TOF) and non-TOF scanners. A deep learning-based brain mask segmentation was implemented, which is shown to be necessary for non-TOF MCCOD generation. MCCOD is shown to be effective in detecting abrupt translation motion errors in slowly varying tracer distribution caused by the motion tracking hardware and can be used to compare different motion estimation methods as well as to improve existing motion information.

Machine-learning classification of astronomical sources: estimating F1-score in the absence of ground truth

ABSTRACT Machine-learning based classifiers have become indispensable in the field of astrophysics, allowing separation of astronomical sources into various classes, with computational efficiency suitable for application to the enormous data volumes that wide-area surveys now typically produce. In the standard supervised classification paradigm, a model is typically trained and validated using data from relatively small areas of sky, before being used to classify sources in other areas of the sky. However, population shifts between the training examples and the sources to be classified can lead to ‘silent’ degradation in model performance, which can be challenging to identify when the ground-truth is not available. In this letter, we present a novel methodology using the nannyml Confidence-Based Performance Estimation (CBPE) method to predict classifier F1-score in the presence of population shifts, but without ground-truth labels. We apply CBPE to the selection of quasars with decision-tree ensemble models, using broad-band photometry, and show that the F1-scores are predicted remarkably well (${\rm MAPE} \sim 10{{\ \rm per\ cent}}$; R2 = 0.74–0.92). We discuss potential use-cases in the domain of astronomy, including machine-learning model and/or hyperparameter selection, and evaluation of the suitability of training data sets for a particular classification problem.

A comparison of uni- and multi-variate methods for identifying brain networks activated by cognitive tasks using intracranial EEG.

Cognitive tasks are commonly used to identify brain networks involved in the underlying cognitive process. However, inferring the brain networks from intracranial EEG data presents several challenges related to the sparse spatial sampling of the brain and the high variability of the EEG trace due to concurrent brain processes. In this manuscript, we use a well-known facial emotion recognition task to compare three different ways of analyzing the contrasts between task conditions: permutation cluster tests, machine learning (ML) classifiers, and a searchlight implementation of multivariate pattern analysis (MVPA) for intracranial sparse data recorded from 13 patients undergoing presurgical evaluation for drug-resistant epilepsy. Using all three methods, we aim at highlighting the brain structures with significant contrast between conditions. In the absence of ground truth, we use the scientific literature to validate our results. The comparison of the three methods’ results shows moderate agreement, measured by the Jaccard coefficient, between the permutation cluster tests and the machine learning [0.33 and 0.52 for the left (LH) and right (RH) hemispheres], and 0.44 and 0.37 for the LH and RH between the permutation cluster tests and MVPA. The agreement between ML and MVPA is higher: 0.65 for the LH and 0.62 for the RH. To put these results in context, we performed a brief review of the literature and we discuss how each brain structure’s involvement in the facial emotion recognition task.

Reliable wavefront reconstruction from a single lateral shearing interferogram using Bayesian convolutional neural network

The reconstruction of the original wavefront from its sheared information is an ill-posed inverse problem in the sense of Hadamard. In this paper, we present a new technique for reconstructing the wavefront from a single lateral shearing interferogram using a Bayesian convolutional neural network. With the synthesized data, the trained network can robustly invert the physical model, recover the spectral leakage problem caused by the shear operation, and reconstruct the desired phase distribution. In addition, a quantitative pixel-level uncertainty measure of the reconstruction results is provided simultaneously, which can be used as an indicator to judge the validity of reconstruction accuracy in the absence of ground truth. Numerical experiments are carried out to investigate the robustness and accuracy of the method, in which the effects of different noise levels and shear distances on reconstruction accuracy are evaluated. We further show that the uncertainty maps are highly indicative of the true error and properly calibrated. Finally, optical tests on reconstructing the topology of object deformation have confirmed that our method can considerably increase the accuracy of wavefront reconstruction as compared with the two baseline methods.

Online in silico validation of disease and gene sets, clusterings or subnetworks with DIGEST.

As the development of new drugs reaches its physical and financial limits, drug repurposing has become more important than ever. For mechanistically grounded drug repurposing, it is crucial to uncover the disease mechanisms and to detect clusters of mechanistically related diseases. Various methods for computing candidate disease mechanisms and disease clusters exist. However, in the absence of ground truth, in silico validation is challenging. This constitutes a major hurdle toward the adoption of in silico prediction tools by experimentalists who are often hesitant to carry out wet-lab validations for predicted candidate mechanisms without clearly quantified initial plausibility. To address this problem, we present DIGEST (in silico validation of disease and gene sets, clusterings or subnetworks), a Python-based validation tool available as a web interface (https://digest-validation.net), as a stand-alone package or over a REST API. DIGEST greatly facilitates in silico validation of gene and disease sets, clusterings or subnetworks via fully automated pipelines comprising disease and gene ID mapping, enrichment analysis, comparisons of shared genes and variants and background distribution estimation. Moreover, functionality is provided to automatically update the external databases used by the pipelines. DIGEST hence allows the user to assess the statistical significance of candidate mechanisms with regard to functional and genetic coherence and enables the computation of empirical $P$-values with just a few mouse clicks.

Evaluation of functional MRI-based human brain parcellation: a review.

Brain parcellations play a crucial role in the analysis of brain imaging data sets, as they can significantly affect the outcome of the analysis. In recent years, several novel approaches for constructing MRI-based brain parcellations have been developed with promising results. In the absence of ground truth, several evaluation approaches have been used to evaluate currently available brain parcellations. In this article, we review and critique methods used for evaluating functional brain parcellations constructed using fMRI data sets. We also describe how some of these evaluation methods have been used to estimate the optimal parcellation granularity. We provide a critical discussion of the current approach to the problem of identifying the optimal brain parcellation that is suited for a given neuroimaging study. We argue that the criteria for an optimal brain parcellation must depend on the application the parcellation is intended for. We describe a teleological approach to the evaluation of brain parcellations, where brain parcellations are evaluated in different contexts and optimal brain parcellations for each context are identified separately. We conclude by discussing several directions for further research that would result in improved evaluation strategies.

Perplexity: evaluating transcript abundance estimation in the absence of ground truth

BackgroundThere has been rapid development of probabilistic models and inference methods for transcript abundance estimation from RNA-seq data. These models aim to accurately estimate transcript-level abundances, to account for different biases in the measurement process, and even to assess uncertainty in resulting estimates that can be propagated to subsequent analyses. The assumed accuracy of the estimates inferred by such methods underpin gene expression based analysis routinely carried out in the lab. Although hyperparameter selection is known to affect the distributions of inferred abundances (e.g. producing smooth versus sparse estimates), strategies for performing model selection in experimental data have been addressed informally at best.ResultsWe derive perplexity for evaluating abundance estimates on fragment sets directly. We adapt perplexity from the analogous metric used to evaluate language and topic models and extend the metric to carefully account for corner cases unique to RNA-seq. In experimental data, estimates with the best perplexity also best correlate with qPCR measurements. In simulated data, perplexity is well behaved and concordant with genome-wide measurements against ground truth and differential expression analysis. Furthermore, we demonstrate theoretically and experimentally that perplexity can be computed for arbitrary transcript abundance estimation models.ConclusionsAlongside the derivation and implementation of perplexity for transcript abundance estimation, our study is the first to make possible model selection for transcript abundance estimation on experimental data in the absence of ground truth.

Reference-free method for investigating classification uncertainty in large-scale land cover datasets

In the last decade, land cover products are produced at a global scale and updated with an unprecedent speed with the development of earth observation and mapping techniques. However, assessing large-scale land cover datasets is always a challenging task because of lack of ground truth and high dependency on manual inspection. To promote the efficiency of assessment, this study introduces a generic framework that identifies potential land cover classification errors without traditional reference data. Depending on rich features in remote sensing data, the overall procedure of can be regarded as a multi-class anomaly detection problem. To improve the performance in dealing with complex classification scheme, a pairing strategy is firstly proposed for pairwise analysis on each two classes, so that the multi-class problem is decomposed into multiple smaller binary-class problems. Secondly, the proximity matrix of each class pair is generated on the basis of extra trees model, which is applicable to mixed numerical and categorical data and provides robust proximity measurement by including multiple levels of randomness. Finally, an improved proximity-based anomaly detection algorithm is applied, and the detection results of each class pair are ensembled to obtain the final anomaly score of each land instance. Experiments show that the overall area under the receiver operating characteristic curve score reaches approximately 0.9 for synthetic datasets and 0.8 for real-life datasets. The proposed method is expected to provide users valuable data for a more efficient assessment in the absence of ground truth in practice.