Knowledge Of Ground Truth Research Articles

Land cover mapping of mixed classes using 2D CNN with multi-frequency SAR data

Synthetic aperture radar (SAR) data obtained at multiple frequencies and polarizations offers valuable complementary information for classifying mixed classes that exhibit similar backscattering response. Although deep learning-based convolutional neural networks (CNNs) effectively extract features from multi-frequency SAR data, the arbitrary ordering of SAR features may hinder optimal convolution of the best feature sub-space for a specific class and underutilize available multi-frequency data. To address this, a novel CNN transforming SAR feature-space from 1-D to 2-D and employing varied dilation-rate convolutions is introduced. This transformation maximizes unique and localized feature combinations, efficiently utilizing the available feature sub-spaces and extracting discriminative features for accurate classifications, addressing the challenge of arbitrary band neighborhoods. Utilizing dual-polarization SAR data from ALOS-2 PALSAR-2 and Sentinel-1 sensors, the proposed CNN achieves an average f-score of 0.97 and a kappa coefficient of 0.97, an improvement of 11 %, 7 % and 3 % in OA compared to the 1-D, 2-D and 3-D CNN classifiers, without feature transformation. The classifier's generalization ability is evaluated using ground truth knowledge of various heterogeneous classes, and the proposed CNN classifier outperforms others in terms of accuracy metrics and generalization ability.

Addressing erroneous scale assumptions in microbe and gene set enrichment analysis.

By applying Differential Set Analysis (DSA) to sequence count data, researchers can determine whether groups of microbes or genes are differentially enriched. Yet sequence count data suffer from a scale limitation: these data lack information about the scale (i.e., size) of the biological system under study, leading some authors to call these data compositional (i.e., proportional). In this article, we show that commonly used DSA methods that rely on normalization make strong, implicit assumptions about the unmeasured system scale. We show that even small errors in these scale assumptions can lead to positive predictive values as low as 9%. To address this problem, we take three novel approaches. First, we introduce a sensitivity analysis framework to identify when modeling results are robust to such errors and when they are suspect. Unlike standard benchmarking studies, this framework does not require ground-truth knowledge and can therefore be applied to both simulated and real data. Second, we introduce a statistical test that provably controls Type-I error at a nominal rate despite errors in scale assumptions. Finally, we discuss how the impact of scale limitations depends on a researcher's scientific goals and provide tools that researchers can use to evaluate whether their goals are at risk from erroneous scale assumptions. Overall, the goal of this article is to catalyze future research into the impact of scale limitations in analyses of sequence count data; to illustrate that scale limitations can lead to inferential errors in practice; yet to also show that rigorous and reproducible scale reliant inference is possible if done carefully.

Transfer and persistence studies of inorganic and organic gunshot residues using synthetic skin membranes

The complex nature of gunshot residue (GSR) transfer and persistence introduces challenges and skepticism in its evidential value. Therefore, this work evaluates the behavior and movement of inorganic and organic gunshot residues to assist in the evidence interpretation. The study encompassed over 650 samples, including 247 collections from human skin after firing a gun and 405 synthetic skin and fabric substrates after depositing a characterized pGSR/OGSR standard. Transfer and persistence experiments were evaluated on different substrates (hands, ears, nostrils, forehead, hair, fabrics, and synthetic skin), at different times after firing (0 to 6 h), and common post-shooting activities (rubbing hands, handshaking, running, washing hands and fabrics). Ground truth knowledge of particle counts and analyte concentrations was used to calculate the recovery for inorganic and organic constituents from clothing and a synthetic skin membrane (StratM®). Authentic shooter skin samples were compared to synthetic skin to establish the validity of skin-substitute models. During controlled experiments, inorganic particles persisted longer than OGSR on inactive samples (9% and < 25% loss at 6 hrs, respectively), but inorganic particles were more prone to secondary transfer than OGSR (up to 35% vs. 0% transfer, respectively). High percentages of particles were lost during vigorous activities like washing hands (99% loss) or rubbing hands (55% loss). In comparison, less loss was observed during the same studies for OGSR (<21%). This study offers a deeper understanding of gunshot residue transfer and persistence mechanisms that can assist analysts and investigators in improving sample collection and interpretation of evidence.

In Silico Study of Local Electrical Impedance Measurements in the Atria - Towards Understanding and Quantifying Dependencies in Human.

Electrical impedance measurements have become an accepted tool for monitoring intracardiac radio frequency ablation. Recently, the long-established generator impedance was joined by novel local impedance measurement capabilities with all electrical circuit terminals being accommodated within the catheter. This work aims at in silico quantification of distinct influencing factors that have remained challenges due to the lack of ground truth knowledge and the superposition of effects in clinical settings. We introduced a highly detailed in silico model of two local impedance enabled catheters, namely IntellaNav MiFi OI and IntellaNav Stablepoint, embedded in a series of clinically relevant environments. Assigning material and frequency specific conductivities and subsequently calculating the spread of the electrical field with the finite element method yielded in silico local impedances. The in silico model was validated by comparison to in vitro measurements of standardized sodium chloride solutions. We then investigated the effect of the withdrawal of the catheter into the transseptal sheath, catheter-tissue interaction, insertion of the catheter into pulmonary veins, and catheter irrigation. All simulated setups were in line with in vitro experiments and in human measurements and gave detailed insight into determinants of local impedance changes as well as the relation between values measured with two different devices. The in silico environment proved to be capable of resembling clinical scenarios and quantifying local impedance changes. The tool can assists the interpretation of measurements in humans and has the potential to support future catheter development.

Spatial Validation of Spectral Unmixing Results: A Case Study of Venice City

Since remote sensing images offer unique access to the distribution of land cover on earth, many countries are investing in this technique to monitor urban sprawl. For this purpose, the most widely used methodology is spectral unmixing which, after identifying the spectra of the mixed-pixel constituents, determines their fractional abundances in the pixel. However, the literature highlights shortcomings in spatial validation due to the lack of detailed ground truth knowledge and proposes five key requirements for accurate reference fractional abundance maps: they should cover most of the area, their spatial resolution should be higher than that of the results, they should be validated using other ground truth data, the full range of abundances should be validated, and errors in co-localization and spatial resampling should be minimized. However, most proposed reference maps met two or three requirements and none met all five. In situ and remote data acquired in Venice were exploited to meet all five requirements. Moreover, to obtain more information about the validation procedure, not only reference spectra, synthetic image, and fractional abundance models (FAMs) that met all the requirements, but also other data, that no previous work exploited, were employed: reference fractional abundance maps that met four out of five requirements, and fractional abundance maps retrieved from the synthetic image. Briefly summarizing the main results obtained from MIVIS data, the average of spectral accuracies in root mean square error was equal to 0.025; using FAMs, the average of spatial accuracies in mean absolute error (MAEk-Totals) was equal to 1.32 and more than 78% of these values were related to sensor characteristics; using reference fractional abundance maps, the average MAEk-Totals value increased to 1.97 because errors in co-localization and spatial-resampling affected about 29% of these values. In conclusion, meeting all requirements and the exploitation of different reference data increase the spatial accuracy, upgrade the validation procedure, and improve the knowledge of accuracy.

Development of scalable lymphatic system in the 4D XCAT phantom: Application to quantitative evaluation of lymphoma PET segmentations.

Digital anthropomorphic phantoms, such as the 4D extended cardiac-torso (XCAT) phantom, are actively used to develop, optimize, and evaluate a variety of imaging applications, allowing for realistic patient modeling and knowledge of ground truth. The XCAT phantom defines the activity and attenuation for a simulated patient, which includes a complete set of organs, muscle, bone, and soft tissue, while also accounting for cardiac and respiratory motion. However, the XCAT phantom does not currently include the lymphatic system, critical for evaluating medical imaging tasks such as sentinel node detection, node density measurement, and radiation dosimetry. In this study, we aimed to develop a scalable lymphatic system in the XCAT phantom, to facilitate improved research of the lymphatic system in medical imaging. Using this scalable lymphatic system, we modeled the lymph node conglomerate pathology that is characteristically observed in primary mediastinal B-cell lymphoma (PMBCL). As an extended application, we evaluated positron emission tomography (PET) image quantification of metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of these simulated lymphomas, though the phantoms may be applied to other imaging modalities and study design paradigms (e.g., image quality, detection). A template model for the lymphatic system was developed based on anatomical data from the Visible Human Project of the National Library of Medicine. The segmented nodes and vessels were fit with non-uniform rational basis spline surfaces, and multichannel large deformation diffeomorphic metric mapping was used to propagate the template to different XCAT anatomies. To model conglomerates observed in PMBCL, lymph nodes were enlarged, converged within the mediastinum, and tracer concentration was increased. We used the phantoms as inputs to a PET simulation tool, which generated images using ordered subsets expectation maximization reconstruction with 2-8mm Gaussian filters. Fixed thresholding (FT) and gradient segmentation were used to determine MTV and TLG. Percent bias (%Bias) and coefficient of variation (COV) were computed as measures of accuracy and precision, respectively, for each MTV and TLG measurement. Using the methodology described above, we introduced a scalable lymphatic system in the XCAT phantom, which allows for the radioactivity and attenuation ground truth to be generated in 116±2.5s using a 2.3GHz processor. Within the Rhinoceros interface, lymph node anatomy and function were modified to create a cohort of 10 phantoms with lymph node conglomerates. Using the lymphoma phantoms to evaluate PET quantification of MTV, mean %Bias values were -9.3%, -41.3%, and 20.9%, while COV values were 4.08%, 7.6%, and 3.4% using 25% FT, 40% FT, and gradient segmentations, respectively. Comparatively for TLG, mean %Bias values were -27.4%, -45.8%, and -16.0%, while COV values were 1.9%, 5.7%, and 1.4%, for the 25% FT, 40% FT, and gradient segmentations, respectively. In this work, we upgraded the XCAT phantom to include a lymphatic system, comprised of a network of 276 scalable lymph nodes and corresponding vessels. As an application, we created a cohort of phantoms with lymph node conglomerates to evaluate lymphoma quantification in PET imaging, which highlights an important application of this work.

Physics-encoded deep learning in identifying battery parameters without direct knowledge of ground truth

We show a method to embed physical laws and on-line observation into machine learning so that irrelevant low-cost battery data can be utilized to identify complex system parameters by machine learning without knowledge of their ground truth as the training data. Lithium diffusivity, a complicated function of lithium concentration, is a crucial parameter for battery performance but difficult to measure directly. We take diffusivity as an example and show that it can be obtained from easily measured sequence of battery voltage over time. In simulations, our results show that this method accurately quantifies not only the diffusivities of both positive and negative electrodes, but also as complex non-linear functions of lithium concentration, purely based on the cell voltage data requiring neither diffusivity nor concentration measurement. Notably, it can accurately predict non-monotonic, many-to-one relations such as “w” shape functions. Moreover, this method is immune to measurement noise and capable of simultaneously estimating multiple parameters. In experiments, our method demonstrates more robust diffusivity estimation than a pure physics-based parameter fitting method and a widely used experimental technique. Our results suggest that the approach enables identifying physical parameters and their interdependence without direct measurements of those parameters.

Comparison of WLAN Probe and Light Sensor-Based Estimators of Bus Occupancy Using Live Deployment Data.

Bus company operators are interested in obtaining knowledge about the number of passengers on their buses—preferably doing so at low deployment costs and in an automated manner, while keeping accuracy high. One solution, widely used in practice, involves deploying a light sensor-based system, counting the people entering and leaving the bus. The light sensor system is simple, but errors accumulate over time, because it is not capable of error correcting. For this reason, the light sensor-based system is compared to a WLAN probe-based system, which has entirely different characteristics. Inaccuracy with the WLAN estimator comes from a need to filter out mobile devices outside the bus and to map the number of detected devices to a number of people. The comparison is performed based on data collected from a real-life deployment in a medium sized German city. The comparison shows the trade-off in selecting either of the two methods. Furthermore, a novel approach for fusion of the light sensor and WLAN estimators is proposed which has a big potential in improving accuracy of both estimators. A fusion approach is proposed that utilizes the different error characteristics for error compensation by calculating compensation terms. The knowledge of Ground Truth is not required as part of this fusion approach for calibration; results show that the approach can find the optimal parameter settings and that it makes this occupancy estimation approach scalable and automated.

SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics.

.SignificanceOptical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings.AimTo facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards.ApproachSIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA’s module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models.ResultsTo showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations.ConclusionsSIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.

Retrieval-Augmented Response Generation for Knowledge-Grounded Conversation in the Wild

Users on the internet usually have conversations on interesting facts or topics along with diverse knowledge from the web. However, most existing knowledge-grounded conversation models consider only a single document regarding the topic of a conversation. The recently proposed retrieval-augmented models generate a response based on multiple documents; however, they ignore the given topic and use only the local context of the conversation. To this end, we introduce a novel retrieval-augmented response generation model that retrieves an appropriate range of documents relevant to both the topic and local context of a conversation and uses them for generating a knowledge-grounded response. Our model first accepts both topic words extracted from the whole conversation and the tokens before the response to yield multiple representations. It then chooses representations of the first N token and ones of keywords from the conversation and document encoders and compares the two groups of representation from the conversation with those groups of the document, respectively. For training, we introduce a new data-weighting scheme to encourage the model to produce knowledge-grounded responses without ground truth knowledge. Both automatic and human evaluation results with a large-scale dataset show that our models can generate more knowledgeable, diverse, and relevant responses compared to the state-of-the-art models.

Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance

We present a method to generate synthetic thorax radiographs with realistic nodules from CT scans, and a perfect ground truth knowledge. We evaluated the detection performance of nine radiologists and two convolutional neural networks in a reader study. Nodules were artificially inserted into the lung of a CT volume and synthetic radiographs were obtained by forward-projecting the volume. Hence, our framework allowed for a detailed evaluation of CAD systems’ and radiologists’ performance due to the availability of accurate ground-truth labels for nodules from synthetic data. Radiographs for network training (U-Net and RetinaNet) were generated from 855 CT scans of a public dataset. For the reader study, 201 radiographs were generated from 21 nodule-free CT scans with altering nodule positions, sizes and nodule counts of inserted nodules. Average true positive detections by nine radiologists were 248.8 nodules, 51.7 false positive predicted nodules and 121.2 false negative predicted nodules. The best performing CAD system achieved 268 true positives, 66 false positives and 102 false negatives. Corresponding weighted alternative free response operating characteristic figure-of-merits (wAFROC FOM) for the radiologists range from 0.54 to 0.87 compared to a value of 0.81 (CI 0.75–0.87) for the best performing CNN. The CNN did not perform significantly better against the combined average of the 9 readers (p = 0.49). Paramediastinal nodules accounted for most false positive and false negative detections by readers, which can be explained by the presence of more tissue in this area.

OMG, I Have to Tweet that! A Study of Factors that Influence Tweet Rates

Many studies have shown that social data such as tweets are a rich source of information about the real-world including, for example, insights into health trends. A key limitation when analyzing Twitter data, however, is that it depends on people self-reporting their own behaviors and observations. In this paper, we present a large-scale quantitative analysis of some of the factors that influence self-reporting bias. In our study, we compare a year of tweets about weather events to ground-truth knowledge about actual weather occurrences. For each weather event we calculate how extreme, how expected, and how big a change the event represents. We calculate the extent to which these factors can explain the daily variations in tweet rates about weather events. We find that we can build global models that take into account basic weather information, together with extremeness, expectation and change calculations to account for over 40% of the variability in tweet rates. We build location-specific (i.e., a model per each metropolitan area) models that account for an average of 70% of the variability in tweet rates.

Biodiversity of Forest Elephants (Loxodonta Cyclotis) Diet in the Ogooué Leketi National Park, Congo Brazzaville

The Ogooué Leketi National Park (OLNP) is located within the Batéké-Léconi-Léfini Landscape in the central basin of the Congo River. The Ogooué Leketi Elephant Project (OLEP) area is important for biodiversity conservation from its significant populations of forest species (forest elephant, gorillas, chimpanzees, duikers, monkeys, etc.) combined with savanna species (Grimm’s duiker, side-striped jackal, etc). Elephant sign is highest 0.9/km in the northwest of the Landscape in the border area of the Batéké Plateau National Park in Gabon where a number of mineral-rich clearings, or bais, attract forest elephants and others wildlife. Elephants travel a long distance and consume diverse plants and occasionally animals’ species from one clearing to others throughout the forest. This paper intends to provide an overall preliminary list of plants consumed by forest elephants across their feeding trials. The results of the study points to the fact that, elephants were found to be feeding on 258 different vegetal species and three animal species recorded from twenty-six elephants feeding trials from March 2013 to December 2014 through different methods. Among these plant species, 156 were identified by us and checked by botanists in both the CERVE at Brazzaville and herbarium of Kinshasa University. Elephants were not only eating plants but they were also consuming occasionally some invertebrate such as bees, termites and ants. An analysis of 53 dung piles revealed that 43 dung piles had traces of 26 species of fruit consumed by elephants while 10 dung piles had no traces of fruits. The OLNP is among the least described protected areas in the Republic of Congo, despite its speculated high potential biodiversity. This lack of ground-truth knowledge is attributed to the status of the protected area that Congolese government classified as park in 2018 after 14 years of existence.

Enhancing manufacturing intelligence through an unsupervised data-driven methodology for cyclic industrial processes

Recent trends in intelligent manufacturing are transforming shop floor environments into digital factories, thanks to a pervasive integration of information and communication technologies in production lines. Industrial processes become the source of high-volume heterogeneous data, paving the way to create manufacturing intelligence by means of machine learning and data-driven methodologies. In such settings, predictive diagnostics play a crucial role, as they promise to predict future critical conditions in the production process. Unfortunately, the diffusion of data-driven predictive maintenance methodologies is limited by (i) the absence of timely ground-truth knowledge (i.e., class labels), required in the learning phase of data-driven supervised approaches, and (ii) the limited availability of data-mining expertise among application-domain experts, required to harness the power of machine learning techniques. Innovative data-driven services are needed to support domain experts in (i) applying powerful self-learning intelligent techniques with limited technical expertise and (ii) easily understanding results and choices operated by such intelligent techniques, to increase trust by means of transparency. To this aim, this paper presents UDaMP, an integrated platform to support manufacturing intelligence by providing a transparent, self-tuning, unsupervised discovery and assisted data labelling service for predictive maintenance, specifically targeted at cyclic industrial processes. UDaMP includes (i) production-cycle-aware feature engineering, (ii) unsupervised discovery of production-cycle categories, (iii) self-tuning of the optimal number of categories, (iv) human-readable characterisation of production-cycle categories, and (v) assisted data labelling for domain experts. Scalable clustering algorithms automatically discover groups of production cycles sharing common time-independent properties. A self-tuning strategy is integrated to automatically configure the specific input parameter and select the best approach for the data under analysis. Each cluster is then locally characterised through the data distribution of the top 10 most relevant features to support domain experts in uncovering its meaning. Experimental evaluation of UDaMP has been performed on real-world data collected in two different industrial settings.

Towards Topic-Aware Slide Generation For Academic Papers With Unsupervised Mutual Learning

Slides are commonly used to present information and tell stories. In academic and research communities, slides are typically used to summarize findings in accepted papers for presentation in meetings and conferences. These slides for academic papers usually contain common and essential topics such as major contributions, model design, experiment details and future work. In this paper, we aim to automatically generate slides for academic papers. We first conducted an in-depth analysis of how humans create slides. We then mined frequently used slide topics. Given a topic, our approach extracts relevant sentences in the paper to provide the draft slides. Due to the lack of labeling data, we integrate prior knowledge of ground truth sentences into a log-linear model to create an initial pseudo-target distribution. Two sentence extractors are learned collaboratively and bootstrap the performance of each other. Evaluation results on a labeled test set show that our model can extract more relevant sentences than baseline methods. Human evaluation also shows slides generated by our model can serve as a good basis for preparing the final presentations.

Permuted Spectral and Permuted Spectral-Spatial CNN Models for PolSAR-Multispectral Data based Land Cover Classification

ABSTRACT It is a challenge to develop methods which can process the polarimetric synthetic aperture radar (PolSAR) and multispectral (MS) data modalities together without losing information from either for remote sensing applications. This paper presents a study which attempts to introduce novel deep learning-based remote sensing data processing frameworks that utilize convolutional neural networks (CNNs) in both spatial and spectral domains to perform land cover (LC) classification with PolSAR-MS data. Also since earth observation remotely sensed data have usually larger spectral depth than normal camera image data, exploiting the spectral information in remote sensing (RS) data is crucial as well. In fact, convolutions in the sub-spectral space are intuitive and alternative to the process of feature selection. Recently, researchers have gained success in exploiting the spectral information of RS data, especially the hyperspectral data with CNNs. In this paper, exploitation of the spectral information in the PolSAR-MS data via a permuted localized spectral convolution along with localized spatial convolution is proposed. Further, the study in this paper also establishes the significance of performing permuted localized spectral convolutions over non-localized or localized spectral convolutions. Two models are proposed, namely a permuted local spectral convolutional network (Perm-LS-CNN) and a permuted local spectral-spatial convolutional network (Perm-LSS-CNN). These models are trained on ground truth class data points measured directly on the terrain. The evaluation of the generalization performance is done using ground truth knowledge on selected well-known regions in the study areas. Comparison with other popular machine learning classifiers shows that the Perm-LSS-CNN model provides better classification results in terms of both accuracy and generalization.

Quantifying accuracy and heterogeneity in single-molecule super-resolution microscopy

The resolution and accuracy of single-molecule localization microscopes (SMLMs) are routinely benchmarked using simulated data, calibration rulers, or comparisons to secondary imaging modalities. However, these methods cannot quantify the nanoscale accuracy of an arbitrary SMLM dataset. Here, we show that by computing localization stability under a well-chosen perturbation with accurate knowledge of the imaging system, we can robustly measure the confidence of individual localizations without ground-truth knowledge of the sample. We demonstrate that our method, termed Wasserstein-induced flux (WIF), measures the accuracy of various reconstruction algorithms directly on experimental 2D and 3D data of microtubules and amyloid fibrils. We further show that WIF confidences can be used to evaluate the mismatch between computational models and imaging data, enhance the accuracy and resolution of reconstructed structures, and discover hidden molecular heterogeneities. As a computational methodology, WIF is broadly applicable to any SMLM dataset, imaging system, and localization algorithm.

Evaluation of Golden-Angle-Sampled Dynamic Contrast-Enhanced MRI Reconstruction Using Objective Image Quality Measures: A Simulated Phantom Study.

We aim to extend the use of image quality metrics (IQMs) from static magnetic resonance imaging (MRI) applications to dynamic MRI studies. We assessed the use of 2 IQMs, the root mean square error and structural similarity index, in evaluating the reconstruction of quantitative dynamic contrast-enhanced (DCE) MRI data acquired using golden-angle sampling and compressed sensing (CS). To address the difficulty of obtaining ground-truth knowledge of parameters describing dynamics in real patient data, we developed a Matlab simulation framework to assess quantitative CS-DCE-MRI. We began by validating the response of each IQM to the CS-MRI reconstruction process using static data and the performance of our simulation framework with simple dynamic data. We then extended the simulations to the more realistic extended Tofts model. When assessing the Tofts model, we tested 4 different methods of selecting a reference image for the IQMs. Results from the retrospective static CS-MRI reconstructions showed that each IQM is responsive to the CS-MRI reconstruction process. Simulations of a simple contrast evolution model validated the performance of our framework. Despite the complexity of the Tofts model, both IQM scores correlated well with the recovery accuracy of a central model parameter for all reference cases studied. This finding may form the basis of algorithms for automated selection of image reconstruction aspects, such as temporal resolution, in golden-angle-sampled CS-DCE-MRI. These further suggest that objective measures of image quality may find use in general dynamic MRI applications.

Comparative Analysis of Supervised and Unsupervised Approaches Applied to Large-Scale “In The Wild” Face Verification

Deep learning-based feature extraction methods and transfer learning have become common approaches in the field of pattern recognition. Deep convolutional neural networks trained using tripled-based loss functions allow for the generation of face embeddings, which can be directly applied to face verification and clustering. Knowledge about the ground truth of face identities might improve the effectiveness of the final classification algorithm; however, it is also possible to use ground truth clusters previously discovered using an unsupervised approach. The aim of this paper is to evaluate the potential improvement of classification results of state-of-the-art supervised classification methods trained with and without ground truth knowledge. In this study, we use two sufficiently large data sets containing more than 200,000 “taken in the wild” images, each with various resolutions, visual quality, and face poses which, in our opinion, guarantee the statistical significance of the results. We examine several clustering and supervised pattern recognition algorithms and find that knowledge about the ground truth has a very small influence on the Fowlkes–Mallows score (FMS) of the classification algorithm. In the case of the classification algorithm that obtained the highest accuracy in our experiment, the FMS improved by only 5.3% (from 0.749 to 0.791) in the first data set and by 6.6% (from 0.652 to 0.718) in the second data set. Our results show that, beside highly secure systems in which face verification is a key component, face identities discovered by unsupervised approaches can be safely used for training supervised classifiers. We also found that the Silhouette Coefficient (SC) of unsupervised clustering is positively correlated with the Adjusted Rand Index, V-measure score, and Fowlkes–Mallows score and, so, we can use the SC as an indicator of clustering performance when the ground truth of face identities is not known. All of these conclusions are important findings for large-scale face verification problems. The reason for this is the fact that skipping the verification of people’s identities before supervised training saves a lot of time and resources.

Cross-Validation Approach to Evaluate Clustering Algorithms: An Experimental Study Using Multi-Label Datasets

Clustering validation is one of the most important and challenging parts of clustering analysis, as there is no ground truth knowledge to compare the results with. Up till now, the evaluation methods for clustering algorithms have been used for determining the optimal number of clusters in the data, assessing the quality of clustering results through various validity criteria, comparison of results with other clustering schemes, etc. It is also often practically important to build a model on a large amount of training data and then apply the model repeatedly to smaller amounts of new data. This is similar to assigning new data points to existing clusters which are constructed on the training set. However, very little practical guidance is available to measure the prediction strength of the constructed model to predict cluster labels for new samples. In this study, we proposed an extension of the cross-validation procedure to evaluate the quality of the clustering model in predicting cluster membership for new data points. The performance score was measured in terms of the root mean squared error based on the information from multiple labels of the training and testing samples. The principal component analysis (PCA) followed by k-means clustering algorithm was used to evaluate the proposed method. The clustering model was tested using three benchmark multi-label datasets and has shown promising results with overall RMSE of less than 0.075 and MAPE of less than 12.5% in three datasets.