Data Instances Research Articles

CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation

Obtaining well-calibrated photometric redshift probability densities for galaxies without a spectroscopic measurement remains a challenge. Deep learning discriminative models, typically fed with multi-band galaxy images, can produce outputs that mimic probability densities and achieve state-of-the-art accuracy. However, several previous studies have found that such models may be affected by miscalibration, an issue that would result in discrepancies between the model outputs and the actual distributions of true redshifts. Our work develops a novel method called the C ontrastive L earning and A daptive KNN for P hotometric Redshift (CLAP) that resolves this issue. It leverages supervised contrastive learning (SCL) and $k$-nearest neighbours (KNN) to construct and calibrate raw probability density estimates, and implements a refitting procedure to resume end-to-end discriminative models ready to produce final estimates for large-scale imaging data, bypassing the intensive computation required for KNN. The harmonic mean is adopted to combine an ensemble of estimates from multiple realisations for improving accuracy. Our experiments demonstrate that CLAP takes advantage of both deep learning and KNN, outperforming benchmark methods on the calibration of probability density estimates and retaining high accuracy and computational efficiency. With reference to CLAP, a deeper investigation on miscalibration for conventional deep learning is presented. We point out that miscalibration is particularly sensitive to the method-induced excessive correlations among data instances in addition to the unaccounted-for epistemic uncertainties. Reducing the uncertainties may not guarantee the removal of miscalibration due to the presence of such excessive correlations, yet this is a problem for conventional methods rather than CLAP. These discussions underscore the robustness of CLAP for obtaining photometric redshift probability densities required by astrophysical and cosmological applications. This is the first paper in our series on CLAP.

Accelerating spin Hall conductivity predictions via machine learning

AbstractAccurately predicting the spin Hall conductivity (SHC) is crucial for designing novel spintronic devices that leverage the spin Hall effect. First‐principles calculations of SHCs are computationally intensive and unsuitable for quick high‐throughput screening. Here, we have developed a residual crystal graph convolutional neural network (Res‐CGCNN) deep learning model to classify and predict SHCs solely based on the structural and compositional information. This is enabled by having access to 9249 instances of SHCs data and incorporating extra residual networks into the standard CGCNN framework. We found that Res‐CGCNN surpasses CGCNN, achieving a mean absolute error of 115.4 (ℏ/e) (S/cm) for regression and an area under the receiver operating characteristic curve of 0.86 for classification. Additionally, we utilized Res‐CGCNN to conduct high‐throughput screenings of materials in the Materials Project database that were absent in the training set. This led to the prediction of several previously unreported materials displaying large SHCs exceeding 1000 (ℏ/e) (S/cm), which were validated through first‐principles calculations. This study represents the inaugural endeavor to construct a machine learning model capable of effectively capturing the intricate nonlinear relationship between SHCs and crystal structure and composition, serving as a useful tool for the efficient screening and design of materials exhibiting high SHCs.

Topology sensing of FANET under missing data

The topological structure of a flying ad hoc network (FANET) is crucial to understand, explain, and predict the behavior of unmanned aerial vehicle (UAV) swarms. Most studies focusing on topology sensing use perfect observations and complete datasets. However, the received signal dataset, being non-cooperative, commonly encounters instances of missing data, causing the performance of the existing algorithms to degrade. We investigate the issue of topology sensing of FANET based on external observations and propose a topology sensing method for FANET with missing data while introducing link-prediction methods to correct the topology inference results. First, we employ multi-dimensional Hawkes processes to model the communication event sequence in the network. Subsequently, to solve the problem in which the binary decision threshold is difficult to determine and cannot be adapted to the application scenario, we propose an extended multi-dimensional Hawkes model suitable for FANET and use the maximum likelihood estimation method for topology inference. Finally, to solve the problem of the low accuracy of inference results owing to missing data, we perform community detection on the observation network and combine the community detection and inference results to construct a mixed connection probability matrix, based on which we perform topology correction. The results of the analysis show that the topology sensing method proposed in this study is robust against missing data, indicating that it is an effective solution for solving this problem.

A Generic Transformation Approach for Complex Laboratory Data Using the Fast Healthcare Interoperability Resources Mapping Language: Method Development and Implementation.

Reaching meaningful interoperability between proprietary health care systems is a ubiquitous task in medical informatics, where communication servers are traditionally used for referring and transforming data from the source to target systems. The Mirth Connect Server, an open-source communication server, offers, in addition to the exchange functionality, functions for simultaneous manipulation of data. The standard Fast Healthcare Interoperability Resources (FHIR) has recently become increasingly prevalent in national health care systems. FHIR specifies its own standardized mechanisms for transforming data structures using StructureMaps and the FHIR mapping language (FML). In this study, a generic approach is developed, which allows for the application of declarative mapping rules defined using FML in an exchangeable manner. A transformation engine is required to execute the mapping rules. FHIR natively defines resources to support the conversion of instance data, such as an FHIR StructureMap. This resource encodes all information required to transform data from a source system to a target system. In our approach, this information is defined in an implementation-independent manner using FML. Once the mapping has been defined, executable Mirth channels are automatically generated from the resources containing the mapping in JavaScript format. These channels can then be deployed to the Mirth Connect Server. The resulting tool is called FML2Mirth, a Java-based transformer that derives Mirth channels from detailed declarative mapping rules based on the underlying StructureMaps. Implementation of the translate functionality is provided by the integration of a terminology server, and to achieve conformity with existing profiles, validation via the FHIR validator is built in. The system was evaluated for its practical use by transforming Labordatenträger version 2 (LDTv.2) laboratory results into Medical Information Object (Medizinisches Informationsobjekt) laboratory reports in accordance with the National Association of Statutory Health Insurance Physicians' specifications and into the HL7 (Health Level Seven) Europe Laboratory Report. The system could generate complex structures, but LDTv.2 lacks some information to fully comply with the specification. The tool for the auto-generation of Mirth channels was successfully presented. Our tests reveal the feasibility of using the complex structures of the mapping language in combination with a terminology server to transform instance data. Although the Mirth Server and the FHIR are well established in medical informatics, the combination offers space for more research, especially with regard to FML. Simultaneously, it can be stated that the mapping language still has implementation-related shortcomings that can be compensated by Mirth Connect as a base technology.

Generative learning for forecasting the dynamics of high-dimensional complex systems

We introduce generative models for accelerating simulations of high-dimensional systems through learning and evolving their effective dynamics. In the proposed Generative Learning of Effective Dynamics (G-LED), instances of high dimensional data are down sampled to a lower dimensional manifold that is evolved through an auto-regressive attention mechanism. In turn, Bayesian diffusion models, that map this low-dimensional manifold onto its corresponding high-dimensional space, operate on batches of physics correlated, time sequences of data and capture the statistics of the system dynamics. We demonstrate the capabilities and drawbacks of G-LED in simulations of several benchmark systems, including the Kuramoto-Sivashinsky (KS) equation, two-dimensional high Reynolds number flow over a backward-facing step, and simulations of three-dimensional turbulent channel flow. The results demonstrate that generative learning offers new frontiers for the accurate forecasting of the statistical properties of high-dimensional systems at a reduced computational cost.

Quarta: quantum supervised and unsupervised learning for binary classification in domain-incremental learning

Quantum machine learning recently gained prominence due to the promise of quantum computers in solving machine learning problems that are intractable on a classical computer. Nevertheless, several studies on problems which remain challenging for classical computing algorithms are emerging. One of these is classifying continuously incoming data instances in incremental fashion, which is studied in this paper through a hybrid computational solution that combines classical and quantum techniques. Hybrid approaches represents one of the current ways for the use of quantum computation in practical applications. In this paper, we show how typical issues of domain-incremental learning can be equally addressed with the properties of quantum mechanics, until to offer often better results. We propose the framework QUARTA to combine algorithms of quantum supervised learning, that is, variational quantum circuits, and techniques used in quantum unsupervised learning, that is, distance estimation. We aim at keeping the classification capabilities, which have learned on previously processed data instances, preserved as much as possible, and then acquiring new knowledge on new data instances. Experiments are performed on real-world datasets with quantum simulators.

FCSSL: fusion enhanced contrastive self-supervised learning method for parallel MRI reconstruction

Objective. The implementation of deep learning in magnetic resonance imaging (MRI) has significantly advanced the reduction of data acquisition times. However, these techniques face substantial limitations in scenarios where acquiring fully sampled datasets is unfeasible or costly.Approach. To tackle this problem, we propose a fusion enhanced contrastive self-supervised learning (FCSSL) method for parallel MRI reconstruction, eliminating the need for fully sampledk-space training dataset and coil sensitivity maps. First, we introduce a strategy based on two pairs of re-undersampling masks within a contrastive learning framework, aimed at enhancing the representational capacity to achieve higher quality reconstruction. Subsequently, a novel adaptive fusion network, trained in a self-supervised learning manner, is designed to integrate the reconstruction results of the framework.Results. Experimental results on knee datasets under different sampling masks demonstrate that the proposed FCSSL achieves superior reconstruction performance compared to other self-supervised learning methods. Moreover,the performance of FCSSL approaches that of the supervised methods, especially under the 2DRU and RADU masks, but no need for fully sample data. The proposed FCSSL, trained under the 3× 1DRU and 2DRU masks, can effectively generalize to unseen 1D and 2D undersampling masks, respectively. For target domain data that exhibit significant differences from source domain data, the proposed model, fine-tuned with just a few dozen instances of undersampled data in the target domain, achieves reconstruction performance comparable to that achieved by the model trained with the entire set of undersampled data.Significance. The novel FCSSL model offers a viable solution for reconstructing high-quality MR images without needing fully sampled datasets, thereby overcoming a major hurdle in scenarios where acquiring fully sampled MR data is difficult.

When two chains meet: Optimizing the design of blockchain-enabled supply chain networks

Applications of the blockchain technology are rapidly emerging across sectors. By redefining information flow, blockchain technology promises to instill trust among supply chain members, improve transaction efficiency, thus reshaping the structure and configuration of supply chains. The practical exploratory use of blockchain has shown immense potential in supply chain management. In order to explore the new changes brought by blockchain technology to supply chain network design issues, this study focuses on the supply chain network design problem in the context of blockchain technology. Specifically, we consider a supply chain network comprised of a number of distribution centers (DCs) and retailers. The operation of this physical network is supported by blockchain-based information infrastructure. As the interrelated relationship between these two networks, it is necessary to optimize them simultaneously to improve the profitability and information exchange level. We propose an objective function based on previous studies to maximize the profits of the entire supply chain network, with decision variables including DC locations, retailer assignment, inventory replenishment policy, and blockchain adoption level. At the same time, to characterize the interrelation between the two networks, the demand and certain cost parameters of the model become endogenous depending on the blockchain adoption level, which significantly increases the complexity of the solution algorithm. We reformulate the model and use the polymatroid cutting plane approach to address this issue. We conducted calculations and numerical analysis. The computational results show that the proposed method can solve practically sized problems, and integrating blockchain design reduces the system-wide cost. We discuss managerial insights based on a range of numerical studies and solve a case study using publicly available instance data.

From shallows to depths: unveiling hybrid synthetic data modeling for enhanced learning with privacy considerations in naturally imbalanced datasets

Data serves as the foundational element that drives model development and performance in machine learning and deep learning. The learning algorithms are as good as data on which they are trained on. Data constrained environments pose serious threat to the effectiveness of learning algorithms. Data limitations stem from a range of factors, including regulatory restrictions, privacy concerns, and the inherent scarcity of relevant data. This constrained availability of data often leads to class imbalance problem in the context of classification tasks. To address the challenges of limited data, the algorithmic generated data, also known as synthetic data, is gaining significant traction as a cost-effective, readily available, and secure alternative. Synthetic data generation techniques can be employed to enhance dataset size by augmenting data samples and to address class imbalance by increasing the number of minority class instances. These techniques generally fall into two main categories: distance-based shallow models and probability estimation-based deep generative models. Shallow interpolation-based models generate new data points within the local space between existing data points, while deep density estimation based models generate new data by learning the whole distribution of data. . In the context of smaller datasets, deep generative models often struggle to accurately estimate the probability distribution of whole data. To effectively represent the global data distribution, these deep models require initial starting data samples to guide their approximation. This paper examines the potential of integration of shallow and deep generative models in the data generation pipeline for effective synthetic data augmentation which furthers enhanced learning and generalization of downstream tasks. In this work, we present a hybrid approach of tabular data generation involving mixed type data attributes (continuous, discrete) and pay special attention to data imbalance and insufficient data problems. We introduce the Hybrid Data Balancing and Augmentation Approach for Mixed Tabular Data (HDBA-MTD), specifically designed to synthesize samples for underrepresented labels of output class and address issues of insufficient data instances. This approach enhances training data diversity, thereby paying special attention to the downstream classification and generalization performance. Experiments are carried out using benchmark datasets to assess the practicality of the presented hybrid model in real-world scenarios. This work has also attempted to quantify the privacy preservability for real data concerning ethical considerations and data security circumstances. The evaluation and analysis of these experiments show that the present hybrid model performs favorably compared to other current hybrid synthetic data generation methods.

VoiceStyle: Voice-Based Face Generation via Cross-Modal Prototype Contrastive Learning

Can we predict a person’s appearance solely based on their voice? This article explores this question by focusing on generating a face from an unheard voice segment. Our proposed method, VoiceStyle, combines cross-modal representation learning with generation modeling, enabling us to incorporate voice semantic cues into the generated face. In the first stage, we introduce cross-modal prototype contrastive (CMPC) learning to establish the association between voice and face. Recognizing the presence of false negative and deviate positive instances in real-world unlabeled data, we not only use voice–face pairs in the same video but also construct additional semantic positive pairs through unsupervised clustering, enhancing the learning process. Moreover, we recalibrate instances based on their similarity to cluster centers in the other modality. In the second stage, we harness the powerful generative capabilities of StyleGAN to produce faces. We optimize the latent code in StyleGAN’s latent space, guided by the learned voice–face alignment. To address the importance of selecting an appropriate starting point for optimization, we aim to automatically find an optimal starting point by utilizing the face prototype derived from the voice input. The entire pipeline can be implemented in a self-supervised manner, eliminating the need for manually labeled annotations. Through extensive experiments, we demonstrate the effectiveness and performance of our VoiceStyle method in both cross-modal representation learning and voice-based face generation.

Sensitive Data Flagging within Data Quality reports: R and Regex Integration for Effective Text Flagging in Large Datasets

ObjectiveWe’re introducing a comprehensive approach to enhance sensitive data flagging within our Data Quality (DQ) tool. Within de-identified health records, sensitive data such as user ids, post codes, phone numbers, and addresses pose significant privacy risks if exposed in their raw form. To address this challenge, we propose using all necessary regex patterns and free text field checks for sensitive data flagging with the objective of efficient detection within large datasets and ultimately reporting such data. ApproachA regex map has been developed along free text field thresholds, distribution counts and percentage checks to allow a generic approach. This ensures that any new regexes can be added to the flagging process without significant code changes. Once identified, sensitive data instances, including fields that contain free text are flagged and displayed in an html report that is human readable by DQ reviewers. ResultsThrough a combination of SQL and R regex based text processing, our approach allows a seamless identification and flagging of sensitive data within datasets. Sensitive data for large tables of +25 million records with over 50 columns are getting flagged in less than 150 seconds (approx. 2 minutes). ConclusionOur development offers a practical solution for sensitive data flagging in complex datasets where team members can implement robust sensitive data flagging by adding more regex formulas in the codebase. ImplicationsDQ reviewers can inspect the sensitive flagged fields in a human readable way, providing an extra measure of confidence when determining the quality of de-identified health records.

A Novel Approach for Model Interpretability and Domain Aware Fine-Tuning in AdaBoost

AbstractThe success of machine learning in real-world use cases has increased its demand in mission-critical applications such as autonomous vehicles, healthcare and medical diagnosis, aviation and flight safety, natural disaster prediction, early warning systems, etc. Adaptive Boosting (AdaBoost) is an ensemble learning method that has gained much traction in such applications. Inherently being a non-interpretable model, the interpretability of the AdaBoost algorithm has been a research topic for many years. Furthermore, most of the research being conducted till now is aimed at explaining AdaBoost using perturbation-based techniques. The paper presents a technique to interpret the AdaBoost algorithm from a data perspective using deletion diagnostics and Cook’s distance. The technique achieves interpretability by detecting the most influential data instances and their impact on the feature importance of the model. This interpretability enables domain experts to accurately modify the significance of specific features in a trained AdaBoost model depending on the data instances. Unlike explaining AdaBoost using perturbation-based techniques, interpreting from a data perspective will enable it to debug data-related biases, errors and to impart the knowledge of the domain experts into the model through domain aware fine-tuning. Experimental studies were conducted with diverse real-world multi-feature datasets to demonstrate interpretability and knowledge integration through domain-aware fine-tuning.

Experience: A Comparative Analysis of Multivariate Time-Series Generative Models: A Case Study on Human Activity Data

Human activity recognition (HAR) is an active research field that has seen great success in recent years due to advances in sensory data collection methods and activity recognition systems. Deep artificial intelligence (AI) models have contributed to the success of HAR systems lately although still suffering from limitations such as data scarcity, the high costs of labelling data instances and datasets’ imbalance and bias. The temporal nature of human activity data, represented as time series data, impose an additional challenge to using AI models in HAR because most state-of-the-art models do not account for the time component of the data instances. These limitations have inspired the time-series research community to design generative models for sequential data but very little work has been done to evaluate the quality of such models. In this work, we conduct a comparative quality analysis of three generative models for time-series data, using a case study in which we aim to generate sensory human activity data from a seed public dataset. Additionally, we adapt and clearly explain four evaluation methods of synthetic time-series data from the literature and apply them to assess the quality of the synthetic activity data we generate. We show experimentally that high quality human activity data can be generated using deep generative models, and the synthetic data can thus be used in HAR systems to augment real activity data. We also demonstrate that the chosen evaluation methods effectively ensure that the generated data meets the essential quality benchmarks of realism, diversity, coherence and utility. Our findings suggest that using deep generative models to produce synthetic human activity data can potentially address challenges related to data scarcity, biases, and expensive labeling. This holds promise for enhancing the efficiency and reliability of HAR systems.

Deep learning-based prediction of indication for cracked tooth extraction using panoramic radiography

BackgroundWe aimed to determine the feasibility of utilizing deep learning-based predictions of the indications for cracked tooth extraction using panoramic radiography.MethodsPanoramic radiographs of 418 teeth (group 1: 209 normal teeth; group 2: 209 cracked teeth) were evaluated for the training and testing of a deep learning model. We evaluated the performance of the cracked diagnosis model for individual teeth using InceptionV3, ResNet50, and EfficientNetB0. The cracked tooth diagnosis model underwent fivefold cross-validation with 418 data instances divided into training, validation, and test sets at a ratio of 3:1:1.ResultsTo evaluate the feasibility, the sensitivity, specificity, accuracy, and F1 score of the deep learning models were calculated, with values of 90.43–94.26%, 52.63–60.77%, 72.01–75.84%, and 76.36–79.00%, respectively.ConclusionWe found that the indications for cracked tooth extraction can be predicted to a certain extent through a deep learning model using panoramic radiography.

Improving the sustainability of WiFi-enabled indoor localization systems through meta-heuristic based instance selection approach

Due to rapid urban development, many indoor and semi-indoor structures, such as underground malls, metro stations, and bus-stops, have emerged. Traditional GNSS outdoor navigation fails in these locations. To meet urban citizens’ needs, we require sustainable ubiquitous localization solutions, leveraging existing WiFi infrastructure. In such applications, processing data at the edge within constrained environments is imperative. Many prior studies neglect the practical challenge of implementing machine learning-based edge device localization, as they assume reliance on cloud servers. Selecting well-distributed data instances that preserve location class boundaries is essential for effective training in such conditions. In this work, we propose an instance selection approach that is modeled by applying a modified Binary Particle Swarm Optimization (BPSO) technique. The k-differentiating neighbor-based measure is incorporated to determine instance hardness while exploring the solution space through BPSO. The work is implemented on three publicly available benchmark datasets that are collected from two university campuses and one shopping mall. The proposed work is found to have achieved an instance reduction of 35% with only 1%–2% decrease in the accuracy measurement and an appreciable error deviation metric of 2.78 m.

Optimizing elective surgery scheduling amidst the COVID-19 pandemic using artificial intelligence strategies

The COVID-19 pandemic profoundly affects elective surgery and healthcare resources. Efficient management of resources, like ward capacity and operating theaters, is crucial. The operations research community explores solutions, notably leveraging artificial intelligence, to address scheduling challenges amid COVID-19 restrictions. In this situation, applying AI becomes essential to getting the best results. In this paper, we address the problem of daily scheduling elective surgeries while accounting for hospital ward capacity. It is possible to reduce this issue to a scheduling puzzle that, given a variety of restrictions, resembles a four-stage hybrid flow shop. These limitations include the availability of resources, patient flow control, wait time avoidance, patient prioritizing, and resource coordination. With the crucial aid of artificial intelligence, our main goal is to assign patients to different surgical resources to minimize the length of time they spend on average in the hospital ward. We suggest putting into practice effective optimization strategies that make use of AI-based algorithms, particularly the variable neighborhood search (VNS) and variable neighborhood descent (VND) algorithms, which are inextricably linked with artificial intelligence concepts. Our studies demonstrate the effectiveness and efficiency of the general VNS in addressing the daily elective surgical scheduling issue (SSP) with the priceless assistance of artificial intelligence. The experiments are based on novel data instances that were inspired by current literature guidelines. The test results conclusively demonstrate the ability of our algorithms to find virtually perfect solutions. Moreover, our results highlight that the use of these methods, strengthened by AI, can significantly increase the size of the solved issue by a remarkable factor of 19.54. In light of the current COVID-19 pandemic, AI thus becomes a key factor in optimizing the scheduling of elective surgeries and the allocation of resources.

Synthesising Diverse and Discriminatory Sets of Instances using Novelty Search in Combinatorial Domains.

Gathering sufficient instance data to either train algorithm-selection models or understand algorithm footprints within an instance space can be challenging. We propose an approach to generating synthetic instances that are tailored to perform well with respect to a target algorithm belonging to a predefined portfolio but are also diverse with respect to their features. Our approach uses a novelty search algorithm with a linearly weighted fitness function that balances novelty and performance to generate a large set of diverse and discriminatory instances in a single run of the algorithm. We consider two definitions of novelty: (1) with respect to discriminatory performance within a portfolio of solvers; (2) with respect to the features of the evolved instances. We evaluate the proposed method with respect to its ability to generate diverse and discriminatory instances in two domains (knapsack and bin-packing), comparing to another well-known quality diversity method, Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) and an evolutionary algorithm that only evolves for discriminatory behaviour. The results demonstrate that the novelty search method outperforms its competitors in terms of coverage of the space and its ability to generate instances that are diverse regarding the relative size of the "performance gap" between the target solver and the remaining solvers in the portfolio. Moreover, for the Knapsack domain, we also show that we are able to generate novel instances in regions of an instance space not covered by existing benchmarks using a portfolio of state-of-the-art solvers. Finally, we demonstrate that the method is robust to different portfolios of solvers (stochastic approaches, deterministic heuristics and state-of-the-art methods), thereby providing further evidence of its generality.

A dual algorithmic approach to deal with multiclass imbalanced classification problems

Many real-world applications involve multiclass classification problems, and often the data across classes is not evenly distributed. Due to this disproportion, supervised learning models tend to classify instances towards the class with the maximum number of instances, which is a severe issue that needs to be addressed. In multiclass imbalanced data classification, machine learning researchers try to reduce the learning model's bias towards the class with a high sample count. Researchers attempt to reduce this unfairness by either balancing the data before the classifier learns it, modifying the classifier's learning phase to pay more attention to the class with a minimum number of instances, or a combination of both. The existing algorithmic approaches find it difficult to understand the clear boundary between the samples of different classes due to unfair class distribution and overlapping issues. As a result, the minority class recognition rate is poor. A new algorithmic approach is proposed that uses dual decision trees. One is used to create an induced dataset using a PCA based grouping approach and by assigning weights to the data samples followed by another decision tree to learn and predict from the induced dataset. The distinct feature of this algorithmic approach is that it recognizes the data instances without altering their underlying data distribution and is applicable for all categories of multiclass imbalanced datasets. Five multiclass imbalanced datasets from UCI were used to classify the data using our proposed algorithm, and the results revealed that the duo-decision tree approach pays better attention to both the minor and major class samples.

A Grid-Based Method for Removing Overlaps of Dimensionality Reduction Scatterplot Layouts.

Dimensionality Reduction (DR) scatterplot layouts have become a ubiquitous visualization tool for analyzing multidimensional datasets. Despite their popularity, such scatterplots suffer from occlusion, especially when informative glyphs are used to represent data instances, potentially obfuscating critical information for the analysis under execution. Different strategies have been devised to address this issue, either producing overlap-free layouts that lack the powerful capabilities of contemporary DR techniques in uncovering interesting data patterns or eliminating overlaps as a post-processing strategy. Despite the good results of post-processing techniques, most of the best methods typically expand or distort the scatterplot area, thus reducing glyphs' size (sometimes) to unreadable dimensions, defeating the purpose of removing overlaps. This article presents Distance Grid (DGrid), a novel post-processing strategy to remove overlaps from DR layouts that faithfully preserves the original layout's characteristics and bounds the minimum glyph sizes. We show that DGrid surpasses the state-of-the-art in overlap removal (through an extensive comparative evaluation considering multiple different metrics) while also being one of the fastest techniques, especially for large datasets. A user study with 51 participants also shows that DGrid is consistently ranked among the top techniques for preserving the original scatterplots' visual characteristics and the aesthetics of the final results.

Uncovering Harmonization Potential in Health Care Data Through Iterative Refinement of Fast Healthcare Interoperability Resources Profiles Based on Retrospective Discrepancy Analysis: Case Study.

Cross-institutional interoperability between health care providers remains a recurring challenge worldwide. The German Medical Informatics Initiative, a collaboration of 37 university hospitals in Germany, aims to enable interoperability between partner sites by defining Fast Healthcare Interoperability Resources (FHIR) profiles for the cross-institutional exchange of health care data, the Core Data Set (CDS). The current CDS and its extension modules define elements representing patients' health care records. All university hospitals in Germany have made significant progress in providing routine data in a standardized format based on the CDS. In addition, the central research platform for health, the German Portal for Medical Research Data feasibility tool, allows medical researchers to query the available CDS data items across many participating hospitals. In this study, we aimed to evaluate a novel approach of combining the current top-down generated FHIR profiles with the bottom-up generated knowledge gained by the analysis of respective instance data. This allowed us to derive options for iteratively refining FHIR profiles using the information obtained from a discrepancy analysis. We developed an FHIR validation pipeline and opted to derive more restrictive profiles from the original CDS profiles. This decision was driven by the need to align more closely with the specific assumptions and requirements of the central feasibility platform's search ontology. While the original CDS profiles offer a generic framework adaptable for a broad spectrum of medical informatics use cases, they lack the specificity to model the nuanced criteria essential for medical researchers. A key example of this is the necessity to represent specific laboratory codings and values interdependencies accurately. The validation results allow us to identify discrepancies between the instance data at the clinical sites and the profiles specified by the feasibility platform and addressed in the future. A total of 20 university hospitals participated in this study. Historical factors, lack of harmonization, a wide range of source systems, and case sensitivity of coding are some of the causes for the discrepancies identified. While in our case study, Conditions, Procedures, and Medications have a high degree of uniformity in the coding of instance data due to legislative requirements for billing in Germany, we found that laboratory values pose a significant data harmonization challenge due to their interdependency between coding and value. While the CDS achieves interoperability, different challenges for federated data access arise, requiring more specificity in the profiles to make assumptions on the instance data. We further argue that further harmonization of the instance data can significantly lower required retrospective harmonization efforts. We recognize that discrepancies cannot be resolved solely at the clinical site; therefore, our findings have a wide range of implications and will require action on multiple levels and by various stakeholders.