Inhomogeneous Datasets Research Articles

Unveiling Causality in Stateful Enterprise Knowledge Graphs: An Exploration of Process-Driven Object Relationships

The exponential growth of enterprise data generation demands innovative approaches for efficient information management across highly interconnected domains. Particularly, the process of systems engineering involves inhomogeneous data sets requiring a high degree of flexibility and traceability throughout its lifecycle. While structured data resides comfortably in dedicated systems, managing unstructured information within annotations, documents and emails often comes with challenging holistic comprehension of linking, context and revisioning. Graph technology holds promise in establishing adaptable connections among decentralized data sets. However, leveraging graph databases comes with challenges. Schema standardization, crucial for framing complex processes, is a difficult undertaking due to evolving data interrelationships and changing requirements. This paper proposes a two-fold solution. First, we introduce a conceptual schema that decouples stateful object descriptions from its headers and furthermore documents transactions. This allows flexible object associations, conditional state versioning and improved traceability while maintaining manageability. Second, we leverage the Business Process Model and Notation (BPMN) for process design, service orchestration and automated graph manipulations. Through application to the publicly available Northwind data set using neo4j and Flowable BPM, we illustrate our methodology’s efficacy and the seamless integration of type-safer data science into dynamic knowledge graphs, contributing to the future of enterprise data management.

Performance Improvement of Fuzzy C-Means Clustering Algorithm by Optimized Early Stopping for Inhomogeneous Datasets

Performance Improvement of Fuzzy C-Means Clustering Algorithm by Optimized Early Stopping for Inhomogeneous Datasets

Verification of the master curve concept (ASTM E1921) and inhomogeneity analysis of a German RPV weld for various loading rates

Fracture-mechanical investigations were carried out on the weld metal of a German reactor pressure vessel steel 22NiMoCr3-7. The crack tip loading rate and test temperature were varied in the brittle-ductile transition region, while the results were evaluated using the Master Curve methodology (ASTME1921), including inhomogeneity analyses in Appendix X5. The highly inhomogeneous datasets could be uniformly described with a bimodal distribution function with good accuracy compared to the standard procedure. However, this agreement decreases with increasing crack tip loading rate. In contrast, a modified Master Curve with a shape factor of p = 0.03/°C progressively matches the data better with increasing crack tip loading rate. The weld metal has a significantly higher toughness than the base material. SEM analysis suggests that the inhomogeneity is due to the stochastic distribution of sharp microcracks, while the observed inhomogeneity is additionally superimposed by dynamic effects (adiabatic heating and local crack arrest) at increased crack tip loading rates. It is argued that the microstructural inhomogeneity is hereby »neutralized« and could instead be interpreted as a possible artifact of the standard Master Curve under dynamic conditions.

Graph-based mobility profiling

The decarbonization of the transport system requires a better understanding of human mobility behavior to optimally plan and evaluate sustainable transport options (such as Mobility as a Service). Current analysis frameworks often rely on specific datasets or data-specific assumptions and hence are difficult to generalize to other datasets or studies. In this work, we present a workflow to identify groups of users with similar mobility behavior that appear across several datasets. Our method does not depend on a specific clustering algorithm, is robust against the choice of hyperparameters, does not require specific labels in the dataset, and is not limited to specific types of tracking data. This allows the extraction of stable mobility profiles based on several small and inhomogeneous tracking data sets. Our method consists of the following main steps: Representing individual mobility using location-based graphs; extraction of graph-based mobility features; clustering using different hyperparameter configurations; group identification using statistical testing. The method is applied to six tracking datasets (Geolife, Green Class 1 + 2, yumuv and two Foursquare datasets) with a total of 1070 users that visit about 3′000’000 different locations with a total tracking duration of over 200′000 days. We can identify and interpret five mobility profiles that appear in all datasets and show how these profiles can be used to analyze longitudinal and cross-sectional tracking studies.

Application of Hierarchical Agglomerative Clustering (HAC) for Systemic Classification of Pop-Up Housing (PUH) Environments

This paper is the result of the first-phase, inter-disciplinary work of a multi-disciplinary research project (“Urban pop-up housing environments and their potential as local innovation systems”) consisting of energy engineers and waste managers, landscape architects and spatial planners, innovation researchers and technology assessors. The project is aiming at globally analyzing and describing existing pop-up housings (PUH), developing modeling and assessment tools for sustainable, energy-efficient and socially innovative temporary housing solutions (THS), especially for sustainable and resilient urban structures. The present paper presents an effective application of hierarchical agglomerative clustering (HAC) for analyses of large datasets typically derived from field studies. As can be shown, the method, although well-known and successfully established in (soft) computing science, can also be used very constructively as a potential urban planning tool. The main aim of the underlying multi-disciplinary research project was to deeply analyze and structure THS and PUE. Multiple aspects are to be considered when it comes to the characterization and classification of such environments. A thorough (global) web survey of PUH and analysis of scientific literature concerning descriptive work of PUH and THS has been performed. Moreover, out of several tested different approaches and methods for classifying PUH, hierarchical clustering algorithms functioned well when properly selected metrics and cut-off criteria were applied. To be specific, the ‘Minkowski’-metric and the ‘Calinski-Harabasz’-criteria, as clustering indices, have shown the best overall results in clustering the inhomogeneous data concerning PUH. Several additional algorithms/functions derived from the field of hierarchical clustering have also been tested to exploit their potential in interpreting and graphically analyzing particular structures and dependencies in the resulting clusters. Hereby, (math.) the significance ‘S’ and (math.) proportion ‘P’ have been concluded to yield the best interpretable and comprehensible results when it comes to analyzing the given set (objects n = 85) of researched PUH-objects together with their properties (n > 190). The resulting easily readable graphs clearly demonstrate the applicability and usability of hierarchical clustering- and their derivative algorithms for scientifically profound building classification tasks in Urban Planning by effectively managing huge inhomogeneous building datasets.

Informed Local Smoothing in 3D Implicit Geological Modeling

Geological models are commonly used to represent geological structures in 3D space. A wide range of methods exists to create these models, with much scientific work focusing recently on implicit representation methods, which perform an interpolation of a three-dimensional field where the relevant boundaries are then isosurfaces in this field. However, this method has well-known problems with inhomogeneous data distributions: if regions with densely sampled data points exist, modeling artifacts are common. We present here an approach to overcome this deficiency through a combination of an implicit interpolation algorithm with a local smoothing approach. The approach is based on the concepts of nugget effect and filtered kriging known from conventional geostatistics. It reduces the impact of regularly occurring modeling artifacts that result from data uncertainty and data configuration and additionally aims to improve model robustness for scale-dependent fit-for-purpose modeling. Local smoothing can either be manually adjusted, inferred from quantified uncertainties associated with input data or derived automatically from data configuration. The application for different datasets with varying configuration and noise is presented for a low complexity geologic model. The results show that the approach enables a reduction of artifacts, but may require a careful choice of parameter settings for very inhomogeneous data sets.

Two c’s in a pod: cosmology-independent measurement of the Type Ia supernova colour–luminosity relation with a sibling pair

ABSTRACT Using Zwicky Transient Facility (ZTF) observations, we identify a pair of ‘sibling’ Type Ia supernovae (SNe Ia), i.e. hosted by the same galaxy at z = 0.0541. They exploded within 200 d from each other at a separation of $0.6\,\mathrm{ arcsec}$ corresponding to a projected distance of only 0.6 kpc. Performing salt2 light-curve fits to the gri ZTF photometry, we show that for these equally distant ‘standardizable candles’, there is a difference of 2 mag in their rest-frame B-band peaks, and the fainter supernova (SN) has a significantly red salt2 colour c = 0.57 ± 0.04, while the stretch values x1 of the two SNe are similar, suggesting that the fainter SN is attenuated by dust in the interstellar medium of the host galaxy. We use these measurements to infer the salt2 colour standardization parameter, β = 3.5 ± 0.3, independent of the underlying cosmology and Malmquist bias. Assuming the colour excess is entirely due to dust, the result differs by 2σ from the average Milky Way total-to-selective extinction ratio, but is in good agreement with the colour–brightness corrections empirically derived from the most recent SN Ia Hubble–Lemaitre diagram fits. Thus we suggest that SN ‘siblings’, which will increasingly be discovered in the coming years, can be used to probe the validity of the colour and light-curve shape corrections using in SN Ia cosmology while avoiding important systematic effects in their inference from global multiparameter fits to inhomogeneous data sets, and also help constrain the role of interstellar dust in SN Ia cosmology.

Homogenization of incoming solar radiation measurements over Poland with satellite and climate reanalysis data

Well-maintained and regularly calibrated measuring instruments provide the most accurate solar radiation data. This extremely valuable research material makes it possible, among others, to analyse variability in solar radiation over the long term and its dependence on other atmospheric state elements such as cloud cover and atmospheric aerosol concentration. Unfortunately, ground-based measurements of solar radiation are often subject to various errors which are very difficult to detect. This is why quality control procedures and homogenisation of data are essential and should be performed prior to further analyses. This paper presents a method for quality control and homogenization of solar radiation data, which builds on the bias-based quality control (BQC) method (Urraca et al., 2017), and is tailored specially for detecting single erroneous daily values, and very long periods of small errors. The method was tested for 16 ground-based stations located in Poland for the period 1991–2015. In comparison with the number of errors detected by the BQC method, the number of detected errors increased significantly: 130 to 2890 more erroneous days were detected at each station. Consequently, the number of inhomogeneous data sets was reduced from 8 to 3 stations. The values on the days considered as erroneous were replaced with debiased values originating from the Surface Solar Radiation Data Set – Heliosat, Edition 2 (SARAH-2). The presented methodology can be also of use in any other places, especially those with many single erroneous days and no metadata publicly available.

A survey for variable young stars with small telescopes – IV. Rotation periods of YSOs in IC 5070

ABSTRACT Studying rotational variability of young stars is enabling us to investigate a multitude of properties of young star-disc systems. We utilize high cadence, multiwavelength optical time series data from the Hunting Outbursting Young Stars citizen science project to identify periodic variables in the Pelican Nebula (IC 5070). A double blind study using nine different period-finding algorithms was conducted and a sample of 59 periodic variables was identified. We find that a combination of four period finding algorithms can achieve a completeness of 85 per cent and a contamination of 30 per cent in identifying periods in inhomogeneous data sets. The best performing methods are periodograms that rely on fitting a sine curve. Utilizing Gaia EDR3 data, we have identified an unbiased sample of 40 periodic young stellar objects (YSOs), without using any colour or magnitude selections. With a 98.9 per cent probability, we can exclude a homogeneous YSO period distribution. Instead, we find a bi-modal distribution with peaks at 3 and 8 d. The sample has a disc fraction of 50 per cent, and its statistical properties are in agreement with other similarly aged YSOs populations. In particular, we confirm that the presence of the disc is linked to predominantly slow rotation and find a probability of 4.8 × 10−3 that the observed relation between period and presence of a disc has occurred by chance. In our sample of periodic variables, we also find pulsating giants, an eclipsing binary, and potential YSOs in the foreground of IC 5070.

Critical comments on publications by S. Hoffmann and N. Vogt on historical novae/supernovae and their candidates

AbstractWe critically discuss recent articles by S. Hoffmann and N. Vogt on historical novae and supernovae (SNe) as well as their list of “24 most promising events” “with rather high probability to be a nova” (Hoffmann et al., AN, 2020, 341, 79 (P3)). Their alleged positional accuracy of previously suggested historical nova/SN records is based on inhomogeneous datasets (Vogt et al.), but then used for the nova search in Hoffmann et al., AN, 2020, 341, 79 (P3). Their claim that previously only “point coordinates” for nova/SN candidates were published, is fabricated. Their estimate of expected nova detection rates is off by a factor of 10 due to mis‐calculation. They accept counterparts down to 4–7 mag at peak, which is against the consensus for the typical limit of naked‐eye discovery. When they discuss previously suggested identifications of historical novae, which they all doubt, they do not present new facts (Hoffmann, MNRAS, 2019, 490, 4194 (P2)). Their catalog of “24 most promising events” for novae (Hoffmann et al., AN, 2020, 341, 79 (P3)) neglects important recent literature (e.g. Pankenier et al., Archeoastronomy in East Asia, New York, Cambria, 2008 and Stephenson and Green, JHA, 2009, 40, 31), the claimed methods are not followed, etc. At least half of their short‐list candidates were and are to be considered comets. For many of the others, duration of more than one night and/or a precise position is missing and/or the sources were treated mistakenly. Two “highlights,” a fabricated SN AD 667–8 and a presumable recurrent nova in AD 891, are already rejected in detail in Neuhäuser et al., MNRAS, 2021a, 501, L1—in both cases, all evidence speaks in favor of comets. There remains only one reliable case, where close to one (possible) historically reported position, a nova shell was already found (AD 1437, Shara et al., Nature, 2017b, 548, 558). Since the proposed positional search areas are not justified due to unfounded textual interpretations (e.g. in fact comets), misunderstandings of historical Chinese astronomy (e.g. incorrect asterism), follow‐up observations cannot be recommended.

Extended detrended fluctuation analysis: effects of nonstationarity and application to sleep data

Extended detrended fluctuation analysis (EDFA) is a recently proposed modification of the conventional method, which provides a characterization of complex time series with varying nonstationarity. It evaluates two scaling exponents for a better quantification of inhomogeneous datasets. Here, we study the effect of different types of nonstationarity on these exponents, including trend, switching between processes with distinct statistical properties and energy variability. Using the simulated signals, we show that the first two types of nonstationarity have the strongest effect for anticorrelated processes and complicate their diagnosis. Nonstationarity in energy is more crucial for time series with positive long-range correlations. Next, we apply EDFA to rat experiments to study the activation of brain lymphatic drainage during sleep. Our analysis reveals significant distinctions in EDFA’s measures between the background electrical activity of the brain and the stage of sleep. The latter offers an indirect way to identify and characterize the nightly activation of the drainage and clearance of brain tissue.

Segmentation of unbalanced and in-homogeneous point clouds and its application to 3D scanned trees

Segmentation of 3D point clouds is still an open issue in the case of unbalanced and in-homogeneous data-sets. In the application context of the modeling of botanical trees, a fundamental challenge consists in separating the leaves from the wood. Based on deep learning and a class decision process, we propose an innovative method designed to separate leaf points from wood points in terrestrial LiDAR point clouds of trees. Although simple, our approach learns trees characteristic point patterns efficiently and robustly. To train our 3D deep learning model, we constructed a 3D labeled point cloud data-set of different tree species. Experiments show that our 3D deep representation together with our geometric approach leads to significant improvement over the state-of-the-art methods in segmentation task.

Extended detrended fluctuation analysis of electroencephalograms signals during sleep and the opening of the blood-brain barrier.

Detrended fluctuation analysis (DFA) is widely used to characterize long-range power-law correlations in complex signals. However, it has restrictions when nonstationarity is not limited only to slow variations in the mean value. To improve the characterization of inhomogeneous datasets, we have proposed the extended DFA (EDFA), which is a modification of the conventional method that evaluates an additional scaling exponent to take into account the features of time-varying nonstationary behavior. Based on EDFA, here, we analyze rat electroencephalograms to identify specific changes in the slow-wave dynamics of brain electrical activity associated with two different conditions, such as the opening of the blood-brain barrier and sleep, which are both characterized by the activation of the brain drainage function. We show that these conditions cause a similar reduction in the scaling exponents of EDFA. Such a similarity may represent an informative marker of fluid homeostasis of the central nervous system.

Impact of missing data on the efficiency of homogenisation: experiments with ACMANTv3

The impact of missing data on the efficiency of homogenisation with ACMANTv3 is examined with simulated monthly surface air temperature test datasets. The homogeneous database is derived from an earlier benchmarking of daily temperature data in the USA, and then outliers and inhomogeneities (IHs) are randomly inserted into the time series. Three inhomogeneous datasets are generated and used, one with relatively few and small IHs, another one with IHs of medium frequency and size, and a third one with large and frequent IHs. All of the inserted IHs are changes to the means. Most of the IHs are single sudden shifts or pair of shifts resulting in platform-shaped biases. Each test dataset consists of 158 time series of 100 years length, and their mean spatial correlation is 0.68–0.88. For examining the impacts of missing data, seven experiments are performed, in which 18 series are left complete, while variable quantities (10–70%) of the data of the other 140 series are removed. The results show that data gaps have a greater impact on the monthly root mean squared error (RMSE) than the annual RMSE and trend bias. When data with a large ratio of gaps is homogenised, the reduction of the upper 5% of the monthly RMSE is the least successful, but even there, the efficiency remains positive. In terms of reducing the annual RMSE and trend bias, the efficiency is 54–91%. The inclusion of short and incomplete series with sufficient spatial correlation in all cases improves the efficiency of homogenisation with ACMANTv3.

Detection limits should be a thing of the past in gamma-ray spectrometry in general as well as in neutron activation analysis.

In gamma-ray spectrometry with high-resolution detectors, full-energy peaks are often to be detected by a peak-search algorithm, with a threshold for detection. Detection limits can be derived from this. Detection limits are often computed along with measured activities or concentrations. When an analyte is not detected, the detection limit remains as the only available information. This leads to inhomogeneous datasets that are difficult or impossible to process correctly without introducing artefacts or biases. Here, it is proposed to determine peak areas at predetermined energies. An unbiased result with its uncertainty always results, obviating the “detection limit” concept.

Synthetic modelling of the light-travel time effect of circumbinary planets

AbstractIn recent years several multi-body, circumbinary planets have been proposed to orbit short-period eclipsing binaries. In light of the recent discoveries based on the Kepler data, the existence of such systems seems plausible. However, performing a detailed dynamical analysis reveals that the majority of the proposed planetary systems follow highly unstable orbits. In order to solve the origin of this problem, we have started to model synthetic light-travel time signals of stable planetary systems. In particular, we aim to study the response of the model in various circumstances (e.g red/white noise level, various sampling frequencies, in-homogeneous data sets, baseline dependency.) This work will significantly increase the confidence with which model work is carried out for future systems and help towards an understaning when models break down (e.g resulting in unstable systems).

A Fast, Automated, Polynomial-Based Cosmic Ray Spike–Removal Method for the High-Throughput Processing of Raman Spectra

Raman spectra often contain undesirable, randomly positioned, intense, narrow-bandwidth, positive, unidirectional spectral features generated when cosmic rays strike charge-coupled device cameras. These must be removed prior to analysis, but doing so manually is not feasible for large data sets. We developed a quick, simple, effective, semi-automated procedure to remove cosmic ray spikes from spectral data sets that contain large numbers of relatively homogenous spectra. Although some inhomogeneous spectral data sets can be accommodated--it requires replacing excessively modified spectra with the originals and removing their spikes with a median filter instead--caution is advised when processing such data sets. In addition, the technique is suitable for interpolating missing spectra or replacing aberrant spectra with good spectral estimates. The method is applied to baseline-flattened spectra and relies on fitting a third-order (or higher) polynomial through all the spectra at every wavenumber. Pixel intensities in excess of a threshold of 3× the noise standard deviation above the fit are reduced to the threshold level. Because only two parameters (with readily specified default values) might require further adjustment, the method is easily implemented for semi-automated processing of large spectral sets.

Improved Estimates of the European Winter Windstorm Climate and the Risk of Reinsurance Loss Using Climate Model Data

AbstractCurrent estimates of the European windstorm climate and their associated losses are often hampered by either relatively short, coarse resolution or inhomogeneous datasets. This study tries to overcome some of these shortcomings by estimating the European windstorm climate using dynamical seasonal-to-decadal (s2d) climate forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF). The current s2d models have limited predictive skill of European storminess, making the ensemble forecasts ergodic samples on which to build pseudoclimates of 310–396 yr in length. Extended winter (October–April) windstorm climatologies are created using scalar extreme wind indices considering only data above a high threshold. The method identifies up to 2363 windstorms in s2d data and up to 380 windstorms in the 40-yr ECMWF Re-Analysis (ERA-40). Classical extreme value analysis (EVA) techniques are used to determine the windstorm climatologies. Differences between the ERA-40 and s2d windstorm climatologies require the application of calibration techniques to result in meaningful comparisons. Using a combined dynamical–statistical sampling technique, the largest influence on ERA-40 return period (RP) uncertainties is the sampling variability associated with only 45 seasons of storms. However, both maximum likelihood (ML) and L-moments (LM) methods of fitting a generalized Pareto distribution result in biased parameters and biased RP at sample sizes typically obtained from 45 seasons of reanalysis data. The authors correct the bias in the ML and LM methods and find that the ML-based ERA-40 climatology overestimates the RP of windstorms with RPs between 10 and 300 yr and underestimates the RP of windstorms with RPs greater than 300 yr. A 50-yr event in ERA-40 is approximately a 40-yr event after bias correction. Biases in the LM method result in higher RPs after bias correction although they are small when compared with those of the ML method. The climatologies are linked to the Swiss Reinsurance Company (Swiss Re) European windstorm loss model. New estimates of the risk of loss are compared with those from historical and stochastically generated windstorm fields used by Swiss Re. The resulting loss-frequency relationship matches well with the two independently modeled estimates and clearly demonstrates the added value by using alternative data and methods, as proposed in this study, to estimate the RP of high RP losses.

Improving the Visualization and Interpretation of Inhomogeneous Data Sets

Time and again, geoscientists are faced with data sets that are inhomogeneous, either spatially, temporally, or both. This can lead to difficulties in deciding how best to visualize a data set and, more important, in deciding how to interpret models created from a particular data set. Using modeling of the Earth's magnetic field through archaeological time as an example, this brief report illustrates how Google Earth® may be used to improve the visualization and interpretation of inhomogeneous data sets.

Highly diverse, massive organic data as explored by a composite QSPR strategy: An advanced study of boiling point

An improved strategy of quantitative structure–property relationship (QSPR) studies of diverse and inhomogeneous organic datasets has been proposed. A molecular connectivity term was successively corrected for different structural features encoded in fragmental descriptors. The so-called solvation index 1χs (a weighted Randić index) was used as a “leading” variable and standardized molecular fragments were employed as “corrective” class-specific variables. Performance of the new approach was illustrated by modelling a dataset of experimental normal boiling points of 833 organic compounds belonging to 20 structural classes. Firstly, separate QSPR models were derived for each class and for eight groups of structurally similar classes. Finally, a general model formed by combining all the classes together was derived ( ). The strategy outlined can find application in QSPR analyses of massive, highly diverse databases of organic compounds.