Irregular Data Research Articles

Calculation and comparison of certain physical properties of sample irregular galaxies with the Milky Way galaxy

Irregular galaxies often include a mix of old and young celestial populations, as well as dust and gas. This study makes an early effort at the morphological classification of irregular galaxies. These sources have been chosen from (70) irregular galaxies data samples from a Catalogue survey. We present a statistical analysis of the physical properties for the chosen data from the database and a collection of tools that used to study the physics of galaxies and cosmology HYPERLEDA (http://atlas.obshp.fr/hyperleda/) and NASA /IPAC Extragalactic Database NED (https://ned.ipac.caltech.edu/) survey.

An Interpretable Early Dynamic Sequential Predictor for Sepsis-Induced Coagulopathy Progression in the Real-World Using Machine Learning.

Sepsis-associated coagulation dysfunction greatly increases the mortality of sepsis. Irregular clinical time-series data remains a major challenge for AI medical applications. To early detect and manage sepsis-induced coagulopathy (SIC) and sepsis-associated disseminated intravascular coagulation (DIC), we developed an interpretable real-time sequential warning model toward real-world irregular data. Eight machine learning models including novel algorithms were devised to detect SIC and sepsis-associated DIC 8n (1 ≤ n ≤ 6) hours prior to its onset. Models were developed on Xi'an Jiaotong University Medical College (XJTUMC) and verified on Beth Israel Deaconess Medical Center (BIDMC). A total of 12,154 SIC and 7,878 International Society on Thrombosis and Haemostasis (ISTH) overt-DIC labels were annotated according to the SIC and ISTH overt-DIC scoring systems in train set. The area under the receiver operating characteristic curve (AUROC) were used as model evaluation metrics. The eXtreme Gradient Boosting (XGBoost) model can predict SIC and sepsis-associated DIC events up to 48 h earlier with an AUROC of 0.929 and 0.910, respectively, and even reached 0.973 and 0.955 at 8 h earlier, achieving the highest performance to date. The novel ODE-RNN model achieved continuous prediction at arbitrary time points, and with an AUROC of 0.962 and 0.936 for SIC and DIC predicted 8 h earlier, respectively. In conclusion, our model can predict the sepsis-associated SIC and DIC onset up to 48 h in advance, which helps maximize the time window for early management by physicians.

The emerging graph neural networks for intelligent fault diagnostics and prognostics: A guideline and a benchmark study

Deep learning (DL)-based methods have advanced the field of Prognostics and Health Management (PHM) in recent years, because of their powerful feature representation ability. The data in PHM are typically regular data represented in the Euclidean space. Nevertheless, there are an increasing number of applications that consider the relationships and interdependencies of data and represent the data in the form of graphs. Such kind of irregular data in non-Euclidean space pose a huge challenge to the existing DL-based methods, making some important operations (e.g., convolutions) easily applied to Euclidean space but difficult to model graph data in non-Euclidean space. Recently, graph neural networks (GNNs), as the emerging neural networks, have been utilized to model and analyze the graph data. However, there still lacks a guideline on leveraging GNNs for realizing intelligent fault diagnostics and prognostics. To fill this research gap, a practical guideline is proposed in this paper, and a novel intelligent fault diagnostics and prognostics framework based on GNN is established to illustrate how the proposed guideline works. In this framework, three types of graph construction methods are provided, and seven kinds of graph convolutional networks (GCNs) with four different graph pooling methods are investigated. To afford benchmark results for helping further study, a comprehensive evaluation of these models is performed on eight datasets, including six fault diagnosis datasets and two prognosis datasets. Finally, four issues related to the performance of GCNs are discussed and potential research directions are provided. The code library is available at: https://github.com/HazeDT/PHMGNNBenchmark.

Unifying organization and operation of irregular spatio-temporal field data with a tensor-based approach

Unifying organization and operation of irregular spatio-temporal field data with a tensor-based approach

Primitive element driven unified organization and operation of irregular spatiotemporal field data a tensor approach

Primitive element driven unified organization and operation of irregular spatiotemporal field data a tensor approach

Unifying organization and operation of irregular spatio-temporal field data with a tensor-based approach

Unifying organization and operation of irregular spatio-temporal field data with a tensor-based approach

A load-balancing strategy for data domain decomposition in parallel programming libraries of raster-based geocomputation

ABSTRACT Parallel programming libraries have been proposed to simplify programming for parallel raster-based geocomputation through hiding parallel programming details for users. However, the strategy of data domain decomposition used in existing libraries often leads to load imbalance owing to inherent characteristics of geocomputation including not only irregular spatial data distribution, but also spatial variation in the amount of computation, thereby impeding their parallel performances. This paper thus proposes a load-balancing strategy of data domain decomposition in parallel programming libraries for raster-based geocomputation based on the concept of spatial computational domain, which characterizes the distribution of computational intensity based on geocomputation characteristics. By implementing the proposed strategy with the message passing interface (MPI), a set of parallel raster-based geocomputation operators across different parallel computing platforms (known as PaRGO V2) was upgraded to improve load-balancing parallelization. The proposed strategy was evaluated by parallelizing two typical geocomputation algorithms (i.e. inverse distance weight interpolation and fuzzy c-means clustering) using PaRGO V2 with uneven distributed computational intensity. The results show that the proposed strategy with PaRGO V2, compared with the previously adopted data domain decomposition strategy, yielded significant improvements to the load balance (i.e. better parallel performance).

Development of a mixed-source, single pesticide database for use in ecological risk assessment: quality control and data standardization practices

Inclusion of pesticide monitoring data in pesticide risk assessment is important yet challenging for several reasons, including infrequent or irregular data collection, disparate sources procedures and associated monitoring periods, and interpretation of the data itself in a policy context. These challenges alone, left unaddressed, will likely introduce unintentional and unforeseen risk assessment conclusions. While individual water quality monitoring programs report standard operating procedures and quality control practices for their own data, cross-checking data for duplicated data from one database to another does not routinely occur. Consequently, we developed a novel quality control and assurance methodology to identify errors and duplicated records toward creating an aggregated, single pesticide database toward use in ecological risk assessment. This methodology includes (1) standardization and reformatting practices, (2) data error and duplicate record identification protocols, (3) missing or inconsistent limit of detection and quantification reporting, and (4) site metadata scoring and ranking procedures to flag likely duplicate records. We applied this methodology to develop an aggregated (multiple-source), national-scale database for atrazine from a diverse set of surface water monitoring programs. The resultant database resolved and/or removed approximately 31% of the total ~ 385,000 records that were due to duplicated records. Identification of sample replicates was also developed. While the quality control and assurances methodologies developed in this work were applied to atrazine, they generally demonstrate how a properly constructed and aggregated single pesticide database would benefit from the methods described herein before use in subsequent statistical and data analysis or risk assessment.

Filling the gaps in Leishmania naiffi and Leishmania guyanensis genome plasticity

Insufficient and irregular data reports on Leishmaniasis, issuing from the developing world, have left much to be desired in terms of understanding the molecular signatures producing distinct infectious phenotypes of the disease. Herein, we report on the complete genome sequencing of Leishmania naiffi and Leishmania guyanensis, sampled from patients in regions of Colombia and Venezuela. In this study, the isolates of cutaneous lesions from both species presented limited structural variation at the chromosomal level, low gene copy number variation, and high genetic heterogeneity. We compared these sequences to the reference genomes hitherto related from Brazil and French Guyana. Although of the same species, we note a consequential genomic disparity between the Venezuelan and French Guyanese isolates of L. guyanensis. Although less significant on the global schema of cutaneous and mucosal disease, such genomic studies of L. naiffi and L. guyanensis substantiate the gaps in understanding of the molecular architecture and multivariate clinical pictures of Leishmaniasis, on an international scale.

Performance of machine learning methods in predicting water quality index based on irregular data set: application on Illizi region (Algerian southeast)

Groundwater quality appraisal is one of the most crucial tasks to ensure safe drinking water sources. Concurrently, a water quality index (WQI) requires some water quality parameters. Conventionally, WQI computation consumes time and is often found with various errors during subindex calculation. To this end, 8 artificial intelligence algorithms, e.g., multilinear regression (MLR), random forest (RF), M5P tree (M5P), random subspace (RSS), additive regression (AR), artificial neural network (ANN), support vector regression (SVR), and locally weighted linear regression (LWLR), were employed to generate WQI prediction in Illizi region, southeast Algeria. Using the best subset regression, 12 different input combinations were developed and the strategy of work was based on two scenarios. The first scenario aims to reduce the time consumption in WQI computation, where all parameters were used as inputs. The second scenario intends to show the water quality variation in the critical cases when the necessary analyses are unavailable, whereas all inputs were reduced based on sensitivity analysis. The models were appraised using several statistical metrics including correlation coefficient (R), mean absolute error (MAE), root mean square error (RMSE), relative absolute error (RAE), and root relative square error (RRSE). The results reveal that TDS and TH are the key drivers influencing WQI in the study area. The comparison of performance evaluation metric shows that the MLR model has the higher accuracy compared to other models in the first scenario in terms of 1, 1.4572*10–08, 2.1418*10–08, 1.2573*10–10%, and 3.1708*10–08% for R, MAE, RMSE, RAE, and RRSE, respectively. The second scenario was executed with less error rate by using the RF model with 0.9984, 1.9942, 3.2488, 4.693, and 5.9642 for R, MAE, RMSE, RAE, and RRSE, respectively. The outcomes of this paper would be of interest to water planners in terms of WQI for improving sustainable management plans of groundwater resources.

Convergence of nonlinear numerical approximations for an elliptic linear problem with irregular data

This work is devoted to the study of the approximation, using two nonlinear numerical methods, of a linear elliptic problem with measure data and heterogeneous anisotropic diffusion matrix. Both methods show convergence properties to a continuous solution of the problem in a weak sense, through the change of variable u = ψ(v), where ψ is a well chosen diffeomorphism between (−1, 1) and ℝ, and v is valued in (−1, 1). We first study a nonlinear finite element approximation on any simplicial grid. We prove the existence of a discrete solution, and, under standard regularity conditions, we prove its convergence to a weak solution of the problem by applying Hölder and Sobolev inequalities. Some numerical results, in 2D and 3D cases where the solution does not belong to H1 (Ω), show that this method can provide accurate results. We then construct a numerical scheme which presents a convergence property to the entropy weak solution of the problem in the case where the right-hand side belongs to L1. This is achieved owing to a nonlinear control volume finite element (CVFE) method, keeping the same nonlinear reformulation, and adding an upstream weighting evaluation and a nonlinear p-Laplace vanishing stabilisation term.

Information consistent degree‐based clustering method for large‐scale group decision‐making with linear uncertainty distributions information

Clustering analysis is a key technique in reducing the dimensionality of high volume irregular data containing large-scale group decision-making (LSGDM) information. Uncertainty theory is suitable for subjective estimation or situation, such as lack of historical data, and it can be employed to effectively express the uncertainty of trust and preference information in LSGDM problems. This paper studies the dimensionality reduction and subgroup optimization in LSGDM by utilizing linear uncertain variables in social networks. A clustering method is proposed to decompose the large group into several subgroups of higher consilience degrees and higher preference similarities, and lower the dimension of information for LSGDM. In the clustering process, two measurement attributes, trust relationship and preference relationship of decision-makers, are combined, and information consistent degree is utilized as the clustering indicator. This approach does not need to preset the threshold and the number of subgroups, and can be employed to obtain subgroups with similar preferences and stable trust relationship. Through the clustering reliability evaluation of subgroups, the rationality of large-scale group clustering results is verified. Subgroup consensus contribution is used to identify superior subgroups and quantify the role of subgroups in improving the consensus level. An example of emergency decision-making and comparative analysis is provided to explain the feasibility and advantages of the proposed method.

Time Series Analysis of Land Cover Change in Dry Mountains: Insights from the Tajik Pamirs

Greening and browning trends in vegetation have been observed in many regions of the world in recent decades. However, few studies focused on dry mountains. Here, we analyze trends of land cover change in the Western Pamirs, Tajikistan. We aim to gain a deeper understanding of these changes and thus improve remote sensing studies in dry mountainous areas. The study area is characterized by a complex set of attributes, making it a prime example for this purpose. We used generalized additive mixed models for the trend estimation of a 32-year Landsat time series (1988–2020) of the modified soil adjusted vegetation index, vegetation data, and environmental and socio-demographic data. With this approach, we were able to cope with the typical challenges that occur in the remote sensing analysis of dry and mountainous areas, including background noise and irregular data. We found that greening and browning trends coexist and that they vary according to the land cover class, topography, and geographical distribution. Greening was detected predominantly in agricultural and forestry areas, indicating direct anthropogenic drivers of change. At other sites, greening corresponds well with increasing temperature. Browning was frequently linked to disastrous events, which are promoted by increasing temperatures.

DTransE: Distributed Translating Embedding for Knowledge Graph

Knowledge graphs play an important role in many applications, such as link prediction and question answering. Translating embedding for knowledge graphs is done with the aim of encoding structured information on entities and their rich relations in a low-dimensional embedding space. TransE is one of the most important methods in translation-based models, and uses translation invariance to implement translating embedding for knowledge graphs. In this line of work, translating embedding models represent the relation as a translation from the head entity to the tail entity and have achieved impressive results. Currently, the TransE model is only developed on single-node machines. Unfortunately, the computing and storage capacities of a single machine can easily reach their limits as knowledge graphs become larger and more complex, which limits the application scope of TransE. In order to solve this problem, we propose a distributed TransE method, known as DTransE, which can utilize distributed computing resources to calculate knowledge graph embeddings. However, building a distributed TransE is complicated and involves challenges of knowledge graph partitioning and computation. To solve these challenges, we provide a high-quality edge partitioning algorithm for the power-law graph by considering the high-degree and low-degree vertices with adaptive weights, which can balance the workload. By using the unactivated Gather-Apply-Scatter model on TransE, the processes periodically exchange messages in a loop. The irregular data distribution among the processes is also optimized to further accelerate communication. As far as we know, this is the first work on a distributed TransE method. We use link prediction to evaluate the DTransE in a distributed environment. Experiments show that, compared to the original TransE method, our proposed DTransE is, on average, 24.5 times faster with a minimum loss of accuracy; compared to the state-of-the-art parallel TransE implementation, DTransE is two times faster on average.

Reconstruction of irregular missing seismic data using conditional generative adversarial networks

During acquisition, due to economic and natural reasons, irregular missing seismic data are always observed. To improve accuracy in subsequent processing, the missing data should be interpolated. A conditional generative adversarial network (cGAN) consisting of two networks, a generator and a discriminator, is a deep-learning model that can be used to interpolate the missing data. However, because cGAN is typically data set oriented, the trained network is unable to interpolate a data set from an area different from that of the training data set. We design a cGAN based on Pix2Pix GAN to interpolate irregular missing seismic data. A synthetic data set synthesized from two models is used to train the network. Furthermore, we add a Gaussian-noise layer in the discriminator to fix a vanishing gradient, allowing us to train a more powerful generator. Two synthetic data sets synthesized by two new geologic models and two field data sets are used to test the trained cGAN. The test results and the calculated recovered signal-to-noise ratios indicate that although the cGAN is trained using synthetic data, the network can reconstruct irregular missing field seismic data with high accuracy using the Gaussian-noise layer. We test the performances of cGANs trained with different patch sizes in the discriminator to determine the best structure, and we train the networks using different training data sets for different missing rates, demonstrating the best training data set. Compared with conventional methods, the cGAN-based interpolation method does not need different parameter selections for different data sets to obtain the best interpolation data. Furthermore, it is also an efficient technique as the cost is because of the training, and after training, the processing time is negligible.

Towards Robust Calculation of Interannual CO2 Growth Signal from TCCON (Total Carbon Column Observing Network)

The CO2 growth rate is one of the key geophysical quantities reflecting the dynamics of climate change as atmospheric CO2 growth is the primary driver of global warming. As recent studies have shown that TCCON (Total Carbon Column Observing Network) measurement footprints embrace quasi-global coverage, we examined the sensitivity of TCCON to the global CO2 growth. To this end, we used the aggregated TCCON observations (2006-2019) to retrieve Annual Growth Rate of CO2 (AGR) at global scales. The global AGR estimates from TCCON (AGRTCCON) are robust and independent, from (a) the station-wise seasonality, from (b) the differences in time series across the TCCON stations, and from (c) the type of TCCON stations used in the calculation (“background” or “contaminated” by neighboring CO2 sources). The AGRTCCON potential error, due to the irregular data sampling is relatively low (2.4–17.9%). In 2006–2019, global AGRTCCON ranged from the minimum of 1.59 ± 2.27 ppm (2009) to the maximum of 3.27 ± 0.82 ppm (2016), whereas the uncertainties express sub-annual variability and the data gap effects. The global AGRTCCON magnitude is similar to the reference AGR from satellite data (AGRSAT = 1.57–2.94 ppm) and the surface-based estimates of Global Carbon Budget (AGRGCB = 1.57–2.85). The highest global CO2 growth rate (2015/2016), caused by the record El Niño, was nearly perfectly reproduced by the TCCON (AGRTCCON = 3.27 ± 0.82 ppm vs. AGRSAT = 3.23 ± 0.50 ppm). The overall agreement between global AGRTCCON with the AGR references was yet weakened (r = 0.37 for TCCON vs. SAT; r = 0.50 for TCCON vs. GCB) due to two years (2008, 2015). We identified the drivers of this disagreement; in 2008, when only few stations were available worldwide, the AGRTCCON uncertainties were excessively high (AGRTCCON = 2.64 ppm with 3.92 ppm or 148% uncertainty). Moreover, in 2008 and 2015, the ENSO-driven bias between global AGRTCCON and the AGR references were detected. TCCON-to-reference agreement is dramatically increased if the years with ENSO-related biases (2008, 2015) are forfeited (r = 0.67 for TCCON vs. SAT, r = 0.82 for TCCON vs. GCB). To conclude, this is the first study that showed promising ability of aggregated TCCON signal to capture global CO2 growth. As the TCCON coverage is expanding, and new versions of TCCON data are being published, multiple data sampling strategies, dynamically changing TCCON global measurement footprint, and the irregular sensitivity of AGRTCCON to strong ENSO events; all should be analyzed to transform the current efforts into a first operational algorithm for retrieving global CO2 growth from TCCON data.

Modelling bluetongue risk in Kazakhstan

BackgroundBluetongue is a serious disease of ruminants caused by the bluetongue virus (BTV). BTV is transmitted by biting midges (Culicoides spp.). Serological evidence from livestock and the presence of at least one competent vector species of Culicoides suggests that transmission of BTV is possible and may have occurred in Kazakhstan.MethodsWe estimated the risk of transmission using a mathematical model of the reproduction number R0 for bluetongue. This model depends on livestock density and climatic factors which affect vector density. Data on climate and livestock numbers from the 2466 local communities were used. This, together with previously published model parameters, was used to estimate R0 for each month of the year. We plotted the results on isopleth maps of Kazakhstan using interpolation to smooth the irregular data. We also mapped the estimated proportion of the population requiring vaccination to prevent outbreaks of bluetongue.ResultsThe results suggest that transmission of bluetongue in Kazakhstan is not possible in the winter from October to March. Assuming there are vector-competent species of Culicoides endemic in Kazakhstan, then low levels of risk first appear in the south of Kazakhstan in April before spreading north and intensifying, reaching maximum levels in northern Kazakhstan in July. The risk declined in September and had disappeared by October.ConclusionThese results should aid in surveillance efforts for the detection and control of bluetongue in Kazakhstan by indicating where and when outbreaks of bluetongue are most likely to occur. The results also indicate where vaccination efforts should be focussed to prevent outbreaks of disease.Graphical abstract

Data complexity of daily natural gas consumption: Measurement and impact on forecasting performance

Data complexity has a great impact on daily natural gas consumption forecasting. However, due to the existence of irregular data, complex periodic change, and volatility data, the conventional methods, such as Lyapunov exponent and sample entropy, are failed to assess the complexity of the consumption data. Thus, this paper proposes a hybrid method of complexity measure, named CMLS. The novel method combined correlation coefficient analysis, missing data detect, Lyapunov exponent, and skewness analysis. Compared with Lyapunov exponent and sample entropy, CMLS is more stable and insensitive to the length of data in complexity measures. Additionally, for revealing the relationship between data complexity and forecasting performance, we design three case studies including 56 sets of daily natural gas consumption, and forecast with three advanced models. The results show that the forecasting performance various a lot in different complexity level. Particularly in very hard level, the daily natural gas consumption data is very hard to be forecasted and the R2 of forecasts are all negative. This paper serves as an initial study seeks to reveal the impact of data complexity on forecasting performance. The findings can help forecasters to evaluate the performance and difficulty of natural gas consumption forecasting.

Using machine learning methods to predict hepatic encephalopathy in cirrhotic patients with unbalanced data

ObjectiveHepatic encephalopathy (HE) is among the most common complications of cirrhosis. Data for cirrhosis with HE is typically unbalanced. Traditional statistical methods and machine learning algorithms thus cannot identify a few classes. In this paper, we use machine learning algorithms to construct a risk prediction model for liver cirrhosis complicated by HE to improve the efficiency of its prediction. MethodWe collected medical data from 1,256 patients with cirrhosis and performed preprocessing to extract 81 features from these irregular data. To predict HE in cirrhotic patients, we compared several classification methods: logistic regression, weighted random forest (WRF), SVM, and weighted SVM (WSVM). We also used an additional 722 patients with cirrhosis for external validation of the model. ResultsThe WRF, WSVM, and logistic regression models exhibited better recognition ability for patients with HE than traditional machine learning models (sensitivity> 0.70), but their ability to identify patients with uncomplicated HE was slightly lower (specificity approximately 85%). The comprehensive evaluation index of the traditional model was higher than those of other models (G-means> 0.80 and F-measure> 0.40). For the WRF, the G-means (0.82), F-measure (0.46), and AUC (0.82) were superior to those of the logistic regression and WSVM models, which means that it can better predict the incidence of HE in patients. ConclusionThe WRF model is more suitable for the classification of unbalanced medical data and can be used to construct a risk prediction and evaluation system for liver cirrhosis complicated with HE. The probabilistic prediction models of WRF can help clinicians identify high-risk patients with HE.

Statistical analysis of beach profiles – A spatiotemporal functional approach

Beach profile data sets provide valuable insight into the morphological evolution of sandy shorelines. However, beach monitoring schemes often show large variability in temporal and spatial intervals between beach profiles. Moreover, beach profiles are often incomplete (i.e. only a part of the profile is measured) and data gaps are unavoidable. The resulting irregular sets of beach profiles complicate statistical analysis and previous studies on the morphological evolution and the effects of external influences have often omitted incomplete beach profiles. In this perspective, a statistical model is suggested to study beach profiles and to identify the effects of external influences. To be precise, the statistical model can be used (1) to determine the temporal and spatial variability of beach profiles while accounting for autoregressive dependencies in space and time, (2) to identify effects of external influences, (3) to predict complete beach profiles at unknown locations (i.e., interpolation between beach profiles), and (4) to forecast complete beach profiles accounting for external influences, such as storm events or nourishments. To illustrate the applicability of this model to irregular beach profile data, this state-of-the-art functional, spatiotemporal model was applied to beach profiles of the island of Sylt, Germany. In a first case study on submerged beach profiles, a decreasing temporal dependency between the profiles in the offshore direction was revealed, highlighting that less frequent measurements of offshore areas would suffice. A second analysis of the emerged beach profiles revealed the general effect of storm conditions (wave heights > 5 m) on subsequently measured beach profiles, which was statistically significant, and the profiles eroded with approximately 0.2–0.7 m in height. In summary, this study proposes and explores the application of a state-of-the-art statistical model to investigate beach profile changes from increasingly diverse and large profile data in coastal engineering and management.