Distributed Data Sets Research Articles

Anomaly Identification Model for Telecom Users Based on Machine Learning Model Fusion

With the development of economic globalization and modern information and communication technology, the situation of communication fraud is becoming more and more serious. How to identify fraudulent calls accurately and effectively has become an urgent task in current telecommunications operations. Affected by the sample set and the current state of the art, the current machine learning methods used to identify the imbalanced distribution dataset of positive and negative samples have low recognition accuracy. Therefore, in this paper, we propose a new hybrid model solution that uses feature construction, feature selection and imbalanced classes handling. A stacking model fusion algorithm composed of a two-layer stacking framework with several state-of-the-art machine learning classifiers is adopted. The results show that the risk user identification model based on mobile network communication behavior established by our stacking model fusion algorithm can accurately predict the category labels of telecom users and improve the risk of telecom users. The generalization performance of the identification is high, which provides a certain reference for the telecommunications industry to identify risk users based on mobile network communication behaviors.

DISCONA: distributed sample compression for nearest neighbor algorithm

Sample compression using \U0001d716-net effectively reduces the number of labeled instances required for accurate classification with nearest neighbor algorithms. However, one-shot construction of an \U0001d716-net can be extremely challenging in large-scale distributed data sets. We explore two approaches for distributed sample compression: one where local \U0001d716-net is constructed for each data partition and then merged during an aggregation phase, and one where a single backbone of an \U0001d716-net is constructed from one partition and aggregates target label distributions from other partitions. Both approaches are applied to the problem of malware detection in a complex, real-world data set of Android apps using the nearest neighbor algorithm. Examination of the compression rate, computational efficiency, and predictive power shows that a single backbone of an \U0001d716-net attains favorable performance while achieving a compression rate of 99%.

Conditional quantum circuit Born machine based on a hybrid quantum–classical​ framework

As a branch of machine learning, generative models are widely used in supervised and unsupervised learning. To speedup certain machine learning tasks, quantum generative adversarial networks, quantum circuit Born machine (QCBM), and quantum Boltzmann machine have been proposed. These generative models can implement some specific generative tasks but have no control over the modes of the generated data. To make the generative model more intelligent and controllable, additional conditional information (such as category labels for MNIST digits) can be added to the model to guide the generation of data. A more in-depth study was carried out based on the QCBM, and a conditional quantum circuit Born machine (CQCBM) based on a hybrid quantum–classical (HQC) framework was proposed. The conditional information was encoded by adding extra qubits to guide the model training process. Experiments were conducted on both mixed Gaussian distribution and MNIST handwritten digit dataset. Numerical and experimental results show that the proposed CQCBM is able to generate the target distribution while satisfying the conditional constraints well. Compared to other conditional quantum generative models only applied to Bars and Stripes (BAS) or Chessboard datasets, the proposed model also performed well on more difficult image-generating tasks.

Optimized Plant Diversity and Carbon Storage for Priority Protection Areas in China

Biodiversity and carbon storage are two key ecosystem functions that are crucial to protect and maintain ecosystem balance. However, there is often little overlap between hotspots for these two different conservation purposes. Additionally, it is not well understood how these different spatial metrics affect these functions in protected ecosystems in China. Here, we explored the relationships between plant diversity metrics and carbon storage by using a large vascular plant distribution dataset, as well as soil fragile organic carbon and biomass carbon datasets in specific spatial areas across China. We also defined priority protection areas (PPAs) using a conservation prioritization method, where 30% of the study areas displayed the highest combined conservation value in carbon storage and plant species richness (SR), phylogenetic diversity (PD), phylogenetic endemism (PE) and evolutionary distinctness (ED). Our results indicated that the correlations between biodiversity metrics and carbon storage were very weak in spatial relationship. However, by including both of these functions in conservation targets, the PPAs could account for more than 95% of the species and evolutionary diversity (PD, ED), and stored large amounts of carbon. Additionally, we broadly divided the PPAs into win–win, high plant diversity, and high carbon areas based on the overlap of biodiversity and carbon storage hotspots. Altogether, our results highlight the importance of understanding and optimizing conservation efforts for different ecosystem functions in different PPAs. Ultimately, this work establishes an urgent need to expand protection in these areas to support mutual biodiversity and carbon storage beneficial solutions.

An Image Turing Test on Realistic Gastroscopy Images Generated by Using the Progressive Growing of Generative Adversarial Networks

Generative adversarial networks (GAN) in medicine are valuable techniques for augmenting unbalanced rare data, anomaly detection, and avoiding patient privacy issues. However, there were limits to generating high-quality endoscopic images with various characteristics, such as peristalsis, viewpoints, light sources, and mucous patterns. This study used the progressive growing of GAN (PGGAN) within the normal distribution dataset to confirm the ability to generate high-quality gastrointestinal images and investigated what barriers PGGAN has to generate endoscopic images. We trained the PGGAN with 107,060 gastroscopy images from 4165 normal patients to generate highly realistic 5122 pixel-sized images. For the evaluation, visual Turing tests were conducted on 100 real and 100 synthetic images to distinguish the authenticity of images by 19 endoscopists. The endoscopists were divided into three groups based on their years of clinical experience for subgroup analysis. The overall accuracy, sensitivity, and specificity of the 19 endoscopist groups were 61.3%, 70.3%, and 52.4%, respectively. The mean accuracy of the three endoscopist groups was 62.4 [Group I], 59.8 [Group II], and 59.1% [Group III], which was not considered a significant difference. There were no statistically significant differences in the location of the stomach. However, the real images with the anatomical landmark pylorus had higher detection sensitivity. The images generated by PGGAN showed highly realistic depictions that were difficult to distinguish, regardless of their expertise as endoscopists. However, it was necessary to establish GANs that could better represent the rugal folds and mucous membrane texture.

Integrated ultra scale-down and multivariate analysis of flocculation and centrifugation for enhanced primary recovery

A linked ultra scale-down (USD) flocculation and centrifugation system facilitates rapid bioprocess development and evaluation. These techniques allow studies to be undertaken at high throughput using as little as 50 mL of feed material. They enable the investigation of multiple conditions, thus generating a relatively large amount of data.This study establishes USD flocculation and centrifugation and joins a sequential multivariate data analysis (MVDA) to evaluate the multiparameter effects on primary recovery performance. MVDA techniques such as Principal Component Analysis (PCA) and Partial Least Squares (PLS) were used to handle the complex data sets and investigate the relationships between process parameters and responses. A strategy to assess floc characteristics using PCA was proposed to eliminate the visual inspection, and aid in the analysis, of particle size distribution (PSD) datasets. The PSD of non-sheared and sheared flocs provide good indicators of floc centrifuge performance.The findings show that this USD system can be used to forecast pilot scale performance. The sequential analysis demonstrated that the produced flocs are shear-sensitive in which feed preparation and process shear significantly impact the centrifugation of those flocs. Strong flocs may not necessarily result from high Camp number values (≥ 105) where optimisation of the flocculation chemical parameters is required.The novel integration of the USD systems with MVDA is a powerful platform to optimise and expand process know-how.

Multidimensional Scaling of Varietal Data in Sedimentary Provenance Analysis

AbstractVarietal studies of sedimentary provenance use the properties of individual minerals or mineral groups. These are recorded as lists of numerical tables that can be difficult to interpret. Multidimensional Scaling (MDS) is a popular multivariate ordination technique for analyzing other types of provenance data based on, for example, detrital geochronology or petrography. Applying MDS to varietal data would allow them to be treated on an equal footing with those other provenance proxies. MDS requires a method to quantify the dissimilarity between two samples. This paper introduces three ways to do so. The first method (“treatment‐by‐row”) turns lists of (compositional) data tables into lists of vectors, using principal component analysis. These lists of vectors can then be treated as “distributional” data and subjected to MDS analysis using dissimilarity measures such as the Kolmogorov‐Smirnov statistic. The second method (“treatment‐by‐column”) turns lists of compositional data tables into multiple lists of vectors, each representing a single component of the varietal data. These multiple distributional data sets are subsequently subjected to Procrustes analysis or 3‐way MDS. The third method uses the Wasserstein‐2 distance to jointly compare the rows and columns of varietal data. This arguably makes the best use of the data but acts more like a “black box” than the other two methods. Applying the three methods to a detrital titanite data set from Colombia yields similar results. After converting varietal data to dissimilarity matrices, they can be combined with other types of provenance data, again using Procrustes analysis or 3‐way MDS.

S-Type Random k Satisfiability Logic in Discrete Hopfield Neural Network Using Probability Distribution: Performance Optimization and Analysis

Recently, a variety of non-systematic satisfiability studies on Discrete Hopfield Neural Networks have been introduced to overcome a lack of interpretation. Although a flexible structure was established to assist in the generation of a wide range of spatial solutions that converge on global minima, the fundamental problem is that the existing logic completely ignores the probability dataset’s distribution and features, as well as the literal status distribution. Thus, this study considers a new type of non-systematic logic termed S-type Random k Satisfiability, which employs a creative layer of a Discrete Hopfield Neural Network, and which plays a significant role in the identification of the prevailing attribute likelihood of a binomial distribution dataset. The goal of the probability logic phase is to establish the logical structure and assign negative literals based on two given statistical parameters. The performance of the proposed logic structure was investigated using the comparison of a proposed metric to current state-of-the-art logical rules; consequently, was found that the models have a high value in two parameters that efficiently introduce a logical structure in the probability logic phase. Additionally, by implementing a Discrete Hopfield Neural Network, it has been observed that the cost function experiences a reduction. A new form of synaptic weight assessment via statistical methods was applied to investigate the effect of the two proposed parameters in the logic structure. Overall, the investigation demonstrated that controlling the two proposed parameters has a good effect on synaptic weight management and the generation of global minima solutions.

Multivariate Tiered Approach To Highlight the Link between Large-Scale Integrated Pesticide Concentrations from Polar Organic Chemical Integrative Samplers and Watershed Land Uses.

This paper presents a multi-step methodology to identify relationships between integrative pesticide quantifications and land uses on a given watershed of the Adour-Garonne Basin (Southwestern France). In fact, a large amount of pesticide concentration data was collected from 51 sites located in the Adour-Garonne Basin for a 1 year monitoring period in 2016. The sampling devices used here were polar organic chemical integrative samplers (POCIS), which provided time-weighted average concentration estimates. For each study site, its associated watershed and land cover distribution were determined using Corine Land Cover 2012 (CLC 2012) and Geographic Information System (GIS). The large-scale data were analyzed using multivariate statistical analyses, such as hierarchical cluster analysis (HCA) and principal component analysis (PCA). HCA grouped the 51 sites into five clusters with similar primary land uses. Next, the integrated pesticide concentration and land use distribution data sets were analyzed in a PCA. The key variables responsible for discriminating the sample sites showed distribution patterns consistent with specific land uses. To confirm these observations, pesticide fingerprints from sites with contrasting land uses were compared using a waffle method. The overall multivariate approach allowed for the identification of contamination sources related to their likely initial use, at the watershed level, that could be useful for preventing or containing pesticide pollution beyond simply acting on areas at risk.

A trait-based approach to determining principles of plant biogeography.

Lineage-specific traits determine how plants interact with their surrounding environment. Unrelated species may evolve similar phenotypic characteristics to tolerate, persist in, and invade environments with certain characteristics, resulting in some traits becoming relatively more common in certain types of habitats. Analyses of these general patterns of geographical trait distribution have led to the proposal of general principles to explain how plants diversify in space over time. Trait-environment correlation analyses quantify to what extent unrelated lineages have similar evolutionary responses to a given type of habitat. In this synthesis, I give a short historical overview on trait-environment correlation analyses, from some key observations from classic naturalists to modern approaches using trait evolution models, large phylogenies, and massive data sets of traits and distributions. I discuss some limitations of modern approaches, including the need for more realistic models, the lack of data from tropical areas, and the necessary focus on trait scoring that goes beyond macromorphology. Overcoming these limitations will allow the field to explore new questions related to trait lability and niche evolution and to better identify generalities and exceptions in how plants diversify in space over time.

Confirmatory Factor Analyses in Psychological Test Adaptation and Development

Confirmatory Factor Analyses in Psychological Test Adaptation and Development

Rodent trapping studies as an overlooked information source for understanding endemic and novel zoonotic spillover.

Rodents, a diverse, globally distributed and ecologically important order of mammals are nevertheless important reservoirs of known and novel zoonotic pathogens. Ongoing anthropogenic land use change is altering these species' abundance and distribution, which among zoonotic host species may increase the risk of zoonoses spillover events. A better understanding of the current distribution of rodent species is required to guide attempts to mitigate against potentially increased zoonotic disease hazard and risk. However, available species distribution and host-pathogen association datasets (e.g. IUCN, GBIF, CLOVER) are often taxonomically and spatially biased. Here, we synthesise data from West Africa from 127 rodent trapping studies, published between 1964-2022, as an additional source of information to characterise the range and presence of rodent species and identify the subgroup of species that are potential or known pathogen hosts. We identify that these rodent trapping studies, although biased towards human dominated landscapes across West Africa, can usefully complement current rodent species distribution datasets and we calculate the discrepancies between these datasets. For five regionally important zoonotic pathogens (Arenaviridae spp., Borrelia spp., Lassa mammarenavirus, Leptospira spp. and Toxoplasma gondii), we identify host-pathogen associations that have not been previously reported in host-association datasets. Finally, for these five pathogen groups, we find that the proportion of a rodent hosts range that have been sampled remains small with geographic clustering. A priority should be to sample rodent hosts across a greater geographic range to better characterise current and future risk of zoonotic spillover events. In the interim, studies of spatial pathogen risk informed by rodent distributions must incorporate a measure of the current sampling biases. The current synthesis of contextually rich rodent trapping data enriches available information from IUCN, GBIF and CLOVER which can support a more complete understanding of the hazard of zoonotic spillover events.

River reach types as large-scale biodiversity proxies for management: The case of the Greater Mekong Region

Large-scale development projects such as hydropower dams in the Greater Mekong Region (GMR) exert high pressure on freshwater resources. Environmental impact assessments in the region can help to understand the possible impacts of these projects, yet these assessments typically require biodiversity data that can be both costly and time-intensive to acquire. As a substitute, researchers often assume that river or ecosystem classes based on geophysical characteristics can be used as biodiversity proxies in large-scale assessments to account for a lack of biodiversity data. However, only limited research exists that compares the spatial distribution of river classes and fish species, and therefore it remains unclear how well river classes can represent fish assemblages or, more generally, biodiversity. To address this question, we here compared a new river reach classification, which used regional expert knowledge to build the classes, with a dataset of fish species distribution in the GMR. We conducted a Redundancy Analysis to estimate how much variability in the fish species data can be explained by the river reach types. The results show a moderate correlation between the datasets (adjusted R2 of 0.44). Based on these findings, we elaborate on the role of spatial hierarchy in fish species distribution and discuss possible implications for management and policies in the GMR.

Developing a Virtual Flowability Sensor for Monitoring a Pharmaceutical Dry Granulation Line

Current technologies to measure granule flowability involve at-line methods that can take hours to perform. This is problematic for a continuous dry granulation tableting line, where the quality assurance and control of the final tablet products depend on real-time monitoring and control of powder flowability. Hence, a real-time alternative is needed for measuring the flowability of the granular products coming out of the roller compactor, which is the unit operation immediately preceding the tablet press. Since particle analyzers have the potential to take inline measurements of the size and shape of granules, they can potentially serve as real-time flowability sensors, given that the size and shape measurements can be used to reliably predict flowability measurements.This paper reports on the use of Partial Least Squares (PLS) regression to utilize distributions of size and shape measurements in predicting the output of three different types of flowability measurements: rotary drum flow, orifice flow, and tapped density analysis. The prediction performance of PLS had a coefficient of determination ranging from 0.80 to 0.97, which is the best reported performance in the literature. This is attributed to the ability of PLS to handle high collinearity in the datasets and the inclusion of multiple shape characteristics—eccentricity, form factor, and elliptical form factor—into the model. The latter calls for a change in industry perspective, which normally dismisses the importance of shape in favor of size; and the former suggests the use of PLS as a better way to reduce the dimensionality of distribution datasets, instead of the widely used practice of pre-selecting distribution percentiles.

Study of experimental method for warm dense matter ionization distribution based on x-ray fluorescence spectroscopy

Warm dense matter (WDM), a state of matter which lies at the frontiers between condensed matter and plasma, is one of the main research objects of high energy density physics (HEDP). Compared to the isolated atom, the electron structure of WDM will change because of the influence of density and temperature effect. Both the accurate theoretical represent and the accurate experimental study of WDM electron structure are challenging, as it is strongly coupled and partially degenerated. In this paper, an experimental method for studying the ionization distribution of warm dense matter based on x-ray fluorescence spectroscopy is developed. In the experiment, warm dense titanium with several tens eV and near solid density is created by a simultaneous drive from high energy xray heating and shock compression in a special designed hohlraum. Then, using the characteristic line spectrum emitted by the laser irradiation on pump material (Vanadium) as pump source, the titanium emits fluorescence. The x-ray fluorescence spectroscopy of titanium with different states (cold sample, 1.8-4.5 g/<i>cm</i><sup>3</sup> and 1-25 eV) is diagnosed by changing the experimental strategy. The experimental results indicate that the line profiles of <i>K<sub>α</sub></i> and <i>K<sub>β</sub></i> fluorescence spectrum of the heated sample change obviously relative to that of the cold sample. Associating a theoretical calculation from two-step Hartree-Fock-Slater (TSHFS) method, the reason for the variation of the line profile is the change of ionization distribution mainly caused by temperature rise. The future work will focus on optimizing the experimental method of x-ray fluorescence spectroscopy, such as improving the spectrum resolution, characterizing the temperature and density experimentally, obtaining a set of ionization distribution data, and then study the influence of dense environment on electronic structure.

Better Modeling Out-of-Distribution Regression on Distributed Acoustic Sensor Data Using Anchored Hidden State Mixup

Generalizing the application of machine learning models to situations where the statistical distribution of training and test data are different has been a complex problem. Our contributions in this article are threefold: 1) we introduce an anchored-based out-of-distribution (OOD) Regression Mixup algorithm, leveraging manifold hidden state mixup and observation similarities to form a novel regularization penalty; 2) we provide a first of its kind high-resolution distributed acoustic sensor dataset that is suitable for testing OOD regression modeling, allowing other researchers to benchmark progress in this area; and 3) we demonstrate with an extensive evaluation the generalization performance of the proposed method against existing approaches and then show that our method achieves state-of-the-art performance. We also demonstrate a wider applicability of the proposed method by exhibiting improved generalization performances on other types of regression datasets, including Udacity and Rotation-MNIST datasets.

CGAN-Based Collaborative Intrusion Detection for UAV Networks: A Blockchain-Empowered Distributed Federated Learning Approach

Numerous resource-constrained Internet of Things (IoT) devices make the edge IoT consisting of unmanned aerial vehicles (UAVs) vulnerable to network intrusion. Therefore, it is critical to design an effective intrusion detection system (IDS). However, the differences in local data sets among UAVs show small samples and uneven distribution, further reducing the detection accuracy of network intrusion. This article proposes a conditional generative adversarial net (CGAN)-based collaborative intrusion detection algorithm with blockchain-empowered distributed federated learning to solve the above problems. This study introduces long short-term memory (LSTM) into the CGAN training to improve the effect of generative networks. Based on the feature extraction ability of LSTM networks, the generated data with CGAN are used as augmented data and applied in the detection and classification of intrusion data. Distributed federated learning with differential privacy ensures data security and privacy and allows collaborative training of CGAN models using multiple distributed data sets. Blockchain stores and shares the training models to ensure security when the global model’s aggregation and updating. The proposed method has good generalization ability, which can greatly improve the detection of intrusion data.

A dataset of spatial distribution of bioclimatic variables in China at 1 km resolution

Bioclimatic variables are indicators reflecting the integrated relationship between living things and climate. They are often used to interprete the relationships between species, vegetations and climate in global change research, and further simulate the geographical distribution patterns of both species and vegetations, as well as their functional characteristics. Regional bioclimate datasets, however, have been rarely reported. Based on an ANUSPIN interpolated dataset (covering temperature, precipitation and sunshine percentage) of 1km-resolution climate variables in China at 30-year basis averaged from 1951 to 1980 and from 1981 to 2010, respectively, we calculated 9 kinds bioclimatic variables in this study, namely mean temperature of the coldest month, mean temperature of the warmest month, absolute maximum temperature, absolute minimum temperature, annual growing degree days above 0°C and 5°C, growing season precipitation, annual drought index and annual moisture index. We plotted their spatial distribution map and analyzed their spatial pattern and trend statistically. Comparative analysis shows that the variation range of corresponding variables is very narrow, and the statistical variables are nearly the same. Therefore, the error of this dataset mainly comes from the spatial distribution dataset of basic climatic factors, and the secondary error in the process is tiny.This dataset provides reasonable environmentally mechanistic explanations for research on the relationships between species, vegetations and climate, and offers a convenient and diverse way for researchers to use bioclimatic variables to simulate species distribution patterns, vegetation structures and functions.

A dataset of glacier crevasse distribution in the ablation areas of Hailuogou Glacier, Mt. Gongga (2016–2021)

The distribution of crevasses in mountain glaciers is of great significance for the study on glacier dynamics, ablation, and glacial instability-induced disasters. From 2016 to 2021, based on repeated aerial surveys during ablation seasons in the Hailuogou Glacier ablation areas on the eastern slope of Gongga Mountain, we obtained the annual high-resolution orthophoto images of the 4 km ice tongue of the glacier during the ablation period. Based on these co-registered images, the crevasses in Hailuogou ice tongue were visually interpreted and delineated from 2016 to 2021. We preliminarily analyzed the distribution characteristics and inter-annual variation of the crevasses. This dataset contains spatial information attributes like the crevasse length and orientation. It is the first set of survey results related to the structural glaciology of mountain glaciers in western China and it can provide data reference for the study on glacial hydrology, mass balance and dynamic models.

A dataset of benthic material distribution of coastal coral reefs in Hainan during 2018–2020

Based on the Sentinel-2 high resolution remote sensing image data, we adopted the object-oriented multi-scale segmentation technology and the nearest neighbor algorithm to classify benthic materials of offshore coral reefs in Wenchang City, Danzhou City, Sanya City of Hainan Province during 2018-2020. Then, combining with the remote sensing images of Gaofen-2 satellite and field validation data we verified the accuracy of the classification results. The dataset can be used to analyze the temporal and spatial changes of benthic materials in coastal coral reefs and assess the health status of coral reefs. Moreover, it can serve as a data foundation for the management and protection of coral reefs in Hainan Island.