Vector Data Research Articles

Fitting Penalized Estimator for Sparse Covariance Matrix with Left-Censored Data by the EM Algorithm

Estimating the sparse covariance matrix can effectively identify important features and patterns, and traditional estimation methods require complete data vectors on all subjects. When data are left-censored due to detection limits, common strategies such as excluding censored individuals or replacing censored values with suitable constants may result in large biases. In this paper, we propose two penalized log-likelihood estimators, incorporating the L1 penalty and SCAD penalty, for estimating the sparse covariance matrix of a multivariate normal distribution in the presence of left-censored data. However, the fitting of these penalized estimators poses challenges due to the observed log-likelihood involving high-dimensional integration over the censored variables. To address this issue, we treat censored data as a special case of incomplete data and employ the Expectation Maximization algorithm combined with the coordinate descent algorithm to efficiently fit the two penalized estimators. Through simulation studies, we demonstrate that both penalized estimators achieve greater estimation accuracy compared to methods that replace censored values with constants. Moreover, the SCAD penalized estimator generally outperforms the L1 penalized estimator. Our method is used to analyze the proteomic datasets.

Method for Testing Combinational Circuits by Multiple Diagnostic Features Using Weight-Based Sum Codes Properties

The paper puts forth a methodology for the arrangement of calculation control at the outputs of combinational digital devices based on utilizing multiple diagnostic features. The first diagnostic feature is the belonging of the formed codewords to a pre-selected weight-based sum code. The second and third diagnostic features are the control of calculations by the belonging of calculated functions describing data and checking bits of sum codes to the class of self-dual and related functions. The arrangement of calculation control by multiple diagnostic features is founded upon the implementation of Boolean signal correction during the synthesis of the embedded control circuit. This is conducted in consideration of the established characteristics of weight-based sum codes, wherein the functions describing their checking bits exhibit identical values on pairs of data vectors inversed in all bits (these are the so-called self-quasidual functions). The utilization of Boolean signal correction on the orthogonal to all input variable combinations enables the additional determination of correction function values. This ensures the self-duality of functions describing data bits and, consequently, the self-quasiduality of functions describing checking bits. The paper proposes a structure for arranging the calculation control in accordance with the three specified diagnostic features. Furthermore, it develops the algorithm of correction function additional determination that underlies the synthesis of the embedded control circuit. It should be noted that the algorithm does not require the analysis of the values of functions calculated at the outputs of the object of diagnostics; instead, it automatically allows for the performance of additional determinations. This facilitates its utilization when integrated with computer-aided logic design systems. The paper presents a case study of the process of obtaining correction functions and simulates the operation of the synthesized self-checking device, thereby demonstrating the effectiveness of the proposed method.

Joint Modelling of Astrophysical Systematics for Cosmology with LSST Cosmic Shear

Abstract We present a novel framework for jointly modelling the weak lensing source galaxy redshift distribution and the intrinsic alignment (IA) of galaxies through a shared luminosity function (LF). In the context of a Rubin Observatory’s Legacy Survey of Space and Time (LSST) Year 1 and Year 10 cosmic shear analysis, we show that our novel approach produces cosmological parameter constraints which are comparable to standard methods, while offering more physical insight into IA and selection effects. We clarify the relationship between individual parameters of a Schechter luminosity function and the redshift distribution of a magnitude-limited sample, showing the consequences of marginalizing over these parameters when modelling intrinsic alignments in standard cosmic shear analyses. We explore the impact of the shape of the luminosity function on the cosmic shear data vector, and we outline the potential of this method to naturally model selection functions in redshift distribution estimation. Although this work focuses on LSST cosmic shear, the proposed joint modelling framework is broadly applicable to weak lensing surveys.

Exploring data flow design and vectorization with oneAPI for streaming applications on CPU+GPU

In recent times, oneAPI has emerged as a competitive framework to optimize streaming applications on heterogeneous CPU+GPU architectures, since it provides portability and performance thanks to the SYCL programming language and efficient parallel libraries as oneTBB. However, this approach opens up a wealth of implementations alternatives in this type of applications: from how to design the data flow to how to exploit data parallelism. Choosing the best alternative is not trivial, so in this paper we analyze them and contribute with an analytical model based on queue theory that helps in the on-line selection of the alternative that maximizes the throughput and the occupancy of the CPU and GPU compute units. We explore the design space offered by: a) different APIs to define the data flow (parallel_pipeline and Flow Graph from oneTBB, and SYCL events from SYCL); b) alternative kernel implementations to express data parallelism (SYCL, AVX and std::simd); and c) the mapping of the kernels into the available computing resources (CPU cores and GPU). The results show that the std::simd library can be 1.54x faster, 3% more energy efficient, and requires 7.36x less programming effort than AVX, and that implementations that enable asynchronous offloading of tasks to the devices as those based on SYCL events and Flow Graph APIs outperform the other APIs, being up to 1.10x faster and up to 1.18x more energy efficient.

Accounting the Ways of Weighing the Bits of the Data Vector when Constructing a Code with Summation in the Ring of Deductions Modulo M = 4 in the Synthesis of Self-Checking Discrete Devices Based on Boolean Signals Correction

The article shows that in the synthesis of concurrent error-detection circuit based on Boolean signals correction, weighted codes with summation in the ring of deductions modulo M = 4 (weight-based Bose — Lin codes) with various arrays of weighting coefficients can be effectively used. The codes with four data and two test symbols used in organizing the control of calculations in a group of six outputs of the diagnostic object are considered in more detail. Four arrays of weight coefficients are highlighted: [1, 1, 1, 2], [1, 1, 2, 3], [1, 2, 2, 3], [2, 2, 2, 3] — the use of which in the construction of a weight-based Bose — Lin code leads to the formation of for each the test vector has exactly one data vector of the following types: <00ab>, <01 ab>, <10ab>, <11 ab>, where a, b ' {0, 1}. This property allows the use of codes with the specified arrays of weighting coefficients in the synthesis of concurrent error-detection circuit with the conversion of only part of the signals from the diagnostic object — two signals involved in the formation of the lower bits of the data vector. The article presents the results of experimental studies of weight-based Bose — Lin codes in the synthesis of concurrent error-detection circuit based on the Boolean signals correction using two algorithms. The first algorithm is based on the use of converting only those signals from the diagnostic object that are involved in the formation of test symbols. The second algorithm is based on the conversion of only those signals from the diagnostic object that participate in the formation of the two lowest digits of the data vector. It has been experimentally established that self-checking devices synthesized using a code with an array of weighting coefficients have the highest efficiency in terms of structural redundancy [2, 2, 2, 3] (on average, the value of the structural redundancy index has been reached at 73 % of the duplication structure). Next, according to this indicator, there are devices synthesized using a code with an array of weighting coefficients [1, 2, 2, 3] (on average, 76—77 % of the duplication structure). The use of arrays of weighting coefficients [1, 1, 2, 3] and [1, 1, 1, 2] in the construction of the code gives a slightly lower effect (on average 79—81 % from the duplication structure). It is shown that in order to ensure complete self-checking of device structures, it will be necessary to use permutations of outputs within controlled groups of outputs of the initial diagnostic object and between groups. The results of the study can be taken into account when developing self-checking digital devices, as well as software tools for their computer-aided design.

A comparison of shrinkage estimators of the cosmological precision matrix

Abstract The determination of the covariance matrix and its inverse, the precision matrix, is critical in the statistical analysis of cosmological measurements. The covariance matrix is typically estimated with a limited number of simulations at great computational cost before inversion into the precision matrix; therefore, it can be ill-conditioned and overly noisy when the sample size n used for estimation is not much larger than the data vector dimension. In this work, we consider a class of methods known as shrinkage estimation for the precision matrix, which combines an empirical estimate with a target that is either analytical or stochastic. These methods include linear and non-linear shrinkage applied to the covariance matrix (the latter represented by the so-called NERCOME estimator), and the direct linear shrinkage estimation of the precision matrix which we introduce in a cosmological setting. By performing Bayesian parameter inference and using metrics like matrix loss functions, the Kullback–Leibler divergence and the eigenvalue spectrum, we compare their performance against the standard sample estimator with varying sample size n. We have found the shrinkage estimators to significantly improve the posterior distribution at low n, especially for the linear shrinkage estimators either inverted from the covariance matrix or applied directly to the precision matrix, with an empirical target constructed from the sample estimate. Our results are particularly relevant to the analyses of Stage-IV spectroscopic galaxy surveys such as the Dark Energy Spectroscopic Instrument (DESI) and Euclid, whose statistical power can be limited by the computational cost of obtaining an accurate precision matrix estimate.

Uncertainty-Aware Deep Neural Representations for Visual Analysis of Vector Field Data.

The widespread use of Deep Neural Networks (DNNs) has recently resulted in their application to challenging scientific visualization tasks. While advanced DNNs demonstrate impressive generalization abilities, understanding factors like prediction quality, confidence, robustness, and uncertainty is crucial. These insights aid application scientists in making informed decisions. However, DNNs lack inherent mechanisms to measure prediction uncertainty, prompting the creation of distinct frameworks for constructing robust uncertainty-aware models tailored to various visualization tasks. In this work, we develop uncertainty-aware implicit neural representations to model steady-state vector fields effectively. We comprehensively evaluate the efficacy of two principled deep uncertainty estimation techniques: (1) Deep Ensemble and (2) Monte Carlo Dropout, aimed at enabling uncertainty-informed visual analysis of features within steady vector field data. Our detailed exploration using several vector data sets indicate that uncertaintyaware models generate informative visualization results of vector field features. Furthermore, incorporating prediction uncertainty improves the resilience and interpretability of our DNN model, rendering it applicable for the analysis of non-trivial vector field data sets.

Sample selection using multi-task autoencoders in federated learning with non-IID data

Federated learning is a machine learning paradigm in which multiple devices collaboratively train a model under the supervision of a central server while ensuring data privacy. However, its performance is often hindered by redundant, malicious, or abnormal samples, leading to model degradation and inefficiency. To overcome these issues, we propose novel sample selection methods for image classification, employing a multi-task autoencoder to estimate sample contributions through loss and feature analysis. Our approach incorporates unsupervised outlier detection, using one-class support vector machine (OCSVM), isolation forest (IF), and adaptive loss threshold (AT) methods managed by a central server to filter noisy samples on clients. We also propose a multi-class deep support vector data description (SVDD) loss controlled by a central server to enhance feature-based sample selection. We validate our methods on CIFAR10 and MNIST datasets across varying numbers of clients, non-IID distributions, and noise levels up to 40%. The results show significant accuracy improvements with loss-based sample selection, achieving gains of up to 7.02% on CIFAR10 with OCSVM and 1.83% on MNIST with AT. Additionally, our federated SVDD loss further improves feature-based sample selection, yielding accuracy gains of up to 0.99% on CIFAR10 with OCSVM. These results show the effectiveness of our methods in improving model accuracy across various client counts and noise conditions.

Text vectorization in sentiment analysis: A comparative study of TF-IDF and Word2Vec from Amazon Fine Food Reviews

Sentiment analysis is a practical tool for marketing and branding teams. Companies can collect and analyze opinions or reviews from social media platforms, blog posts, and other numerous forums. It may help them acquire positive feedback to reinforce strengths or identify negative emotions to make improvements. The research is to compare two text vectorization methods in opinion mining: Term Frequency-Inverse Document Frequency (TF-IDF) and Word2Vec, using Amazon Fine Food Reviews dataset. This study will use these two methods to vectorize preprocessed text data and also input the vectorized data to the emotion classification model, analyzing the performance of two methods in the emotion classification task. The consequence indicates that the former outperforms the latter in handling large datasets, particularly in distinguishing between different sentiment categories, but latter is superior in capturing the semantic relationship of words. Therefore, it is suggested that the advantages of the two methods be combined in practical applications to improve the accuracy and efficiency.

Enhanced anomaly detection of industrial control systems via graph-driven spatio-temporal adversarial deep support vector data description

Enhanced anomaly detection of industrial control systems via graph-driven spatio-temporal adversarial deep support vector data description

Generalized Fused Lasso for Treatment Pooling in Network Meta-Analysis.

This work develops a generalized fused lasso (GFL) approach to fitting contrast-based network meta-analysis (NMA) models. The GFL method penalizes all pairwise differences between treatment effects, resulting in the pooling of treatments that are not sufficiently different. This approach offers an intriguing avenue for potentially mitigating biases in treatment rankings and reducing sparsity in networks. To fit contrast-based NMA models within the GFL framework, we formulate the models as generalized least squares problems, where the precision matrix depends on the standard error in the data, the estimated between-study heterogeneity and the correlation between contrasts in multi-arm studies. By utilizing a Cholesky decomposition of the precision matrix, we linearly transform the data vector and design matrix to frame NMA within the GFL framework. We demonstrate how to construct the GFL penalty such that every pairwise difference is penalized similarly. The model is straightforward to implement in R via the "genlasso" package, and runs instantaneously, contrary to other regularization approaches that are Bayesian. A two-step GFL-NMA approach is recommended to obtain measures of uncertainty associated with the (pooled) relative treatment effects. Two simulation studies confirm the GFL approach's ability to pool treatments that have the same (or similar) effects while also revealing when incorrect pooling may occur, and its potential benefits against alternative methods. The novel GFL-NMA method is successfully applied to a real-world dataset on diabetes where the standard NMA model was not favored compared to the best-fitting GFL-NMA model with AICc selection of the tuning parameter ( .

Accuracy Evaluation Method for Vector Data Based on Hexagonal Discrete Global Grid

With the continuous advancement of technology for obtaining geographic spatial data, the accumulated volume of such data has been increasing, thus imposing higher demands on the storage, organization, and management of such data. As a new form of data management, the Discrete Global Grid System (DGGS) provides standardized descriptions and the exchange of geographic information on a global scale, enabling the efficient storage and application of large-scale global spatial data. Constituting a traditional type of GIS spatial data, vector data have advantages such as clear positions, implicit attributes, and suitability for map output. The representation of vector data in the global discrete grid network based on an equal-area projection, such as the hexagonal grid, fundamentally solves problems such as data redundancy, geometric deformation, and data discontinuity that arise when representing multiple vector data in a gridded format. This paper proposes different gridding methods for various types of vector data, and a quantifiable accuracy evaluation system is established from the perspectives of geographical deviation, geometric features, and topological relationships, to evaluate the accuracy of the gridded vector data, covering all types of gridded vector data based on the hexagonal grid. The evaluation method is generally applicable to all hexagonal-grid-based gridded vector data, and can be generalized based on application scenarios, for evaluating the usability of hexagonal grid vector data.

Improved Deep Support Vector Data Description Model Using Feature Patching for Industrial Anomaly Detection.

In industrial contexts, anomaly detection is crucial for ensuring quality control and maintaining operational efficiency in manufacturing processes. Leveraging high-level features extracted from ImageNet-trained networks and the robust capabilities of the Deep Support Vector Data Description (SVDD) model for anomaly detection, this paper proposes an improved Deep SVDD model, termed Feature-Patching SVDD (FPSVDD), designed for unsupervised anomaly detection in industrial applications. This model integrates a feature-patching technique with the Deep SVDD framework. Features are extracted from a pre-trained backbone network on ImageNet, and each extracted feature is split into multiple small patches of appropriate size. This approach effectively captures both macro-structural information and fine-grained local information from the extracted features, enhancing the model's sensitivity to anomalies. The feature patches are then aggregated and concatenated for further training with the Deep SVDD model. Experimental results on both the MvTec AD and CIFAR-10 datasets demonstrate that our model outperforms current mainstream approaches and provides significant improvements in anomaly detection performance, which is vital for industrial quality assurance and defect detection in real-time manufacturing scenarios.

Adaptive Neuro-Fuzzy System for Detection of Wind Turbine Blade Defects

Wind turbines are the most frequently used objects of renewable energy today. However, issues that arise during their operation can greatly affect their effectiveness. Blade erosion, cracks, and other defects can slash turbine performance while also forcing maintenance costs to soar. Modern defect detection applications have significant computing resources needed for training and insufficient accuracy. The goal of this study is to develop the improved adaptive neuro-fuzzy inference system (ANFIS) for wind turbine defect detection, which will reduce computing resources and increase its accuracy. Unmanned aerial vehicles are deployed to photograph the turbines, and these images are beamed back and processed for early defect detection. The proposed adaptive neuro-fuzzy inference system processes the data vectors with lower complexity and higher accuracy. For this purpose, the authors explored grid partitioning and subtractive clustering methods and selected the last one because it uses three rules only for fault detection, ensuring low computational costs and enabling the discovery of wind turbine defects quickly and efficiently. Moreover, the proposed ANFIS is implemented in a controller, which has an accuracy of 91%, that is 1.4 higher than the accuracy of the existing similar controller.

Niamey: The ages of a city

Over the last few decades, countries in the South have been undergoing rapid urbanization, as if to make up for lost time. Sub-Saharan Africa is characterized by a very low urbanization rate compared to0 the rest of the world. Although the African continent reached its urban transition in 2015, Niger remains by far the least urbanized country, with a rate of 17%. The city of Niamey is the main urban center, with an estimated population of 1,449,801 hbts in 2023, spread over an area of around 33,100 ha. The aim of this study is to analyze the spatial expansion of the city of Niamey from 1984 to 2023. The main data used in this study are raster images from the United States Geological Survey (USGS), vector data from Open Sources Map (OSM) and GoogleEarth, secondary data from the National Institute of Statistics (INS) and field observation. This study enabled us to conclude that between 1984 and 2023, the city of Niamey underwent very strong spatial expansion. The city grew from 4,690 ha to 33,100 ha, i.e. 28,410 ha absorbed in 39 years, with exceptional growth between 2014 and 2023, when the urban area doubled. Its population has risen from 397,437 at the time of the 1988 general population and housing census to an estimated 1,449,801 in 2023 (INS), an increase of 1,052,364 in 35 years. Between these two dates, population density fell from 87.7 to 43.8 inhabitants/km2, i.e. half that of 1984. This spatial expansion has resulted in unprecedented peri-urbanization.

Bearing life prediction method based on novel health indicators and improved similarity curve matching

Abstract To tackle the issues of low accuracy in similarity curve matching and the challenges in measuring the similarity of degraded curve segments in rolling bearing life prediction methods, we propose a novel bearing remaining useful life (RUL) prediction method utilizing new health indicators and improved similarity curve matching techniques. Initially, time and frequency domain features are extracted from the bearing vibration signals. Subsequently, the multi-dimensional features are meticulously screened using monotonicity, Spearman's correlation, and uncertainty metrics to construct the health indicator (HI) and the health indicator library for the bearing. Subsequently, the support vector data description (SVDD) combined with the PID search algorithm is employed to ascertain the initial prediction time of the bearing and to construct the corresponding health indicator library. Following this, both global and local feature capture are conducted using dual dynamic time warping (DDTW) in conjunction with a sliding window to predict the remaining service life of rolling bearings.
The proposed method is validated on the PHM2012 and XJTU-SY datasets and benchmarked against the latest research. The results demonstrate that our method significantly enhances the prediction accuracy of the remaining useful life of bearings, underscoring the effectiveness and superiority of the proposed approach.and superiority of the proposed approach.

Protecting location privacy: An approach with error-based transformation and mercator projection-based transformation for geospatial vector data

Protecting location privacy: An approach with error-based transformation and mercator projection-based transformation for geospatial vector data

Quantifying the rise of animals during the Ediacaran–Cambrian using ichnodissimilarity

Abstract The trace fossil record provides important insights into the evolution of early animals during the Ediacaran/Cambrian transition, with changes in ichnodiversity through time and between environments informing on the diversification of major body plans, behaviors, and niches. To quantify variation in the diversity of trace fossils across this critical interval, we propose a measure of trace fossil dissimilarity (ichnodissimilarity) based on vector calculation. Furthermore, by comparing discrepancies between the angular bisector and mean vector of two sets of vectorized fossil data, we are able to weigh the relative contribution of increases and decreases in the variation of occurrences of taxa. We used this metric to analyze an expansive dataset of Ediacaran/Cambrian trace fossils. The results allowed us to quantify the diversification of traces across this transition, informing on the timing of first appearance of different behaviors (e.g., foraging, grazing, and resting) and functional groups. By interpreting the results in the context of environmental changes and advancements in motility and sensory capabilities, we were able to pinpoint the onset and sequence of the Fortunian diversification event, Cambrian information revolution, and agronomic revolution, shedding light on the evolution of organismal body plans, behaviors, and locomotion during the Ediacaran/Cambrian transition. We identified two phases of origination and expansion during the divergence of early animal traces. Furthermore, by analyzing shallow- and deep-marine trace fossils, we were able to uncover evidence for a more rapid diversification of traces in shallow-marine environments, with progressive niche partitioning through the Ediacaran to Cambrian.

Selection of Grid Road Networks Based on Raster Data

In cartography, generalization is a key process used to simplify complex geographic information, making it suitable for display at different scales while maintaining its essential meaning. When representing high-density road networks on a fixed screen area, overcrowding and loss of clarity often occur. To solve these problems, a road selection operation can be applied. However, traditional methods have primarily focused on structured vector road networks, leaving unstructured raster road networks largely unaddressed. This study introduces a novel technique, Adaptive Road Width Selection (ARWS), designed to improve the multiscale visualization of compact road systems using unstructured raster datasets. The ARWS method begins by segmenting the original raster road network into multilevel superpixels of varying sizes, reflecting the road widths, through neighborhood analysis. Next, road superpixel matching and selection are performed based on the minimum angle and maximum distance rules, alongside shortest-path calculations. Finally, redundant intersection pixels are eliminated to generate the selection results. The proposed ARWS method was evaluated using road network data from Shenzhen, China, producing effective multiscale visualization outcomes. Unlike conventional techniques relying on structured vector data, ARWS excels in preserving the semantic attributes, overall structure, local connectivity, and integrity of road networks. It addresses the challenges of multiscale visualization in dense road networks, offering a robust solution for unstructured raster data.

Fast Solution of Sylvester‐Structured Systems for Spatial Source Separation of the Cosmic Microwave Background

ABSTRACTImplementation of many statistical methods for large, multivariate data sets requires one to solve a linear system that, depending on the method, is of the dimension of the number of observations or each individual data vector. This is often the limiting factor in scaling the method with data size and complexity. In this paper, we illustrate the use of Krylov subspace methods to address this issue in a statistical solution to a source separation problem in cosmology where the data size is prohibitively large for direct solution of the required system. Two distinct approaches, adapted from techniques in the literature, are described: one that uses the method of conjugate gradients directly to the Kronecker‐structured problem and another that reformulates the system as a Sylvester matrix equation. We show that both approaches produce an accurate solution within an acceptable computation time and with practical memory requirements for the data size that is currently available.