Real Datasets Research Articles

Enhanced Estimation of Population Variance under Simple Random Sampling with an Application to Real Data

In this paper, we present an enhanced class of estimators for finite population variance under simple random sampling that incorporates an auxiliary information. The equations for bias and mean square error of the proposed estimators are produced up to the first order of approximation. Some well-known estimators are found as a special case of the proposed estimators for suitably chosen values of the scalars. The effectiveness of the proposed estimators is calculated in relation to the existing estimators both theoretically and empirically using five real data sets. The empirical results exhibit the superiority of the suggested estimators over to the existing estimators in each population.. KEYWORDS :Auxiliary variable, Mean square error, Variance estimation, Class of estimators.

On Modi-Exponential Distribution under Conventional Type-I and Type-II Censoring

In this article, a new Modi-Exponential distribution is considered under Type-I and Type-II censoring. Some statistical and reliability characteristics of the distribution are specified. Statistical inferences of the Modi-Exponential parameters based on Type-I and Type-II censored data are discussed. The maximum likelihood estimation method is used for estimation of unknown parameters along with reliability characteristics. Asymptotic confidence intervals are also obtained for interval estimation. Monte-Carlo Simulation study is performed to study the performances of estimators. Further, two real data sets are provided to illustrate the results.. KEYWORDS :Censoring, Maximum likelihood estimation, Confidence bounds, Simulation.

Improving coronary heart disease prediction with real-life dataset: a stacked generalization framework with maximum clinical attributes and SMOTE balancing for imbalanced data

Improving coronary heart disease prediction with real-life dataset: a stacked generalization framework with maximum clinical attributes and SMOTE balancing for imbalanced data

MSTG: Multi-Scale Transformer with Gradient for joint spatio-temporal enhancement

Despite the widespread availability of HDR (High Dynamic Range) display devices, the majority of video sources are still stored in SDR (Standard Dynamic Range) format, leading to an urgent need for UHD (Ultra High Definition) video reconstruction. A simple solution is to divide the task into two sub-tasks: video super-resolution (SR) and video Bit-Depth Enhancement (BDE). While some joint enhancement tasks led by SR have been explored in multiple dimensions, the quantitative dimension has often been overlooked. In this paper, we address the first joint task of BDE and SR and propose a succinct network called MSTG. We conduct a systematic analysis to explain how this effect can be achieved and overcome the core challenge arising from distortions exacerbated by the differing optimization directions of BDE and SR. This challenge is attributed to indistinguishable contours and detailed textures. To tackle this, we employ a dual-branch module that leverages both gradient and image information to discern between BDE and SR. Furthermore, we propose a novel model named Cross-scale Transformer (CTF) embedded with Cross-scale Attention (CSA). This model facilitates information aggregation while adaptively utilizing structural similarity features. It jointly extracts, aligns, and fuses frame features at multiple scales, forming the MST module. Experiments demonstrate that the proposed algorithm outperforms the cascaded video enhancement methods by large margins on both synthetic and realistic datasets.

Enhanced Compression of k-Mer Sets with Counters via de Bruijn Graphs.

An essential task in computational genomics involves transforming input sequences into their constituent k-mers. The quest for an efficient representation of k-mer sets is crucial for enhancing the scalability of bioinformatic analyses. One widely used method involves converting the k-mer set into a de Bruijn graph (dBG), followed by seeking a compact graph representation via the smallest path cover. This study introduces USTAR* (Unitig STitch Advanced constRuction), a tool designed to compress both a set of k-mers and their associated counts. USTAR leverages the connectivity and density of dBGs, enabling a more efficient path selection for constructing the path cover. The efficacy of USTAR is demonstrated through its application in compressing real read data sets. USTAR improves the compression achieved by UST (Unitig STitch), the best algorithm, by percentages ranging from 2.3% to 26.4%, depending on the k-mer size, and it is up to times faster.

Mining Spatial Co-Location Patterns With a Mixed Prevalence Measure.

A co-location pattern indicates a subset of spatial features whose instances are frequently located together in proximate geographical space. Most previous studies of spatial co-location pattern mining concern what percentage of instances per feature are involved in the table instance of a pattern, but neglect the heterogeneity in the number of feature instances and the distribution of instances. As a result, the deviation may be occurred in the interest measure of co-locations. In this article, we propose a novel mixed prevalence index (MPI) incorporating the effect of feature-level and instance-level heterogeneity on the prevalence measure, which can address some dilemmas in existing interest measures. Luckily, MPI possesses the partial antimonotone property. In virtue of this property, a branch-based search algorithm equipped with some optimizing strategies of MPI calculation is proposed, namely, Branch-Opt-MPI. Comprehensive experiments are conducted on both real and synthetic spatial datasets. Experimental results reveal the superiority of MPI compared to other interest measures and also validate the efficiency and scalability of the Branch-Opt-MPI. Particularly, the Branch-Opt-MPI performs more efficiently than baselines for several times or even orders of magnitude in dense data.

A benchmark of health insurance fraud detection using machine learning techniques

Health insurance fraud is a complex problem that also has a significant financial impact. Recently, with the availability of large volumes of data and the evolution of computing power, machine learning techniques have become the preferred method for fraud detection. However, the main difficulty facing researchers in this field is the lack of real data sets and the absence of reliable fraud labels. Most published studies use aggregated provider-level or simulated data to test fraud detection algorithms, which may not deliver accurate results. The present study aims to provide a more accurate assessment of fraud detection methods by using real detailed health insurance claims data to compare six of the most common supervised classification algorithms including neural networks and the use of two categorical feature preparation methods. The study was conducted under the guidance of insurance experts, who provided the fraud label inference rules and reviewed the results. A comprehensive description of the benchmarking process and an interpretation of the results are provided in this paper. The results show that supervised classification can be used effectively to detect health insurance fraud, improving detection accuracy by a factor of 4.2 (84% recall for a positive rate of 20%).<p> </p>

RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining.

As common weather, rain streaks adversely degrade the image quality and tend to negatively affect the performance of outdoor computer vision systems. Hence, removing rains from an image has become an important issue in the field. To handle such an ill-posed single image deraining task, in this article, we specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet), which embeds the intrinsic priors of rain streaks and has clear interpretability. In specific, we first establish a rain convolutional dictionary (RCD) model for representing rain streaks and utilize the proximal gradient descent technique to design an iterative algorithm only containing simple operators for solving the model. By unfolding it, we then build the RCDNet in which every network module has clear physical meanings and corresponds to each operation involved in the algorithm. This good interpretability greatly facilitates an easy visualization and analysis of what happens inside the network and why it works well in the inference process. Moreover, taking into account the domain gap issue in real scenarios, we further design a novel dynamic RCDNet, where the rain kernels can be dynamically inferred corresponding to input rainy images and then help shrink the space for rain layer estimation with few rain maps, so as to ensure a fine generalization performance in the inconsistent scenarios of rain types between training and testing data. By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted, faithfully characterizing the features of both rain and clean background layers and, thus, naturally leading to better deraining performance. Comprehensive experiments implemented on a series of representative synthetic and real datasets substantiate the superiority of our method, especially on its well generality to diverse testing scenarios and good interpretability for all its modules, compared with state-of-the-art single image derainers both visually and quantitatively. Code is available at https://github.com/hongwang01/DRCDNet.

Origin-destination demand prediction of public transit using graph convolutional neural network

The insight into origin–destination (OD) demand patterns aids transport planners in making the public transit system more efficient and attractive. This may encourage individuals to shift from private vehicles to public transit, easing the burden on traffic and its negative impacts. Hence, to know how OD demand is going to vary in future, a state-of-the-art OD demand prediction model needs to be developed. Previously, studies have developed zone-based prediction models which may not be appropriate for predicting OD demand within a route of public transit. Additionally, spatial correlations between the stops of public transit must be included in the model for improved forecasting accuracy. Hence, in an effort to fulfil these gaps, a Graph Convolutional Neural Network (GCN) is developed to forecast the OD demand of public bus transit with nodes being the bus stops and links between them representing the passenger flow between the stops. Land use around the bus stops is retrieved as a node feature and included in the model to account for the spatial correlation between the stops. The model is trained using a real-life dataset from the public bus service of Davangere city located in India. Land use around the bus stops is extracted from the Davangere city master plan, procured from the urban development authority. The developed model is compared with conventional models and the findings show that the GCN model performs better in terms of prediction accuracy than the baseline models. Additionally, at the stop level, the performance of the model remained stable due to the inclusion of land use data compared to conventional models where land use data was not considered.

A new pulsar timing model for scalar-tensor gravity with applications to PSR J2222-0137 and pulsar-black hole binaries

Context. Scalar-tensor gravity (STG) theories are well-motivated alternatives to general relativity (GR). One class of STG theories, Damour–Esposito–Farèse (DEF) gravity, has a massless scalar field with two arbitrary coupling parameters. We are interested in this theory because, despite its simplicity, it predicts a wealth of different phenomena, such as dipolar gravitational wave emission and spontaneous scalarisation of neutron stars (NSs). These phenomena of DEF gravity can be tested by timing binary radio pulsars. In the methods used so far, intermediate phenomenological post-Keplerian (PK) parameters are measured by fitting the corresponding timing model to the timing data whose values are then compared to the predictions from the alternative theory being tested. However, this approach loses information between intermediate steps and does not account for possible correlations between PK parameters. Aims. We aim to develop a new binary pulsar timing model ‘DDSTG’ (called after Damour, Deruelle and STG) to enable more precise tests of STG theories based on a minimal set of binary parameters. The expressions for PK parameters in DEF gravity are self-consistently incorporated into the model. PK parameters depend on two masses which are now directly fitted to the data without intermediate steps. The new technique takes into account all possible correlations between PK parameters naturally. Methods. Grids of physical parameters of NSs were calculated in the framework of DEF gravity for a set of 11 equations of state. Automatic differentiation (AutoDiff) technique was employed, which aids in the calculation of gravitational form factors of NSs with a higher precision than in previous works. The pulsar timing program TEMPO was selected as a framework for the realisation of the DDSTG model. The implemented model is applicable to any type of pulsar companions. We also simulated realistic future radio-timing datasets for a number of large radio observatories for the binary pulsar PSR J2222-0137 and three generic pulsar-black hole (PSR-BH) systems. Results. We applied the DDSTG model to the most recently published observational data for PSR J2222-0137. The obtained limits on DEF gravity parameters for this system confirm and improve previous results. New limits are also the most reliable because DEF gravity is directly fitted to the data. We argue that future observations of PSR J2222-0137 can significantly improve the limits and that PSR-BH systems have the potential to place the tightest limits in certain areas of the DEF gravity parameter space.

An extended Rayleigh Weibull model with actuarial measures and applications

In this article, we discuss a new extension of the Rayleigh-Weibull model using the Marshall-Olkin family of distributions. The proposed model is called the Marshall-Olkin-Rayleigh-Weibull (MORW) model. Various statistical properties of the MORW distribution are discussed, including explicit expressions for quantiles, moments, incomplete and conditional moments, some inequality measures, moment generating function, moments of the residual and reversed residual life, the Rényi entropy, and order statistics. Six different estimation methods are considered to investigate the behavior of the model parameters within the MORW model. A Monte Carlo simulation study is conducted to evaluate the performance of these different estimators. In addition, some actuarial measures, such as the Value-at-Risk, the expected shortfall, the tail Value-at-Risk, the tail variance, the tail variance premium, the tail conditional expectation, and the tail standard deviation premium, are calculated. Finally, applying the model to real data sets illustrates the applicability and usefulness of the MORW distribution.

ViTR-Net: An unsupervised lightweight transformer network for cable surface defect detection and adaptive classification

As a crucial load-bearing component of the cable-stayed bridge, the cable requires surface defect detection to maintain its safety. The current deep-learning-based stay-cable surface defect detection still faces challenges, including insufficient actual defect samples, cumbersome labeling processes, and an effective method for further evaluation and classification of defects. To address these problems, an unsupervised vision transformer-based lightweight reconstruction network called ViTR-Net is proposed, which only uses normal stay-cable surface images to achieve end-to-end localization of defect areas in the image. ViTR-Net is a generative-based method that uses a hybrid network of convolutional neural networks and visual transformers as a feature extraction network. A feature correction fusion module is designed by integrating differential convolution and attention mechanisms to compensate for potential detail loss caused by feature extraction. Subsequently, a severity index for defects is established to assess the severity of various defects, and the detected defects are adaptively classified using the affinity propagation clustering algorithm. ViTR-Net achieves the highest defect detection accuracy and exquisite defect segmentation results on a realistic cable defect dataset through experimental validation. The proposed model can realize the surface defect assessment and adaptive classification of the stay cable, which has a good application prospect in the actual maintenance process of the stay cable.

Exploiting residual errors in nonlinear online prediction

We introduce a novel online (or sequential) nonlinear prediction approach that incorporates the residuals, i.e., prediction errors in the past observations, as additional features for the current data. Including the past error terms in an online prediction algorithm naturally improves prediction performance significantly since this information is essential for an algorithm to adjust itself based on its past errors. These terms are well exploited in many linear statistical models such as ARMA, SES, and Holts-Winters models. However, the past error terms are rarely or in a certain sense not optimally exploited in nonlinear prediction models since training them requires complex nonlinear state-space modeling. To this end, for the first time in the literature, we introduce a nonlinear prediction framework that utilizes not only the current features but also the past error terms as additional features, thereby exploiting the residual state information in the error terms, i.e., the model’s performance on the past samples. Since the new feature vectors contain error terms that change with every update, our algorithm jointly optimizes the model parameters and the feature vectors simultaneously. We achieve this by introducing new update equations that handle the effects resulting from the changes in the feature vectors in an online manner. We use soft decision trees and neural networks as the nonlinear prediction algorithms since these are the most widely used methods in highly publicized competitions. However, as we show, our methods are generic and any algorithm supporting gradient calculations can be straightforwardly used. We show through our experiments on the well-known real-life competition datasets that our method significantly outperforms the state-of-the-art. We also provide the implementation of our approach including the source code to facilitate reproducibility (https://github.com/ahmetberkerkoc/SDT-ARMA).

Bridging realities: training visuo-haptic object recognition models for robots using 3D virtual simulations

This paper proposes an approach for training visuo-haptic object recognition models for robots using synthetic datasets generated by 3D virtual simulations. In robotics, where visual object recognition has witnessed considerable progress due to an abundance of image datasets, the scarcity of diverse haptic samples has resulted in a noticeable gap in research on machine learning incorporating the haptic sense. Our proposed methodology addresses this challenge by utilizing 3D virtual simulations to create realistic synthetic datasets, offering a scalable and cost-effective solution to integrate haptic and visual cues for object recognition seamlessly. Acknowledging the importance of multimodal perception, particularly in robotic applications, our research not only closes the existing gap but envisions a future where intelligent agents possess a holistic understanding of their environment derived from both visual and haptic senses. Our experiments show that synthetic datasets can be used for training object recognition in haptic and visual modes by incorporating noise, performing some preprocessing, data augmentation, or domain adaptation. This work contributes to the advancement of multimodal machine learning toward a more nuanced and comprehensive robotic perception.

Efficient and Provable Effective Resistance Computation on Large Graphs: An Index-based Approach

Effective resistance (ER) is a fundamental metric for measuring node similarities in a graph, and it finds applications in various domains including graph clustering, recommendation systems, link prediction, and graph neural networks. The state-of-the-art algorithm for computing effective resistance relies on a landmark technique, which involves selecting a node that is easy to reach by all the other nodes as a landmark. The performance of this technique heavily depends on the chosen landmark node. However, in many real-life graphs, it is not always possible to find an easily reachable landmark node, which can significantly hinder the algorithm's efficiency. To overcome this problem, we propose a novel multiple landmarks technique which involves selecting a set of landmark nodes V l such that the other nodes in the graph can easily reach any one of a landmark node in V l . Specifically, we first propose several new formulas to compute ER with multiple landmarks, utilizing the concept of Schur complement. These new formulas allow us to pre-compute and maintain several small-sized matrices related to V l as a compact index. With this powerful index technique, we demonstrate that both single-pair and single-source ER queries can be efficiently answered using a newly-developed V l -absorbed random walk sampling or V l -absorbed push technique. Comprehensive theoretical analysis shows that all proposed index-based algorithms achieve provable performance guarantees for both single-pair and single-source ER queries. Extensive experiments on 5 real-life datasets demonstrate the high efficiency of our multiple landmarks-based index techniques. For instance, our algorithms, with a 1.5 GB index size, can be up to 4 orders of magnitude faster than the state-of-the-art algorithms while achieving the same accuracy on a large road network.

Mean Estimators for Sensitive Variables Using Scrambled Response Models

The most popular methods for sensitive study are regression estimation methods that use standard regression coefficients. When it comes to survey researchers, mean estimation is an important issue. The vast majority of population mean estimation methods that can be found in sampling theory are meant only to be utilized with non-sensitive data-sets. When the variable that is important is sensitive, such as drug usage, illegal income, abortion, exam cheating, the amount of income tax due, employee rule-breaking, etc., these estimation methods cannot operate efficiently. In this article, we propose a novel method for computing the population mean using exponential technique of robust-type regression estimators for scrambled response model (SRM) under simple random sampling (SRS). The mean square error (MSE) equation is generated using a first-order approximation and examined with existing estimating techniques in order to evaluate the efficacy of the new approach. Additionally, the proposed estimator's percentage relative efficiency (PRE) is determined compared to other estimators. The effectiveness of the proposed method is demonstrated using real data sets. According to the results, the suggested estimator performs better than other estimators in the literature.

InstaSynth: Opportunities and Challenges in Generating Synthetic Instagram Data with ChatGPT for Sponsored Content Detection

Large Language Models (LLMs) raise concerns about lowering the cost of generating texts that could be used for unethical or illegal purposes, especially on social media. This paper investigates the promise of such models to help enforce legal requirements related to the disclosure of sponsored content online. We investigate the use of LLMs for generating synthetic Instagram captions with two objectives: The first objective (fidelity) is to produce realistic synthetic datasets. For this, we implement content-level and network-level metrics to assess whether synthetic captions are realistic. The second objective (utility) is to create synthetic data useful for sponsored content detection. For this, we evaluate the effectiveness of the generated synthetic data for training classifiers to identify undisclosed advertisements on Instagram. Our investigations show that the objectives of fidelity and utility may conflict and that prompt engineering is a useful but insufficient strategy. Additionally, we find that while individual synthetic posts may appear realistic, collectively they lack diversity, topic connectivity, and realistic user interaction patterns.

Candl: cosmic microwave background analysis with a differentiable likelihood

We present candl, an automatically differentiable python likelihood for analysing cosmic microwave background power spectrum measurements. candl is powered by JAX, which makes it fast and easy to calculate derivatives of the likelihood. This facilitates, for example, robust Fisher matrices without finite-difference methods. We show the benefits of candl through a series of example calculations, covering forecasting, robustness tests, and gradient-based Markov chain Monte Carlo sampling. These also include optimising the band power bin width to minimise parameter errors of a realistic mock data set. Moreover, we calculate the correlation of parameter constraints from correlated and partially overlapping subsets of the SPT-3G 2018 TT/TE/EE data release. In a traditional analysis framework, these tasks are slow and require careful fine-tuning to obtain stable results. As such, a fully differentiable pipeline allows for a higher level of scrutiny; we argue that this is the paradigm shift required to leverage incoming data from ground-based experiments, which will significantly improve the cosmological parameter constraints from the Planck mission. candl comes with the latest primary and lensing power spectrum data from the South Pole Telescope and Atacama Cosmology Telescope collaborations and will be used as part of the upcoming SPT-3G TT/TE/EE and ϕϕ data releases. Along with the core code, we release a series of auxiliary tools, which simplify common analysis tasks and interface the likelihood with other cosmological software. candl is pip-installable and publicly available on Github.

A survey of BWT variants for string collections.

In recent years, the focus of bioinformatics research has moved from individual sequences to collections of sequences. Given the fundamental role of the Burrows-Wheeler Transform (BWT) in string processing, a number of dedicated tools have been developed for computing the BWT of string collections. While the focus has been on improving efficiency, both in space and time, the exact definition of the BWT employed has not been at the center of attention. As we show in this paper, the different tools in use often compute non-equivalent BWT variants: the resulting transforms can differ from each other significantly, including the number r of runs, a central parameter of the BWT. Moreover, with many tools, the transform depends on the input order of the collection. In other words, on the same dataset, the same tool may output different transforms if the dataset is given in a different order. We studied 18 dedicated tools for computing the BWT of string collections and were able to identify 6 different BWT variants computed by these tools. We review the differences between these BWT variants, both from a theoretical and from a practical point of view, comparing them on 8 real-life biological datasets with different characteristics. We find that the differences can be extensive, depending on the datasets, and are largest on collections of many similar short sequences. The parameter r, the number of runs of the BWT, also shows notable variation between the different BWT variants; on our datasets, it varied by a multiplicative factor of up to 4.2. Source code and scripts to replicate the results and download the data used in the article are available at https://github.com/davidecenzato/BWT-variants-for-string-collections. Supplementary data are available at Bioinformatics online.

Efficient Class of Variance Estimators for Population using Supplementary Information in Stratified Random Sampling

This paper addresses an efficient class of variance estimators for population using stratified random sampling. The suggested class of estimators using supplementary information has been studied in different circumstances. The expressions of bias and mean square error (MSE) of the proposed estimators are derived up to the first degree of approximation. The theoretical comparison of the proposed and considered estimators is also discussed, which shows that the proposed estimators are more efficient than the existing estimators. Theoretical findings are validated by three different types of real data sets and simulation studies. The numerical results of the proposed and existing estimators are compared in terms of mean square error, percentage relative efficiency and diagrams. It is observed that all the proposed estimators outperform the existing estimators. For instance, the traditional unbiased estimator Ozel et.al [6] and other existing estimators. Lastly, appropriate recommendations have been provided for researchers to use these suggested estimators to solve real-world issues.