Sample Size Of Dataset Research Articles

Exploring K-Means Clustering Efficiency: Accuracy and Computational Time across Multiple Datasets

Exploring K-Means Clustering Efficiency: Accuracy and Computational Time across Multiple Datasets

Calibrating and Visualizing Some Bootstrap Confidence Regions

When the bootstrap sample size is moderate, bootstrap confidence regions tend to have undercoverage. Improving the coverage is known as calibrating the confidence region. Consider testing H0:θ=θ0 versus H1:θ≠θ0. We reject H0 only if θ0 is not contained in a large-sample 95% confidence region. If the confidence region has 3% undercoverage for the data set sample size, then the type I error is 8% instead of the nominal 5%. Hence, calibrating confidence regions is also useful for testing hypotheses. Several bootstrap confidence regions are also prediction regions for a future value of a bootstrap statistic. A new bootstrap confidence region uses a simple prediction region calibration technique to improve the coverage. The DD plot for visualizing prediction regions can also be used to visualize some bootstrap confidence regions.

TabDEG: Classifying differentially expressed genes from RNA-seq data based on feature extraction and deep learning framework.

Traditional differential expression genes (DEGs) identification models have limitations in small sample size datasets because they require meeting distribution assumptions, otherwise resulting high false positive/negative rates due to sample variation. In contrast, tabular data model based on deep learning (DL) frameworks do not need to consider the data distribution types and sample variation. However, applying DL to RNA-Seq data is still a challenge due to the lack of proper labeling and the small sample size compared to the number of genes. Data augmentation (DA) extracts data features using different methods and procedures, which can significantly increase complementary pseudo-values from limited data without significant additional cost. Based on this, we combine DA and DL framework-based tabular data model, propose a model TabDEG, to predict DEGs and their up-regulation/down-regulation directions from gene expression data obtained from the Cancer Genome Atlas database. Compared to five counterpart methods, TabDEG has high sensitivity and low misclassification rates. Experiment shows that TabDEG is robust and effective in enhancing data features to facilitate classification of high-dimensional small sample size datasets and validates that TabDEG-predicted DEGs are mapped to important gene ontology terms and pathways associated with cancer.

Harmonizing heterogeneous transcriptomics datasets for machine learning-based analysis to identify spaceflown murine liver-specific changes

NASA has employed high-throughput molecular assays to identify sub-cellular changes impacting human physiology during spaceflight. Machine learning (ML) methods hold the promise to improve our ability to identify important signals within highly dimensional molecular data. However, the inherent limitation of study subject numbers within a spaceflight mission minimizes the utility of ML approaches. To overcome the sample power limitations, data from multiple spaceflight missions must be aggregated while appropriately addressing intra- and inter-study variabilities. Here we describe an approach to log transform, scale and normalize data from six heterogeneous, mouse liver-derived transcriptomics datasets (ntotal = 137) which enabled ML-methods to classify spaceflown vs. ground control animals (AUC ≥ 0.87) while mitigating the variability from mission-of-origin. Concordance was found between liver-specific biological processes identified from harmonized ML-based analysis and study-by-study classical omics analysis. This work demonstrates the feasibility of applying ML methods on integrated, heterogeneous datasets of small sample size.

Sample Size Considerations for Fine-Tuning Large Language Models for Named Entity Recognition Tasks: Methodological Study.

Large language models (LLMs) have the potential to support promising new applications in health informatics. However, practical data on sample size considerations for fine-tuning LLMs to perform specific tasks in biomedical and health policy contexts are lacking. This study aims to evaluate sample size and sample selection techniques for fine-tuning LLMs to support improved named entity recognition (NER) for a custom data set of conflicts of interest disclosure statements. A random sample of 200 disclosure statements was prepared for annotation. All "PERSON" and "ORG" entities were identified by each of the 2 raters, and once appropriate agreement was established, the annotators independently annotated an additional 290 disclosure statements. From the 490 annotated documents, 2500 stratified random samples in different size ranges were drawn. The 2500 training set subsamples were used to fine-tune a selection of language models across 2 model architectures (Bidirectional Encoder Representations from Transformers [BERT] and Generative Pre-trained Transformer [GPT]) for improved NER, and multiple regression was used to assess the relationship between sample size (sentences), entity density (entities per sentence [EPS]), and trained model performance (F1-score). Additionally, single-predictor threshold regression models were used to evaluate the possibility of diminishing marginal returns from increased sample size or entity density. Fine-tuned models ranged in topline NER performance from F1-score=0.79 to F1-score=0.96 across architectures. Two-predictor multiple linear regression models were statistically significant with multiple R2 ranging from 0.6057 to 0.7896 (all P<.001). EPS and the number of sentences were significant predictors of F1-scores in all cases ( P<.001), except for the GPT-2_large model, where EPS was not a significant predictor (P=.184). Model thresholds indicate points of diminishing marginal return from increased training data set sample size measured by the number of sentences, with point estimates ranging from 439 sentences for RoBERTa_large to 527 sentences for GPT-2_large. Likewise, the threshold regression models indicate a diminishing marginal return for EPS with point estimates between 1.36 and 1.38. Relatively modest sample sizes can be used to fine-tune LLMs for NER tasks applied to biomedical text, and training data entity density should representatively approximate entity density in production data. Training data quality and a model architecture's intended use (text generation vs text processing or classification) may be as, or more, important as training data volume and model parameter size.

ADCP suspended sediment transport monitoring using acoustic particle radius

Monitoring suspended sediment concentration (SSC) can be challenging as direct sampling methods are labour intensive and indirect measurements based on optical or acoustic backscatter are sensitive to changes in particle properties. Regardless, using ADCP backscatter to predict SSC is promising because of the possibility to capture suspended sediment transport by combining with flow measurements. To reduce sensitivity of established backscatter-SSC relations to changing particle size, a methodology is proposed where acoustic particle radius is derived using multi-frequency backscatter measurements obtained with a Nortek Signature1000 ADCP equipped with a vertical beam echosounder. Considering acoustic particle radii in an adapted backscatter-SSC model shows promising improvement in correlations with water sample reference measurements compared to the traditional single frequency approach based on a field test. Follow-up assessment is required to overcome limitations in dataset sample size and to investigate further improvements of the method. Still, application of the method can significantly enhance capability of ADCPs predicting SSC and – since backscatter is recorded over depth in conjunction with flow measurements – the ability to monitor suspended sediment transport using a single instrument.

Sparse L0-norm least squares support vector machine with feature selection

Least squares support vector machine (LSSVM) is a powerful classification tool based on hyperplanes. But the classical LSSVM does not perform well on small sample size data sets (SSS) because it lacks feature selection capability. To address this issue, we introduce an L0-norm regularization term into the objective function of LSSVM, and propose a new sparse LSSVM model called L0-LSSVM. However, the introduction of the L0-norm makes the resulting optimization problem nonconvex and nonsmooth. To overcome these challenges, instead of using the traditional convex or non-convex approximation of the L0-norm, we design an efficient algorithm for L0-LSSVM based on alternating direction method of multipliers (ADMM) where the L0-norm is handled by employing the proximity operator. Numerical results demonstrate that L0-LSSVM not only exhibits excellent generalization performance and feature selection capability but also offers a substantial improvement in computational speed.

ADscreen: A speech processing-based screening system for automatic identification of patients with Alzheimer's disease and related dementia

Alzheimer's disease and related dementias (ADRD) present a looming public health crisis, affecting roughly 5 million people and 11 % of older adults in the United States. Despite nationwide efforts for timely diagnosis of patients with ADRD, >50 % of them are not diagnosed and unaware of their disease. To address this challenge, we developed ADscreen, an innovative speech-processing based ADRD screening algorithm for the protective identification of patients with ADRD. ADscreen consists of five major components: (i) noise reduction for reducing background noises from the audio-recorded patient speech, (ii) modeling the patient's ability in phonetic motor planning using acoustic parameters of the patient's voice, (iii) modeling the patient's ability in semantic and syntactic levels of language organization using linguistic parameters of the patient speech, (iv) extracting vocal and semantic psycholinguistic cues from the patient speech, and (v) building and evaluating the screening algorithm. To identify important speech parameters (features) associated with ADRD, we used the Joint Mutual Information Maximization (JMIM), an effective feature selection method for high dimensional, small sample size datasets. Modeling the relationship between speech parameters and the outcome variable (presence/absence of ADRD) was conducted using three different machine learning (ML) architectures with the capability of joining informative acoustic and linguistic with contextual word embedding vectors obtained from the DistilBERT (Bidirectional Encoder Representations from Transformers). We evaluated the performance of the ADscreen on an audio-recorded patients' speech (verbal description) for the Cookie-Theft picture description task, which is publicly available in the dementia databank. The joint fusion of acoustic and linguistic parameters with contextual word embedding vectors of DistilBERT achieved F1-score = 84.64 (standard deviation [std] = ±3.58) and AUC-ROC = 92.53 (std = ±3.34) for training dataset, and F1-score = 89.55 and AUC-ROC = 93.89 for the test dataset. In summary, ADscreen has a strong potential to be integrated with clinical workflow to address the need for an ADRD screening tool so that patients with cognitive impairment can receive appropriate and timely care.

Research on Fire Detection in Laboratories Based on CNN and Transfer Learning

To solve the problem that model may not be convergent owing to datasets with small sample sizes results from the secrecy of labs, a fire detection method for lab fire detection based on CNN (Convolutional Neural Network) and transfer leaning is proposed. Experiments of transfer learning were conducted with configuration of super parameters and a small lab fire dataset, whose pre-trained models are GoogLeNet, ResNet and MobileNet trained with ImageNet dataset. We analyze and assess the loss function curve and the performances of the above models. Result shows that MoblieNet model is chosen to be the optimum model for fire detection, providing a reference to lab fire detection with small sample size datasets. Different from artificial fire surveillance, the proposed method based on CNN and transfer learning realize real-time, autonomy and efficient fire detection.

Selection and validation of novel stable reference genes for qPCR analysis in EMT and MET

Quantitative real-time polymerase chain reaction is a powerful tool for quantifying gene expression. The relative quantification relies on normalizing the data to reference genes or internal controls not modulated by the experimental conditions. The most widely used internal controls occasionally show changed expression patterns in different experimental settings, such as the mesenchymal to epithelial transition. Thus, identifying appropriate internal controls is of utmost importance. We analyzed multiple RNA-Seq datasets using a combination of statistical approaches such as percent relative range and coefficient of variance to define a list of candidate internal control genes, which was then validated experimentally and by using in silico analyses as well. We identified a group of genes as strong internal control candidates with high stability compared to the classical ones. We also presented evidence for the superiority of the percent relative range method for calculating expression stability in data sets with larger sample sizes. We used multiple methods to analyze data collected from several RNA-Seq datasets; we identified Rbm17 and Katna1 as the most stable reference genes in EMT/MET studies. The percent relative range approach surpasses other methods when analyzing datasets of larger sample sizes.

Gene filtering strategies for machine learning guided biomarker discovery using neonatal sepsis RNA-seq data.

Machine learning (ML) algorithms are powerful tools that are increasingly being used for sepsis biomarker discovery in RNA-Seq data. RNA-Seq datasets contain multiple sources and types of noise (operator, technical and non-systematic) that may bias ML classification. Normalisation and independent gene filtering approaches described in RNA-Seq workflows account for some of this variability and are typically only targeted at differential expression analysis rather than ML applications. Pre-processing normalisation steps significantly reduce the number of variables in the data and thereby increase the power of statistical testing, but can potentially discard valuable and insightful classification features. A systematic assessment of applying transcript level filtering on the robustness and stability of ML based RNA-seq classification remains to be fully explored. In this report we examine the impact of filtering out low count transcripts and those with influential outliers read counts on downstream ML analysis for sepsis biomarker discovery using elastic net regularised logistic regression, L1-reguarlised support vector machines and random forests. We demonstrate that applying a systematic objective strategy for removal of uninformative and potentially biasing biomarkers representing up to 60% of transcripts in different sample size datasets, including two illustrative neonatal sepsis cohorts, leads to substantial improvements in classification performance, higher stability of the resulting gene signatures, and better agreement with previously reported sepsis biomarkers. We also demonstrate that the performance uplift from gene filtering depends on the ML classifier chosen, with L1-regularlised support vector machines showing the greatest performance improvements with our experimental data.

Predicting Human Bioavailability of Subcutaneously Administered Fusion Proteins and Monoclonal Antibodies Using Human Intravenous Clearance or Antibody Isoelectric Point

There has been an increasing trend towards subcutaneous (SC) delivery of fusion proteins and monoclonal antibodies (mAbs) in recent years versus intravenous (IV) administration. The prediction of bioavailability is one of the major barriers in clinical translation of SC-administered therapeutic proteins due to a lack of reliable in vitro and preclinical in vivo predictive models. In this study, we explored the relationships between human SC bioavailability and physicochemical or pharmacokinetic properties of 19 Fc- or albumin-fusion proteins and 98 monoclonal antibodies. An inverse linear correlation was observed between human SC bioavailability and intravenous clearance (CL) or isoelectric point (pI). Multivariate regression models were developed using intravenous CL and pI of a training set (N = 59) as independent variables. The predictive models of mAbs were validated with an independent test set (N = 33). Two linear regression models resulted in 24 (73%) and 27 (82%) among 33 predictions within 0.8- to 1.2-fold deviations. Due to the small sample size of dataset, regression model validation was not conducted for fusion proteins. Overall, this study demonstrated that CL- and pI-based multivariate regression models could be used to predict human SC bioavailability of mAbs.

Machine learning models trained on synthetic datasets of multiple sample sizes for the use of predicting blood pressure from clinical data in a national dataset.

The potential for synthetic data to act as a replacement for real data in research has attracted attention in recent months due to the prospect of increasing access to data and overcoming data privacy concerns when sharing data. The field of generative artificial intelligence and synthetic data is still early in its development, with a research gap evidencing that synthetic data can adequately be used to train algorithms that can be used on real data. This study compares the performance of a series machine learning models trained on real data and synthetic data, based on the National Diet and Nutrition Survey (NDNS). Features identified to be potentially of relevance by directed acyclic graphs were isolated from the NDNS dataset and used to construct synthetic datasets and impute missing data. Recursive feature elimination identified only four variables needed to predict mean arterial blood pressure: age, sex, weight and height. Bayesian generalised linear regression, random forest and neural network models were constructed based on these four variables to predict blood pressure. Models were trained on the real data training set (n = 2408), a synthetic data training set (n = 2408) and larger synthetic data training set (n = 4816) and a combination of the real and synthetic data training set (n = 4816). The same test set (n = 424) was used for each model. Synthetic datasets demonstrated a high degree of fidelity with the real dataset. There was no significant difference between the performance of models trained on real, synthetic or combined datasets. Mean average error across all models and all training data ranged from 8.12 To 8.33. This indicates that synthetic data was capable of training equally accurate machine learning models as real data. Further research is needed on a variety of datasets to confirm the utility of synthetic data to replace the use of potentially identifiable patient data. There is also further urgent research needed into evidencing that synthetic data can truly protect patient privacy against adversarial attempts to re-identify real individuals from the synthetic dataset.

Real-time classification for Φ-OTDR vibration events in the case of small sample size datasets

Efficient classification of vibration signals detected by phase-sensitive optical time domain reflectometer (Φ-OTDR) based on small samples is an effective method to reduce the false alarm rate without GPU or large data sets. This paper proposes a fiber optic system vibration event recognition method based on a combination of image segmentation pre-processing, texture, statistical, morphological feature extraction, and weighted support vector machine (WSVM), which can effectively classify-five types of vibration events in high-speed railway perimeter intrusion detection with small sample data and no parallel processing units. Erosion and dilation operations are applied to vibration signal image feature enhancement in image pre-processing. The vibration signal region and background are separated by the maximum inter-class variance method, then 35 features of the vibration signal region are calculated and finally employed to construct a WSVM. Experiments show that the method achieves 99 FPS and 98.8% accuracy on the test set with 330 vibration images as the training set to build the model without GPU and in the presence of interference signals. It provides a generalized Φ-OTDR vibration event recognition method for small samples.

Data-driven performance analysis of a residential building applying artificial neural network (ANN) and multi-objective genetic algorithm (GA)

Residential buildings account for nearly 60% of electrical energy consumption in hot climates as in Kuwait. As a result, this study proposes a reliable multi-objective model to obtain the optimal design for a typical residential Kuwaiti building by integrating an Artificial Neural Network (ANN) model with Genetic Algorithm (GA) optimization method. The ANN model was investigated and verified using the results of building performance simulations applying EnergyPlus software. The effects of sample size of dataset on performance of ANN were evaluated. The final optimal building design was optimized using the GA method after ensuring the convergence of the final ANN model. Several design and operation parameters were considered as decision variables, while cooling energy consumption, discomfort hours, and equivalent carbon emissions were selected as objective functions. In addition, sensitivity analysis was conducted to evaluate the impacts of decision variables on objective functions. The sensitivity results indicated that insulation highly affect energy consumption and carbon emission, while cooling setpoint played a key role in discomfort hours. Furthermore, Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method was applied for decision-making among Pareto optimal solutions. The results showed that the optimal solution suggested by TOPSIS methods using ANN-based model provided a substantial reduction in energy consumption, discomfort hours, and carbon emission up to 39.3%, 62.8%, and 40.5% compared with the base case, respectively. It is recommended that further research be undertaken in considering uncertainty parameters on optimization process and applying the developed framework for building in different climate.

A unified model for the sparse optimal scoring problem

Optimal scoring (OS), an equivalent form of linear discriminant analysis (LDA), is an important supervised learning method and dimensionality reduction tool. However, it is still a challenge for the classical OS on small sample size (SSS) datasets. In this paper, to find sparse discriminant vectors, we propose a unified model for sparse optimal scoring (SOS) by virtue of the generalized ℓq-norm (0≤q≤1). To overcome the difficulty in treating the generalized ℓq-norm, we propose an efficient alternative direction method of multipliers (ADMM), where proximity operator of ℓq-norm is employed for different q values. Meanwhile, the convergence results of our method are also established. Numerical experiments on artificial and benchmark datasets demonstrate the effectiveness and feasibility of our proposed method.

Interpreting models interpreting brain dynamics

Brain dynamics are highly complex and yet hold the key to understanding brain function and dysfunction. The dynamics captured by resting-state functional magnetic resonance imaging data are noisy, high-dimensional, and not readily interpretable. The typical approach of reducing this data to low-dimensional features and focusing on the most predictive features comes with strong assumptions and can miss essential aspects of the underlying dynamics. In contrast, introspection of discriminatively trained deep learning models may uncover disorder-relevant elements of the signal at the level of individual time points and spatial locations. Yet, the difficulty of reliable training on high-dimensional low sample size datasets and the unclear relevance of the resulting predictive markers prevent the widespread use of deep learning in functional neuroimaging. In this work, we introduce a deep learning framework to learn from high-dimensional dynamical data while maintaining stable, ecologically valid interpretations. Results successfully demonstrate that the proposed framework enables learning the dynamics of resting-state fMRI directly from small data and capturing compact, stable interpretations of features predictive of function and dysfunction.

Fault Classification of Rolling Element Bearing in Machine Learning Domain

Rolling element bearings are crucial components of rotating machinery used in various industries, including aerospace, navigation, machine tools, etc. Therefore, it is essential to establish suitable techniques for condition monitoring and fault diagnosis of bearings to avoid breakdowns and damages during operation for overall industrial sustainability. Vibration-based condition monitoring has been the most employed technique in this perspective. Many researchers have investigated the vibration response of rolling element bearings having inner race defects, outer race defects, or rolling element defects using conventional techniques in past decades. However, Machine Learning (ML) has emerged as another way of bearing fault diagnosis. In this work, fault signatures of ball bearings are classified using a total of 6 (with 24 subcategories) ML models, and a comparative performance of these models is presented. The ML classifiers are trained with extracted time-domain and frequency-domain features using open-source Case Western Reserve University (CWRU) bearing data. Two datasets of different sample size and number of samples of vibration data corresponding to a healthy ball bearing, a defective bearing with inner race defect, a ball defect, and an outer race defect, running at a particular set of working conditions, are considered. The accuracy of ML models is compared to identify the best model for classifying the faults under consideration.

Investigating Performance of Composite Quantile Regression with and without Penalization

The performance of composite quantile regression with and without penalization was investigated and results compared to Lasso Quantile Regression (LQR), Lasso Regression (LR) and Ridge Regression (RR). The mean square error, Akiake information criteria and mean absolute percentage error were used as the comparative criteria. The comparison was illustrated using real dataset and simulated data sets of sample sizes n=30,100,300,500 and 1000. Five consecutive quantiles; 0.19, 0.39, 0.59, 0.79 and 0.99 were used for the quantile regression methods while the lasso regression and ridge regression were based on the mean effect. Another set of quantiles; 0.25, 0.5, 0.75 and 0.95 were also accessed based on only CQR and CQR_AL. The results shows that the composite quantile regression without penalization (CQR) and composite quantile regression with penalization (CQR_AL) achieved same results with the lowest variance on estimated effects and was best fitted model for very large data sets (n=100, 300, 500 &amp; 1000). For the real life data with sample size of 318, the CQR_AL showed the least MSE=421.7653 and AIC=1930.136, but its prediction accuracy was low at 19.03%. Considering sample size 30, the LQR at the 0.59th quantile had the lowest variance with MSE of 1.3940 and was best fitted with AIC of 340.20, also it was noticed that COR_AL achieved the next lowest value for MSE and AIC. These results led to the conclusion that CQR_AL and CQR can be used alternatively with large data sets. But when the sample size is small the LQR is most suited, but if a combined quantile effect is sort the CQR_AL should be opted for.

A machine learning approach to small area estimation: predicting the health, housing and well-being of the population of Netherlands

BackgroundLocal policymakers require information about public health, housing and well-being at small geographical areas. A municipality can for example use this information to organize targeted activities with the aim of improving the well-being of their residents. Surveys are often used to gather data, but many neighborhoods can have only few or even zero respondents. In that case, estimating the status of the local population directly from survey responses is prone to be unreliable.MethodsSmall Area Estimation (SAE) is a technique to provide estimates at small geographical levels with only few or even zero respondents. In classical individual-level SAE, a complex statistical regression model is fitted to the survey responses by using auxiliary administrative data for the population as predictors, the missing responses are then predicted and aggregated to the desired geographical level. In this paper we compare gradient boosted trees (XGBoost), a well-known machine learning technique, to a structured additive regression model (STAR) designed for the specific problem of estimating public health and well-being in the whole population of the Netherlands.ResultsWe compare the accuracy and performance of these models using out-of-sample predictions with five-fold Cross Validation (5CV). We do this for three data sets of different sample sizes and outcome types. Compared to the STAR model, gradient boosted trees are able to improve both the accuracy of the predictions and the total time taken to get these predictions. Even though the models appear quite similar in overall accuracy, the small area predictions at neighborhood level sometimes differ significantly. It may therefore make sense to pursue slightly more accurate models for better predictions into small areas. However, one of the biggest benefits is that XGBoost does not require prior knowledge or model specification. Data preparation and modelling is much easier, since the method automatically handles missing data, non-linear responses, interactions and accounts for spatial correlation structures.ConclusionsIn this paper we provide new nationwide estimates of health, housing and well-being indicators at neighborhood level in the Netherlands, see ’Online materials’. We demonstrate that machine learning provides a good alternative to complex statistical regression modelling for small area estimation in terms of accuracy, robustness, speed and data preparation. These results can be used to make appropriate policy decisions at a local level and make recommendations about which estimation methods are beneficial in terms of accuracy, time and budget constraints.