Increase In Training Set Size Research Articles

Homogenizing SOHO/EIT and SDO/AIA 171 Å Images: A Deep-learning Approach

Extreme-ultraviolet (EUV) images of the Sun are becoming an integral part of space weather prediction tasks. However, having different surveys requires the development of instrument-specific prediction algorithms. As an alternative, it is possible to combine multiple surveys to create a homogeneous data set. In this study, we utilize the temporal overlap of Solar and Heliospheric Observatory Extreme ultraviolet Imaging Telescope and Solar Dynamics Observatory Atmospheric Imaging Assembly 171 Å surveys to train an ensemble of deep-learning models for creating a single homogeneous survey of EUV images for two solar cycles. Prior applications of deep learning have focused on validating the homogeneity of the output while overlooking the systematic estimation of uncertainty. We use an approach called “approximate Bayesian ensembling” to generate an ensemble of models whose uncertainty mimics that of a fully Bayesian neural network at a fraction of the cost. We find that ensemble uncertainty goes down as the training set size increases. Additionally, we show that the model ensemble adds immense value to the prediction by showing higher uncertainty in test data that are not well represented in the training data.

Effects of marker density and training set size on the genomic selection accuracy for predicting sheathblight resistance in rice (Oryza sativa L.)

The success of genomic selection mainly depends on the extent of linkage disequilibrium between markers and quantitative trait loci, size of training set, heritability of the trait etc. The extent of linkage disequilibrium depends on the genetic structure of the population and marker density. This study was conducted to determine the effects of marker density and size of training set on prediction accuracy using 1545 recombinant inbred lines derived from eleven bi-parental rice populations. All RILs were genotyped with 6564 SNPs and screened in two hot spot locations to assess reaction against sheath blight.Bayesian B model was used to train the statistical model for calculation of marker effects and genomic estimated breeding values. To evaluate the genomic prediction accuracy, various levels of training set size (300, 500, 700, 900 and 1200 lines) and marker density (500, 800, 1100, 1400, 1700, 2000, 4000 and 6000 markers) were considered. In our study, the prediction accuracy increased with increase in training set size, however, average prediction accuracy of 0.717 was obtained for the training set comprising of 900 lines before reaching plateau with marginal increase in prediction accuracy with higher training set sizes. The predictive ability increased dramatically with more SNPs included in the analysis until 2000 markers with average prediction accuracy of 0.681, no significant improvement beyond this was observed. The results indicate that training set with approximately 900 lines and 2000 uniformly distributed SNP markers with good amount of polymorphism across populations would be enough to reach achievable accuracy to predict sheath blight resistance in rice.

Validation of mathematical models for the prediction of organs‐at‐risk dosimetric metrics in high‐dose‐rate gynecologic interstitial brachytherapy

Given the complicated nature of an interstitial gynecologic brachytherapy treatment plan, the use of a quantitative tool to evaluate the quality of the achieved metrics compared to clinical practice would be advantageous. For this purpose, predictive mathematical models to predict the D2cc of rectum and bladder in interstitial gynecologic brachytherapy are discussed and validated. Previous plans were used to establish the relationship between D2cc and the overlapping volume of the organ at risk with the targeted area (C0) or a 1-cm expansion of the target area (C1). Three mathematical models were evaluated: D2cc = α*C1 + β (LIN); D2cc = α - exp(-β*C0) (EXP); and a mixed approach (MIX), where both C0 and C1 were inputs of the model. The parameters of the models were optimized on a training set of patient data, and the predictive error of each model (predicted D2cc - real D2cc) was calculated on a validation set of patient data. The data of 20 patients were used to perform a K-fold cross validation analysis, with K = 2, 4, 6, 8, 10, and 20. MIX was associated with the smallest mean prediction error <6.4% for an 18-patient training set; LIN had an error <8.5%; EXP had an error <8.3%. Best case scenario analysis shows that an error ≤ 5% can be achieved for a ten-patient training set with MIX, an error ≤ 7.4% for LIN, and an error ≤ 6.9% for EXP. The error decreases with the increase in training set size, with the most marked decrease observed for MIX. The MIX model can predict the D2cc of the organs at risk with an error lower than 5% with a training set of ten patients or greater. The model can be used in the development of quality assurance tools to identify treatment plans with suboptimal sparing of the organs at risk. It can also be used to improve preplanning and in the development of real-time intraoperative planning tools.

SU‐E‐T‐340: Predictive Models for Dosimetry of the Organs‐At‐Risk in Interstitial Gynecologic Brachytherapy

Purpose: In this work, mathematical models for the prediction of the D2cc of rectum and bladder in interstitial gynecologic brachytherapy based on geometric information, without the generation of a treatment plan, are discussed and validated. Methods: Previous plans were used to establish the relationship between D2cc and the overlapping volume of the organ‐at‐risk with the targeted area (C0) or a 1‐cm expansion of the target area (C1). Three mathematical models were evaluated: D2cc = a*C1+b (LIN); D2cc = a — exp(‐b*C0) (EXP); and a mixed approach (MIX), where both C0 and C1 were inputs of the model. The model's parameters were optimized on a training set, and the model predictive error (predicted D2cc — real D2cc) was calculated on a validation set. The data of 20 patients treated with interstitial gynecologic brachytherapy were used to perform a K‐fold cross validation analysis, with K = 2, 4, 6,8, 10, and 20. Results: MIX was associated with the smallest mean prediction error with error &lt;6.4% for an 18‐patient training set; LIN had an error &lt;8.5%; EXP had an error &lt; 8.3%. Best‐case‐scenario analysis showed that an error &lt;5% could be achieved for a 10‐patient training set with MIX, error &lt;7.5% for LIN and error &lt;7.0% for EXP. The error decreases with the increase in training set size, with the most marked decrease observed for MIX. Conclusion: Ten training cases allow the MIX model to predict D2cc for the organs‐at‐risk with an error &lt;5%. While clearly specific to clinical practice, the model can be used in the development of quality assurance tools to identify treatment plans with sub‐optimal sparing of the organs‐at‐risk. It can also be used to improve preplanning and in the development of real‐time intra‐operative planning tools.

Domain-aware Evaluation of Named Entity Recognition Systems for Croatian

We provide an evaluation of the currently available named entity recognition systems for Croatian. The evaluation puts special emphasis on domain dependence. To this goal, we manually annotated a dataset of approximately 1 million tokens of Croatian text from various domains within the newspaper text genre. The dataset was annotated using a three-class named entity tagset – denoting personal names, locations and organizations. We give insight to feature selection, domain sensitivity and effects of increase in training set size for statistical named entity recognition using the state-of-the-art Stanford NER system. We also sketch a comparison of publicly available named entity recognition systems for Croatian considering domain dependence, regardless of their underlying paradigms. Our top-performing system achieved an F1-score of 0.884 in a mixed-domain testing scenario, scoring 0.925 and 0.843 in the two domains separated for the experiment. The system shows consistency in state-of-the-art scores for detecting names of persons, locations and organizations.

On the statistical assessment of classifiers using DNA microarray data.

BackgroundIn this paper we present a method for the statistical assessment of cancer predictors which make use of gene expression profiles. The methodology is applied to a new data set of microarray gene expression data collected in Casa Sollievo della Sofferenza Hospital, Foggia – Italy. The data set is made up of normal (22) and tumor (25) specimens extracted from 25 patients affected by colon cancer. We propose to give answers to some questions which are relevant for the automatic diagnosis of cancer such as: Is the size of the available data set sufficient to build accurate classifiers? What is the statistical significance of the associated error rates? In what ways can accuracy be considered dependant on the adopted classification scheme? How many genes are correlated with the pathology and how many are sufficient for an accurate colon cancer classification? The method we propose answers these questions whilst avoiding the potential pitfalls hidden in the analysis and interpretation of microarray data.ResultsWe estimate the generalization error, evaluated through the Leave-K-Out Cross Validation error, for three different classification schemes by varying the number of training examples and the number of the genes used. The statistical significance of the error rate is measured by using a permutation test. We provide a statistical analysis in terms of the frequencies of the genes involved in the classification. Using the whole set of genes, we found that the Weighted Voting Algorithm (WVA) classifier learns the distinction between normal and tumor specimens with 25 training examples, providing e = 21% (p = 0.045) as an error rate. This remains constant even when the number of examples increases. Moreover, Regularized Least Squares (RLS) and Support Vector Machines (SVM) classifiers can learn with only 15 training examples, with an error rate of e = 19% (p = 0.035) and e = 18% (p = 0.037) respectively. Moreover, the error rate decreases as the training set size increases, reaching its best performances with 35 training examples. In this case, RLS and SVM have error rates of e = 14% (p = 0.027) and e = 11% (p = 0.019). Concerning the number of genes, we found about 6000 genes (p < 0.05) correlated with the pathology, resulting from the signal-to-noise statistic. Moreover the performances of RLS and SVM classifiers do not change when 74% of genes is used. They progressively reduce up to e = 16% (p < 0.05) when only 2 genes are employed. The biological relevance of a set of genes determined by our statistical analysis and the major roles they play in colorectal tumorigenesis is discussed.ConclusionsThe method proposed provides statistically significant answers to precise questions relevant for the diagnosis and prognosis of cancer. We found that, with as few as 15 examples, it is possible to train statistically significant classifiers for colon cancer diagnosis. As for the definition of the number of genes sufficient for a reliable classification of colon cancer, our results suggest that it depends on the accuracy required.

Boosting and Other Ensemble Methods

We compare the performance of three types of neural network-based ensemble techniques to that of a single neural network. The ensemble algorithms are two versions of boosting and committees of neural networks trained independently. For each of the four algorithms, we experimentally determine the test and training error curves in an optical character recognition (OCR) problem as both a function of training set size and computational cost using three architectures. We show that a single machine is best for small training set size while for large training set size some version of boosting is best. However, for a given computational cost, boosting is always best. Furthermore, we show a surprising result for the original boosting algorithm: namely, that as the training set size increases, the training error decreases until it asymptotes to the test error rate. This has potential implications in the search for better training algorithms.

Visual field analysis using artificial neural networks.

There have been several reports on the application of artificial neural networks (ANNs) to visual field classification. While these have demonstrated that neural networks can be used with good results they have not explored the effects that the training set can have upon network performance nor emphasized the unique value of ANNs in visual field analysis. This paper considers the problem of differentiating normal and glaucomatous visual fields and explores different training set characteristics using field data collected from a Henson CFS2000 perimeter. Training set properties including size, balance between normals and glaucomas, extent of field loss and the spatial location of glaucomatous defects are explored. A multilayer network with 132 input nodes, 20 hidden layer nodes and 2 output nodes in trained using an error backpropagation algorithm. Both sensitivity and specificity are measured during testing. The results demonstrate that large random sets are better than small random sets since sensitivity improves with size and specificity is not adversely affected. The variability in performance also reduces as training set size increases. In addition, sets that are biased towards glaucoma examples are more sensitive and less specific, while sets biased with normal examples are more specific and less sensitive than balanced sets. Thus large training sets with class balance are generally desirable for good sensitivities and specificities. The actual glaucoma examples contained in the set are also important. A training set deficient in examples has no detrimental effect on sensitivity or specificity. The spatial distribution of defects is also crucial. Spatially biased sets are unable to recognize defects that occur in locations where no previous defect has been presented while more balanced sets lead to better performance. In conclusion the 'ideal' training set should contain many examples of early defects that represent the full range of locations where these defects may occur.