Single Training Set Research Articles

Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization

Deep learning applications for assessing medical images are limited because the datasets are often small and imbalanced. The use of synthetic data has been proposed in the literature, but neither a robust comparison of the different methods nor generalizability has been reported. Our approach integrates a retinal image quality assessment model and StyleGAN2 architecture to enhance Age-related Macular Degeneration (AMD) detection capabilities and improve generalizability. This work compares ten different Generative Adversarial Network (GAN) architectures to generate synthetic eye-fundus images with and without AMD. We combined subsets of three public databases (iChallenge-AMD, ODIR-2019, and RIADD) to form a single training and test set. We employed the STARE dataset for external validation, ensuring a comprehensive assessment of the proposed approach. The results show that StyleGAN2 reached the lowest Fréchet Inception Distance (166.17), and clinicians could not accurately differentiate between real and synthetic images. ResNet-18 architecture obtained the best performance with 85% accuracy and outperformed the two human experts (80% and 75%) in detecting AMD fundus images. The accuracy rates were 82.8% for the test set and 81.3% for the STARE dataset, demonstrating the model’s generalizability. The proposed methodology for synthetic medical image generation has been validated for robustness and accuracy, with free access to its code for further research and development in this field.

Improving generalization capability of deep learning-based nuclei instance segmentation by non-deterministic train time and deterministic test time stain normalization

With the advent of digital pathology and microscopic systems that can scan and save whole slide histological images automatically, there is a growing trend to use computerized methods to analyze acquired images. Among different histopathological image analysis tasks, nuclei instance segmentation plays a fundamental role in a wide range of clinical and research applications. While many semi- and fully-automatic computerized methods have been proposed for nuclei instance segmentation, deep learning (DL)-based approaches have been shown to deliver the best performances. However, the performance of such approaches usually degrades when tested on unseen datasets.In this work, we propose a novel method to improve the generalization capability of a DL-based automatic segmentation approach. Besides utilizing one of the state-of-the-art DL-based models as a baseline, our method incorporates non-deterministic train time and deterministic test time stain normalization, and ensembling to boost the segmentation performance. We trained the model with one single training set and evaluated its segmentation performance on seven test datasets. Our results show that the proposed method provides up to 4.9%, 5.4%, and 5.9% better average performance in segmenting nuclei based on Dice score, aggregated Jaccard index, and panoptic quality score, respectively, compared to the baseline segmentation model.

Abstract 5391: Machine learning of cancer type and tissue of origin from proteomes of 1,277 human tissue samples and 975 cancer cell lines

Abstract Introduction Cancer type is determined through tumor morphology, aided by immunohistochemical staining. The development of machine learning (ML) models using histology slides has powered the image-based prediction of the site of origin in cancer of unknown primary (CUP). Here, we used ML on proteomic data to predict cancer types and tissue of origin from a sample cohort consisting of 1,277 human tissue samples spanning 44 cancer types. The training proteome datasets included two independent sets of proteomes acquired from a pan-cancer cell line collection and a subset of the tissue cohort for online ML. Methods All samples were processed using data-independent acquisition mass spectrometry (DIA-MS). Two proteomic profiles from the pan-cancer cell line cohort were generated using two independent sample preparation methods. These were normalized by Combat and merged by averaging the protein abundance, yielding a single training set (D1) with 975 cell lines and 9,688 proteins. Similary, 1,277 tissue samples were processed by DIA-MS, quantifying 9,501 proteins. Celligner was used to align the cell lines (D1) with the tissue cohort. Half of the tissue proteomes were used as a second training set (D2) for online ML and a hold-out test set was constructed by taking the other half of the tissue cohort (T1). A multinomial logistic regression was used to predict cancer and tissue types. Top-k accuracy, as the evaluation metric, computes how often the correct cancer and tissue type class is among the top k classes predicted. Results As a proof of concept, we defined six cancer types (adenocarcinoma, sarcoma, squamous carcinoma, lymphoma, melanoma and small cell carcinoma) and seven adenocarcinoma tissues of origin (breast, colorectal, liver, lung, ovary, stomach/esophagus and pancreas) for an ML experiment. We learned a classifier using the cell lines (D1) as the baseline training set, and consecutively added 10% of D2 to D1 for online ML. We tested the baseline model and each subsequent new model on the test set T1. We observed a monotonic performance increase from 0.89 (baseline; Top-1 accuracy) to 0.97 (all D2 were used) when predicting the six cancer types. We observed an analogous trend when predicting the seven tissue types (from 0.64 to 0.84). These results suggest that cancer cell lines can be used to predict cancer type and adenocarcinoma tissue of origin. Conclusion Our proteomic-based ML model can predict cancer type and adenocarcinoma tissue of origin in concordance with existing histopathological classification. It can also assign multiple probabilities to tumor type and tissue of origin, potentially enabling the classification of CUP in future work. By adding tissue samples stepwise to the existing model, its predictive performance can be further enhanced. This reflects a real-world knowledgebase that will continue to increase in predictive power as additional data are added. Citation Format: Zhaoxiang Cai, Zainab Noor, Adel T. Aref, Emma L. Boys, Dylan Xavier, Natasha Lucas, Steven G. Williams, Jennifer M. Koh, Rebecca C. Poulos, Peter G. Hains, Phillip J. Robinson, Rosemary Balleine, Roger R. Reddel, Qing Zhong. Machine learning of cancer type and tissue of origin from proteomes of 1,277 human tissue samples and 975 cancer cell lines. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5391.

Instance Selection-Based Surrogate-Assisted Genetic Programming for Feature Learning in Image Classification.

Genetic programming (GP) has been applied to feature learning for image classification and achieved promising results. However, many GP-based feature learning algorithms are computationally expensive due to a large number of expensive fitness evaluations, especially when using a large number of training instances/images. Instance selection aims to select a small subset of training instances, which can reduce the computational cost. Surrogate-assisted evolutionary algorithms often replace expensive fitness evaluations by building surrogate models. This article proposes an instance selection-based surrogate-assisted GP for fast feature learning in image classification. The instance selection method selects multiple small subsets of images from the original training set to form surrogate training sets of different sizes. The proposed approach gradually uses these surrogate training sets to reduce the overall computational cost using a static or dynamic strategy. At each generation, the proposed approach evaluates the entire population on the small surrogate training sets and only evaluates ten current best individuals on the entire training set. The features learned by the proposed approach are fed into linear support vector machines for classification. Extensive experiments show that the proposed approach can not only significantly reduce the computational cost but also improve the generalisation performance over the baseline method, which uses the entire training set for fitness evaluations, on 11 different image datasets. The comparisons with other state-of-the-art GP and non-GP methods further demonstrate the effectiveness of the proposed approach. Further analysis shows that using multiple surrogate training sets in the proposed approach achieves better performance than using a single surrogate training set and using a random instance selection method.

Technical note: Evaluation of a V-Net autosegmentation algorithm for pediatric CT scans: Performance, generalizability, and application to patient-specific CT dosimetry.

This study developed and evaluated a fully convolutional network (FCN) for pediatric CT organ segmentation and investigated the generalizability of the FCN across image heterogeneities such as CT scanner model protocols and patient age. We also evaluated the autosegmentation models as part of a software tool for patient-specific CT dose estimation. A collection of 359 pediatric CT datasets with expert organ contours were used for model development and evaluation. Autosegmentation models were trained for each organ using a modified FCN 3D V-Net. An independent test set of 60 patients was withheld for testing. To evaluate the impact of CT scanner model protocol and patient age heterogeneities, separate models were trained using a subset of scanner model protocols and pediatric age groups. Train and test sets were split to answer questions about the generalizability of pediatric FCN autosegmentation models to unseen age groups and scanner model protocols, as well as the merit of scanner model protocol or age-group-specific models. Finally, the organ contours resulting from the autosegmentation models were applied to patient-specific dose maps to evaluate the impact of segmentation errors on organ doseestimation. Results demonstrate that the autosegmentation models generalize to CT scanner acquisition and reconstruction methods which were not present in the training dataset. While models are not equally generalizable across age groups, age-group-specific models do not hold any advantage over combining heterogeneous age groups into a single training set. Dice similarity coefficient (DSC) and mean surface distance results are presented for 19 organ structures, for example, median DSC of 0.52 (duodenum), 0.74 (pancreas), 0.92 (stomach), and 0.96 (heart). The FCN models achieve a mean dose error within 5% of expert segmentations for all 19 organs except for the spinal canal, where the mean error was 6.31%. Overall, these results are promising for the adoption of FCN autosegmentation models for pediatric CT, including applications for patient-specific CT doseestimation.

Integrating Multiple Datasets and Machine Learning Algorithms for Satellite-Based Bathymetry in Seaports

Water depth estimation in seaports is essential for effective port management. This paper presents an empirical approach for water depth determination from satellite imagery through the integration of multiple datasets and machine learning algorithms. The implementation details of the proposed approach are provided and compared against different existing machine learning algorithms with a single training set. For a single training set and a single machine learning method, our analysis shows that the proposed depth estimation method provides a better root-mean-square error (RMSE) and a higher coefficient of determination (R2) under turbid water conditions, with overall RMSE and R2 improvements of 1 cm and 0.7, respectively. The developed method may be employed in monitoring dredging activities, especially in areas with polluted water, mud and/or a high sediment content.

WITHDRAWN: Contactless attendance system using Siamese neural network based face recognition

Abstract One of the most important and fast growing biometrical technological advancement is face recognition, used for identification of the person without any human intervention. In the proposed work, one shot learning has been used on Siamese convolution neural network (SCNN) for face recognition and attendance of the employee is entered in the MySQL database. Application Programmable interface used for uploading database is APACHE Web Server. The advantage of one shot learning algorithm is that it requires single training set for similarity measure. Get well soon message is send to absent employees through Gmail using send mail transfer protocol. Twelve images have been successfully tested and the system observed state-of –art performance with all face recognized correctly.

Boosting Deep Open World Recognition by Clustering

While convolutional neural networks have brought significant advances in robot vision, their ability is often limited to closed world scenarios, where the number of semantic concepts to be recognized is determined by the available training set. Since it is practically impossible to capture all possible semantic concepts present in the real world in a single training set, we need to break the closed world assumption, equipping our robot with the capability to act in an open world. To provide such ability, a robot vision system should be able to (i) identify whether an instance does not belong to the set of known categories (i.e. open set recognition), and (ii) extend its knowledge to learn new classes over time (i.e. incremental learning). In this work, we show how we can boost the performance of deep open world recognition algorithms by means of a new loss formulation enforcing a global to local clustering of class-specific features. In particular, a first loss term, i.e. global clustering, forces the network to map samples closer to the class centroid they belong to while the second one, local clustering, shapes the representation space in such a way that samples of the same class get closer in the representation space while pushing away neighbours belonging to other classes. Moreover, we propose a strategy to learn class-specific rejection thresholds, instead of heuristically estimating a single global threshold, as in previous works. Experiments on RGB-D Object and Core50 datasets show the effectiveness of our approach.

Repeated holdout validation for weighted quantile sum regression

Weighted Quantile Sum (WQS) regression is a method commonly used in environmental epidemiology to assess the impact of chemical mixtures in relation to a health outcome of interest. Data are partitioned into a single training and test set to reduce sample-specific chemical weights. However, in typical epidemiology sample sizes, this may produce unstable chemical weights and WQS index estimates, and investigators may resort to training and testing on the same data. To solve this problem, we propose repeated holdout validation whereby data are randomly partitioned 100 times, producing a distribution of validated results. Taking the mean as the final estimate, confidence estimates may also be calculated for inference. Further, this method helps characterize the variability in chemical weights, aiding in the identification of chemicals of concern. This is important since it may direct future research into specific chemicals.Using data from 718 mother-child pairs in the Swedish Environmental Longitudinal, Mother and Child, Asthma and Allergy (SELMA) study, we assessed the association between prenatal exposure to 26 endocrine disrupting chemicals and child Intelligence Quotient (IQ). Results using a single partition were unstable, varying by random seed. The WQS index estimate was significant when all data was used (e.g. no partition) (β = −2.2 CI = −3.43, −0.98), but attenuated and nonsignificant using repeated holdout validation (β = −0.82 CI = −2.11, 0.45). When implementing WQS in epidemiologic studies with limited sample sizes, repeated holdout validation is a viable alternative to using a single, or no partitioning. Repeated holdout can both stabilize results and help characterize the uncertainty in identifying chemicals of concern, while maintaining some of the the rigor of holdout validation.•Repeated holdout validation improves the stability of WQS estimates in finite study samples•Uncertainty in identifying toxic chemicals of concern is acknowledged and characterized

Multi-domain adversarial training of neural network acoustic models for distant speech recognition

Building deep neural network acoustic models directly based on far-field speech from multiple recording environments with different acoustic properties is an increasingly popular approach to address the problem of distant speech recognition. The currently common approach to building such multi-condition (multi-domain) models is to compile available data from all different environments into a single train set, discarding information regarding the specific environment to which each utterance belongs. We propose a novel strategy for training neural network acoustic models based on adversarial training which makes use of environment labels during training. By adjusting the parameters of the initial layers of the network adversarially with respect to a domain classifier trained to recognize the recording environments, we enforce better invariance to the diversity of recording conditions. We provide a motivating study on the mechanism by which a deep network learns environmental invariance, and discuss some relations with existing approaches for improving the robustness of DNN models. The proposed multi-domain adversarial training is evaluated on an end-to-end speech recognition task based on the AMI meeting corpus, achieving a relative character error rate reduction of +3.3% with respect to a conventional multi-condition trained baseline and +25.4% with respect to a clean-trained baseline.

Gene expression based cancer classification

Cancer classification based on molecular level investigation has gained the interest of researches as it provides a systematic, accurate and objective diagnosis for different cancer types. Several recent researches have been studying the problem of cancer classification using data mining methods, machine learning algorithms and statistical methods to reach an efficient analysis for gene expression profiles.Studying the characteristics of thousands of genes simultaneously offered a deep insight into cancer classification problem. It introduced an abundant amount of data ready to be explored. It has also been applied in a wide range of applications such as drug discovery, cancer prediction and diagnosis which is a very important issue for cancer treatment. Besides, it helps in understanding the function of genes and the interaction between genes in normal and abnormal conditions. That is done by monitoring the behavior of genes -gene expression data- under different conditions.In this paper, an effective ensemble approach is proposed. Ensemble classifiers increase not only the performance of the classification, but also the confidence of the results. The motivations beyond using ensemble classifiers are that the results are less dependent on peculiarities of a single training set and because the ensemble system outperforms the performance of the best base classifier in the ensemble.

A Novel Eigenface based Species Recognition System

To develop a species recognition system for a resettable trap using novel species identification techniques. Classical Eigenface based identification techniques are widely used in human faced detection domain. In this research Eigen faced based technique is used to identify the feral animals like Possums, cat and Weasels. When traditional Eigen faced technique is applied to detect these animal, detection rate is extremely poor (Possums 55%, Cats 33% and weasels 45%), due to their orientation of the heads and fur patterns. In this research, Eigenface based image recognition technique's detection rate was improved by adding different training sets to the system. Traditional Eigenface detection domain one training set is used, but it was discovered single training set was not adequate to detect small animal. This is because smaller animals like possums, cats and weasels tend to have different color group, different texture and hard to obtain face up images. Therefore it was decided to divide the training set into different sub groups. This sub training sets are used to train system and search for match. This method improved the detection rate up to 83% for possum, 50% for cats and 63% weasels. General Terms Euclidean distance, face recognition, animal detection

A human platelet calcium calculator trained by pairwise agonist scanning.

Since platelet intracellular calcium mobilization [Ca(t)]i controls granule release, cyclooxygenase-1 and integrin activation, and phosphatidylserine exposure, blood clotting simulations require prediction of platelet [Ca(t)]i in response to combinatorial agonists. Pairwise Agonist Scanning (PAS) deployed all single and pairwise combinations of six agonists (ADP, convulxin, thrombin, U46619, iloprost and GSNO used at 0.1, 1, and 10xEC50; 154 conditions including a null condition) to stimulate platelet P2Y1/P2Y12 GPVI, PAR1/PAR4, TP, IP receptors, and guanylate cyclase, respectively, in Factor Xa-inhibited (250 nM apixaban), diluted platelet rich plasma that had been loaded with the calcium dye Fluo-4 NW. PAS of 10 healthy donors provided [Ca(t)]i data for training 10 neural networks (NN, 2-layer/12-nodes) per donor. Trinary stimulations were then conducted at all 0.1x and 1xEC50 doses (160 conditions) as was a sampling of 45 higher ordered combinations (four to six agonists). The NN-ensemble average was a calcium calculator that accurately predicted [Ca (t)]i beyond the single and binary training set for trinary stimulations (R = 0.924). The 160 trinary synergy scores, a normalized metric of signaling crosstalk, were also well predicted (R = 0.850) as were the calcium dynamics (R = 0.871) and high-dimensional synergy scores (R = 0.695) for the 45 higher ordered conditions. The calculator even predicted sequential addition experiments (n = 54 conditions, R = 0.921). NN-ensemble is a fast calcium calculator, ideal for multiscale clotting simulations that include spatiotemporal concentrations of ADP, collagen, thrombin, thromboxane, prostacyclin, and nitric oxide.

A Localization Algorithm of Nodes Based on Hypersphere Granular Computing in Wireless Sensor Networks

This paper proposes a node localization algorithm based on super chondritic calculation, super chondritic calculation scanning of a training set is able to complete the training process, the training has characteristics of fast, the problem of node localization for wireless sensor networks. Firstly, according to the related parameter information for the location of wireless sensor nodes, a multi input, multi output problem of training set, and then through the methods of grid division, location area, the original training set into the classification of multi input, single output training set, the super chondritic algorithm by scanning training, in order to get the relevant parameters, and the estimation of the unknown node location. Simulation results show that the proposed algorithm has better positioning accuracy.

Detection of static groups and crowds gathered in open spaces by texture classification

A surveillance system specifically developed to manage crowded scenes is described in this paper. In particular we focused on static crowds, composed by groups of people gathered and stayed in the same place for a while. The detection and spatial localization of static crowd situations is performed by means of a One Class Support Vector Machine, working on texture features extracted at patch level. Spatial regions containing crowds are identified and filtered using motion information to prevent noise and false alarms due to moving flows of people. By means of one class classification and inner texture descriptors, we are able to obtain, from a single training set, a sufficiently general crowd model that can be used for all the scenarios that shares a similar viewpoint. Tests on public datasets and real setups validate the proposed system.

Design and Analysis of Classifier Learning Experiments in Bioinformatics: Survey and Case Studies

In many bioinformatics applications, it is important to assess and compare the performances of algorithms trained from data, to be able to draw conclusions unaffected by chance and are therefore significant. Both the design of such experiments and the analysis of the resulting data using statistical tests should be done carefully for the results to carry significance. In this paper, we first review the performance measures used in classification, the basics of experiment design and statistical tests. We then give the results of our survey over 1,500 papers published in the last two years in three bioinformatics journals (including this one). Although the basics of experiment design are well understood, such as resampling instead of using a single training set and the use of different performance metrics instead of error, only 21 percent of the papers use any statistical test for comparison. In the third part, we analyze four different scenarios which we encounter frequently in the bioinformatics literature, discussing the proper statistical methodology as well as showing an example case study for each. With the supplementary software, we hope that the guidelines we discuss will play an important role in future studies.

Window consensus PCA for multiblock statistical process control: adaption to small and time‐dependent normal operating condition regions, illustrated by online high performance liquid chromatography of a three‐stage continuous process

AbstractA method for multiblock statistical process control is described, involving the computation of Q and D statistics both for individual blocks and for the overall process using window consensus principal components analysis (WCPCA). The approach overcomes two common problems. The first is a small normal operating conditions (NOC) region, which is done by determining the Q‐statistic limits and D statistics using leave‐one‐out (LOO) residuals and scores, rather than employing the residuals and scores of a single training set model obtained from the entire NOC region. The second overcomes the problem of temporal drift of the process and/or measurement technique by updating the NOC covariance matrix to adapt to normal process changes. The unifying multiblock statistical process control and relevant statistics are adapted to cope with these issues and are illustrated in this paper using CPCA as applied to online high performance liquid chromatography (HPLC) of a three‐stage continuous process. Copyright © 2010 John Wiley & Sons, Ltd.

Reinforcement learning design for cancer clinical trials

We develop reinforcement learning trials for discovering individualized treatment regimens for life-threatening diseases such as cancer. A temporal-difference learning method called Q-learning is utilized that involves learning an optimal policy from a single training set of finite longitudinal patient trajectories. Approximating the Q-function with time-indexed parameters can be achieved by using support vector regression or extremely randomized trees. Within this framework, we demonstrate that the procedure can extract optimal strategies directly from clinical data without relying on the identification of any accurate mathematical models, unlike approaches based on adaptive design. We show that reinforcement learning has tremendous potential in clinical research because it can select actions that improve outcomes by taking into account delayed effects even when the relationship between actions and outcomes is not fully known. To support our claims, the methodology's practical utility is illustrated in a simulation analysis. In the immediate future, we will apply this general strategy to studying and identifying new treatments for advanced metastatic stage IIIB/IV non-small cell lung cancer, which usually includes multiple lines of chemotherapy treatment. Moreover, there is significant potential of the proposed methodology for developing personalized treatment strategies in other cancers, in cystic fibrosis, and in other life-threatening diseases.

Combining multiple positive training sets to generate confidence scores for protein–protein interactions

Motivation: High-throughput experimental and computational methods are generating a wealth of protein–protein interaction data for a variety of organisms. However, data produced by current state-of-the-art methods include many false positives, which can hinder the analyses needed to derive biological insights. One way to address this problem is to assign confidence scores that reflect the reliability and biological significance of each interaction. Most previously described scoring methods use a set of likely true positives to train a model to score all interactions in a dataset. A single positive training set, however, may be biased and not representative of true interaction space.Results: We demonstrate a method to score protein interactions by utilizing multiple independent sets of training positives to reduce the potential bias inherent in using a single training set. We used a set of benchmark yeast protein interactions to show that our approach outperforms other scoring methods. Our approach can also score interactions across data types, which makes it more widely applicable than many previously proposed methods. We applied the method to protein interaction data from both Drosophila melanogaster and Homo sapiens. Independent evaluations show that the resulting confidence scores accurately reflect the biological significance of the interactions.Contact: rfinley@wayne.eduSupplementary information: Supplementary data are available at Bioinformatics Online.

Application of a hybrid model on short‐term load forecasting based on support vector machines (SVM)

To overcome the shortcoming of single train set of support vector machines (SVM), a novel hybrid model based on dual support vector machines (DSVM) is presented in this paper. The first SVM takes the recent samples in the vicinity of the demand day as its train samples. It can capture the most recent dynamic changing load. The second one takes the same season's load samples in historic years that have the similar attributes with the demand days as its train samples to reflect the season‐period rule. Final results can be archived by converging both SVM. The raw dataset related to experiments was obtained from the EUNITE network. The experiments have proved that the accuracy of proposed model is better than traditional one in general and showed this model's feasibility in practical application.