Ten-fold Cross-validation Experiment Research Articles

A Novel Method for Automatic Identification of Breathing State

Sputum deposition blocks the airways of patients and leads to blood oxygen desaturation. Medical staff must periodically check the breathing state of intubated patients. This process increases staff workload. In this paper, we describe a system designed to acquire respiratory sounds from intubated subjects, extract the audio features, and classify these sounds to detect the presence of sputum. Our method uses 13 features extracted from the time-frequency spectrum of the respiratory sounds. To test our system, 220 respiratory sound samples were collected. Half of the samples were collected from patients with sputum present, and the remainder were collected from patients with no sputum present. Testing was performed based on ten-fold cross-validation. In the ten-fold cross-validation experiment, the logistic classifier identified breath sounds with sputum present with a sensitivity of 93.36% and a specificity of 93.36%. The feature extraction and classification methods are useful and reliable for sputum detection. This approach differs from waveform research and can provide a better visualization of sputum conditions. The proposed system can be used in the ICU to inform medical staff when sputum is present in a patient’s trachea.

Incorporating Topic Information in a Global Feature Selection Schema for Authorship Attribution

Authorship attribution (AA) is a stylometric analysis task of finding the author of an anonymous/disputed text document. In AA, the performance improvement of class-based feature selection schemas, such as Chi-square, and Gini index over frequency-based feature selection schemas, such as document frequency, common n-grams, and inverted document frequency has been shown to be limited. In AA, the feature selection process is significantly affected by topic distributions. In this paper, we assess the performance of a global feature selection approach into which the document’s topic category is incorporated to scale the existing feature weights. In this approach, the common features of an author among different topics indicate higher relevance for the author and thus have higher weights. On the other hand, features with biased topic distributions are assumed to have high topic relevance and lower weights. In this approach, the global topic measure and the author specific topic measure are combined in order to scale the existing selection weights of the features. The ten-fold cross-validation experiment result on a multi-topic dataset with a random topic distribution indicates that our approach improves the performance of Chi-square, modified Gini index, and common n-grams schemas significantly in the best performing configurations of the classifiers.

Classification of Exacerbation Frequency in the COPDGene Cohort Using Deep Learning With Deep Belief Networks.

This study aims to develop an automatic classifier based on deep learning for exacerbation frequency in patients with chronic obstructive pulmonary disease (COPD). A three-layer deep belief network (DBN) with two hidden layers and one visible layer was employed to develop classification models and the models' robustness to exacerbation was analyzed. Subjects from the COPDGene cohort were labeled with exacerbation frequency, defined as the number of exacerbation events per year. A total of 10300 subjects with 361 features each were included in the analysis. After feature selection and parameter optimization, the proposed classification method achieved an accuracy of 91.99%, using a ten-fold cross validation experiment. The analysis of DBN weights showed that there was a good visual spatial relationship between the underlying critical features of different layers. Our findings show that the most sensitive features obtained from the DBN weights are consistent with the consensus showed by clinical rules and standards for COPD diagnostics. We, thus, demonstrate that DBN is a competitive tool for exacerbation risk assessment for patients suffering from COPD.

Predicting Protein-Protein Interaction Sites Using Sequence Descriptors and Site Propensity of Neighboring Amino Acids.

Information about the interface sites of Protein–Protein Interactions (PPIs) is useful for many biological research works. However, despite the advancement of experimental techniques, the identification of PPI sites still remains as a challenging task. Using a statistical learning technique, we proposed a computational tool for predicting PPI interaction sites. As an alternative to similar approaches requiring structural information, the proposed method takes all of the input from protein sequences. In addition to typical sequence features, our method takes into consideration that interaction sites are not randomly distributed over the protein sequence. We characterized this positional preference using protein complexes with known structures, proposed a numerical index to estimate the propensity and then incorporated the index into a learning system. The resulting predictor, without using structural information, yields an area under the ROC curve (AUC) of 0.675, recall of 0.597, precision of 0.311 and accuracy of 0.583 on a ten-fold cross-validation experiment. This performance is comparable to the previous approach in which structural information was used. Upon introducing the B-factor data to our predictor, we demonstrated that the AUC can be further improved to 0.750. The tool is accessible at http://bsaltools.ym.edu.tw/predppis.

Automated fetal brain segmentation from 2D MRI slices for motion correction

Motion correction is a key element for imaging the fetal brain in-utero using Magnetic Resonance Imaging (MRI). Maternal breathing can introduce motion, but a larger effect is frequently due to fetal movement within the womb. Consequently, imaging is frequently performed slice-by-slice using single shot techniques, which are then combined into volumetric images using slice-to-volume reconstruction methods (SVR). For successful SVR, a key preprocessing step is to isolate fetal brain tissues from maternal anatomy before correcting for the motion of the fetal head. This has hitherto been a manual or semi-automatic procedure. We propose an automatic method to localize and segment the brain of the fetus when the image data is acquired as stacks of 2D slices with anatomy misaligned due to fetal motion. We combine this segmentation process with a robust motion correction method, enabling the segmentation to be refined as the reconstruction proceeds. The fetal brain localization process uses Maximally Stable Extremal Regions (MSER), which are classified using a Bag-of-Words model with Scale-Invariant Feature Transform (SIFT) features. The segmentation process is a patch-based propagation of the MSER regions selected during detection, combined with a Conditional Random Field (CRF). The gestational age (GA) is used to incorporate prior knowledge about the size and volume of the fetal brain into the detection and segmentation process. The method was tested in a ten-fold cross-validation experiment on 66 datasets of healthy fetuses whose GA ranged from 22 to 39weeks. In 85% of the tested cases, our proposed method produced a motion corrected volume of a relevant quality for clinical diagnosis, thus removing the need for manually delineating the contours of the brain before motion correction. Our method automatically generated as a side-product a segmentation of the reconstructed fetal brain with a mean Dice score of 93%, which can be used for further processing.

Phased searching with NEAT in a Time-Scaled Framework: Experiments on a computer-aided detection system for lung nodules

In the field of computer-aided detection (CAD) systems for lung nodules in computed tomography (CT) scans, many image features are presented and many artificial neural network (ANN) classifiers with various structural topologies are analyzed; frequently, the classifier topologies are selected by trial-and-error experiments. To avoid these trial and error approaches, we present a novel classifier that evolves ANNs using genetic algorithms, called "Phased Searching with NEAT in a Time or Generation-Scaled Framework", integrating feature selection with the classification task. We analyzed our method's performance on 360 CT scans from the public Lung Image Database Consortium database. We compare our method's performance with other more-established classifiers, namely regular NEAT, Feature-Deselective NEAT (FD-NEAT), fixed-topology ANNs, and support vector machines (SVMs) using ten-fold cross-validation experiments of all 360 scans. The results show that the proposed "Phased Searching" method performs better and faster than regular NEAT, better than FD-NEAT, and achieves sensitivities at 3 and 4 false positives (FP) per scan that are comparable with the fixed-topology ANN and SVM classifiers, but with fewer input features. It achieves a detection sensitivity of 83.0±9.7% with an average of 4FP/scan, for nodules with a diameter greater than or equal to 3mm. It also evolves networks with shorter evolution times and with lower complexities than regular NEAT (p=0.026 and p<0.001, respectively). Analysis on the average and best network complexities evolved by regular NEAT and by our approach shows that our approach searches for good solutions in lower dimensional search spaces, and evolves networks without superfluous structure. We have presented a novel approach that combines feature selection with the evolution of ANN topology and weights. Compared with the original threshold-based Phased Searching method of Green, our method requires fewer parameters and converges to the optimal network complexity required for the classification task at hand. The results of the ten-fold cross-validation experiments also show that our proposed CAD system for lung nodule detection performs well with respect to other methods in the literature.

Dealing with orthographic variation in a tagger-lemmatizer for fourteenth century Dutch charters

In this paper we describe a tagger-lemmatizer for fourteenth century Dutch charters (as found in the corpus van Reenen/Mulder), with a special focus on the treatment of the extensive orthographic variation in this material. We show that despite the difficulties caused by the variation, we are still able to reach about 95 % accuracy in a tenfold cross-validation experiment for both tagging and lemmatization. We can deal effectively with the variation in tokenization (as applied by the authors) by pre-normalization (retokenization). For variation in spelling, however, we choose to expand our lexicon with predicted spelling variants. For those forms which can also not be found in this expanded lexicon, we first derive the word class and subsequently search for the most similar lexicon word. Interestingly, our techniques for recognizing spelling variants turn out to be vital for lemmatization accuracy, but much less important for tagging accuracy.

Evaluating current automatic de-identification methods with Veteran’s health administration clinical documents

BackgroundThe increased use and adoption of Electronic Health Records (EHR) causes a tremendous growth in digital information useful for clinicians, researchers and many other operational purposes. However, this information is rich in Protected Health Information (PHI), which severely restricts its access and possible uses. A number of investigators have developed methods for automatically de-identifying EHR documents by removing PHI, as specified in the Health Insurance Portability and Accountability Act “Safe Harbor” method.This study focuses on the evaluation of existing automated text de-identification methods and tools, as applied to Veterans Health Administration (VHA) clinical documents, to assess which methods perform better with each category of PHI found in our clinical notes; and when new methods are needed to improve performance.MethodsWe installed and evaluated five text de-identification systems “out-of-the-box” using a corpus of VHA clinical documents. The systems based on machine learning methods were trained with the 2006 i2b2 de-identification corpora and evaluated with our VHA corpus, and also evaluated with a ten-fold cross-validation experiment using our VHA corpus. We counted exact, partial, and fully contained matches with reference annotations, considering each PHI type separately, or only one unique ‘PHI’ category. Performance of the systems was assessed using recall (equivalent to sensitivity) and precision (equivalent to positive predictive value) metrics, as well as the F2-measure.ResultsOverall, systems based on rules and pattern matching achieved better recall, and precision was always better with systems based on machine learning approaches. The highest “out-of-the-box” F2-measure was 67% for partial matches; the best precision and recall were 95% and 78%, respectively. Finally, the ten-fold cross validation experiment allowed for an increase of the F2-measure to 79% with partial matches.ConclusionsThe “out-of-the-box” evaluation of text de-identification systems provided us with compelling insight about the best methods for de-identification of VHA clinical documents. The errors analysis demonstrated an important need for customization to PHI formats specific to VHA documents. This study informed the planning and development of a “best-of-breed” automatic de-identification application for VHA clinical text.

Comparison of Several Texture Features for Tumor Detection in CE Images

Capsule endoscopy (CE) has been widely used as a new technology to diagnose gastrointestinal tract diseases, especially for small intestine. However, the large number of images in each test is a great burden for physicians. As such, computer aided detection (CAD) scheme is needed to relieve the workload of clinicians. In this paper, automatic differentiation of tumor CE image and normal CE image is investigated through comparative textural feature analysis. Four different color textures are studied in this work, i.e., texture spectrum histogram, color wavelet covariance, rotation invariant uniform local binary pattern and curvelet based local binary pattern. With support vector machine being the classifier, the discrimination ability of these four different color textures for tumor detection in CE images is extensively compared in RGB, Lab and HSI color space through ten-fold cross-validation experiments on our CE image data. It is found that HSI color space is the most suitable color space for all these texture based CAD systems. Moreover, the best performance achieved is 83.50% in terms of average accuracy, which is obtained by the scheme based on rotation invariant uniform local binary pattern.

Extractive Speech Summarization Using Shallow Rhetorical Structure Modeling

We propose an extractive summarization approach with a novel shallow rhetorical structure learning framework for speech summarization. One of the most under-utilized features in extractive summarization is hierarchical structure information-semantically cohesive units that are hidden in spoken documents. We first present empirical evidence that rhetorical structure is the underlying semantic information, which is rendered in linguistic and acoustic/prosodic forms in lecture speech. A segmental summarization method, where the document is partitioned into rhetorical units by K-means clustering, is first proposed to test this hypothesis. We show that this system produces summaries at 67.36% ROUGE-L F-measure, a 4.29% absolute increase in performance compared with that of the baseline system. We then propose Rhetorical-State Hidden Markov Models (RSHMMs) to automatically decode the underlying hierarchical rhetorical structure in speech. Tenfold cross validation experiments are carried out on conference speeches. We show that system based on RSHMMs gives a 71.31% ROUGE-L F-measure, a 8.24% absolute increase in lecture speech summarization performance compared with the baseline system without using RSHMM. Our method equally outperforms the baseline with a conventional discourse feature. We also present a thorough investigation of the relative contribution of different features and show that, for lecture speech, speaker-normalized acoustic features give the most contribution at 68.5% ROUGE-L F-measure, compared to 62.9% ROUGE-L F-measure for linguistic features, and 59.2% ROUGE-L F-measure for un-normalized acoustic features. This shows that the individual speaking style of each speaker is highly relevant to the summarization.

Protein subcellular localization prediction of eukaryotes using a knowledge-based approach

BackgroundThe study of protein subcellular localization (PSL) is important for elucidating protein functions involved in various cellular processes. However, determining the localization sites of a protein through wet-lab experiments can be time-consuming and labor-intensive. Thus, computational approaches become highly desirable. Most of the PSL prediction systems are established for single-localized proteins. However, a significant number of eukaryotic proteins are known to be localized into multiple subcellular organelles. Many studies have shown that proteins may simultaneously locate or move between different cellular compartments and be involved in different biological processes with different roles.ResultsIn this study, we propose a knowledge based method, called KnowPredsite, to predict the localization site(s) of both single-localized and multi-localized proteins. Based on the local similarity, we can identify the "related sequences" for prediction. We construct a knowledge base to record the possible sequence variations for protein sequences. When predicting the localization annotation of a query protein, we search against the knowledge base and used a scoring mechanism to determine the predicted sites. We downloaded the dataset from ngLOC, which consisted of ten distinct subcellular organelles from 1923 species, and performed ten-fold cross validation experiments to evaluate KnowPredsite's performance. The experiment results show that KnowPredsite achieves higher prediction accuracy than ngLOC and Blast-hit method. For single-localized proteins, the overall accuracy of KnowPredsite is 91.7%. For multi-localized proteins, the overall accuracy of KnowPredsite is 72.1%, which is significantly higher than that of ngLOC by 12.4%. Notably, half of the proteins in the dataset that cannot find any Blast hit sequence above a specified threshold can still be correctly predicted by KnowPredsite.ConclusionKnowPredsite demonstrates the power of identifying related sequences in the knowledge base. The experiment results show that even though the sequence similarity is low, the local similarity is effective for prediction. Experiment results show that KnowPredsite is a highly accurate prediction method for both single- and multi-localized proteins. It is worth-mentioning the prediction process of KnowPredsite is transparent and biologically interpretable and it shows a set of template sequences to generate the prediction result. The KnowPredsite prediction server is available at http://bio-cluster.iis.sinica.edu.tw/kbloc/.

Using Random Forests and Logistic Regression for Performance Prediction of Latin American ADRS and Banks

In the paper, random forests and logistic regressions’ support of financial analysis functions’ predictive tool to forecast corporate performance and rank accounting and corporate variables according to their impact on performance is demonstrated. Ten-fold cross-validation experiments are conducted on one sample each of Latin American depository receipts (ADRs) and Latin American banks. Random forests indicate that the most important variables that affect ADRs performance are size and the law-and-order tradition; the most important variables that affect banks are size, long-term assets to deposits, number of directors, and efficiency of the legal system. The interpretation of predictive models for a small sample improved when the capacity of random forests to rank and predict with the parameters of a logistic regression were combined.

Spoken language annotation and data-driven modelling of phone-level pronunciation in discourse context

A detailed description of the discourse context of a word can be used for predicting word pronunciation in discourse context and also enables studies of the interplay between various types of information on e.g. phone-level pronunciation. The work presented in this paper is aimed at modelling systematic variation in the phone-level realisation of words inherent to a language variety. A data-driven approach based on access to detailed discourse context descriptions is used. The discourse context descriptions are constructed through annotation of spoken language with a large variety of linguistic and related variables in multiple layers. Decision tree pronunciation models are induced from the annotation. The effects of using different types and different amounts of information for model induction are explored. Models generated in a tenfold cross-validation experiment produce on average 8.2% errors on the phone level when they are trained on all available information. Models trained on phoneme level information only have an average phone error rate of 14.2%. This means that including information above the phoneme level in the context description can improve model performance by 42.2%.

Using Adaboost for Equity Investment Scorecards

The objective of this paper is to demonstrate how the boosting approach can be used to define a data-driven board Balanced Scorecard (BSC) with applications to Latin American markets and S&P 500 companies. We compare our results using Adaboost with logistic regression, bagging, and random forests. We conduct tenfold cross-validation experiments on one sample of Latin American Depository Receipts (ADRs), on another sample of Latin American banks, and on the S&P 500 companies. We find that if the dataset is uniform (similar types of companies and same source of information), as is the case with the Latin American ADRs dataset, the results of Adaboost are similar to the results of bagging and random forests. Only when the dataset shows significant non-uniformity does bagging improve the results. Additionally, the uniformity of the dataset affects the interpretability of the results. Using Adaboost, we were able to generate alternating decision trees (ADTs) that explained the relationship between corporate governance variables, performance and efficiency. We also proposed an algorithm to build a representative ADT based on crossvalidation experiments. The representative ADT selected the most important indicators for the board BSC. Additionally, the thresholds of the representative ADT established targets or ranges of values to be used in the board BSC. As a final result, we propose a quantitative strategic management system combining Adaboost with the BSC for board-level or investment decisions.