Tree-based Classification Research Articles

TACOS: a novel approach for accurate prediction of cell-specific long noncoding RNAs subcellular localization.

Long noncoding RNAs (lncRNAs) are primarily regulated by their cellular localization, which is responsible for their molecular functions, including cell cycle regulation and genome rearrangements. Accurately identifying the subcellular location of lncRNAs from sequence information is crucial for a better understanding of their biological functions and mechanisms. In contrast to traditional experimental methods, bioinformatics or computational methods can be applied for the annotation of lncRNA subcellular locations in humans more effectively. In the past, several machine learning-based methods have been developed to identify lncRNA subcellular localization, but relevant work for identifying cell-specific localization of human lncRNA remains limited. In this study, we present the first application of the tree-based stacking approach, TACOS, which allows users to identify the subcellular localization of human lncRNA in 10 different cell types. Specifically, we conducted comprehensive evaluations of six tree-based classifiers with 10 different feature descriptors, using a newly constructed balanced training dataset for each cell type. Subsequently, the strengths of the AdaBoost baseline models were integrated via a stacking approach, with an appropriate tree-based classifier for the final prediction. TACOS displayed consistent performance in both the cross-validation and independent assessments compared with the other two approaches employed in this study. The user-friendly online TACOS web server can be accessed at https://balalab-skku.org/TACOS.

Forecasting Bitcoin price direction with random forests: How important are interest rates, inflation, and market volatility?

Bitcoin has grown in popularity and has now attracted the attention of individual and institutional investors. Accurate Bitcoin price direction forecasts are important for determining the trend in Bitcoin prices and asset allocation. This paper addresses several unanswered questions. How important are business cycle variables like interest rates, inflation, and market volatility for forecasting Bitcoin prices? Does the importance of these variables change across time? Are the most important macroeconomic variables for forecasting Bitcoin prices the same as those for gold prices? To answer these questions, we utilize tree-based machine learning classifiers, along with traditional logit econometric models. The analysis reveals several important findings. First, random forests predict Bitcoin and gold price directions with a higher degree of accuracy than logit models. Prediction accuracy for bagging and random forests is between 75% and 80% for a five-day prediction. For 10-day to 20-day forecasts bagging and random forests record accuracies greater than 85%. Second, technical indicators are the most important features for predicting Bitcoin and gold price direction, suggesting some degree of market inefficiency. Third, oil price volatility is important for predicting Bitcoin and gold prices indicating that Bitcoin is a substitute for gold in diversifying this type of volatility. By comparison, gold prices are more influenced by inflation than Bitcoin prices, indicating that gold can be used as a hedge or diversification asset against inflation.

Characterizing Infant Mortality Using Data Mining - A Case Study in Two Brazilian States - Santa Catarina and Amapá.

Infant mortality is characterized by the death of young children under the age of one, and it is an issue affecting millions of children in the world. The objective of this article is to employ concepts of knowledge discovery in databases, specifically of machine learning in the data mining phase, to characterize infant mortality in two states of Brazil: Santa Catarina, with the lowest infant mortality rate of the country's states, and Amapá, with the highest. The classifiers C4.5, JRip, Random Forest, SVM, and Multilayer Perceptron were used, and a brief comparison of the results obtained by the classifiers in both states is made. In addition, the dataset preprocessing is detailed, which includes attribute selection and class balancing. The results show that the features APGAR5, WEIGHT, and CONGENITAL ANOMALY stood out the most from the rules generated by the tree-based classifiers.

Unemployment in Rural Europe: A Machine Learning Perspective

This paper aims to provide policy-relevant findings that can contribute to the resilience of rural regions by discovering the main individual-level factors related to unemployment in those areas through the use of a set of machine learning techniques. Unemployment status is predicted using tree-based classification models: namely, classification tree, bootstrap aggregation, random forest, gradient boosting, and stochastic gradient boosting. The results are further analyzed using inferential techniques such as SHAP value analysis. Results suggest that access to training programmes can mitigate the labor market inequalities caused by differences in education levels, gender, age, alongside with parental education levels. The results also show how such inequalities are even larger for various subgroups detected by the employed algorithms.

Machine Learning Approach to Predict Clinical Pregnancy Potential in Women Undergoing IVF Program

Objective: Hidden knowledge could be discovered within a large practical data of in vitro fertilization (IVF) practice. In this study, Machine learning-based data mining techniques were utilized to construct a reliable prediction model for clinical pregnancy in IVF. Study Design: A retrospective cohort multicenter study involving 4.570 IVF cycles. All patients underwent fresh embryo transfer at either the cleavage or blastocyst stage between January 2015 and December 2019. The experiment focused on utilizing tree-based classifiers to generate and compare the most effective prediction model that could predict a clinical pregnancy through clinical data. Additionally, each classifier is optimized via a genetic algorithm technique, along with the selection of variables. Results: Both the decision tree and random forest showed similar performance that was much better than the gradient boost. The two superior classifiers achieved a balanced accuracy of roughly 0.62. Additionally, each prediction model was shown to work optimally with different combinations of variables, with some variables being consistently included, such as female age, and some consistently excluded, which provides an insight into the relationship between the variables and each prediction model. Conclusion: Machine learning algorithm remains effective for the purpose of data mining and knowledge extraction in IVF clinical datasets through which a relatively reliable prediction system for clinical pregnancy could be constructed, provided the available data is sufficient.

Multistep-ahead forecasting for maximum and minimum air temperatures using a new hybrid intelligence tree-based filter classifier

Multistep-ahead forecasting for maximum and minimum air temperatures using a new hybrid intelligence tree-based filter classifier

Forecasting solar stock prices using tree-based machine learning classification: How important are silver prices?

Solar energy is one of the fastest growing sources of electricity generation. Forecasting solar stock prices is important for investors and venture capitalists interested in the renewable energy sector. This paper uses tree-based machine learning methods to forecast the direction of solar stock prices. The feature set used in prediction includes a selection of well-known technical indicators, silver prices, silver price volatility, and oil price volatility. The solar stock price direction prediction accuracy of random forests, bagging, support vector machines, and extremely randomized trees is much higher than that of logit. For a forecast horizon of between 8 and 20 days, random forests, bagging, support vector machines, and extremely randomized trees achieve a prediction accuracy greater than 85%. Although not as prominent as technical indicators like MA200, WAD, and MA20, oil price volatility and silver price volatility are also important predictors. An investment portfolio trading strategy based on trading signals generated from the extremely randomized trees stock price direction prediction outperforms a simple buy and hold strategy. These results demonstrate the accuracy of using tree-based machine learning methods to forecast the direction of solar stock prices and adds to the broader literature on using machine learning techniques to forecast stock prices.

PD53-07 DETECTION OF EARLY-STAGE BLADDER CANCER USING PLASMA EXTRACELLULAR VESICLE PROTEIN BIOMARKERS

You have accessJournal of UrologyCME1 May 2022PD53-07 DETECTION OF EARLY-STAGE BLADDER CANCER USING PLASMA EXTRACELLULAR VESICLE PROTEIN BIOMARKERS Juan Hinestrosa, Jean Lewis, Gregor Schroder, Heath Balcer, and Rajaram Krishnan Juan HinestrosaJuan Hinestrosa More articles by this author , Jean LewisJean Lewis More articles by this author , Gregor SchroderGregor Schroder More articles by this author , Heath BalcerHeath Balcer More articles by this author , and Rajaram KrishnanRajaram Krishnan More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002630.07AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: There is a need for earlier bladder cancer screening to reduce lifetime costs and improve patient outcomes. Urinary protein biomarkers have shown some promise, but reported sensitivities appear compromised by low and/or variable biomarker concentrations. In this study, we investigated the potential of using extracellular vesicles (EV) circulating in the blood to detect stage I and II bladder cancer. Using an alternating current electrokinetic (ACE) platform to purify EVs from plasma, we show how a multi-marker EV protein-based methodology can be used to discriminate bladder cancer cases from controls in stage I and II cases. METHODS: Plasma samples were obtained from a biorepository. We used an ACE microelectrode array to isolate plasma EVs of stage I and II bladder cancer cases (N=58) or controls (N=387). EV proteins were quantified using multiplex protein immunoassays for 54 cancer-related proteins. EV protein expression patterns were analyzed using a boosted decision tree-based binary classification algorithm. To identify biomarker subsets that optimized sensitivity of the model, the permutation importance at sensitivity >99% was calculated for each feature. To evaluate biomarker and algorithm performance, a round of 10 repetitions of 5-fold cross validation was done using both the optimized hyper-parameters and the features selected via the permutation importance. RESULTS: We found that relative concentrations of a set of protein biomarkers carried by circulating plasma EVs was sufficient to discriminate bladder cancer cases from controls, with enough sensitivity to detect stage I and II cancers. The EV protein-based classifier had an overall area under curve (AUC) of 0.964 (95% CI: 0.947–0.981) with a sensitivity of 56.9% (95% CI: 44.1%–68.8%) at 99.2% specificity (95% CI: 97.8%-99.7%). At a lower 95.6% specificity (95% CI: 93.1%-97.2%) the sensitivity was 72.4% (95% CI: 59.8%-82.3%). CONCLUSIONS: Testing of EV-associated plasma proteins may enable early-stage I and II bladder cancer detection where treatment is simpler and does not require bladder resection. The classifier’s performance in this pilot, case-control study showed encouraging results for the initial 54 biomarkers studied. Since we have observed much higher specificities for other early-stage cancer types, the performance for bladder cancer will likely be improved by inclusion of additional biomarker proteins. Future efforts will include examining additional cancer types and evaluating classifier performance using samples from donors with related benign conditions with the aim of a pan-cancer early detection assay. Source of Funding: Biological Dynamics © 2022 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 207Issue Supplement 5May 2022Page: e912 Advertisement Copyright & Permissions© 2022 by American Urological Association Education and Research, Inc.MetricsAuthor Information Juan Hinestrosa More articles by this author Jean Lewis More articles by this author Gregor Schroder More articles by this author Heath Balcer More articles by this author Rajaram Krishnan More articles by this author Expand All Advertisement PDF DownloadLoading ...

Computational Intelligence for Observation and Monitoring: A Case Study of Imbalanced Hyperspectral Image Data Classification.

Imbalance in hyperspectral images creates a crisis in its analysis and classification operation. Resampling techniques are utilized to minimize the data imbalance. Although only a limited number of resampling methods were explored in the previous research, a small quantity of work has been done. In this study, we propose a novel illustrative study of the performance of the existing resampling techniques, viz. oversampling, undersampling, and hybrid sampling, for removing the imbalance from the minor samples of the hyperspectral dataset. The balanced dataset is classified in the next step, using the tree-based ensemble classifiers by including the spectral and spatial features. Finally, the comparative study is performed based on the statistical analysis of the outcome obtained from those classifiers that are discussed in the results section. In addition, we applied a new ensemble hybrid classifier named random rotation forest to our dataset. Three benchmark hyperspectral datasets: Indian Pines, Salinas Valley, and Pavia University, are applied for performing the experiments. We have taken precision, recall, F score, Cohen kappa, and overall accuracy as assessment metrics to evaluate our model. The obtained result shows that SMOTE, Tomek Links, and their combinations stand out to be the more optimized resampling strategies. Moreover, the ensemble classifiers such as rotation forest and random rotation ensemble provide more accuracy than others of their kind.

Multi-label chain clustering-classification and regression predictive models for patient punctuality and turnaround time in outpatient primary care settings

This study develops two multi-label chain machine learning predictive models to anticipate patient punctuality and turnaround time. The first model uses an integrated model of clustering and classification where the check-in, service, and checkout times are clustered into three categories using the K-means algorithm. Then, patient punctuality and established clusters are used to develop a multi-label chain predictive model that utilizes Logistic Regression, Multi-Layer Perceptron, and tree-based classifiers. The second model predicts patient punctuality and turnaround time using a multi-label chain regression model that utilizes Linear Regression, Huber Regressor, ADR Regression, Multi-Layer Perceptron, and tree-based regressors. It was found that a patient’s age is a key driver for both patient punctuality and turnaround time. Also, there is a significant association between patient punctuality and turnaround time. The first proposed model predicted the punctuality and turnaround time with an average best F1-score of about 70.4% and 71.9%, respectively. The second model produced acceptable results with the best average R-squared of 0.67 for punctuality and 0.68 for turnaround time. The models can reduce the complexity and time of predicting several numerical outputs, enhance the interpretation of the results, and improve the understanding of the results by non-technical staff.

A genetic programming-based convolutional neural network for image quality evaluations

Monitoring the perceptual quality of digital images is fundamentally important since digital image transmissions through the Internet continue to increase exponentially. Many automatic image quality evaluation (IQE) metrics have been developed based on image features correlated to image distortions; however, those metrics are only effective on particular image distortion types. In recent years, convolutional neural network (CNNs) have been developed for IQEs. These CNNs first capture image features from distorted images; image qualities are predicted based on the captured image features. Since the CNN weights are randomly initialized and are updated with respect to a loss function, image features which are strongly correlated to image quality are not guaranteed to be captured. In this paper, a hybrid deep neural network (DNN) is proposed by integrating image quality metrics to capture image features which are correlated to image quality; the approach guarantees that significant image features are included to predict image quality. Also, a tree-based classifier namely geometric semantic genetic programming is proposed to perform the overall predictions by incorporating CNN predictions and image features; the approach is simpler than the fully connected network but is able to model the nonlinear image qualities. The performance of the proposed hybrid DNN is evaluated by an image quality database with 3000 distorted images. The mean correlation achieved by the proposed hybrid DNN is 0.57 which is higher than the other tested methods. Experimental results with the t- test, F-test and Tueky’s range tests show that the proposed hybrid DNN achieves more accurate image predictions with a 99.9% confidence level, compared to the state-of-the-art IQE metics and the most recently developed CNN for IQEs.

Change detection filter technique-based fault analysis of HVDC transmission line

PurposeThis research paper aims to investigate the change detection filter technique with a decision tree-based event (fault type) classifier for recognizing and categorizing power system disturbances on the high-voltage DC (HVDC) transmission link.Design/methodology/approachA change detection filter is used to the average and differential current components, which detects the point of fault initiation and records a change detection point (CDP). The half-cycle differential and average currents on both sides of the CDP are sent through the signal processing unit, which produces the respective target. The extracted target indices are sent through a decision tree-based fault classifier mechanism for fault classification.FindingsIn comparison with conventional differential current protection systems, the developed framework is faster in fault detection and classification and provides great accuracy. The new technology allows for prompt identification of the fault category, allowing electrical grids to be restored as quickly as possible to minimize economic losses. This novel technology enhances efficiency in terms of reducing computing complexity.Research limitations/implicationsSetting a threshold value for identification is one of the limitations. To bring the designed system into stability condition before creating faults on it is another limitation. Reducing the computational burden is one of the limitations.Practical implicationsCreating a practical system in laboratory is difficult as it is a HVDC transmission line. Apart from that, installing rectifier and converter section for HVDC transmission line is difficult in a laboratory setting.Originality/valueThe suggested scheme’s importance and accuracy have been rigorously validated for the standard HVDC transmission system, subjected to various types of DC fault, and the results show the proposed algorithm would be a feasible alternative to real-time applications.

Silhouettes and Quasi Residual Plots for Neural Nets and Tree-based Classifiers

Classification by neural nets and by tree-based methods are powerful tools of machine learning. There exist interesting visualizations of the inner workings of these and other classifiers. Here we pursue a different goal, which is to visualize the cases being classified, either in training data or in test data. An important aspect is whether a case has been classified to its given class (label) or whether the classifier wants to assign it to a different class. This is reflected in the (conditional and posterior) probability of the alternative class (PAC). A high PAC indicates label bias, that is, the possibility that the case was mislabeled. The PAC is used to construct a silhouette plot which is similar in spirit to the silhouette plot for cluster analysis. The average silhouette width can be used to compare different classifications of the same dataset. We will also draw quasi residual plots of the PAC versus a data feature, which may lead to more insight in the data. One of these data features is how far each case lies from its given class. The graphical displays are illustrated and interpreted on datasets containing images, mixed features, and tweets. Supplementary materials for this article are available online.

Power output optimization of electric vehicles smart charging hubs using deep reinforcement learning

Since most branches of the distribution grid may already be close to their maximum capacity, smart management when charging electric vehicles (EVs) is becoming more and more crucial. In fact, office buildings might not be able to handle several transactions at the same time, especially considering the next generation of fast chargers which are very power expensive. Thus, an efficient charging policy needs to be found. This paper proposes the scheduling of real-time EVs charging through deep reinforcement learning (DRL) techniques. DRL has been chosen because it can adaptively learn from interacting with the surrounding environment. The focus of the optimization is to ensure the completion of the charging transactions in a timely manner, while shifting the load from the times of peak demand. The novelty of the proposed approach lies in its innovative framework: pools of electric vehicles with different characteristics are categorized using a clustering algorithm, a tree-based classifier has been developed to sort new instances of EVs, and a multilayer perceptron artificial deep neural network has been trained to predict the expected duration of each charging session. These features are used as inputs to the DRL agent, and are mapped into actions that adjust the maximum power associated to each charging station. The model has been compared to a traditional charging algorithm and increasingly challenging scenarios have been considered. Results have shown that the developed algorithm fails less than the baseline, with a reduction of the load due to EVs charging of 80% during peak times.

Machine learning based met data anomaly labelling

Abstract Data preprocessing is the initial and utmost critical phase of wind resource assessment and wind power curve performance analysis. Without ensuring high quality and site representative data availability, an evaluation for the wind resource potential of a wind farm site means investment decision making involving a great deal of uncertainty. The current practice in the field is to use fix rules and via data scanning by manual labouring of data field experts. Although rule-based (if-else, interval-based) prefiltering applications can be found in some commercial software, and these rules are also limited for addressing the needed preprocessing requirements fully. Therefore, this process is time consuming and causes inefficient resource allocation considering the manual scanning process performed by data scientists & experts. In this study, reviewed data by experts are used as data with correct labels, and then efficient classifiers are generated by applying machine learning algorithms to unfiltered and filtered data. It is reported that tree-based classifiers are performing better.

Machine learning methods for constructing probabilisticFermi-LAT catalogs

Context.Classification of sources is one of the most important tasks in astronomy. Sources detected in one wavelength band, for example using gamma rays, may have several possible associations in other wavebands, or there may be no plausible association candidates.Aims.In this work we aim to determine the probabilistic classification of unassociated sources in the thirdFermiLarge Area Telescope (LAT) point source catalog (3FGL) and the fourthFermiLAT data release 2 point source catalog (4FGL-DR2) using two classes – pulsars and active galactic nuclei (AGNs) – or three classes – pulsars, AGNs, and “OTHER” sources.Methods.We use several machine learning (ML) methods to determine a probabilistic classification ofFermi-LAT sources. We evaluate the dependence of results on the meta-parameters of the ML methods, such as the maximal depth of the trees in tree-based classification methods and the number of neurons in neural networks.Results.We determine a probabilistic classification of both associated and unassociated sources in the 3FGL and 4FGL-DR2 catalogs. We cross-check the accuracy by comparing the predicted classes of unassociated sources in 3FGL with their associations in 4FGL-DR2 for cases where such associations exist. We find that in the two-class case it is important to correct for the presence of OTHER sources among the unassociated ones in order to realistically estimate the number of pulsars and AGNs. We find that the three-class classification, despite different types of sources in the OTHER class, has a similar performance as the two-class classification in terms of reliability diagrams and, at the same time, it does not require adjustment due to presence of the OTHER sources among the unassociated sources. We show an example of the use of the probabilistic catalogs for population studies, which include associated and unassociated sources.

Optimizing Attack Detection for High Dimensionality and Imbalanced Data with SMOTE, Chi-Square and Random Forest Classifier

The rapid growth of the network generates a very large and varied amount of traffic which has an impact on data and information security. This study resolves two common problems in attack detection, namely high dimensionality and high-class imbalance of the network traffic. This study used the ISCX CICIDS-2017 dataset. This study used the ISCX CICIDS-2017 dataset. The CICIDS-2017 dataset is imbalance that contains very diverse types of traffic including normal traffic and several types of attacks (multi-class). This study proposes a combination of the Chi-Square feature selection technique with the Tree-Based Classifier Random Forest. In the experiment first the Chi-Square Correlation Based feature selection technique was applied to the imbalance dataset. The selected features are then validated using several Random Forest algorithms. The test was also performed comparisons with other classification algorithms such as Naïve Bayes, Bayes Network, J48, REPTree, and Adaboost. This study also examines the implementation of SMOTE to overcome the problem of high calass imbalance. The test results also show that the proposed ensemble method has a very good performance from the Accuracy, TPR, FPR, Precision, F-Measure, and ROC values

Tree-based dynamic classifier chains

Classifier chains are an effective technique for modeling label dependencies in multi-label classification. However, the method requires a fixed, static order of the labels. While in theory, any order is sufficient, in practice, this order has a substantial impact on the quality of the final prediction. Dynamic classifier chains denote the idea that for each instance to classify, the order in which the labels are predicted is dynamically chosen. The complexity of a naïve implementation of such an approach is prohibitive, because it would require to train a sequence of classifiers for every possible permutation of the labels. To tackle this problem efficiently, we propose a new approach based on random decision trees which can dynamically select the label ordering for each prediction. We show empirically that a dynamic selection of the next label improves over the use of a static ordering under an otherwise unchanged random decision tree model. In addition, we also demonstrate an alternative approach based on extreme gradient boosted trees, which allows for a more target-oriented training of dynamic classifier chains. Our results show that this variant outperforms random decision trees and other tree-based multi-label classification methods. More importantly, the dynamic selection strategy allows to considerably speed up training and prediction.

Predicting Earthquake-Induced Soil Liquefaction Based on Machine Learning Classifiers: A Comparative Multi-Dataset Study

The study and prediction of soil liquefaction is an important and complex issue in geotechnical earthquake engineering. This paper attempts to compare the predictability of soil liquefaction potential between several machine learning classification models, which includes some tree-based classifiers, multilayer perceptron (MLP) neural networks, Support Vector Machine (SVM), some state-of-the-art ensemble methods, K nearest neighbors method, classical Naive Bayesian classifier and logistic regression. Three data sets covering shear-wave velocity measurements, cone penetration testing (CPT), and real historic earthquakes cases are employed to train and evaluate the machine learning classifiers. In order to make the best use of large varieties of statistical and machine learning classification algorithms, it is necessary to give a comparative evaluation of the model performance before model selection and offer advice on a unified stable model for all sorts of collected datasets. In the comparative study, data preprocessing is first performed to ensure the dataset into all machine models is of good quality. Then all three datasets with different input features are passed into the machine learning algorithms to obtain its confusion matrix and some evaluation indices. Reliable assessment of model performance is done through a repeated sub-sampling process. Experimental results are also supported by ROC curves. The results of this study indicated that although most machine learning methods are able to represent the complex relationship between seismic proper seismic properties of soils and corresponding liquefaction potential, ensemble learning has achieved more successful results in all three datasets test and can be a fairly promising approach on prediction of earthquake-induced soil liquefaction.

Efficiently identifying individuals at high risk for treatment resistance in major depressive disorder using electronic health records

BackgroundWith the emergence of evidence-based treatments for treatment-resistant depression, strategies to identify individuals at greater risk for treatment resistance early in the course of illness could have clinical utility. We sought to develop and validate a model to predict treatment resistance in major depressive disorder using coded clinical data from the electronic health record. MethodsWe identified individuals from a large health system with a diagnosis of major depressive disorder receiving an index antidepressant prescription, and used a tree-based machine learning classifier to build a risk stratification model to identify those likely to experience treatment resistance. The resulting model was validated in a second health system. ResultsIn the second health system, the extra trees model yielded an AUC of 0.652 (95% CI: 0.623–0.682); with sensitivity constrained at 0.80, specificity was 0.358 (95% CI: 0.300–0.413). Lift in the top quintile was 1.99 (95% CI: 1.76–2.22). Including additional data for the 4 weeks following treatment initiation did not meaningfully improve model performance. LimitationsThe extent to which these models generalize across additional health systems will require further investigation. ConclusionElectronic health records facilitated stratification of risk for treatment-resistant depression and demonstrated generalizability to a second health system. Efforts to improve upon such models using additional measures, and to understand their performance in real-world clinical settings, are warranted.