Conditional Random Forest Research Articles

Carbon price forecasting models based on big data analytics

After the establishment of the European Union's Emissions Trading System (EU-ETS) carbon pricing attracted many researchers. This paper aims to develop a prediction model that anticipates future carbon prices given a real-world data set. We treat the carbon pricing issue as part of big data analytics to achieve this goal. We apply three fundamental methodologies to characterize the carbon price. First method is the artificial neural network, which mimics the principle of human brain to process relevant data. As a second approach, we apply the decision tree algorithm. This algorithm is structured through making multiple binary decisions, and it is mostly used for classification. We employ two different decision tree algorithms, namely traditional and conditional, to determine the type of decision tree that gives better results in terms of prediction. Finally, we exploit the random forest, which is a more complex algorithm compared to the decision tree. Similar to the decision tree, we test both traditional and conditional random forest algorithms to analyze their performances. We use Brent crude futures, coal, electricity and natural gas prices, and DAX and S&P Clean Energy Index as explanatory variables. We analyze the variables' effects on carbon price forecasting. According to our results, S&P Clean Energy Index is the most influential variable in explaining the changes in carbon price, followed by DAX Index and coal price. Moreover, we conclude that the traditional random forest is the best algorithm based on all indicators. We provide the details of these methods and their comparisons.AbbreviationsANNArtificial Neural NetworkCDTConditional Decision TreeCRFConditional Random ForestDAXStock market index consisting of 30 major German companiesDTDecision TreeETSEmissions Trading SystemEU-ETSEuropean Union Emissions Trading SystemFFNNFeedforward Neural NetworkGHGGreenhouse GasesIncNodePurityIncrease in Node PurityMAEMean Average ErrorMAPEMean Average Percentage ErrorMSEMean Square ErrorRBFNRadial Basis Function NetworkRFRandom ForestRMSERoot Mean Square ErrorRSR-Squared (Coefficient of determination)TDTTraditional Decision TreeTRFTraditional Random ForestVARVector Auto RegressionVMDVariational Mode Decomposition% IncMSEPercentage increase in mean square error

Real-time pose invariant spontaneous smile detection using conditional random regression forests

Detecting spontaneous smile in unconstrained environment is a challenging problem mainly due to the large intra-class variations caused by head poses. This paper presents a real-time smile detection method based on conditional random regression forests. Since the relation between image patches and smile intensity is modelled conditional to head pose, the proposed smile detection method is not sensitive to head poses. To achieve high smile detection performance, techniques including regression forest, multiple-label dataset augmentation and non-informative patch removement are employed. Experimental results show that the proposed method achieves competitive performance to state-of-the-art deep neural network based methods on two challenging real-world datasets, although using hand-crafted features. A dynamical forest ensemble scheme is also presented to make a trade-off between smile detection performance and processing speed. In contrast to deep neural networks, the proposed method can run in real-time on general hardware without GPU.

Hidden Smile Correlation Discovery Across Subjects Using Random Walk with Restart

Fine-grained smile analysis is a complicated and challenging process. Understanding other party’s smiles is one of the key tasks associated with realizing the implicit messages transmitted by the human. Considering this kind of message transmission is a major feature of human communication, understanding smiling has great potential value to promote the development of humanoid robots and animated software agents. Therefore, a fine-grained smile analysis system is proposed to uncover the hidden smile correlation across subjects. The system incorporates head pose as prior knowledge and employs conditional random forest to detect fiducial points on face. After that, a steady-state probability defined by a succession of Markov random steps is used to indicate the relevance score between smiles across subjects. We demonstrate performance of the proposed system on both constrained and unconstrained face datasets. The experimental results show that the proposed system is able to classify 4 smile levels and uncover hidden smile correlations across subjects successfully.

A machine learning approach to assess price sensitivity with application to automobile loan segmentation

Price sensitivity is an outstanding business issue in companies and organizations that aim to undertake optimal managerial decisions for increasing sales and / or revenue. Hence, price sensitivity assessment has become an in fashion problem that has attracted the attention of a wide variety of actors and business units within the organizations. In this paper we propose a machine learning approach to assess price sensitivity for an automobile loan portfolio in order to get a segmentation revealing the existence of groups with differential price sensitivity, defined by their differential purchase responses against changes in the loan interest rate. The proposed method combines the power of conditional inference trees, random forests and model based recursive partitioning algorithms to implement a process for price group finding, variable selection and price sensitivity segmentation in order uncover such differential groups and characterize them by asset and product characteristics and by customer attributes as well. The resulting segmentation will define high sensitivity groups, where interest rate reductions can be recommended in order to increase sales, as well as nearly insensitive groups for which a price strategy that increases the interest rate is expected to have slight impact on loan disbursements.

Agentive (para)synthetic compounds in Russian: a quantitative study of rival constructions

The paper compares two rival word-formation constructions giving rise to compound agent nouns in Russian, i.e., (para)synthetic compounds formed with the agentive suffixes -ec and -tel’, such as basnopisec ‘fable writer’ and bytopisatel’ ‘everyday-life writer’. To understand what makes these constructions different from one another, compounds in -ec and -tel’ are analyzed based on a number of formal and semantic criteria, i.e., the part of speech and semantic role of the non-verbal element of the compound, the transitivity and formal aspect of the verbal base of the compound, the animacy of the compound’s referent, and the semantics of the compound. The study is supported by statistical analyses, i.e., conditional inference trees and random forests, which help discriminate the behavior of rival constructions and determine which parameters are more relevant for the comparison. To understand whether diachronic and/or stylistic factors also affect the survival of rival constructions, the data are checked in the Russian National Corpus, which allows retrieving information about the texts in which compounds occur, such as their creation date and textual genre. Finally, the productivity of rival word-formation constructions in modern Russian is discussed both in terms of diachronic changes and in terms of restrictions that the two constructions are subject to. The analyses carried out demonstrate that the two constructions show significant differences regarding their semantics, but also their diachronic and stylistic distribution, as well as their productivity, which prevents one construction from completely ousting the other in modern Russian.

To Predict Visual Deterioration According to the Degree of Intracranial Hypertension in Patients with Cerebral Venous Sinus Thrombosis

Background: Visual damage is one of the most common complications of cerebral venous sinus thrombosis (CVST)-associated intracranial hypertension (IH). This study is aimed at stratifying the risk of IH-induced visual damage in an attempt to predict its deterioration and prevent high-risk patients from irreversible eyesight impairment promptly. Methods: A total of 94 patients with confirmed diagnosis of CVST were eligible for enrollment in this study. According to cerebrospinal fluid pressure at admission, the involved patients were classified into mild IH (< 250 mmH₂O), moderate IH (250–330 mmH₂O), and severe IH (≥330 mmH₂O) groups. Results: The ratio of visual deterioration in the severe IH group was 75%, which was significantly higher than in either the moderate (44.4%) or the mild groups (14.3%). As regards subjects without visual symptoms at admission, visual deterioration occurred in 9.4 ± 4.5 days after admission in the severe group while it occurred in 30.5 ± 16.8 days in the moderate group (p = 0.024). The conditional inference tree and random forest revealed that severe IH might be considered as an index of visual deterioration. Visual field defect, fading eyesight, and papilledema were significantly worse in patients with severe IH as compared to patients with mild or moderate IH, all p < 0.01. Conclusions: IH ≥330 mmH₂O may be a cut-off value to predict the deterioration of visual damage in CVST, revealing that ophthalmologic interventions should be considered in a timely manner in this condition, particularly when recanalization of cerebral venous sinus cannot be achieved within a short time.

Stratification of the risk of bipolar disorder recurrences in pregnancy and postpartum.

Pregnancy and childbirth are a period of high risk for women with bipolar disorder and involve difficult decisions particularly about continuing or stopping medications.AimsTo explore what clinical predictors may help to individualise the risk of perinatal recurrence in women with bipolar disorder. Information was gathered retrospectively by semi-structured interview, questionnaires and case-note review from 887 women with bipolar disorder who have had children. Clinical predictors were selected using backwards stepwise logistic regression, conditional permutation random forests and reinforcement learning trees. Previous perinatal history of affective psychosis or depression was the most significant predictor of a perinatal recurrence (odds ratio (OR) = 8.5, 95% CI 5.04-14.82 and OR = 3.6, 95% CI 2.55-5.07 respectively) but even parous women with bipolar disorder without a previous perinatal mood episode were at risk following a subsequent pregnancy, with 7% developing postpartum psychosis. Previous perinatal history of affective psychosis or depression is the most important predictor of perinatal recurrence in women with bipolar disorder and can be used to individualise risk assessments.Declaration of interestNone.

Synthetic RapidEye data used for the detection of area-based spruce tree mortality induced by bark beetles

Tree mortality caused by outbreaks of the bark beetle Ips typographus (L.) plays an important role in the natural dynamics of Norway spruce (Picea abies L.) stands, which could cause far-reaching changes in the occurrence and duration of vegetation phenology. Field-based early detection of tree disturbances is hampered by logistic, terrain, and technical shortcomings, and by the inability to continuously monitor disturbances over large areas. Despite achievements in remote mapping of bark-beetle-induced tree mortalities, early warning has been mostly unsuccessful mainly because of the lack of spectral sensitivity and discrepancies in definitions of field- and image-based disturbance classes. Here we applied a method based on inter-annual phenology of Norway spruce stands derived from synthetic multispectral data to part of the Bavarian Forest National Park in Germany. We fused temporally continuous Moderate Resolution Imaging Spectroradiometer and discrete RapidEye data using a flexible spatiotemporal data fusion method to achieve validated 8-day RapidEye-like composites of normalized difference vegetation index for 2011. We assumed that the dead trees delineated on 2012 aerial photographs were those in which bark beetle infestations were initiated in 2011. Samples were drawn with variable-sized buffering to represent the areas prone to infestations and their surroundings. We applied a conditional inference random forest to select the best image date among the entire 46 synthetic datasets to best discriminate between the core infestation patches and their surroundings from the subsequent year. Of the discrete time points identified, day 281 of the year represented the highest discrepancy between aerial image-based dead trees and their surroundings. Classification results were significantly correlated with beetle count data obtained using pheromone traps. Our method provided valuable information for management purposes and enabled wall-to-wall mapping of stands prone to infestation and its uncertainty. The results offer potential implications for rapid and cost-effective monitoring of bark beetle outbreaks using satellite data, which would be of great benefit for both management and research tasks.

Data‐driven recovery of hand depth using CRRF on stereo images

Hand pose is emerging as an important interface for human–computer interaction. This study presents a data‐driven method to estimate a high‐quality depth map of a hand from a stereoscopic camera input by introducing a novel superpixel‐based regression framework that takes advantage of the smoothness of the depth surface of the hand. To this end, the authors introduce conditional regressive random forest (CRRF), a method that combines a conditional random field (CRF) and an RRF to model the mapping from a stereo red, green and blue image pair to a depth image. The RRF provides a unary term that adaptively selects different stereo‐matching measures as it implicitly determines matching pixels in a coarse‐to‐fine manner. While the RRF makes depth prediction for each superpixel independently, the CRF unifies the prediction of depth by modelling pairwise interactions between adjacent superpixels. Experimental results show that CRRF can generate a depth image more accurately than the leading contemporary techniques using an inexpensive stereo camera.

SURVEYING BORDERS IN A SPEECH COMMUNITY

This research reports responses to a language questionnaire for an extensive set ofphonological, lexical, and grammatical variables in Kansas City. Linguistic variables aremodeled with conditional inference trees and random forests against demographicfactors like age, race, and sex, as well as several geopolitical factors. Results show that --while Kansas Citians identify many cultural, political, and social barriers that shape life inthe city -- race is the primary social predictor of survey responses. Among sixteen surveyitems with significant predictors, race is most significant for nine. Geopolitical factorsthat Kansas Citians imagine as meaningful do not affect linguistic practice. Based on thisfinding, the article considers the “speech community” as a unit of sociolinguistic analysis,and concludes that the large metropolitan area can be thought of as a single speechcommunity, as long as the sample is racially homogenous.

Syntactic variation in ‘need’-constructions in Estonian dialects

The article contributes new data and findings to the growing field of corpus-based dialect syntax research. The focus of the paper is on variation in ‘need’-constructions (tarvis/vaja olema+ nominal complement/infinitive ‘need to’) based on the corpus of Estonian dialects. Our purpose was to demonstrate the complex nature of syntactic variation, constrained geographically, individually or by language-internal factors. The study takes a corpus-based quantitative approach to observing the geographical spread of linguistic units. We apply conditional inference tree and random forests models to capture the (co)varying parts of the construction studied. Our results show that variation in different parts of constructions is influenced by different factors, both geographical and language-internal. Lexical variation (adverbtarvis‘need’ orvaja‘need’) and omission of the copula are clearly geographically distributed, while omission of the experiencer is determined mainly by language-internal factors. However, the study has also found extensive inter-individual differences.

Prognostic factors for survival in adult patients with recurrent glioblastoma: a decision-tree-based model.

We assessed prognostic factors in relation to OS from progression in recurrent glioblastomas. Retrospective multicentric study enrolling 407 (training set) and 370 (external validation set) adult patients with a recurrent supratentorial glioblastoma treated by surgical resection and standard combined chemoradiotherapy as first-line treatment. Four complementary multivariate prognostic models were evaluated: Cox proportional hazards regression modeling, single-tree recursive partitioning, random survival forest, conditional random forest. Median overall survival from progression was 7.6 months (mean, 10.1; range, 0-86) and 8.0 months (mean, 8.5; range, 0-56) in the training and validation sets, respectively (p = 0.900). Using the Cox model in the training set, independent predictors of poorer overall survival from progression included increasing age at histopathological diagnosis (aHR, 1.47; 95% CI [1.03-2.08]; p = 0.032), RTOG-RPA V-VI classes (aHR, 1.38; 95% CI [1.11-1.73]; p = 0.004), decreasing KPS at progression (aHR, 3.46; 95% CI [2.10-5.72]; p < 0.001), while independent predictors of longer overall survival from progression included surgical resection (aHR, 0.57; 95% CI [0.44-0.73]; p < 0.001) and chemotherapy (aHR, 0.41; 95% CI [0.31-0.55]; p < 0.001). Single-tree recursive partitioning identified KPS at progression, surgical resection at progression, chemotherapy at progression, and RTOG-RPA class at histopathological diagnosis, as main survival predictors in the training set, yielding four risk categories highly predictive of overall survival from progression both in training (p < 0.0001) and validation (p < 0.0001) sets. Both random forest approaches identified KPS at progression as the most important survival predictor. Age, KPS at progression, RTOG-RPA classes, surgical resection at progression and chemotherapy at progression are prognostic for survival in recurrent glioblastomas and should inform the treatment decisions.

Revisiting acoustic correlates of pharyngealization in Jordanian and Moroccan Arabic: Implications for formal representations

This exploratory study of Jordanian and Moroccan Arabic (JA and MA) aims to evaluate whether pharyngealization is associated with an epilaryngeal constriction which causes ‘retraction’ and tense voice quality in surrounding vowels, following the Laryngeal Articulator Model (LAM) (Esling, 2005). Twenty male speakers (10 per dialect) produced vowels preceded by /d or dˤ/. Thirteen acoustic correlates obtained at the onset and midpoint were used to assess this type of constriction. A predictive modeling approach was used; starting with Bayesian Generalized Linear Mixed Effects modeling followed by Conditional Random Forest for classification. Vowels in the pharyngealized context were more open (higher F1, Z1-Z0), more back (lower F2, higher Z3-Z2), more compact (lower Z2-Z1), and showed spectral divergence (higher Z3-Z2). Voice quality results showed these vowels to be produced with a tense voice. High classification rates of 93.5% for JA and 91.1% for MA were obtained and variable importance score showed formant-based measures outperform voice quality ones. This suggests pharyngealization has ‘retraction,’ with a back and down gesture, as a primary correlate followed by [+CONSTRICTED GLOTTIS]. The implications of these results provide strong support for LAM, the feature [+CET], and the use of the epilarynx to describe pharyngealization.

Distributed Intrinsic Functional Connectivity Patterns Predict Diagnostic Status in Large Autism Cohort.

Diagnosis of autism spectrum disorder (ASD) currently relies on behavioral observations because brain markers are unknown. Machine learning approaches can identify patterns in imaging data that predict diagnostic status, but most studies using functional connectivity MRI (fcMRI) data achieved only modest accuracies of 60-80%. We used conditional random forest (CRF), an ensemble learning technique protected against bias from feature correlation (which exists in fcMRI matrices). We selected 252 low-motion resting-state functional MRI scans from the Autism Brain Imaging Data Exchange, including 126 typically developing (TD) and 126 ASD participants, matched for age, nonverbal IQ, and head motion. A matrix of functional connectivities between 220 functionally defined regions of interest was used for diagnostic classification. In several runs, we achieved accuracies of 92-99% for classifiers with >300 features (most informative connections). Features, including pericentral somatosensory and motor regions, were disproportionately informative. Findings differed partially from a previous study in the same sample that used feature selection with random forest (which is biased by feature correlations). External validation in a smaller in-house data set, however, achieved only 67-71% accuracy. The large number of features in optimal models can be attributed to etiological heterogeneity under the clinical ASD umbrella. Lower accuracy in external validation is expected due to differences in unknown composition of ASD variants across samples. High accuracy in the main data set is unlikely due to noise overfitting, but rather indicates optimized characterization of a given cohort.

Phonological properties of word classes and directionality in conversion

In the study of the word-formation process of conversion, one particularly difficult task is to determine the directionality of the process, that is, to decide which word represents the base and which the derived word. One possibility to inform this decision that has received only limited attention is to capitalize on word-class-specific phonological properties. This paper empirically investigates this option for English noun-verb conversion by building on recent findings on phonological differences between these two word classes. A large-scale study of phonological properties is carried out on CELEX data, employing the quantitative techniques of conditional inference trees and random forests. An important result of this analysis is that the accuracy of phonological cues varies widely across different subsamples in the data. Essentially this means that phonological cues can be used as a criterion to determine the directionality of words that are at least two syllables in length. When restricted to this part of the lexicon, phonological properties represent a fairly accurate indicator of source word class and are therefore a useful addition to the linguist's toolkit for determining directionality in conversion. Based on this result, the paper also discusses the relations of phonological properties to other criteria commonly employed to determine directionality.

Málnotkun sem mælikvarði á áhrif málstöðlunar: Skólaritgerðir úr Lærða skólanum í Reykjavík (1846–1904)

The Reykjavík Grammar School (1846–1904) has been widely regarded as a primary force in the implementation of standard language norms in Iceland. The present article attempts to test this hypothesis in a selection of 189 student essays from the grammar school, including a survey of the teachers’ corrections. Three linguistic variables were selected, both known from the prescriptivist tradition and corrected in the es-says: 1) the generic pronoun maður, 2) the finite verb in third (V3) rather than second position (V2), 3) the definite article sá vs. hinn. Based on a series of statistical tools, a log-likelihood test for the generic pronoun, generalised linear mixed-effects models and conditional random forests for verb placement and the definite article, it is argued that the use of non-standard variants correlates significantly with progression of study (grades 1–3 vs. 4–6) and/or graduation score (low vs. high). The small corpus size prevented an analysis along the lines of Hinrichs et al. (2015), who recommend testing whether the (non-)use of one stigmatised ariant also correlates with the (non-)use of other stigmatised variants. Here, the educative variables arguably fulfil a similar function, independently from observing frequency decrease over time. The results thus suggest that prescriptive efforts were, indeed, quite successful.

An ensemble of ordered logistic regression and random forest for child garment size matching

Size fitting is a significant problem for online garment shops. The return rates due to size misfit are very high. We propose an ensemble (with an original and novel definition of the weights) of ordered logistic regression and random forest (RF) for solving the size matching problem, where ordinal data should be classified. These two classifiers are good candidates for combined use due to their complementary characteristics. A multivariate response (an ordered factor and a numeric value assessing the fit) was considered with a conditional random forest. A fit assessment study was carried out with 113 children. They were measured using a 3D body scanner to obtain their anthropometric measurements. Children tested different garments of different sizes, and their fit was assessed by an expert. Promising results have been achieved with our methodology. Two new measures have been introduced based on RF with multivariate responses to gain a better understanding of the data. One of them is an intervention in prediction measure defined locally and globally. It is shown that it is a good alternative to variable importance measures and it can be used for new observations and with multivariate responses. The other proposed tool informs us about the typicality of a case and allows us to determine archetypical observations in each class.

A Machine Learning Approach for Hot-Spot Detection at Protein-Protein Interfaces.

Understanding protein-protein interactions is a key challenge in biochemistry. In this work, we describe a more accurate methodology to predict Hot-Spots (HS) in protein-protein interfaces from their native complex structure compared to previous published Machine Learning (ML) techniques. Our model is trained on a large number of complexes and on a significantly larger number of different structural- and evolutionary sequence-based features. In particular, we added interface size, type of interaction between residues at the interface of the complex, number of different types of residues at the interface and the Position-Specific Scoring Matrix (PSSM), for a total of 79 features. We used twenty-seven algorithms from a simple linear-based function to support-vector machine models with different cost functions. The best model was achieved by the use of the conditional inference random forest (c-forest) algorithm with a dataset pre-processed by the normalization of features and with up-sampling of the minor class. The method has an overall accuracy of 0.80, an F1-score of 0.73, a sensitivity of 0.76 and a specificity of 0.82 for the independent test set.

What Predicts Stroop Performance? A Conditional Random Forest Approach

An experimental science relies on solid and replicable results. The last few years have seen a rich discussion on the reliability and validity of psychological science and whether our experimental findings can falsify our existing theoretical models. But concerns have also arisen that this movement may put a halt to theoretical developments. In this article, we re-analyze the data from an article published in this journal that concluded that lab site did not matter as predictor for Stroop performance, and, therefore, that context was likely to matter little in predicting the outcome of the Stroop task. The authors challenge this conclusion via a new analytical method -- supervised machine learning -- that “lets the data speak”. The authors apply this approach to the Stroop task from Many Labs 3 to illustrate the utility of machine learning, and find surprising results. The authors discuss differences with some conclusions of the original article, which variables need to be controlled for in future inhibitory control tasks, and why psychologists can use machine learning to find surprising, yet solid, results in their own data.

Caries Risk Assessment Item Importance: Risk Designation and Caries Status in Children under Age 6.

Caries risk assessment (CRA) is widely recommended for dental caries management. Little is known regarding how practitioners use individual CRA items to determine risk and which individual items independently predict clinical outcomes in children under age 6-years. Assess the relative importance of pediatric CRA items in dental providers' decision-making regarding patient risk and in association with clinically evident caries, cross-sectionally and longitudinally. CRA information was abstracted retrospectively from electronic patient records of children initially ages 6-72 months at a university pediatric dentistry clinic (N=3810 baseline; N=1315 with follow-up). The 17-item CRA form included caries risk indicators, caries protective items, and clinical indicators. Conditional random forests classification trees were implemented to identify and assign variable importance to CRA items independently associated with baseline high-risk designation, baseline evident tooth decay, and follow-up evident decay. Thirteen individual CRA items, including all clinical indicators and all but one risk indicator, were independently and statistically significantly associated with student/resident providers' caries-risk designation. Provider-assigned baseline risk category was strongly associated with follow-up decay, which increased from low (20.4%), moderate (30.6%), to high/extreme risk patients (68.7%). Of baseline CRA items, before adjustment 12 were associated with baseline decay and 7 with decay at follow-up; however, in the conditional random forests models, only the clinical indicators (evident decay, dental plaque, and recent restoration placement) and one risk indicator (frequent snacking) were independently and statistically significantly associated with future disease, for which baseline evident decay was the strongest predictor. In this predominantly high-risk population under caries-preventive care, more individual CRA items were independently associated with providers' risk determination than with future caries status. These university dental providers considered many items in decision-making regarding patient risk, suggesting that in turn, these comprehensive CRA forms could also aid individualized care, linking risk assessment to disease management.