Multivariate Classification Research Articles

“Identifying and characterizing scene representations relevant for categorization behavior”

Abstract Scene recognition is a core sensory capacity that enables humans to adaptively interact with their environment. Despite substantial progress in the understanding of the neural representations underlying scene recognition, the relevance of these representations for behavior given varying task demands remains unknown. To address this, we aimed to identify behaviorally relevant scene representations, to characterize them in terms of their underlying visual features, and to reveal how they vary across different tasks. We recorded fMRI data while human participants viewed scenes and linked brain responses to behavior in three tasks acquired in separate sessions: man-made/natural categorization, basic-level categorization, and fixation color discrimination. We found correlations between categorization response times and scene-specific brain responses, quantified as the distance to a hyperplane derived from a multivariate classifier. Across tasks, these effects were found in largely distinct parts of the ventral visual stream. This suggests that different scene representations are relevant for behavior depending on the task. Next, using deep neural networks as a proxy for visual feature representations, we found that intermediate layers mediated the relationship between scene representations and behavior for both categorization tasks, indicating a contribution of mid-level visual features to these representations. Finally, we observed opposite patterns of brain-behavior correlations in the man-made/natural and the fixation task, indicating interference of representations with behavior for task demands that do not align with the content of representations. Together, these results reveal the spatial extent, content, and task-dependence of the visual representations that mediate behavior in complex scenes.

Prior memory responses modulate behavior and brain state engagement

Memory encoding and retrieval constitute neurally dissociable brain states and prior behavioral work suggests that these states may linger in time. Thus memory states may influence both the current experience and subsequent events; however, this account has not been directly tested. To test the hypothesis that memory judgments induce brain states that persist for several hundred milliseconds, we recorded scalp electroencephalography while participants completed a recognition task. We used an independently validated multivariate mnemonic state classifier to assess memory state engagement. We replicate previous behavioral findings, yet we find that memory states are modulated by response congruency. We find strong retrieval state engagement on incongruent trials, when the response switches between two consecutive trials. These findings indicate that cortical brain states are influenced by prior judgments and suggest that a non-mnemonic, internal attention state may be recruited in the face of changing demands in a dynamic environment.

Authenticity Verification of Commercial Poppy Seed Oil Using FT-IR Spectroscopy and Multivariate Classification

Authenticating poppy seed oil is essential to ensure product quality and prevent economic and health-related fraud. This study developed a non-targeted approach using FT-IR spectroscopy and pattern recognition analysis to verify the authenticity of poppy seed oil. Thirty-nine poppy seed oil samples were sourced from online stores and local markets in Turkiye. Gas chromatography–Flame Ionization Detector (GC-FID) analysis revealed adulteration in 23% of the samples, characterized by unusual fatty acid composition. Spectra of the oil samples were captured with a portable 5-reflection FT-IR sensor. Soft Independent Model of Class Analogies (SIMCA) was used to create class algorithms, successfully detecting all instances of adulteration. Partial least square regression (PLSR) models were then developed to predict the predominant fatty acid composition, achieving strong external validation performance (RCV = 0.96–0.99). The models exhibited low standard errors of prediction (SEP = 0.03–1.40%) and high predictive reliability (RPD = 2.9–6.1; RER = 8.4–13.1). This rapid, non-destructive method offers a reliable solution for authenticating poppy seed oil and predicting its fatty acid composition, presenting valuable applications for producers and regulatory authorities. This approach aids in regulatory compliance, protection of public health, and strengthening of consumer confidence by ensuring the authenticity of the product.

Multivariate time series classification based on spatial-temporal attention dynamic graph neural network

Multivariate time series classification based on spatial-temporal attention dynamic graph neural network

ST-Tree with interpretability for multivariate time series classification

Multivariate time series classification is of great importance in practical applications and is a challenging task. However, deep neural network models such as Transformers exhibit high accuracy in multivariate time series classification but lack interpretability and fail to provide insights into the decision-making process. On the other hand, traditional approaches based on decision tree classifiers offer clear decision processes but relatively lower accuracy. Swin Transformer (ST) addresses these issues by leveraging self-attention mechanisms to capture both fine-grained local patterns and global patterns. It can also model multi-scale feature representation learning, thereby providing a more comprehensive representation of time series features. To tackle the aforementioned challenges, we propose ST-Tree with interpretability for multivariate time series classification. Specifically, the ST-Tree model combines ST as the backbone network with an additional neural tree model. This integration allows us to fully leverage the advantages of ST in learning time series context while providing interpretable decision processes through the neural tree. This enables researchers to gain clear insights into the model’s decision-making process and extract meaningful interpretations. Through experimental evaluations on 10 UEA datasets, we demonstrate that the ST-Tree model improves accuracy in multivariate time series classification tasks and provides interpretability through visualizing the decision-making process across different datasets.

An adaptive assessment method of power system transient stability considering PMU data loss

AbstractTransient stability assessment (TSA) plays an important role in ensuring the reliable operation of power systems. With the popularity of phasor measurement units (PMUs), data‐driven TSA methods have been widely concerned. However, the performance of TSA model may deteriorate when data loss occurs due to PMU failure. This paper proposes an adaptive assessment method for transient stability of power systems considering PMU data loss. First, considering the importance of temporal features, a collection of PMU clusters is constructed to minimize the failure risk and maintain full observability of the whole buses of the grid. Secondly, a weighted integrated assessment model based on PMU clusters is constructed by using an improved eXplainable Convolutional neural network for Multivariate time series classification (XCM) as a TSA classifier. The model can make full use of time series information to carry out adaptive TSA and maintain the robustness of the assessment performance even when PMU failure occurs. Finally, it is verified in a modified IEEE 39‐bus system with wind power and solar power. The effect of the proposed method shows high accuracy and strong anti‐noise interference ability in case of data loss.

Clinical Outcomes and Predictors of Poor Prognosis in Non-aneurysmal Subarachnoid Hemorrhage: A 10-Year Cohort Analysis.

Introduction A large majority of spontaneoussubarachnoid hemorrhages (SAH) are attributed to aneurysm rupture, though the cause remains unknown in a notable percentage of cases. Non-aneurysmal SAH (naSAH) is generally thought to follow a more benign clinical course than aneurysmal SAH (aSAH); however, similar complications may occur, and poor outcomes are still possible. Given the limited research on naSAH, this study aims to characterize these patients and correlate clinical and radiographic findings with outcomes. Methods A cohort of 149 patients with naSAH was selected from 2014 to 2023. Outcomes were assessed using the modified Rankin Scale (mRS), categorizing results as favorable (mRS 0-2) or unfavorable (mRS 3-6). Descriptive analysis was conducted, dividing the sample into two groups based on blood distribution on computed tomography scan: pretruncal (pnSAH) and non-pretruncal SAH (npnSAH). Associations between variables were tested, and a multivariable logistic regression was performed to identify significant predictors. Results The mean age was 57.54 years, with 79 males (53.0%) and 70 females (47.0%). A favorable outcome was observed in 133 patients (89.3%).Age, chronic hypertension, anticoagulant, and antiplatelettherapy were significant predictors of poor outcome(p < 0.05). A favorable World Federation of Neurosurgical Societies (WFNS) grade (I-III) was recorded in 88.6% of patients and was significantly associated with outcome (p < 0.05). Among the patients with unfavorable outcomes, 87.5%exhibited a npnSAH pattern. Rebleeding was rare, with only one case (0.7%). Acute hydrocephalus was the primary complication observed in naSAH cases (19.5%). Conclusions Patients with a npnSAH pattern were significantly more likely to experience unfavorable outcomes compared to those with a pnSAH pattern. In multivariate regression analysis, WFNS classification, bleeding pattern (pnSAH vs. npnSAH), and acute hydrocephalus were identified as independent predictors of poor outcomes.

A neuronal marker of eye contact spontaneously activated in neurotypical subjects but not in autistic spectrum disorders

Attention to faces and eye contact are key behaviors for establishing social bonds in humans. In Autism Spectrum Disorders (ASD), a disturbance in neurodevelopment, impaired face processing and gaze avoidance are key clinical features for ASD diagnosis. The biological alterations underlying these impairments are not yet clearly established. Using high-density electroencephalography coupled with multi-variate pattern classification and group blind source separation methods we searched for face- and-face components-related neural signals that could best discriminate visual processing of neurotypical subjects (N = 38) from ASD participants (N = 27). We isolated a face-specific neural signal in the superior temporal sulcus peaking at 240 msec after face-stimulus onset. A machine learning algorithm applied on the extracted neural component reached 74% decoding accuracy at the same latencies, discriminating the neurotypical population from ASD subjects in whom this signal was weak. By manipulating attention on different parts of the face, we also found that the power of the evoked signal in neurotypical subjects varied depending on the region observed: it was strong when the eye region fell on the fovea to decrease on regions further away and outside the stimulus face. Such face and face-components selective neural modulations were not found in ASD, although they did show typical early face-related P100 and N170 signals. These results show that specialized cortical mechanisms for face perception show higher responses for eyes when attention is focused on gaze and that these mechanisms may be particularly affected in autism spectrum disorders.

Modality-specific and modality-general representations of subjective value in frontal cortex

Neuroeconomics theories propose that the value associated with diverse rewards or reward-predicting stimuli is encoded along a common reference scale, irrespective of their sensory properties. However, in a dynamic environment with changing stimulus-reward pairings, the brain must also represent the sensory features of rewarding stimuli. The mechanism by which the brain balances these needs—deriving a common reference scale for valuation while maintaining sensitivity to sensory contexts—remains unclear. To investigate this, we conducted an fMRI study with human participants engaged in a dynamic foraging task, which required integrating the reward history of auditory or visual choice options and updating the subjective value for each sensory modality. Univariate fMRI analysis revealed modality-specific value representations in the orbitofrontal cortex (OFC) and modality-general value representations in the ventromedial prefrontal cortex (vmPFC), confirmed by an exploratory multivariate pattern classification approach. Crucially, modality-specific value representations were absent when the task involved instruction-based rather than value-based choices. Effective connectivity analysis showed that modality-specific value representations emerged from selective bidirectional interactions across the auditory and visual sensory cortices, the corresponding OFC clusters, and the vmPFC. These results illustrate how the brain enables a valuation process that is sensitive to the sensory context of rewarding stimuli.

SDKT: Similar Domain Knowledge Transfer for Multivariate Time Series Classification Tasks

ABSTRACTMultivariate time series data classification has a wide range of applications in reality. With rapid development of deep learning, convolutional networks are widely used in this task and have achieved the current best performance. However, due to high difficulty and cost of collecting this type of data, labeled data is still scarce. In some tasks, the model shows overfitting, resulting in relatively poor classification performance. In order to improve the classification performance under such situation, we proposed a novel classification method based on transfer learning—similar domain knowledge transfer (call SDKT for short). Firstly, we designed a multivariate time series domain distance calculation method (call MTSDDC for short), which helped selecting the source domain that is most similar to target domain; Secondly, we used ResNet as a pre‐trained classifier, transferred the parameters of the similar domain network to the target domain network and continue to fine‐tune the parameters. To verify our method, we conducted experiments on several public datasets. Our study has also shown that the transfer effect from the source domain to the target domain is highly negatively correlated with the distance between them, with an average Pearson coefficient of −0.78. For the transfer of most similar source domain, compared to the ResNet model without transfer and the current best model, the average accuracy improvements on the datasets we used are 4.01% and 1.46% respectively.

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Multivariate time series classification (MTSC), which identifies categories of multiple sensor signals recorded in continuous time, is widely used in various fields such as transportation, finance, and medical treatment. The focused challenge remains learning the dependencies between subsequences to capture discriminative patterns while providing convincing explanations. In this paper, we propose a temporal-variable parallel deep learning framework to mine global and local features to achieve a win-win situation in performance and interpretability. Specifically, for harmonizing the inattention blindness of global features, we introduce a graph attention mechanism with global awareness (GAT-g), where the learning of edge representations incorporates both inter-node relationships and the node-to-graph context. Furthermore, for evaluating the feature combinations utility, we exploit game interactions for the first time, which quantifies the utility of feature combination through Shapley values to illustrate the dynamically coordinating representation ability of the model for diverse time series features. In addition, the interpretation module leverages temporal and variable subspace attention distributions to provide instantiated explanations with additive computational complexity, enhancing the comprehension of prediction results. Experimental evaluation on the University of East Anglia (UEA) archive of 30 multivariate time series datasets shows that the proposed method outperforms 12 state-of-the-art methods on 11 datasets.

Identification of Salmonella Serogroups and Distinction Between Typhoidal and Non-Typhoidal Salmonella Based on ATR-FTIR Spectroscopy.

Salmonellosis is the second-most commonly reported foodborne gastrointestinal infection in the European Union and a major contributor to foodborne outbreaks globally. Salmonella serotyping differentiates typhoidal strains requiring antibiotic therapy (e.g., serovars Typhi, Paratyphi A, Paratyphi B-d-tartrate negative, Paratyphi C) from typically self-limiting non-typhoidal Salmonella (NTS) strains, making precise identification essential for appropriate treatment and epidemiological tracking. At the same time, the ability to identify the serogroup of Salmonella, regardless of which of the above two groups it belongs to, provides an important initial epidemiological indication that is useful for case management by competent health authorities. This study evaluates the effectiveness of ATR-FTIR spectroscopy coupled with a machine learning algorithm to identify four key Salmonella enterica serogroups (B, C1, D1-including typhoidal strains such as S. Typhi-and E1) directly from solid monomicrobial cultures without sample pretreatment. The system was paired with I-dOne software v2.2 already able to detect Salmonella spp., possibly leading to the characterisation of both the species and serotype from one colony. The multivariate classification model was trained and validated with 248 strains, with an overall accuracy of >98% over 113 samples. This approach offers a potential rapid alternative for clinical labs without serotyping facilities.

Functional Immunoaffinity 3D Magnetic Core-Shell Nanometallic Structure for High-Efficiency Separation and Label-Free SERS Detection of Exosomes.

Tumor exosomes, known as maternal cell messengers, play an important role in cancer occurrence, proliferation, metastasis, immune escape, drug resistance, and other processes and are an entry point for cancer research. However, there is still a lack of an efficient detection technology for exosomes. In this study, the ultrahigh sensitivity SERS nanoprobe with a three dimensional (3D) magnetic core/Au nanocolumn/Au nanoparticles shell strongly coupling multistage structure (Fe3O4@NR-NPs) was constructed by crystal growth of nanocrystals in the confined space of a central radiating single particle mesoporous molecular sieve channel and strong coupling secondary growth of gold particles. The exosomes were confined onto the "hot spot" of plasmonic nanoparticles and rapidly enriched by CD63 antibody functional-Fe3O4@NR-NPs to achieve high sensitivity detection, with the limit of detection of 1 × 103 particles/mL (S/N = 3). The spectral data set of different exosomes is applied to train for multivariate classification of cell types and to estimate how the normal exosome data resemble cancer cell exosomes by principal component analysis (PCA). Finally, this detection method has also been successfully employed for the detection of exosomes in complex samples; this proves that the proposed SERS-based method is a promising tool for clinical cancer screening.

MagNet: Multilevel Dynamic Wavelet Graph Neural Network for Multivariate Time Series Classification

Multivariate time series classification (MTSC) is a fundamental data mining task, which is widely applied in the fields like health care and energy management. However, the existing MTSC methods are mostly adapted from univariate versions and model the static patterns among series in the time domain. We argue they fail to capture the inter-dependencies across variables and rarely consider the unique dynamic features in multilevel frequencies, which are susceptible to signal noise and lack sufficient feature extraction capability to achieve satisfactory classification accuracy. To address these issues, we propose a novel M ultilevel dyn a mic wavelet g raph neural Net work called MagNet, which effectively captures inherent temporal-frequency dependencies in multivariate time series data in a global view, facilitating the information flow among interrelated variables and leveraging learnable graph neural networks (GNNs) to uncover dynamic frequency dependencies. We propose an orthogonal temporal convolution layer that utilizes soft orthogonal losses to constrain features learned at different frequency components to reduce feature redundancy. Additionally, we introduce a hierarchical graph coarsening operator to address the flat learning challenges in traditional GNNs. Our dynamic wavelet graph neural network and hierarchical coarsening enable deep model stacking and end-to-end learning. Extensive experiments on 30 UEA benchmarks demonstrate that our method outperforms the state-of-the-art baselines in the MTSC tasks.

Total cholesterol and bilirubin levels are associated with neurologic outcomes in patients with out-of-hospital cardiac arrest.

Assessing the neurologic outcomes of patients who experience out of hospital cardiac arrest (OHCA) is challenging. Neurologic outcomes were evaluated using initial nutrition related biochemical markers. We used data from a multicentre retrospective observational study, the Korean Cardiac Arrest Resuscitation Consortium (KoCARC) registry. Among the 666 patients, 217 had good neurologic outcomes, while 449 had poor neurologic outcomes. Multivariate logistic regression and classification and regression tree (CART) analyses were employed. In the multivariate logistic regression analysis, total cholesterol ≥ 158.5mg/dL, total bilirubin ≥ 0.265mg/dL, Sodium < 142.1mEq/L, AST < 200.5 U/L and were identified as significant biomarkers for good neurologic outcomes. In the CART analysis, total cholesterol ≥ 158.5mg/dL and total bilirubin ≥ 0.365mg/dL were found to be significant indicators. In additional analysis, when the total bilirubin level ranged from 0.6 to 0.7mg/dL, the highest rate of a good neurologic outcome was observed at 44.6%, whereas levels below or above this range gradually indicated a lower rate of a good neurologic outcome. We propose that total cholesterol and total bilirubin levels could serve as valuable indicators for predicting neurologic outcomes in patients with OHCA.

Pooled Long-Term Outcomes With Nivolumab Plus Ipilimumab or Nivolumab Alone in Patients With Advanced Melanoma.

Nivolumab (NIVO) + ipilimumab (IPI) combination and NIVO monotherapy have demonstrated durable clinical benefit in patients with unresectable/metastatic melanoma. This analysis describes long-term overall survival (OS) with the combination or monotherapy pooled across all major company-sponsored trials, as well as clinical factors associated with survival, in patients with immune checkpoint inhibitor (ICI) treatment-naïve unresectable/metastatic melanoma. Data were pooled from six CheckMate studies in ICI treatment-naïve patients receiving NIVO + IPI (NIVO 1 mg/kg + IPI 3 mg/kg or NIVO 3 mg/kg + IPI 1 mg/kg) or NIVO monotherapy (3 mg/kg). OS was assessed for each treatment, as well as in select subgroups. Cox proportional multivariate analysis (MVA) and classification and regression tree (CART) analyses were performed within treatment arms. Median follow-up for OS was 45.0 months for patients treated with NIVO + IPI (n = 839) and 35.8 months for patients treated with NIVO (n = 536). OS was longer with NIVO + IPI versus NIVO monotherapy (hazard ratio, 0.78 [95% CI, 0.67 to 0.91]), with 6-year OS rates of 52% versus 41%, respectively. Consistent benefit was observed in BRAF-mutant and BRAF-wild-type patients and those with normal and elevated lactate dehydrogenase (LDH). Numerical difference in OS was also observed across PD-L1 expression levels, although more pronounced with no/low PD-L1 expression. Clinical factors associated with decreased survival in both the MVA and CART analyses were LDH > upper limit of normal with either treatment, age ≥65 years with NIVO + IPI, and the presence of liver metastases with NIVO monotherapy. In this large, pooled nonrandomized retrospective analysis, we observed that NIVO + IPI provides longer OS than NIVO in patients with ICI treatment-naïve advanced melanoma and identifies clinical factors that appear to be associated with survival for each treatment, which may assist with treatment decision making.

A novel two-enhancive aspect module in convolutional neural networks for multivariate time series classification

Multivariate Time Series Classification (MTSC) presents a significant challenge in time series data mining. While many methods have been proposed, Convolutional Neural Networks (CNNs) still face limitations in effectively capturing multivariate dependencies. In this paper, we introduce the Two-Enhancive Aspect Module (TEAM), a novel attention mechanism that integrates both Temporal Attention (TEAM_T) and Channel Attention (TEAM_C) to enhance feature extraction in CNNs. In TEAM_T, the calculation of attention weights considers both the sample’s content and its relative positional placement, forming a pseudo-Gaussian distribution that better reflects the relative importance of time samples. In TEAM_C, the Discrete Cosine Transform (DCT) is used for time-frequency transformation, building channel attention in the frequency domain, it better captures the interdependencies between channels. Evaluated on the UEA benchmark with 15 MTSC datasets, TEAM consistently outperforms 14 baseline methods and 5 alternative attention mechanisms, significantly boosting classification performance and surpassing the current SOTA in mean accuracy.

Investigating the detection of peanuts in chopped nut products using hyperspectral imaging systems

The intentional presence or cross-contamination of peanuts in other nut products can result in severe health problems for consumers allergic to peanuts. It is therefore essential to identify the presence of these allergenic compounds in commercialized nut products prior to their sale. For this purpose, we assessed the performance of a visible near infrared (Vis-NIR) and a shortwave infrared (SWIR) hyperspectral imaging (HSI) systems working in the spectral regions 419–1007 nm and 842–2532 nm, respectively, to identify peanut pieces in different chopped nuts (almonds, hazelnuts and walnuts). Two strategies were evaluated to create the training and validation sets. In Strategy I, these sets were composed of spectra belonging to individual pixels, whereas in Strategy II, the mean spectrum of each individual piece of nut was used. We used partial least squares discriminant analysis (PLS-DA) to develop the classification models, and the results were assessed by means of the values obtained for the sensitivity, specificity, and non-error rate (NER) statistics. The external validation procedure showed excellent classification results, with a NER of 98.3 % and 99.8 % for the Vis-NIR and SWIR systems, respectively, for Strategy I, and 100 % for both systems when Strategy II was followed. These results, therefore, confirm the viability of using HSI technology together with multivariate classification methods to detect peanut pieces in other chopped-nut products.

Clinical and laboratory predictors for bacteremia in critically ill calves.

Sepsis is a main contributor to calf mortality, but diagnosis is difficult. Develop and validate a predictive model for bacteremia in critically ill calves (CIC). A total of 334 CIC, sampled for blood culture. Cross-sectional study. Multivariable logistic regression and classification tree analysis on clinical, ultrasonographic, and laboratory variables were performed on a dataset including all animals. Model validation was done on 30% of the dataset. Similar statistics (except validation) were performed on a subset of the database (n = 143), in which presumed contaminants were excluded. The best performing model to predict bacteremia, taking all detected bacteria into account, included tachypnea, tachycardia, acidemia, hypoglycemia, venous hypoxemia, and hypoproteinemia. Sensitivity and specificity of this model were 70.6% and 98.0%, respectively, but decreased to 61.5% and 91.7% during model validation. The best-performing model, excluding presumed contaminants, included abnormal temperature, heart rate, absence of enteritis, hypocalcemia, and hyperlactatemia as risk factors for bacteremia. Sensitivity and specificity of this model were 71.4% and 93.9%, respectively. Both classification trees performed less well in comparison to logistic regression. The classification tree excluding presumed contaminants, featured hypoglycemia, absence of diarrhea, and hyperlactatemia as risk factors for bacteremia. Sensitivity and specificity were 39.4% and 92.7%, respectively. Hypoglycemia, hyperlactatemia, and hypoproteinemia seem relevant in assessing bacteremia in CIC. The performance of these models based on basic clinical and blood variables remains insufficient to predict bacteremia.

Precise and Accurate Short-term Forecasting of Solar Energetic Particle Events with Multivariate Time-series Classifiers

Solar energetic particle (SEP) events are one of the most crucial aspects of space weather that require continuous monitoring and forecasting using robust methods. We demonstrate a proof of concept of using a data-driven supervised classification framework on a multivariate time-series data set covering solar cycles 22, 23, and 24. We implement ensemble modeling that merges the results from three proton channels (E ≥ 10 MeV, 50 MeV, and 100 MeV) and the long-band X-ray flux (1–8 Å) channel from the Geostationary Operational Environmental Satellite missions. Our task is binary classification, such that the aim of the model is to distinguish strong SEP events from nonevents. Here, strong SEP events are those crossing the Space Weather Prediction Center’s “S1” threshold of solar radiation storm and proton fluxes below that threshold are weak SEP events. In addition, we consider periods of nonoccurrence of SEPs following a flare with magnitudes ≥C6.0 to maintain a natural imbalance of sample distribution. In our data set, there are 244 strong SEP events comprising the positive class. There are 189 weak events and 2460 “SEP-quiet” periods for the negative class. We experiment with summary statistic, one-nearest neighbor, and supervised time-series forest (STSF) classifiers and compare their performance to validate our methods for prediction windows from 5 minutes up to 60 minutes. We find the STSF model to perform better under all circumstances. For an optimal classification threshold of ≈0.3 and a 60 minutes prediction window, we obtain a true skill statistic TSS = 0.850, Heidke skill score HSS = 0.878, and Gilbert skill score GSS = 0.783.