Classification Metrics Research Articles

Comparative Analysis of Machine Learning Models for Prediction of Autism Spectrum Disorder Using Screening Data

Autism spectrum disorder (ASD) is a neurological and developmental disorder that affects how people interact with others, communicate, learn, and behave. ASD prediction is difficult because the diagnostic factors may not be based solely on observation. In this research paper, an in-depth comparative analysis of various machine learning models applied to the task of classifying autism traits was presented. Our study aimed to assess the performance of these models within the context of identifying individuals with autism based on relevant features and data. The machine learning models investigated in this study encompassed Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), k-Nearest Neighbours (KNN), Naive Bayes (NB), and Neural Network (NN). The models were evaluated using six essential classification metrics: accuracy, precision, recall, specificity, F1 score, and AUC score. On the training dataset, our results reveal nuanced performance characteristics. SVM and RF excel in precision and recall, showing promise for accurate autism trait classification. KNN exhibits remarkable specificity, suggesting its potential for minimizing false positives. LR and NB demonstrate balanced performance across multiple metrics, while NN exhibits high precision and recall, albeit with higher computational demands. It was concluded that SVM was the best classification model for autism trait classification.

B-122 Improving the Diagnostic Utility of Multiplexed Assays in Laboratory Medicine with Machine Learning

Abstract Background The application of machine learning (ML) and artificial intelligence (AI) in precision medicine is of great interest and continues to grow. ML approaches offer a flexible and data-centric methodology for modeling diagnostic and prognostic trends, capable of identifying complex non-linear patterns and relationships between predictor variables and outcomes within large data sets. Although ML holds great promise, there are several common pitfalls that must be avoided for ML models to provide clinical utility. In the current study, we provide a rigorous and systematic evaluation of the utility of ML algorithms to predict disease in several representative data sets generated via multiplexed biomarker assays. We compare five major classes of ML algorithms compared to more traditional combinatorial analysis (logistic regression) to offer perspective against the current gold standard. Methods We obtained five data sets from published literature that include measurements of multiplexed biomarkers in patients with obstructive sleep apnea, sepsis, and breast and colon cancers in comparison to healthy controls. These datasets were generated by bead-based and planar multiplexed immunoassays. We chose representative ML algorithms from the classes of support vector machines (SVM), random forests (RF), neural networks (NNet), and extreme gradient boosting (xgBoost) in comparison to logistic regression (LR). Datasets were randomly split into internal validation and external validation sets via 10 iterations of permutation analysis. We tuned and validated the models using 5-fold cross validation, then tested those on external subsets. We report the average of several model classification metrics and associated variances across all resampled data sets. Results For the classification of sepsis using inflammatory cytokines measured on the Luminex bead-based immunoassay, RF outperformed LR in external validation sets by a 9.8% increase in AUROC. For classification of breast cancer patients, xgBoost and RF performed best with 20.0% and 16.9% increases in AUROC, respectively, whereas for classification of colon cancer patients, NNet outperformed LR by 11.5%. Across all datasets, on average, all ML algorithms outperformed LR in the internal validation sets by a 7.9% (6.3% - 9.9%) increase in AUROC. However, in external validation sets, only xgBoost and SVM outperformed LR, where RF and NNet showed signs of overfitting. Conclusions Overall, certain ML algorithms show improvement over LR for diagnostic applications using multiplexed assays, by up to 20% increase in AUROC. Interpretation of ML results must be performed rigorously to prevent overly optimistic and unrealistic conclusions. In the context of laboratory medicine, our results showcase the utility of ML in clinical applications, while highlighting potential disadvantages of ML. This analysis underscores the need for establishing rigorous guidelines to support data analytics during the development of novel multiplexed assays, which is needed to advance precision medicine.

Joint Image Processing with Learning-Driven Data Representation and Model Behavior for Non-Intrusive Anemia Diagnosis in Pediatric Patients.

Anemia diagnosis is crucial for pediatric patients due to its impact on growth and development. Traditional methods, like blood tests, are effective but pose challenges, such as discomfort, infection risk, and frequent monitoring difficulties, underscoring the need for non-intrusive diagnostic methods. In light of this, this study proposes a novel method that combines image processing with learning-driven data representation and model behavior for non-intrusive anemia diagnosis in pediatric patients. The contributions of this study are threefold. First, it uses an image-processing pipeline to extract 181 features from 13 categories, with a feature-selection process identifying the most crucial data for learning. Second, a deep multilayered network based on long short-term memory (LSTM) is utilized to train a model for classifying images into anemic and non-anemic cases, where hyperparameters are optimized using Bayesian approaches. Third, the trained LSTM model is integrated as a layer into a learning model developed based on recurrent expansion rules, forming a part of a new deep network called a recurrent expansion network (RexNet). RexNet is designed to learn data representations akin to traditional deep-learning methods while also understanding the interaction between dependent and independent variables. The proposed approach is applied to three public datasets, namely conjunctival eye images, palmar images, and fingernail images of children aged up to 6 years. RexNet achieves an overall evaluation of 99.83 ± 0.02% across all classification metrics, demonstrating significant improvements in diagnostic results and generalization compared to LSTM networks and existing methods. This highlights RexNet's potential as a promising alternative to traditional blood-based methods for non-intrusive anemia diagnosis.

Deep-Learning-Based Disease Classification in Patients Undergoing Cine Cardiac MRI.

Automated approaches may allow for fast, reproducible clinical assessment of cardiovascular diseases from MRI. To develop an MRI-based deep learning (DL) disease classification algorithm to distinguish among normal subjects (NORM), patients with dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy (HCM), and ischemic heart disease (IHD). Retrospective. A total of 1337 subjects (55% female), comprising normal subjects (N = 568), and patients with DCM (N = 151), HCM (N = 177), and IHD (N = 441). Balanced steady-state free precession cine sequence at 1.5/3.0 T. Bi-ventricular morphological and functional features and global and segmental left ventricular strain features were automatically extracted from short- and long-axis cine images. Variational autoencoder models were trained on the extracted features and compared against consensus disease label provided by two expert readers (13 and 14 years of experience). Adding unlabeled, normal data to the training was explored to increase specificity of NORM class. Tenfold cross-validation for model development; mean, standard deviation (SD) for measurements; classification metrics: area under the curve (AUC), confusion matrix, accuracy, specificity, precision, recall; 95% confidence intervals; Mann-Whitney U test for significance. AUCs of 0.952 for NORM, 0.881 for DCM, 0.908 for HCM, and 0.856 for IHD and overall accuracy of 0.778 were obtained, with specificity of 0.908 for the NORM class using both SAX and LAX features. Longitudinal strain features slightly improved classification metrics by 0.001 to 0.03 points, except for HCM-AUC. Differences in accuracy, metrics for NORM class and HCM-AUC were statistically significant. Cotraining using unlabeled data increased the specificity for the NORM class to 0.961. Cardiac function features automatically extracted from cine MRI have potential to be used for disease classification, especially for normal-abnormal classification. Feature analyses showed that strain features were important for disease labeling. Cotraining using unlabeled data may help to increase specificity for normal-abnormal classification. 3 TECHNICAL EFFICACY: Stage 1.

Research on Prediction Model of Benign and Malignant Breast Cancer Based on Machine Learning

This study uses three machine learning models to perform classification analysis on breast cancer data, aiming to improve diagnostic accuracy. After data preprocessing and standardization, the performance of each model was comprehensively evaluated using classification and regression metrics. The results showed that the logistic regression model performed the best, with an accuracy of 0.9825, while SVM and random forest also showed good performance. The classification effects of each model were visualized through ROC curves and confusion matrices, demonstrating that logistic regression has high application value in breast cancer diagnosis.

An Improved Deep Learning Framework for Multimodal Medical Data Analysis

Lung disease is one of the leading causes of death worldwide. This emphasizes the need for early diagnosis in order to provide appropriate treatment and save lives. Physicians typically require information about patients’ clinical symptoms, various laboratory and pathology tests, along with chest X-rays to confirm the diagnosis of lung disease. In this study, we present a transformer-based multimodal deep learning approach that incorporates imaging and clinical data for effective lung disease diagnosis on a new multimodal medical dataset. The proposed method employs a cross-attention transformer module to merge features from the heterogeneous modalities. Then unified fused features are used for disease classification. The experiments were performed and evaluated on several classification metrics to illustrate the performance of the proposed approach. The study’s results revealed that the proposed method achieved an accuracy of 95% in terms of accurate classification of tuberculosis and outperformed other traditional fusion methods on multimodal tuberculosis data used in this study.

Development, study, and comparison of models of cross-immunity to the influenza virus using statistical methods and machine learning.

The World Health Organization considers the values of antibody titers in the hemagglutination inhibition assay as one of the most important criteria for assessing successful vaccination. Mathematical modeling of cross-immunity allows for identification on a real-time basis of new antigenic variants, which is of paramount importance for human health. This study uses statistical methods and machine learning techniques from simple to complex: logistic regression model, random forest method, and gradient boosting. The calculations used the AAindex matrices in parallel to the Hamming distance. The calculations were carried out with different types and values of antigenic escape thresholds, on four data sets. The results were compared using common binary classification metrics. Significant differentiation is shown depending on the data sets used. The best results were demonstrated by all three models for the forecast autumn season of 2022, which were preliminary trained on the February season of the same year (Auroc 0.934; 0.958; 0.956, respectively). The lowest results were obtained for the entire forecast year 2023, they were set up on data from two seasons of 2022 (Aucroc 0.614; 0.658; 0.775). The dependence of the results on the types of thresholds used and their values turned out to be insignificant. The additional use of AAindex matrices did not significantly improve the results of the models without introducing significant deterioration. More complex models show better results. When developing cross-immunity models, testing on a variety of data sets is important to make strong claims about their prognostic robustness.

Optimizing poultry audio signal classification with deep learning and burn layer fusion

This study introduces a novel deep learning-based approach for classifying poultry audio signals, incorporating a custom Burn Layer to enhance model robustness. The methodology integrates digital audio signal processing, convolutional neural networks (CNNs), and the innovative Burn Layer, which injects controlled random noise during training to reinforce the model's resilience to input signal variations. The proposed architecture is streamlined, with convolutional blocks, densely connected layers, dropout, and an additional Burn Layer to fortify robustness. The model demonstrates efficiency by reducing trainable parameters to 191,235, compared to traditional architectures with over 1.7 million parameters. The proposed model utilizes a Burn Layer with burn intensity as a parameter and an Adamax optimizer to optimize and address the overfitting problem. Thorough evaluation using six standard classification metrics showcases the model's superior performance, achieving exceptional sensitivity (96.77%), specificity (100.00%), precision (100.00%), negative predictive value (NPV) (95.00%), accuracy (98.55%), F1 score (98.36%), and Matthew’s correlation coefficient (MCC) (95.88%). This research contributes valuable insights into the fields of audio signal processing, animal health monitoring, and robust deep-learning classification systems. The proposed model presents a systematic approach for developing and evaluating a deep learning-based poultry audio classification system. It processes raw audio data and labels to generate digital representations, utilizes a Burn Layer for training variability, and constructs a CNN model with convolutional blocks, pooling, and dense layers. The model is optimized using the Adamax algorithm and trained with data augmentation and early-stopping techniques. Rigorous assessment on a test dataset using standard metrics demonstrates the model's robustness and efficiency, with the potential to significantly advance animal health monitoring and disease detection through audio signal analysis.

Bayesian optimized multimodal deep hybrid learning approach for tomato leaf disease classification

Manual identification of tomato leaf diseases is a time-consuming and laborious process that may lead to inaccurate results without professional assistance. Therefore, an automated, early, and precise leaf disease recognition system is essential for farmers to ensure the quality and quantity of tomato production by providing timely interventions to mitigate disease spread. In this study, we have proposed seven robust Bayesian optimized deep hybrid learning models leveraging the synergy between deep learning and machine learning for the automated classification of ten types of tomato leaves (nine diseased and one healthy). We customized the popular Convolutional Neural Network (CNN) algorithm for automatic feature extraction due to its ability to capture spatial hierarchies of features directly from raw data and classical machine learning techniques [Random Forest (RF), XGBoost, GaussianNB (GNB), Support Vector Machines (SVM), Multinomial Logistic Regression (MLR), K-Nearest Neighbor (KNN)], and stacking for classifications. Additionally, the study incorported a Boruta feature filtering layer to capture the statistically significant features. The standard, research-oriented PlantVillage dataset was used for the performance testing, which facilitates benchmarking against prior research and enables meaningful comparisons of classification performance across different approaches. We utilized a variety of statistical classification metrics to demonstrate the robustness of our models. Using the CNN-Stacking model, this study achieved the highest classification performance among the seven hybrid models. On an unseen dataset, this model achieved average precision, recall, f1-score, mcc, and accuracy values of 98.527%, 98.533%, 98.527%, 98.525%, and 98.268%, respectively. Our study requires only 0.174 s of testing time to correctly identify noisy, blurry, and transformed images. This indicates our approach's time efficiency and generalizability in images captured under challenging lighting conditions and with complex backgrounds. Based on the comparative analysis, our approach is superior and computationally inexpensive compared to the existing studies. This work will aid in developing a smartphone app to offer farmers a real-time disease diagnosis tool and management strategies.

Are light curve classification metrics good proxies for SN Ia cosmological constraining power?

When selecting a lc classifier for use as part of a photometric supernova Ia (SN Ia) cosmological analysis, it is common to make decisions based on metrics of classification performance, such as the contamination within the photometrically classified SN Ia sample, rather than a measure of cosmological constraining power. If the former is an appropriate proxy for the latter, this practice would save those designing an analysis pipeline from eliminate the computational expense of a full cosmology forecast in the analysis pipeline design process This study tests the assumption that lc classification metrics are an appropriate proxy for cosmology metrics. We emulated photometric SN Ia cosmology lc samples with controlled contamination rates of individual contaminant classes and evaluated each of them under a set of classification metrics. We then derived cosmological parameter constraints from all samples under two common analysis approaches and quantified the impact of contamination by each contaminant class on the resulting cosmological parameter estimates. We observe that cosmology metrics are sensitive to both the contamination rate and the class of the contaminating population, whereas the classification metrics are shown to be insensitive to the latter. Based on these findings, we discourage any exclusive reliance on classification-based metrics for analysis design decisions, which (counterintuitively) include but are not limited to the classifier choice. Instead, we recommend optimising science analysis pipeline design choices using a metric of the information gained about the physical parameters of interest.

Development of Machine Learning Models for Predicting the 1-Year Risk of Reoperation After Lower Limb Oncological Resection and Endoprosthetic Reconstruction Based on Data From the PARITY Trial.

Oncological resection and reconstruction involving the lower extremities commonly lead to reoperations that impact patient outcomes and healthcare resources. This study aimed to develop a machine learning (ML) model to predict this reoperation risk. This study was conducted according to TRIPOD + AI. Data from the PARITY trial was used to develop ML models to predict the 1-year reoperation risk following lower extremity oncological resection and reconstruction. Six ML algorithms were tuned and calibrated based on fivefold cross-validation. The best-performing model was identified using classification and calibration metrics. The polynomial support vector machine (SVM) model was chosen as the best-performing model. During internal validation, the SVM exhibited an AUC-ROC of 0.73 and a Brier score of 0.17. Using an optimal threshold that balances all quadrants of the confusion matrix, the SVM exhibited a sensitivity of 0.45 and a specificity of 0.81. Using a high-sensitivity threshold, the SVM exhibited a sensitivity of 0.68 and a specificity of 0.68. Total operative time was the most important feature for reoperation risk prediction. The models may facilitate reoperation risk stratification, allowing for better patient counseling and for physicians to implement measures that reduce surgical risks.

Comparison of a machine learning model with a conventional rule-based selective dry cow therapy algorithm for detection of intramammary infections

We trained machine learning models to identify intramammary infections (IMI) in late lactation cows at dry-off to guide antibiotic treatment, and compared their performance to a rule-based algorithm that is currently used on dairy farms in the US. We conducted an observational test-characteristics study using a data set of 3,645 cows approaching dry-off from 68 US dairy herds. The outcome variables of interest were cow-level IMI caused by all pathogens, major pathogens, and Streptococcus and Strep-like organisms (SSLO), which were determined using aerobic culture of aseptic quarter-milk samples and identification of isolates using MALDI-TOF. Individual cow records were extracted from the farm software to create 53 feature variables at the cow and 39 at the herd-level which were derived from cow-level descriptive data, records of clinical mastitis events, results from routine testing of milk for volume and concentrations of somatic cell count (SCC), fat, and protein. ML algorithms evaluated were logistic regression, decision tree, random forest, light gradient-boosting machine, naïve bayes, and neural networks. For comparison, cows were also classified according to a conventional rule-based algorithm that considered a cow as high risk for IMI if she had at one or more high SCC (>200,000 cells/ml) tests or ≥2 cases of clinical mastitis during the lactation of enrollment. Area under the curve (AUC) and Youden's index were used to compare models, in addition to binary classification metrics, including sensitivity, specificity, and predictive values. ML models had slightly higher AUC and Youden's index values than the rule-based algorithm for all IMI outcomes of interest. However, these improvements in prediction accuracy were substantially less than what we had considered necessary for the technology to be a worthwhile alternative to the rule-based algorithm. Therefore, evidence is lacking to support the wholesale use of ML-guided selective dry cow therapy at the moment. We recommend that producers wanting to implement algorithm-guided SDCT use a rule-based method.

Comparison of blood-based liver fibrosis scores in the Mount Sinai Health System, MASLD Registry, and NHANES 2017-2020 study.

Liver fibrosis is a critical public health concern, necessitating early detection to prevent progression. This study evaluates the recently developed LiverRisk score and steatosis-associated Fibrosis Estimator (SAFE) score against established indices for prognostication and/or fibrosis prediction in 4diverse cohorts, including participants with metabolic dysfunction-associated steatotic liver disease (MASLD). We used data from the Mount Sinai Data Warehouse (32,828 participants without liver disease diagnoses), the Mount Sinai MASLD/MASH Longitudinal Registry (422 participants with MASLD), and National Health and Nutrition Examination Survey 2017-2020 (4133 participants representing the general population) to compare LiverRisk score, FIB-4 index, APRI, and SAFE score. Analyses included Cox proportional hazards regressions, Kaplan-Meier estimates, and classification metrics to evaluate performance in prognostication and fibrosis prediction. In Mount Sinai Data Warehouse, LiverRisk score was significantly associated with future liver-related outcomes but did not significantly outperform FIB-4 or APRI for predicting any of the outcomes. In the general population, LiverRisk score and SAFE score outperformed FIB-4 and APRI in identifying fibrosis, but LiverRisk score underperformed among participants who were non-White or had type 2 diabetes. Among participants with MASLD, SAFE score outperformed FIB-4 and APRI in 1 of 2 cohorts, but there were generally few significant performance differences between all 4 scores. LiverRisk score does not consistently outperform existing predictors in diverse populations, and further validation is needed before adoption in settings with significant differences from the original derivation cohorts. It remains necessary to replicate the ability of these scores to predict liver-specific mortality, as well as to develop diagnostic tools for liver fibrosis that are accessible and substantially better than current scores, especially among patients with MASLD and other chronic liver conditions.

MEMPSEP‐I. Forecasting the Probability of Solar Energetic Particle Event Occurrence Using a Multivariate Ensemble of Convolutional Neural Networks

AbstractThe Sun continuously affects the interplanetary environment through a host of interconnected and dynamic physical processes. Solar flares, Coronal Mass Ejections (CMEs), and Solar Energetic Particles (SEPs) are among the key drivers of space weather in the near‐Earth environment and beyond. While some CMEs and flares are associated with intense SEPs, some show little to no SEP association. To date, robust long‐term (hours‐days) forecasting of SEP occurrence and associated properties (e.g., onset, peak intensities) does not effectively exist and the search for such development continues. Through an Operations‐2‐Research support, we developed a self‐contained model that utilizes a comprehensive data set and provides a probabilistic forecast for SEP event occurrence and its properties. The model is named Multivariate Ensemble of Models for Probabilistic Forecast of Solar Energetic Particles (MEMPSEP). MEMPSEP workhorse is an ensemble of Convolutional Neural Networks that ingests a comprehensive data set (MEMPSEP‐III by Moreland et al. (2024, https://doi.org/10.1029/2023SW003765)) of full‐disc magnetogram‐sequences and in situ data from different sources to forecast the occurrence (MEMPSEP‐I—this work) and properties (MEMPSEP‐II by Dayeh et al. (2024, https://doi.org/10.1029/2023SW003697)) of a SEP event. This work focuses on estimating true SEP occurrence probabilities achieving a 2.5% improvement in reliability and a Brier score of 0.14. The outcome provides flexibility for the end‐users to determine their own acceptable level of risk, rather than imposing a detection threshold that optimizes an arbitrary binary classification metric. Furthermore, the model‐ensemble, trained to utilize the large class‐imbalance between events and non‐events, provides a clear measure of uncertainty in our forecast.

Two-step deep learning models for detection and identification of the manufacturers and types of dental implants on panoramic radiographs.

The purpose of this study is to develop two-step deep learning models that can automatically detect implant regions on panoramic radiographs and identify several types of implants. A total of 1,574 panoramic radiographs containing 3675 implants were included. The implant manufacturers were Kyocera, Dentsply Sirona, Straumann, and Nobel Biocare. Model A was created to detect oral implants and identify the manufacturers using You Only Look Once (YOLO) v7. After preparing the image patches that cropped the implant regions detected by model A, model B was created to identify the implant types per manufacturer using EfficientNet. Model A achieved very high performance, with recall of 1.000, precision of 0.979, and F1 score of 0.989. It also had accuracy, recall, precision, and F1 score of 0.98 or higher for the classification of the manufacturers. Model B had high classification metrics above 0.92, exception for Nobel's class 2 (Parallel). In this study, two-step deep learning models were built to detect implant regions, identify four manufacturers, and identify implant types per manufacturer.

Leakage Identification of Underground Structures Using Classification Deep Neural Networks and Transfer Learning.

Water leakage defects often occur in underground structures, leading to accelerated structural aging and threatening structural safety. Leakage identification can detect early diseases of underground structures and provide important guidance for reinforcement and maintenance. Deep learning-based computer vision methods have been rapidly developed and widely used in many fields. However, establishing a deep learning model for underground structure leakage identification usually requires a lot of training data on leakage defects, which is very expensive. To overcome the data shortage, a deep neural network method for leakage identification is developed based on transfer learning in this paper. For comparison, four famous classification models, including VGG16, AlexNet, SqueezeNet, and ResNet18, are constructed. To train the classification models, a transfer learning strategy is developed, and a dataset of underground structure leakage is created. Finally, the classification performance on the leakage dataset of different deep learning models is comparatively studied under different sizes of training data. The results showed that the VGG16, AlexNet, and SqueezeNet models with transfer learning can overall provide higher and more stable classification performance on the leakage dataset than those without transfer learning. The ResNet18 model with transfer learning can overall provide a similar value of classification performance on the leakage dataset than that without transfer learning, but its classification performance is more stable than that without transfer learning. In addition, the SqueezeNet model obtains an overall higher and more stable performance than the comparative models on the leakage dataset for all classification metrics.

Optimizing machine learning algorithms for diabetes data: A metaheuristic approach to balancing and tuning classifiers parameters

Diabetes mellitus poses a global health concern, prompting the development of machine learning algorithms designed to construct a model for the accurate classification of patients, enabling precise diagnoses and early-stage treatment. However, the efficacy of classifying diabetes patients through machine learning relies on datasets, often plagued by imbalance, leading to biased classification and inaccurate diagnoses. Previous research attempts, employing techniques like random sampling (under-sampling and oversampling) and the Synthetic Minority Oversampling Technique (SMOTE), have struggled to achieve optimally balanced datasets. Additionally, setting the best parameters for machine learning classifiers remains a challenging task. To address these issues, this research focuses on devising a methodological metaheuristic optimization, a machine learning algorithm tailored for diabetes data balancing, and classifier hyperparameter tuning. Leveraging Particle Swarm Optimization (PSO) algorithm for diabetes data balancing and a genetic algorithm to select the optimal architecture for various machine learning classifiers. The study compares the performance of the proposed metaheuristic data balancer and classifier architecture parameter tuner using classification metrics (F1 score, Average Precision–Recall (APR), AUC, and accuracy). The PSO balanced dataset emerges as the most effective in classifying diabetes, with an Average Percentage Improvement (API) in classification performance metrics: 20.78% accuracy, 16.79% area under the curve for receiver operating characteristics, and a significant 32.78% enhancement in APR. Moreover, the XGBOOST classifier trained with a genetic algorithm demonstrates minimal computational training time for the Centre for Disease Control and Prevention (CDC) diabetes dataset compared to the artificial neural network and random forest classifier. Notably, the imbalanced CDC diabetes dataset exhibits the least APR compared to random under-sampling and the PSO data balancing technique.

Transformer-based deep learning networks for fault detection, classification, and location prediction in transmission lines

ABSTRACT Fault detection, classification, and location prediction are crucial for maintaining the stability and reliability of modern power systems, reducing economic losses, and enhancing system protection sensitivity. This paper presents a novel Hierarchical Deep Learning Approach (HDLA) for accurate and efficient fault diagnosis in transmission lines. HDLA leverages two-stage transformer-based classification and regression models to perform Fault Detection (FD), Fault Type Classification (FTC), and Fault Location Prediction (FLP) directly from synchronized raw three-phase current and voltage samples. By bypassing the need for feature extraction, HDLA significantly reduces computational complexity while achieving superior performance compared to existing deep learning methods. The efficacy of HDLA is validated on a comprehensive dataset encompassing various fault scenarios with diverse types, locations, resistances, inception angles, and noise levels. The results demonstrate significant improvements in accuracy, recall, precision, and F1-score metrics for classification, and Mean Absolute Errors (MAEs) and Root Mean Square Errors (RMSEs) for prediction, showcasing the effectiveness of HDLA for real-time fault diagnosis in power systems.

Performance of evaluation metrics for classification in imbalanced data

Performance of evaluation metrics for classification in imbalanced data

Comparing discrete choice and machine learning models in predicting destination choice

Destination choice modeling has long been dominated by theory-based discrete choice models. Simultaneously, machine learning has demonstrated improved predictive performance to other fields of discrete choice modeling. The objective of this research was to compare machine learning models and a multinomial logit model in predicting destination choice. The models were assessed on their predictive performance using metrics for both binary classification and probabilistic classification. The results indicate that machine learning models, especially a random forest model, could bring improvements in prediction accuracy. The more data was used in training the models, the better the machine learning models tended to perform compared to the multinomial logit model. With less data, the multinomial logit model performed comparatively well. The findings are relevant for the field of destination choice modeling, where evidence on the use of machine learning models is very limited. In addition, the unbalanced choice sets of destination choice models with multiple non-chosen alternatives increases the need for further research in model fit and parameter tuning.