Reliable Machine Research Articles

Custom Loss Functions in XGBoost Algorithm for Enhanced Critical Error Mitigation in Drill-Wear Analysis of Melamine-Faced Chipboard.

The advancement of machine learning in industrial applications has necessitated the development of tailored solutions to address specific challenges, particularly in multi-class classification tasks. This study delves into the customization of loss functions within the eXtreme Gradient Boosting (XGBoost) algorithm, which is a critical step in enhancing the algorithm's performance for specific applications. Our research is motivated by the need for precision and efficiency in the industrial domain, where the implications of misclassification can be substantial. We focus on the drill-wear analysis of melamine-faced chipboard, a common material in furniture production, to demonstrate the impact of custom loss functions. The paper explores several variants of Weighted Softmax Loss Functions, including Edge Penalty and Adaptive Weighted Softmax Loss, to address the challenges of class imbalance and the heightened importance of accurately classifying edge classes. Our findings reveal that these custom loss functions significantly reduce critical errors in classification without compromising the overall accuracy of the model. This research not only contributes to the field of industrial machine learning by providing a nuanced approach to loss function customization but also underscores the importance of context-specific adaptations in machine learning algorithms. The results showcase the potential of tailored loss functions in balancing precision and efficiency, ensuring reliable and effective machine learning solutions in industrial settings.

A machine learning-based nomogram model for predicting the recurrence of cystitis glandularis.

Cystitis glandularis is a chronic inflammatory disease of the urinary system characterized by high recurrence rates, the reasons for which are still unknown. This study aims to identify potential factors contributing to recurrence and propose a simple and feasible prognostic model through nomogram construction. Patients with confirmed recurrence based on outpatient visits or readmissions were included in this study, which was subsequently divided into training and validation cohorts. Machine learning techniques were utilized to screen for the most important predictors, and these were then employed to construct the nomogram. The reliability of the nomogram was assessed through receiver operating characteristic curve analysis, decision curve analysis, and calibration curves. A total of 252 patients met the screening criteria and were enrolled in this study. Over the 12-month follow-up period, the relapse rate was found to be 57.14% (n = 144). The five final predictors identified through machine learning were urinary infections, urinary calculi, eosinophil count, lymphocyte count, and serum magnesium. The area under curve values for all three time points assessing recurrence exceeded 0.75. Furthermore, both calibration curves and decision curve analyses indicated good performance of the nomogram. We have developed a reliable machine learning-based nomogram for predicting recurrence in cystitis glandularis.

Combining Deep Learning with Signal-image Encoding for Multi-Modal Mental Wellbeing Classification

The quantification of emotional states is an important step to understanding wellbeing. Time series data from multiple modalities such as physiological and motion sensor data have proven to be integral for measuring and quantifying emotions. Monitoring emotional trajectories over long periods of time inherits some critical limitations in relation to the size of the training data. This shortcoming may hinder the development of reliable and accurate machine learning models. To address this problem, this article proposes a framework to tackle the limitation in performing emotional state recognition: (1) encoding time series data into coloured images; (2) leveraging pre-trained object recognition models to apply a Transfer Learning (TL) approach using the images from step 1; (3) utilising a 1D Convolutional Neural Network (CNN) to perform emotion classification from physiological data; (4) concatenating the pre-trained TL model with the 1D CNN. We demonstrate that model performance when inferring real-world wellbeing rated on a 5-point Likert scale can be enhanced using our framework, resulting in up to 98.5% accuracy, outperforming a conventional CNN by 4.5%. Subject-independent models using the same approach resulted in an average of 72.3% accuracy (SD 0.038). The proposed methodology helps improve performance and overcome problems with small training datasets.

Uncertainty driven active learning of coarse grained free energy models

Coarse graining techniques play an essential role in accelerating molecular simulations of systems with large length and time scales. Theoretically grounded bottom-up models are appealing due to their thermodynamic consistency with the underlying all-atom models. In this direction, machine learning approaches hold great promise to fitting complex many-body data. However, training models may require collection of large amounts of expensive data. Moreover, quantifying trained model accuracy is challenging, especially in cases of non-trivial free energy configurations, where training data may be sparse. We demonstrate a path towards uncertainty-aware models of coarse grained free energy surfaces. Specifically, we show that principled Bayesian model uncertainty allows for efficient data collection through an on-the-fly active learning framework and opens the possibility of adaptive transfer of models across different chemical systems. Uncertainties also characterize models’ accuracy of free energy predictions, even when training is performed only on forces. This work helps pave the way towards efficient autonomous training of reliable and uncertainty aware many-body machine learned coarse grain models.

Machine Learning in the Parkinson’s disease smartwatch (PADS) dataset

The utilisation of smart devices, such as smartwatches and smartphones, in the field of movement disorders research has gained significant attention. However, the absence of a comprehensive dataset with movement data and clinical annotations, encompassing a wide range of movement disorders including Parkinson’s disease (PD) and its differential diagnoses (DD), presents a significant gap. The availability of such a dataset is crucial for the development of reliable machine learning (ML) models on smart devices, enabling the detection of diseases and monitoring of treatment efficacy in a home-based setting. We conducted a three-year cross-sectional study at a large tertiary care hospital. A multi-modal smartphone app integrated electronic questionnaires and smartwatch measures during an interactive assessment designed by neurologists to provoke subtle changes in movement pathologies. We captured over 5000 clinical assessment steps from 504 participants, including PD, DD, and healthy controls (HC). After age-matching, an integrative ML approach combining classical signal processing and advanced deep learning techniques was implemented and cross-validated. The models achieved an average balanced accuracy of 91.16% in the classification PD vs. HC, while PD vs. DD scored 72.42%. The numbers suggest promising performance while distinguishing similar disorders remains challenging. The extensive annotations, including details on demographics, medical history, symptoms, and movement steps, provide a comprehensive database to ML techniques and encourage further investigations into phenotypical biomarkers related to movement disorders.

Developing a Reliable Hybrid Machine Learning Model for Objective Soccer Player Valuation

Developing a Reliable Hybrid Machine Learning Model for Objective Soccer Player Valuation

Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale (The Short Version)

Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale (The Short Version)

SBTD: Secured Brain Tumor Detection in IoMT Enabled Smart Healthcare.

Brain tumors are fatal and severely disrupt brain function as they advance. Timely detection and precise monitoring are crucial for improving patient outcomes and survival. A smart healthcare system leveraging the Internet of Medical Things (IoMT) revolutionizes patient care by offering streamlined remote healthcare, especially for individuals with acute medical conditions like brain tumors. However, such systems face significant challenges, such as (1) the increasing prevalence of cyber attacks in the expanding digital healthcare landscape, and (2) the lack of reliability and accuracy in existing tumor detection methods. To address these issues, we propose Secured Brain Tumor Detection (SBTD), the first unified system integrating IoMT with secure tumor detection. SBTD features: (1) a robust security framework, grounded in chaos theory, to safeguard medical data; and (2) a reliable machine learning-based tumor detection framework that accurately localizes tumors using their anatomy. Comprehensive experimental evaluations on different multimodal MRI datasets demonstrate the system's suitability, clinical applicability and superior performance over state-of-the-art algorithms.

High-Efficiency Industrial 130 kW cw Solid-State RF Amplifier for 1.3 GHz

A new 1.3 GHz solid-state high-power RF amplifier (SSA) has been built for the Lighthouse project in close cooperation between Cryoelectra and RI Research Instruments. The amplifier was developed by Cryoelectra as a scalable compact system with an RF-power density of 40kW/m2. Its industrial design is very reliable and easy to maintain. The SSA delivers a continuous RF output power of more than 130 kW with a wall-plug efficiency of 63% and with very low phase noise. The power is generated by 40 patented RF amplifier modules each containing 8 GaN transistor units. Their outputs are combined by a coaxial 8-way combiner in the center of the module. Each module is connected to the 4x9-way cable-free waveguide combiner network and can be exchanged in the event of fault within minutes thanks to quick connectors. A sophisticated control system continuously monitors the state of all components for reliable machine and personnel protection.Despite all the challenges encountered during development and production, the SSA was built and handed over to the client in the shortest possible time. It passed the acceptance test on site exceptionally well.

Use of Support Vector Machine(SVM) in Prediction of Success of Wart Treatment Methods

Abstract: Warts are common noncancerous benign tumors caused by the Human Papilloma Virus. They can be treated using various methods, including cryotherapy and immunotherapy. However, the success rates of these treatment methods are not consistent and can vary from patient to patient. To address this issue, we have developed a reliable machine learning model that can accurately predict the success of immunotherapy and cryotherapy for individual patients based on their demographic and clinical characteristics. By utilizing a dataset of 180 patients who received either immunotherapy or cryotherapy for their warts, we employed a support vector machine classifier with a radial basis function kernel for the immunotherapy treatment method (Nugroho et al., 2018) . This model takes into account factors such as sex, age, time, number of warts, type, area, and response to treatment to make predictions about the likelihood of treatment success. Through this model, healthcare professionals can have a better understanding of which treatment method is more likely to be effective for a specific patient. The accuracy of our machine learning model for predicting the success of immunotherapy and cryotherapy for warts is expected to be high, as previous studies using similar datasets have reported classification accuracies of up to 90 (Rahman et al., 2020). Our model builds upon previous research in the field, such as the fuzzy logic rule-based method developed by Khozeimeh et al and the AdaBoost with classification and regression tree and random forest algorithms employed by Putra et al. Using our machine learning model, healthcare professionals can make more informed decisions about the treatment approach for individual patients with warts, increasing the likelihood of successful outcomes and improving overall patient care. In conclusion, our machine learning model offers a promising approach to predict the success of immunotherapy and cryotherapy treatments for warts.

Spectroscopy-Guided Deep Learning Predicts Solid-Liquid Surface Adsorbate Properties in Unseen Solvents.

Accurately and rapidly acquiring the microscopic properties of a material is crucial for catalysis and electrochemistry. Characterization tools, such as spectroscopy, can be a valuable tool to infer these properties, and when combined with machine learning tools, they can theoretically achieve fast and accurate prediction results. However, on the path to practical applications, training a reliable machine learning model is faced with the challenge of uneven data distribution in a vast array of non-negligible solvent types. Herein, we employ a combination of the first-principles-based approach and data-driven model. Specifically, we utilize density functional theory (DFT) to calculate theoretical spectral data of CO-Ag adsorption in 23 different solvent systems as a data source. Subsequently, we propose a hierarchical knowledge extraction multiexpert neural network (HMNN) to bridge the knowledge gaps among different solvent systems. HMNN undergoes two training tiers: in tier I, it learns fundamental quantitative spectra-property relationships (QSPRs), and in tier II, it inherits the fundamental QSPR knowledge from previous steps through a dynamic integration of expert modules and subsequently captures the solvent differences. The results demonstrate HMNN's superiority in estimating a range of molecular adsorbate properties, with an error range of less than 0.008 eV for zero-shot predictions on unseen solvents. The findings underscore the usability, reliability, and convenience of HMNN and could pave the way for real-time access to microscopic properties by exploiting QSPR.

Development of early prediction model of in-hospital cardiac arrest based on laboratory parameters

BackgroundIn-hospital cardiac arrest (IHCA) is an acute disease with a high fatality rate that burdens individuals, society, and the economy. This study aimed to develop a machine learning (ML) model using routine laboratory parameters to predict the risk of IHCA in rescue-treated patients.MethodsThis retrospective cohort study examined all rescue-treated patients hospitalized at the First Medical Center of the PLA General Hospital in Beijing, China, from January 2016 to December 2020. Five machine learning algorithms, including support vector machine, random forest, extra trees classifier (ETC), decision tree, and logistic regression algorithms, were trained to develop models for predicting IHCA. We included blood counts, biochemical markers, and coagulation markers in the model development. We validated model performance using fivefold cross-validation and used the SHapley Additive exPlanation (SHAP) for model interpretation.ResultsA total of 11,308 participants were included in the study, of which 7779 patients remained. Among these patients, 1796 (23.09%) cases of IHCA occurred. Among five machine learning models for predicting IHCA, the ETC algorithm exhibited better performance, with an AUC of 0.920, compared with the other four machine learning models in the fivefold cross-validation. The SHAP showed that the top ten factors accounting for cardiac arrest in rescue-treated patients are prothrombin activity, platelets, hemoglobin, N-terminal pro-brain natriuretic peptide, neutrophils, prothrombin time, serum albumin, sodium, activated partial thromboplastin time, and potassium.ConclusionsWe developed a reliable machine learning-derived model that integrates readily available laboratory parameters to predict IHCA in patients treated with rescue therapy.

Radiomics and Clinicopathological Characteristics for Predicting Lymph Node Metastasis in Testicular Cancer

Accurate prediction of lymph node metastasis (LNM) in patients with testicular cancer is highly relevant for treatment decision-making and prognostic evaluation. Our study aimed to develop and validate clinical radiomics models for individual preoperative prediction of LNM in patients with testicular cancer. We enrolled 91 patients with clinicopathologically confirmed early-stage testicular cancer, with disease confined to the testes. We included five significant clinical risk factors (age, preoperative serum tumour markers AFP and B-HCG, histotype and BMI) to build the clinical model. After segmenting 273 retroperitoneal lymph nodes, we then combined the clinical risk factors and lymph node radiomics features to establish combined predictive models using Random Forest (RF), Light Gradient Boosting Machine (LGBM), Support Vector Machine Classifier (SVC), and K-Nearest Neighbours (KNN). Model performance was assessed by the area under the receiver operating characteristic (ROC) curve (AUC). Finally, the decision curve analysis (DCA) was used to evaluate the clinical usefulness. The Random Forest combined clinical lymph node radiomics model with the highest AUC of 0.95 (±0.03 SD; 95% CI) was considered the candidate model with decision curve analysis, demonstrating its usefulness for preoperative prediction in the clinical setting. Our study has identified reliable and predictive machine learning techniques for predicting lymph node metastasis in early-stage testicular cancer. Identifying the most effective machine learning approaches for predictive analysis based on radiomics integrating clinical risk factors can expand the applicability of radiomics in precision oncology and cancer treatment.

Machine learning study of the extended drug-target interaction network informed by pain related voltage-gated sodium channels.

Pain is a significant global health issue, and the current treatment options for pain management have limitations in terms of effectiveness, side effects, and potential for addiction. There is a pressing need for improved pain treatments and the development of new drugs. Voltage-gated sodium channels, particularly Nav1.3, Nav1.7, Nav1.8, and Nav1.9, play a crucial role in neuronal excitability and are predominantly expressed in the peripheral nervous system. Targeting these channels may provide a means to treat pain while minimizing central and cardiac adverse effects. In this study, we construct protein-protein interaction (PPI) networks based on pain-related sodium channels and develop a corresponding drug-target interaction network to identify potential lead compounds for pain management. To ensure reliable machine learning predictions, we carefully select 111 inhibitor data sets from a pool of more than 1000 targets in the PPI network. We employ 3 distinct machine learning algorithms combined with advanced natural language processing (NLP)-based embeddings, specifically pretrained transformer and autoencoder representations. Through a systematic screening process, we evaluate the side effects and repurposing potential of more than 150,000 drug candidates targeting Nav1.7 and Nav1.8 sodium channels. In addition, we assess the ADMET (absorption, distribution, metabolism, excretion, and toxicity) properties of these candidates to identify leads with near-optimal characteristics. Our strategy provides an innovative platform for the pharmacological development of pain treatments, offering the potential for improved efficacy and reduced side effects.

XGBoost and Random Forest Algorithms: An in Depth Analysis

Machine learning is playing an increasingly important role in many facets of our lives as technology develops, including forecasting weather, figuring out social media trends, and predicting prices on the world market. This significance invoked the demand of some efficient predicting models that can easily handle complex data and provide maximum accurate results. XGBoost and Random Forest are upgradable ensemble techniques used to solve regression and classification problems that have evolved and proved to be dependable and reliable machine learning challenge solvers. In this research paper, we undertake a comprehensive analysis and comparison of these two prominent machine learning algorithms. The first half of the research includes relevant overview on both of the techniques, significance and evolution of both algorithms. The latter part of this study involves a meticulous comparative analysis between Random Forest and XGBoost, scrutinizing facets such as time complexity, precision, and reliability. We examine their distinctive approaches to handling regression and classification problems while closely examining their subtle handling of training and testing datasets. A thorough quantitative evaluation using a variety of performance metrics, such as the F1-score, Recall, Precision, Mean Squared Erro

Personality Trait Recognition Based on Smartphone Typing Characteristics in the Wild

As governed by personality trait theory, humans tackle problems differently depending on their long-term behavioral characteristics. Computational awareness of personality traits fuels affective computing research, which investigates how to reliably recognize and utilize personality traits. Applications are diverse, including therapy monitoring, learning assistance, and recommender systems. Data-driven approaches are a promising path forward towards personality-aware human-computer interactions. Thereby, central challenges are the non-disruptive data acquisition, the time frame over which data must be collected before predictions become accurate, and the feature-centered data reduction to train reliable and lightweight machine learning models. In this work, we address these challenges by presenting a fully-automatic feature extraction and machine learning pipeline that makes accurate personality trait predictions for the widely-used Five Factor Model from passively-collected, short-term smartphone typing data collected from 76 participants (68 university students) in the wild. Our model allows for personality trait assessments after one day of data collection, demonstrating that, despite being a long-term behavioral trend, personality traits can be inferred accurately from shorter time periods. We demonstrate that our system can accurately predict personality traits on two levels (low and high) with up to 74.5% accuracy and 0.72 AUC for a single day, and up to 84.5% accuracy and 0.79 AUC after subsequent refinement over 10 weeks.

Decomposing texture and semantic for out-of-distribution detection

The out-of-distribution (OOD) detection task assumes samples that follow the distribution of training data as in-distribution (ID), while samples from other data distributions are considered OOD. In recent years, the OOD detection tasks have made significant progress since many studies observed that the distribution mismatch between training and real datasets can severely deteriorate the reliability of AI systems. Nevertheless, the lack of precise interpretation for the in-distribution (ID) limits the application of the OOD detection methods to real-world systems. To tackle this, we decompose the definition of the ID into texture and semantics, motivated by the demands of real-world scenarios. We also design new benchmarks to measure the robustness that OOD detection methods should have. Our proposed benchmark verifies not only the precision but also the robustness of the detection models. It is crucial to measure both factors in OOD detection as they indicate different traits of the model. For instance, precision is relevant to scenarios that detect minor cracks in the conveyor belt of a smart factory, whereas robustness pertains to maintaining performance under diverse weather conditions, as required by autonomous driving. To achieve a good balance between the OOD detection performance and robustness, our method takes a divide-and-conquer approach. Specifically, the proposed model first handles each component of the texture and semantics separately and then fuses these later. This philosophy is empirically proven by a series of benchmarks including both the proposed and the conventional counterpart. By decomposing the prior “unclear” definition of the ID into texture and semantic components, our novel approach better suits the demands of a reliable machine learning system, which requires robustness and consistent performance across varied scenarios. Unlike prior works, our approach does not rely on any extra datasets or labels. This prevents our proposed framework from being dependent on a particular dataset distribution.

Techniques for Developing Reliable Machine Learning Classifiers Applied to Understanding and Predicting Protein:Protein Interaction Hot Spots.

With machine learning now transforming the sciences, successful prediction of biological structure or activity is mainly limited by the extent and quality of data available for training, the astute choice of features for prediction, and thorough assessment of the robustness of prediction on a variety of new cases. In this chapter, we address these issues while developing and sharing protocols to build a robust dataset and rigorously compare several predictive classifiers using the open-source Python machine learning library, scikit-learn. We show how to evaluate whether enough data has been used for training and whether the classifier has been overfit to training data. The most telling experiment is 500-fold repartitioning of the training and test sets, followed by prediction, which gives a good indication of whether a classifier performs consistently well on different datasets. An intuitive method is used to quantify which features are most important for correct prediction.The resulting well-trained classifier, hotspotter, can robustly predict the small subset of amino acid residues on the surface of a protein that are energetically most important for binding a protein partner: the interaction hot spots. Hotspotter has been trained and tested here on a curated dataset assembled from 1046 non-redundant alanine scanning mutation sites with experimentally measured change in binding free energy values from 97 different protein complexes; this dataset is available to download. The accessible surface area of the wild-type residue at a given site and its degree of evolutionary conservation proved the most important features to identify hot spots. A variant classifier was trained and validated for proteins where only the amino acid sequence is available, augmented by secondary structure assignment. This version of hotspotter requiring fewer features is almost as robust as the structure-based classifier. Application to the ACE2 (angiotensin converting enzyme 2)receptor, which mediates COVID-19 virus entry into human cells, identified the critical hot spot triad of ACE2 residues at the center of the small interface with the CoV-2 spike protein. Hotspotter results can be used to guide the strategic design of protein interfaces and ligands and also to identify likely interfacial residues for protein:protein docking.

Impact of signal intensity normalization of MRI on the generalizability of radiomic-based prediction of molecular glioma subtypes

ObjectivesRadiomic features have demonstrated encouraging results for non-invasive detection of molecular biomarkers, but the lack of guidelines for pre-processing MRI-data has led to poor generalizability. Here, we assessed the influence of different MRI-intensity normalization techniques on the performance of radiomics-based models for predicting molecular glioma subtypes.MethodsPreoperative MRI-data from n = 615 patients with newly diagnosed glioma and known isocitrate dehydrogenase (IDH) and 1p/19q status were pre-processed using four different methods: no normalization (naive), N4 bias field correction (N4), N4 followed by either WhiteStripe (N4/WS), or z-score normalization (N4/z-score). A total of 377 Image-Biomarker-Standardisation-Initiative-compliant radiomic features were extracted from each normalized data, and 9 different machine-learning algorithms were trained for multiclass prediction of molecular glioma subtypes (IDH-mutant 1p/19q codeleted vs. IDH-mutant 1p/19q non-codeleted vs. IDH wild type). External testing was performed in public glioma datasets from UCSF (n = 410) and TCGA (n = 160).ResultsSupport vector machine yielded the best performance with macro-average AUCs of 0.84 (naive), 0.84 (N4), 0.87 (N4/WS), and 0.87 (N4/z-score) in the internal test set. Both N4/WS and z-score outperformed the other approaches in the external UCSF and TCGA test sets with macro-average AUCs ranging from 0.85 to 0.87, replicating the performance of the internal test set, in contrast to macro-average AUCs ranging from 0.19 to 0.45 for naive and 0.26 to 0.52 for N4 alone.ConclusionIntensity normalization of MRI data is essential for the generalizability of radiomic-based machine-learning models. Specifically, both N4/WS and N4/z-score approaches allow to preserve the high model performance, yielding generalizable performance when applying the developed radiomic-based machine-learning model in an external heterogeneous, multi-institutional setting.Clinical relevance statementIntensity normalization such as N4/WS or N4/z-score can be used to develop reliable radiomics-based machine learning models from heterogeneous multicentre MRI datasets and provide non-invasive prediction of glioma subtypes.Key Points• MRI-intensity normalization increases the stability of radiomics-based models and leads to better generalizability.• Intensity normalization did not appear relevant when the developed model was applied to homogeneous data from the same institution.• Radiomic-based machine learning algorithms are a promising approach for simultaneous classification of IDH and 1p/19q status of glioma.

Effect of Flattened Structures of Molecules and Materials on Machine Learning Model Training.

A key aspect of producing accurate and reliable machine learning models for the prediction of properties of quantum chemistry (QC) data is identifying possible data characteristics that may negatively influence model training. In previous work, we identified that molecules and materials with a low volume of the convex hull (VCH) of atomic positions may be harmful in model training and a source of prediction outliers. In this paper, we extend this analysis further and develop a biased sampling study to evaluate the influence of VCH on the training data of a model using different structures of molecules and materials. Our study confirms that VCH influences model training and shows the importance of using homogeneous geometric characteristics of structures when building new data sets or selecting training sets from larger QC data sets.