Train Machine Learning Models Research Articles

Optimizing data acquisition: a Bayesian approach for efficient machine learning model training

Acquiring a substantial number of data points for training accurate machine learning (ML) models is a big challenge in scientific fields where data collection is resource-intensive. Here, we propose a novel approach for constructing a minimal yet highly informative database for training ML models in complex multi-dimensional parameter spaces. To achieve this, we mimic the underlying relation between the output and input parameters using Gaussian process regression (GPR). Using a set of known data, GPR provides predictive means and standard deviation for the unknown data. Given the predicted standard deviation by GPR, we select data points using Bayesian optimization to obtain an efficient database for training ML models. We compare the performance of ML models trained on databases obtained through this method, with databases obtained using traditional approaches. Our results demonstrate that the ML models trained on the database obtained using Bayesian optimization approach consistently outperform the other two databases, achieving high accuracy with a significantly smaller number of data points. Our work contributes to the resource-efficient collection of data in high-dimensional complex parameter spaces, to achieve high precision ML predictions.

Generating Synthetic MR Spectroscopic Imaging Data with Generative Adversarial Networks to Train Machine Learning Models.

To develop a new method to generate synthetic MR spectroscopic imaging (MRSI) data for training machine learning models. This study targeted routine MRI examination protocols with single voxel spectroscopy (SVS). A novel model derived from pix2pix generative adversarial networks was proposed to generate synthetic MRSI data using MRI and SVS data as inputs. T1- and T2-weighted, SVS, and reference MRSI data were acquired from healthy brains with clinically available sequences. The proposed model was trained to generate synthetic MRSI data. Quantitative evaluation involved the calculation of the mean squared error (MSE) against the reference and metabolite ratio value. The effect of the location of and the number of the SVS data on the quality of the synthetic MRSI data was investigated using the MSE. The synthetic MRSI data generated from the proposed model were visually closer to the reference. The 95% confidence interval (CI) of the metabolite ratio value of synthetic MRSI data overlapped with the reference for seven of eight metabolite ratios. The MSEs tended to be lower in the same location than in different locations. The MSEs among groups of numbers of SVS data were not significantly different. A new method was developed to generate MRSI data by integrating MRI and SVS data. Our method can potentially increase the volume of MRSI data training for other machine learning models by adding SVS acquisition to routine MRI examinations.

Machine learning assisted designing of polymers and refractive index prediction: Easy and fast screening of polymers from chemical space

For the selection of efficient materials capable of optical applicability, the property of refractive index (RI) of the material is considered extremely significant. Acquisition of RI values via empirical framework is difficult and prolonged task. Consequently, data-driven method provides a rapid alternative for estimating RI values. This study presents a fast framework that is based on machine learning (ML) approach to design novel polymers capable of performing in optical applications. The framework involves training of ML models to predict the RI values of polymers. Subsequently, 10,000 new polymers were generated utilizing the Breaking Retrosynthetically Interesting Chemical Substructures (BRICS) methodology and fast ML model is used to predict the RI values of newly generated polymers. Polymers exhibiting higher RI values were retained. The synthetic accessibility of these selected polymers was also assessed in order to facilitate the future empirical measurements. Furthermore, chemical similarity among the chosen polymers was also investigated and structural diversity was revealed among the selected polymers. This study is introducing fast and easy framework for the designing and screening of efficient materials. This framework can be modified to study other materials and properties.

Synthetic Malware Using Deep Variational Autoencoders and Generative Adversarial Networks

The effectiveness of detecting malicious files heavily relies on the quality of the training dataset, particularly its size and authenticity. However, the lack of high-quality training data remains one of the biggest challenges in achieving widespread adoption of malware detection by trained machine and deep learning models. In response to this challenge, researchers have made initial strides by employing generative techniques to create synthetic malware samples. This work utilizes deep variational autoencoders (VAE) and generative adversarial networks (GAN) to produce malware samples as opcode sequences. The generated malware opcodes are then distinguished from authentic opcode samples using machine and deep learning techniques as validation methods. The primary objective of this study was to compare synthetic malware generated using VAE and GAN technologies. The results showed that neither approach could create synthetic malware that could deceive machine learning classification. However, the WGAN-GP algorithm showed more promise by requiring a higher number of synthetic malware samples in the train set to effectively be detected, proving ita better approach in synthetic malware generation.

Optimizing data acquisition: a Bayesian approach for efficient machine learning model training

Abstract Acquiring a substantial number of data points for training accurate machine learning (ML) models is a big challenge in scientific fields where data collection is resource-intensive. Here, we propose a novel approach for constructing a minimal yet highly informative database for training ML models in complex multi-dimensional parameter spaces. To achieve this, we mimic the underlying relation between the output and input parameters using Gaussian process regression (GPR). Using a set of known data, GPR provides predictive means and standard deviation for the unknown data. Given the predicted standard deviation by GPR, we select data points using Bayesian optimization to obtain an efficient database for training ML models. We compare the performance of ML models trained on databases obtained through this method, with databases obtained using traditional approaches. Our results demonstrate that the ML models trained on the database obtained using Bayesian optimization approach consistently outperform the other two databases, achieving high accuracy with a significantly smaller number of data points. Our work contributes to the resource-efficient collection of data in high-dimensional complex parameter spaces, to achieve high precision machine learning predictions.

Progressive transfer learning for advancing machine learning-based reduced-order modeling.

To maximize knowledge transfer and improve the data requirement for data-driven machine learning (ML) modeling, a progressive transfer learning for reduced-order modeling (p-ROM) framework is proposed. A key concept of p-ROM is to selectively transfer knowledge from previously trained ML models and effectively develop a new ML model(s) for unseen tasks by optimizing information gates in hidden layers. The p-ROM framework is designed to work with any type of data-driven ROMs. For demonstration purposes, we evaluate the p-ROM with specific Barlow Twins ROMs (p-BT-ROMs) to highlight how progress learning can apply to multiple topological and physical problems with an emphasis on a small training set regime. The proposed p-BT-ROM framework has been tested using multiple examples, including transport, flow, and solid mechanics, to illustrate the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. In both similar and different topologies, p-BT-ROM achieves improved model accuracy with much less training data. For instance, p-BT-ROM with four-parent (i.e., pre-trained models) outperforms the no-parent counterpart trained on data nine times larger. The p-ROM framework is poised to significantly enhance the capabilities of ML-based ROM approaches for scientific and engineering applications by mitigating data scarcity through progressively transferring knowledge.

Progressive transfer learning for advancing machine learning-based reduced-order modeling

To maximize knowledge transfer and improve the data requirement for data-driven machine learning (ML) modeling, a progressive transfer learning for reduced-order modeling (p-ROM) framework is proposed. A key concept of p-ROM is to selectively transfer knowledge from previously trained ML models and effectively develop a new ML model(s) for unseen tasks by optimizing information gates in hidden layers. The p-ROM framework is designed to work with any type of data-driven ROMs. For demonstration purposes, we evaluate the p-ROM with specific Barlow Twins ROMs (p-BT-ROMs) to highlight how progress learning can apply to multiple topological and physical problems with an emphasis on a small training set regime. The proposed p-BT-ROM framework has been tested using multiple examples, including transport, flow, and solid mechanics, to illustrate the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. In both similar and different topologies, p-BT-ROM achieves improved model accuracy with much less training data. For instance, p-BT-ROM with four-parent (i.e., pre-trained models) outperforms the no-parent counterpart trained on data nine times larger. The p-ROM framework is poised to significantly enhance the capabilities of ML-based ROM approaches for scientific and engineering applications by mitigating data scarcity through progressively transferring knowledge.

Contrastive Self-supervised Learning for Neurodegenerative Disorder Classification.

Neurodegenerative diseases such as Alzheimer's disease (AD) or frontotemporal lobar degeneration (FTLD) involve specific loss of brain volume, detectable in vivo using T1-weighted MRI scans. Supervised machine learning approaches classifying neurodegenerative diseases require diagnostic-labels for each sample. However, it can be difficult to obtain expert labels for a large amount of data. Self-supervised learning (SSL) offers an alternative for training machine learning models without data-labels. We investigated if the SSL models can applied to distinguish between different neurodegenerative disorders in an interpretable manner. Our method comprises a feature extractor and a downstream classification head. A deep convolutional neural network trained in a contrastive self-supervised way serves as the feature extractor, learning latent representation, while the classifier head is a single-layer perceptron. We used N=2694 T1-weighted MRI scans from four data cohorts: two ADNI datasets, AIBL and FTLDNI, including cognitively normal controls (CN), cases with prodromal and clinical AD, as well as FTLD cases differentiated into its sub-types. Our results showed that the feature extractor trained in a self-supervised way provides generalizable and robust representations for the downstream classification. For AD vs. CN, our model achieves 82% balanced accuracy on the test subset and 80% on an independent holdout dataset. Similarly, the Behavioral variant of frontotemporal dementia (BV) vs. CN model attains an 88% balanced accuracy on the test subset. The average feature attribution heatmaps obtained by the Integrated Gradient method highlighted hallmark regions, i.e., temporal gray matter atrophy for AD, and insular atrophy for BV. In conclusion, our models perform comparably to state-of-the-art supervised deep learning approaches. This suggests that the SSL methodology can successfully make use of unannotated neuroimaging datasets as training data while remaining robust and interpretable.

Bayesian Networks for Prescreening in Depression: Algorithm Development and Validation.

Identifying individuals with depressive symptomatology (DS) promptly and effectively is of paramount importance for providing timely treatment. Machine learning models have shown promise in this area; however, studies often fall short in demonstrating the practical benefits of using these models and fail to provide tangible real-world applications. This study aims to establish a novel methodology for identifying individuals likely to exhibit DS, identify the most influential features in a more explainable way via probabilistic measures, and propose tools that can be used in real-world applications. The study used 3 data sets: PROACTIVE, the Brazilian National Health Survey (PesquisaNacionalde Saúde [PNS]) 2013, and PNS 2019, comprising sociodemographic and health-related features. A Bayesian network was used for feature selection. Selected features were then used to train machine learning models to predict DS, operationalized as a score of ≥10 on the 9-item Patient Health Questionnaire. The study also analyzed the impact of varying sensitivity rates on the reduction of screening interviews compared to a random approach. The methodology allows the users to make an informed trade-off among sensitivity, specificity, and a reduction in the number of interviews. At the thresholds of 0.444, 0.412, and 0.472, determined by maximizing the Youden index, the models achieved sensitivities of 0.717, 0.741, and 0.718, and specificities of 0.644, 0.737, and 0.766 for PROACTIVE, PNS 2013, and PNS 2019, respectively. The area under the receiver operating characteristic curve was 0.736, 0.801, and 0.809 for these 3 data sets, respectively. For the PROACTIVE data set, the most influential features identified were postural balance, shortness of breath, and how old people feel they are. In the PNS 2013 data set, the features were the ability to do usual activities, chest pain, sleep problems, and chronic back problems. The PNS 2019 data set shared 3 of the most influential features with the PNS 2013 data set. However, the difference was the replacement of chronic back problems with verbal abuse. It is important to note that the features contained in the PNS data sets differ from those found in the PROACTIVE data set. An empirical analysis demonstrated that using the proposed model led to a potential reduction in screening interviews of up to 52% while maintaining a sensitivity of 0.80. This study developed a novel methodology for identifying individuals with DS, demonstrating the utility of using Bayesian networks to identify the most significant features. Moreover, this approach has the potential to substantially reduce the number of screening interviews while maintaining high sensitivity, thereby facilitating improved early identification and intervention strategies for individuals experiencing DS.

O-131 A novel approach to predicting the potential of a blastocyst using untargeted spectral analysis of spent culture medium and machine learning

Abstract Study question Is it possible to effectively predict negative outcomes of intrauterine embryo transfer using modified untargeted Raman spectroscopy of the spent culture medium? Summary answer The proposed spectroscopic approach, combined with machine learning methods, allows identifying a group of embryos that are almost unable to implant and develop ongoing pregnancy. What is known already Embryo deselection is usually based on the karyotype analysis of the embryo and originates from the assumption that an abnormal PGT result is associated with negative transfer outcomes. Embryo selection methods aimed at identifying the most capable for implantation embryo are based on morphological, morphometric, and metabolic approaches. However, due to the dependence of the phenomenon (pregnancy) on several factors, predicting the outcome of the transfer solely through variables derived from the embryo does not correspond to modern views on implantation. At the same time, predicting negative transfer outcomes through the information originating from the embryo appears quite rational. Study design, size, duration 424 samples of spent culture medium used, collected from microdrops with individually cultured blastocyst from the internal institutional biosamples repository. Medium samples were collected from July 1, 2021, to June 1, 2023. The class attribute for each sample was the absence or presence of ongoing pregnancy at the 12-14 week of gestational age. The positive class (ongoing pregnancy) proportion in the presented dataset is 33%, and the negative (no ongoing pregnancy) class proportion is 67%. Participants/materials, setting, methods The study material was spent culture medium from blastocysts with known transfer outcomes by the end of first trimester. All media samples were stored at -86 °С until the moment of spectroscopy. Before the study, samples were thawed at room temperature for 15 minutes. Untargeted Raman spectroscopy was done on a signal amplification substrate using a green laser (532 nm). The architecture selection and machine learning model training were performed using the Python library mcfly. Main results and the role of chance After building the model the following results for validation and test sets were obtained: So it is possible to identify a group of embryos that are almost incapable of producing ongoing pregnancy. This group constitutes about 40% of the total number of negative outcomes. The error of determining an embryo as being incapable for implantation and normal pregnancy development is about 6%, what is relatively low number. Limitations, reasons for caution The presented data are not multicenter; they were obtained from samples from one laboratory and using a limited set (only one manufacturer) of culture media. This may lead to reduced or nonexistent relevance of the model for data from other sources and conditions. Wider implications of the findings It is necessary to check the proposed method on an additional set of culture media from other manufactures and laboratories. This may lead to refining the parameters of the built neural network. The assumption of significant clinical value of the developed model needs to be checked through prospective testing. Trial registration number fpc-2023-ivf-01

Detection of disease-specific signatures in B cell repertoires of lymphomas using machine learning.

The classification of B cell lymphomas-mainly based on light microscopy evaluation by a pathologist-requires many years of training. Since the B cell receptor (BCR) of the lymphoma clonotype and the microenvironmental immune architecture are important features discriminating different lymphoma subsets, we asked whether BCR repertoire next-generation sequencing (NGS) of lymphoma-infiltrated tissues in conjunction with machine learning algorithms could have diagnostic utility in the subclassification of these cancers. We trained a random forest and a linear classifier via logistic regression based on patterns of clonal distribution, VDJ gene usage and physico-chemical properties of the top-n most frequently represented clonotypes in the BCR repertoires of 620 paradigmatic lymphoma samples-nodular lymphocyte predominant B cell lymphoma (NLPBL), diffuse large B cell lymphoma (DLBCL) and chronic lymphocytic leukemia (CLL)-alongside with 291 control samples. With regard to DLBCL and CLL, the models demonstrated optimal performance when utilizing only the most prevalent clonotype for classification, while in NLPBL-that has a dominant background of non-malignant bystander cells-a broader array of clonotypes enhanced model accuracy. Surprisingly, the straightforward logistic regression model performed best in this seemingly complex classification problem, suggesting linear separability in our chosen dimensions. It achieved a weighted F1-score of 0.84 on a test cohort including 125 samples from all three lymphoma entities and 58 samples from healthy individuals. Together, we provide proof-of-concept that at least the 3 studied lymphoma entities can be differentiated from each other using BCR repertoire NGS on lymphoma-infiltrated tissues by a trained machine learning model.

An Innovative Approach for Enhancing Relay Coordination in Distribution Systems Through Online Adaptive Strategies Utilizing DNN Machine Learning and a Hybrid GA-SQP Framework

AbstractThe present study addresses the issue of varying fault locations within a distribution system, which leads to fluctuations in short-circuit currents and requires the implementation of adaptive protection strategies for network reliability. This paper presents a novel adaptive protection scheme that specifically considers these fault location variations using directional overcurrent relays (DOCRs). Unlike previous research on adaptive protection, which does not adequately account for fault location variations, this method employs deep neural networks (DNNs) for online fault location detection. In the verification process, the effectiveness of the proposed methodologies was assessed by analyzing the time derivative of a trained machine learning model for fault identification. This approach enables the immediate detection of fault locations within the distribution system and facilitates the transmission of the setting group index to activate preset optimal coordination parameter values for the system relays. Crucially, the proposed method ensures that the coordination constraints remain intact across various adaptive settings, relying on precise fault identification through machine learning. The practical significance of this approach lies in its applicability to real-world systems because the proposed solutions and protective settings can be easily implemented using commercially available relays. To evaluate its effectiveness, the adaptive protection scheme was tested on three distribution networks: IEEE 14-Bus, 15-Bus and 30-Bus. The comparative test results highlight that the proposed method significantly improves the speed of the protection system for distribution networks when compared to existing studies, making it a valuable contribution to enhancing network reliability and performance.

Melanoma image synthesis: a review using generative adversarial networks

Melanoma is a highly malignant skin cancer that may be fatal if not promptly detected and treated. The limited availability of high-quality melanoma images, which are needed for training machine learning models, is one of the obstacles to detecting melanoma. Generative adversarial networks (GANs) have grown in popularity as a strong technique for image synthesis. This research is also targeted at the sustainable development goal (SDG) for health care. In this study, we survey existing GAN-based melanoma image synthesis methods. In this work, we briefly introduce GANs and how they may be used for generating synthetic images. Ensuring healthy lifestyles and promoting well-being for everyone, regardless of age, is the main aim. A comparative study is carried out on how GANs are used in current research to generate melanoma images and how they improve the classification performance of neural networks. Various public and proprietary datasets for training GANs in melanoma image synthesis are also discussed. Lastly, we assess the examined studies' performance using measures like the Frechet Inception distance (FID), Inception score, structural similarity ındex (SSIM), and various classification performance metrics. We compare the evaluated findings and suggest further GAN-based melanoma image-creation research.

Statistical analysis of microstructurally small fatigue crack growth in three-dimensional polycrystals based on high-fidelity numerical simulations

Microstructurally small cracks (MSCs) are sensitive to their local microstructural neighborhoods, resulting in highly variable 3D crack propagation within individual samples and among a population of samples. Statistically quantifying this complex spatiotemporal variability requires collecting and analyzing a large number of 3D MSC growth observations, which remains challenging due to experimental constraints that limit the availability of 3D observations of MSCs in the existing literature. We address this gap by leveraging virtual observations of MSC growth using a state-of-the-art, high-fidelity simulation framework to provide a statistical perspective on 3D MSC growth behavior. The MSC growth simulations were performed in 40 unique but statistically similar polycrystalline microstructures, and the data extracted from the virtual observations were collectively analyzed to quantify variability in geometric, microstructural, and micromechanical aspects of MSCs. Variability in tortuosity, crack surface crystallography, and MSC growth parameters (local crack extension and kink angle) were quantified statistically. The results show that 3D MSC surfaces do not necessarily propagate along {111} crystallographic planes despite crack traces on radial planes being closely aligned with {111} slip plane traces. Also, the 3D MSC surfaces exhibit increasingly tortuous propagation and spatially varying local crack growth rates (with variability as high as 59% among the population). Moreover, a correlation study revealed some of the highly influential microstructural and micromechanical features controlling MSC growth parameters. The quantified variability holds direct implications for advanced materials prognosis in engineering applications, and the identified features from the correlation study can be leveraged to train machine learning models for rapidly predicting MSC growth behavior in the future.

A probabilistic approach to training machine learning models using noisy data

Machine learning (ML) models are increasingly popular in environmental and hydrologic modeling, but they typically contain uncertainties resulting from noisy data (erroneous or outlier data). This paper presents a novel probabilistic approach that combines ML and Markov Chain Monte Carlo simulation to (1) detect and underweight likely noisy data, (2) develop an approach capable of detecting noisy data during model deployment, and (3) interpret the reasons why a data point is deemed noisy to help heuristically distinguish between outliers and erroneous data. The new algorithm recognizes that there is no unique way to split the training data into noisy and clean data, and thus produces an ensemble of plausible splits. The algorithm successfully detected noisy data in synthetic benchmark problems with varying complexity and a real-world public supply water withdrawal dataset. The algorithm is generic and flexible, making it suitable for application across a broad range of hydrologic and environmental disciplines.

Artificial Intelligence-Clinical Decision Support System in Infectious Disease Control: Combatting Multidrug-Resistant Klebsiella pneumoniae with Machine Learning.

The World Health Organization has identified Klebsiella pneumoniae (KP) as a significant threat to global public health. The rising threat of carbapenem-resistant Klebsiella pneumoniae (CRKP) leads to prolonged hospital stays and higher medical costs, necessitating faster diagnostic methods. Traditional antibiotic susceptibility testing (AST) methods demand at least 4 days, requiring 3 days on average for culturing and isolating the bacteria and identifying the species using matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF MS), plus an extra day for interpreting AST results. This lengthy process makes traditional methods too slow for urgent clinical situations requiring rapid decision-making, potentially hindering prompt treatment decisions, especially for fast-spreading infections such as those caused by CRKP. This research leverages a cutting-edge diagnostic method that utilizes an artificial intelligence-clinical decision support system (AI-CDSS). It incorporates machine learning algorithms for the swift and precise detection of carbapenem-resistant and colistin-resistant strains. We selected 4307 KP samples out of a total of 52,827 bacterial samples due to concerns about multi-drug resistance using MALDI-TOF MS and Vitek-2 systems for AST. It involved thorough data preprocessing, feature extraction, and machine learning model training fine-tuned with GridSearchCV and 5-fold cross-validation, resulting in high predictive accuracy, as demonstrated by the receiver operating characteristic and area under the curve (AUC) scores, laying the groundwork for our AI-CDSS. MALDI-TOF MS analysis revealed distinct intensity profiles differentiating CRKP and susceptible strains, as well as colistin-resistant Klebsiella pneumoniae (CoRKP) and susceptible strains. The Random Forest Classifier demonstrated superior discriminatory power, with an AUC of 0.96 for detecting CRKP and 0.98 for detecting CoRKP. Integrating MALDI-TOF MS with machine learning in an AI-CDSS has greatly expedited the detection of KP resistance by approximately 1 day. This system offers timely guidance, potentially enhancing clinical decision-making and improving treatment outcomes for KP infections.

Integrating Domain Knowledge in Multi-Source Classification Tasks

This work presents an extended investigation into multi-domain learning techniques within the context of image and audio classification, with a focus on the latter. In machine learning, collections of data obtained or generated under similar conditions are referred to as domains or data sources. However, the distinct acquisition or generation conditions of these data sources are often overlooked, despite their potential to significantly impact model generalization. Multi-domain learning addresses this challenge by seeking effective methods to train models to perform adequately across all domains seen during the training process. Our study explores a range of model-agnostic multi-domain learning techniques that leverage explicit domain information alongside class labels. Specifically, we delve into three distinct methodologies: a general approach termed Stew, which involves mixing all available data indiscriminately; and two batch domain-regularization methods: Balanced Domains and Loss Sum. These methods are evaluated through several experiments conducted on datasets featuring multiple data sources for audio and image classification tasks. Our findings underscore the importance of considering domain-specific information during the training process. We demonstrate that the application of the Loss Sum method yields notable improvements in model performance (0.79 F1-Score) compared to conventional approaches that blend data from all available domains (0.62 F1-Score). By examining the impact of different multi-domain learning techniques on classification tasks, this study contributes to a deeper understanding of effective strategies for leveraging domain knowledge in machine learning model training.

Data-driven discovery of novel metal organic frameworks with superior ammonia adsorption capacity

Ammonia (NH3) has been a subject of great interest due to its important roles in diverse technological applications. However, high toxicity and corrosiveness of NH3 has made it an important task to develop an efficient carrier to safely capture NH3 with high capacity. Here, we employ a machine learning (ML) model to discover high-performance metal organic frameworks (MOFs) that will work as efficient NH3 carriers. By constructing databases at two distinct conditions, adsorption and desorption, through Grand Canonical Monte Carlo (GCMC) simulations to train ML models, we identify eight novel MOFs as potentially efficient NH3 carriers through screening the large-scale MOF databases with the trained models and GCMC verification. The identified MOFs exhibit the average NH3 working capacity exceeding 1100 mg/g, and subsequent molecular dynamics simulations demonstrate mechanical stability of the predicted MOFs. Moreover, analyses of the diffusion mechanism within the proposed MOFs underscore the strong dependence of NH₃ gas diffusivity on the structural details of the materials.

Developing Machine Learning-Based Predictive Models for Hallux Valgus Recurrence Based on Measurements From Radiographs.

Machine learning (ML) is increasingly used to predict the prognosis of numerous diseases. This retrospective analysis aimed to develop a prediction model using ML algorithms and to identify predictors associated with the recurrence of hallux valgus (HV) following surgery. A total of 198 symptomatic feet that underwent chevron osteotomy combined with a distal soft tissue procedure were enrolled and analyzed from 2 independent medical centers. The feet were grouped according to nonrecurrence or recurrence based on 1-year follow-up outcomes. Preoperative weightbearing radiographs and immediate postoperative nonweightbearing radiographs were obtained for each HV foot. Radiographic measurements (eg, HV angle and intermetatarsal angle) were acquired and used for ML model training. A total of 9 commonly used ML models were trained on the data obtained from one institute (108 feet), and tested on the other data set from another independent institute (90 feet) for external validation. Optimal feature sets for each model were identified based on a 2000-resample bootstrap-based internal validation via an exhaustive search. The performance of each model was then tested on the external validation set. The area under the curve (AUC), classification accuracy, sensitivity, and specificity of each model were calculated to evaluate the performance of each model. The support vector machine (SVM) model showed the highest predictive accuracy compared to other methods, with an AUC of 0.88 and an accuracy of 75.6%. Preoperative hallux valgus angle, tibial sesamoid position, postoperative intermetatarsal angle, and postoperative tibial sesamoid position were identified as the most selected features by several ML models. ML classifiers such as SVM could predict the recurrence of HV (an HVA >20 degrees) at a 1-year follow-up while identifying associated predictors in a multivariate manner. This study holds the potential for foot and ankle surgeons to effectively identify individuals at higher risk of HV recurrence postsurgery.

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

In additive manufacturing (AM), process defects such as keyhole pores are difficult to anticipate, affecting the quality and integrity of the AM-produced materials. Hence, considerable efforts have aimed to predict these process defects by training machine learning (ML) models using passive measurements such as acoustic emissions. This work considered a dataset in which keyhole pores of a laser powder bed fusion (LPBF) experiment were identified using X-ray radiography and then registered both in space and time to acoustic measurements recorded during the LPBF experiment. Due to AM’s intrinsic process controls, where a pore-forming event is relatively rare, the acoustic datasets collected during monitoring include more non-pores than pores. In other words, the dataset for ML model development is imbalanced. Moreover, this imbalanced and sparse data phenomenon remains ubiquitous across many AM monitoring schemes since training data is nontrivial to collect. Hence, we propose a machine learning approach to improve this dataset imbalance and enhance the prediction accuracy of pore-labeled data. Specifically, we investigate how data augmentation helps predict pores and non-pores better. This imbalance is improved using recent advances in data augmentation called Mixup, a weak-supervised learning method. Convolutional neural networks (CNNs) are trained on original and augmented datasets, and an appreciable increase in performance is reported when testing on five different experimental trials. When ML models are trained on original and augmented datasets, they achieve an accuracy of 95% and 99% on test datasets, respectively. We also provide information on how dataset size affects model performance. Lastly, we investigate the optimal Mixup parameters for augmentation in the context of CNN performance.