Feature Machine Learning Research Articles

Precision healthcare: A deep dive into machine learning algorithms and feature selection strategies for accurate heart disease prediction

This paper presents a comprehensive exploration of machine learning algorithms (MLAs) and feature selection techniques for accurate heart disease prediction (HDP) in modern healthcare. By focusing on diverse datasets encompassing various challenges, the research sheds light on optimal strategies for early detection. MLAs such as Decision Trees (DT), Random Forests (RF), Support Vector Machines (SVM), Gaussian Naive Bayes (NB), and others were studied, with precision and recall metrics emphasized for robust predictions. Our study addresses challenges in real-world data through data cleaning and one-hot encoding, enhancing the integrity of our predictive models. Feature extraction techniques—Recursive Feature Extraction (RFE), Principal Component Analysis (PCA), and univariate feature selection—play a crucial role in identifying relevant features and reducing data dimensionality. Our findings showcase the impact of these techniques on improving prediction accuracy. Optimized models for each dataset have been achieved through grid search hyperparameter tuning, with configurations meticulously outlined. Notably, a remarkable 99.12 % accuracy was achieved on the first Kaggle dataset, showcasing the potential for accurate HDP. Model robustness across diverse datasets was highlighted, with caution against overfitting. The study emphasizes the need for validation of unseen data and encourages ongoing research for generalizability. Serving as a practical guide, this research aids researchers and practitioners in HDP model development, influencing clinical decisions and healthcare resource allocation. By providing insights into effective algorithms and techniques, the paper contributes to reducing heart disease-related morbidity and mortality, supporting the healthcare community's ongoing efforts.

Comparison of Different Fusion Radiomics for Predicting Benign and Malignant Sacral Tumors: A Pilot Study.

Differentiating between benign and malignant sacral tumors is crucial for determining appropriate treatment options. This study aims to develop two benchmark fusion models and a deep learning radiomic nomogram (DLRN) capable of distinguishing between benign and malignant sacral tumors using multiple imaging modalities. We reviewed axial T2-weighted imaging (T2WI) and non-contrast computed tomography (NCCT) of 134 patients pathologically confirmed as sacral tumors. The two benchmark fusion models were developed using fusion deep learning (DL) features and fusion classical machine learning (CML) features from multiple imaging modalities, employing logistic regression, K-nearest neighbor classification, and extremely randomized trees. The two benchmark models exhibiting the most robust predictive performance were merged with clinical data to formulate the DLRN. Performance assessment involved computing the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, accuracy, negative predictive value (NPV), and positive predictive value (PPV). The DL benchmark fusion model demonstrated superior performance compared to the CML fusion model. The DLRN, identified as the optimal model, exhibited the highest predictive performance, achieving an accuracy of 0.889 and an AUC of 0.961 in the test sets. Calibration curves were utilized to evaluate the predictive capability of the models, and decision curve analysis (DCA) was conducted to assess the clinical net benefit of the DLR model. The DLRN could serve as a practical predictive tool, capable of distinguishing between benign and malignant sacral tumors, offering valuable information for risk counseling, and aiding in clinical treatment decisions.

Approach to Detecting Malicious Bots in the Vkontakte Social Network and Assessing Their Parameters

The emergence of new varieties of bots in social networks and the improvement of their capabilities to imitate the natural behavior of real users represent a significant problem in the field of protection of social networks and online communities. This paper proposes a new approach to detecting and assessing the parameters of bots within the social network «VKontakte». The basis of the proposed approach is the creation of datasets using the method of «controlled purchase» of bots, which allows one to assess bots’ characteristics such as price, quality, and speed of action of bots, and using the Turing Test to assess how much users trust bots. In combination with traditional machine learning methods and features extracted from interaction graphs, text messages, and statistical distributions, it becomes possible to not only detect bots accurately but also predict their characteristics. This paper demonstrates that the trained machine learning model, based on the proposed approach, is robust to imbalanced data and can identify most types of bots as it has only a minor correlation with their main characteristics. The proposed approach can be used within the choice of countermeasures for the protection of social networks and for historical analysis, which allows not only to confirm the presence of bots but also to characterize the specifics of the attack.

Building datasets to support information extraction and structure parsing from electronic theses and dissertations

Despite the millions of electronic theses and dissertations (ETDs) publicly available online, digital library services for ETDs have not evolved past simple search and browse at the metadata level. We need better digital library services that allow users to discover and explore the content buried in these long documents. Recent advances in machine learning have shown promising results for decomposing documents into their constituent parts, but these models and techniques require data for training and evaluation. In this article, we present high-quality datasets to train, evaluate, and compare machine learning methods in tasks that are specifically suited to identify and extract key elements of ETD documents. We explain how we construct the datasets by manual labeling the data or by deriving labeled data through synthetic processes. We demonstrate how our datasets can be used to develop downstream applications and to evaluate, retrain, or fine-tune pre-trained machine learning models. We describe our ongoing work to compile benchmark datasets and exploit machine learning techniques to build intelligent digital libraries for ETDs.

Physics-Informed Design of Hybrid Pulse Power Characterization Tests for Rechargeable Batteries

Industry-standard diagnostic methods for rechargeable batteries, such as hybrid pulse power characterization (HPPC) tests for hybrid electric vehicles, provide some indications of state of health (SoH), but lack a physical basis to guide protocol design and identify degradation mechanisms. We develop a physics-based theoretical framework for HPPC tests, which are able to accurately determine specific mechanisms for battery degradation in porous electrode simulations. We show that voltage pulses are generally preferable to current pulses, since voltage-resolved linearization more rapidly quantifies degradation without sacrificing accuracy or allowing significant state changes during the measurement. In addition, asymmetric amounts of information gain between charge /discharge pulses are found from differences in electrode kinetic scales. We demonstrate our approach of physics-informed HPPC on simulated Li-ion batteries with nickel-rich cathodes and graphite anodes. Multivariable optimization by physics-informed HPPC rapidly determines kinetic parameters that correlate with degradation phenomena at the anode, such as solid-electrolyte interphase (SEI) growth and lithium plating, as well as at the cathode, such as oxidation-induced cation disorder. If validated experimentally, standardized voltage protocols for HPPC tests could play a pivotal role in expediting battery SoH assessment and accelerating materials design by providing new electrochemical features for interpretable machine learning of battery degradation.

Automated Classification of Indian Mango Varieties Using Machine Learning and MobileNet-v2 Deep Features

Automated Classification of Indian Mango Varieties Using Machine Learning and MobileNet-v2 Deep Features

Calibrating Deep Learning Classifiers for Patient-Independent Electroencephalogram Seizure Forecasting.

The recent scientific literature abounds in proposals of seizure forecasting methods that exploit machine learning to automatically analyze electroencephalogram (EEG) signals. Deep learning algorithms seem to achieve a particularly remarkable performance, suggesting that the implementation of clinical devices for seizure prediction might be within reach. However, most of the research evaluated the robustness of automatic forecasting methods through randomized cross-validation techniques, while clinical applications require much more stringent validation based on patient-independent testing. In this study, we show that automatic seizure forecasting can be performed, to some extent, even on independent patients who have never been seen during the training phase, thanks to the implementation of a simple calibration pipeline that can fine-tune deep learning models, even on a single epileptic event recorded from a new patient. We evaluate our calibration procedure using two datasets containing EEG signals recorded from a large cohort of epileptic subjects, demonstrating that the forecast accuracy of deep learning methods can increase on average by more than 20%, and that performance improves systematically in all independent patients. We further show that our calibration procedure works best for deep learning models, but can also be successfully applied to machine learning algorithms based on engineered signal features. Although our method still requires at least one epileptic event per patient to calibrate the forecasting model, we conclude that focusing on realistic validation methods allows to more reliably compare different machine learning approaches for seizure prediction, enabling the implementation of robust and effective forecasting systems that can be used in daily healthcare practice.

Machine Learning-Based Prediction of Reduction Potentials for PtIV Complexes.

Some of the well-known drawbacks of clinically approved PtII complexes can be overcome using six-coordinate PtIV complexes as inert prodrugs, which release the corresponding four-coordinate active PtII species upon reduction by cellular reducing agents. Therefore, the key factor of PtIV prodrug mechanism of action is their tendency to be reduced which, when the involved mechanism is of outer-sphere type, is measured by the value of the reduction potential. Machine learning (ML) models can be used to effectively capture intricate relationships within PtIV complex data, leading to highly accurate predictions of reduction potentials and other properties, and offering significant insights into their electrochemical behavior and potential applications. In this study, a machine learning-based approach for predicting the reduction potentials of PtIV complexes based on relevant molecular descriptors is presented. Leveraging a data set of experimentally determined reduction potentials and a diverse range of molecular descriptors, the proposed model demonstrates remarkable predictive accuracy (MSE = 0.016 V2, RMSE = 0.13 V, R2 = 0.92). Ab initio calculations and a set of different machine learning algorithms and feature engineering techniques have been employed to systematically explore the relationship between molecular structure and similarity and reduction potential. Specifically, it has been investigated whether the reduction potential of these compounds can be described by combining ML models across different combinations of constitutional, topological, and electronic molecular descriptors. Our results not only provide insights into the crucial factors influencing reduction potentials but also offer a rapid and effective tool for the rational design of PtIV complexes with tailored electrochemical properties for pharmaceutical applications. This approach has the potential to significantly expedite the development and screening of novel PtIV prodrug candidates. The analysis of principal components and key features extracted from the model highlights the significance of structural descriptors of the 2D Atom Pairs type and the lowest unoccupied molecular orbital energy. Specifically, with just 20 appropriately selected descriptors, a notable separation of complexes based on their reduction potential value is achieved.

Machine learning-based wood anatomy identification: towards anatomical feature recognition

Summary Computer vision-based wood identification has been successfully applied to recognize tree species using digital images of wood sections or surfaces. However, this image-to-species approach can only recognize a limited number of species due to two main reasons: 1) the lack of a good reference database requiring high-quality standardized images from multiple individuals of hundreds or even thousands of traded timber species, and 2) species not included in the reference database cannot be identified without expert knowledge. Another bottleneck is that the feature extraction process used by these species recognition approaches is a black box, thereby creating a discrepancy between machine learning features and wood anatomical features. This discrepancy prevents wood anatomists from understanding how these machine-learning algorithms work. Here, we survey currently existing methods used in feature extraction, classification, and deep learning methods applied in wood identification along with their pitfalls and opportunities. As an example of how the field could move forward, we launch the idea of building an image-to-features-to-species identification approach based on microscopic wood images as well as text files comprising wood anatomical descriptions. If we can manage machine learning-based algorithms to recognize the main wood anatomical traits that experts use to identify species in a (semi-)automated way, this would boost wood identification in two ways: (1) extensive reference databases for each species would become less crucial as the databases are ordered at the trait level, (2) timber identification would become more feasible for species that have not yet been included in the reference database as long as wood anatomical descriptions are available.

Fraud risk assessment in car insurance using claims graph features in machine learning

The article proposes a process for claims assessment in car insurance, which makes it possible to calculate the fraud rate on the annual set of claims using a reduced set of attributes and graph vertex properties. This approach improves the security of insurance companies’ assets against fraudulent attacks. A method for constructing a claims graph and extracting additional features from it for evaluation is described. It is shown that in order to build a graph, it is not necessary to have data on the connection of the claim participants. Two tests were carried out on a real opensource datasets with labelling of fraudulent cases. The results of the first one show the increase in classification metrics when using attributes obtained from the graph. The application of the proposed approach resulted in doubling the area under the Precision-Recall curve. The experimental results demonstrated high quality metrics for fraud detection, with a Recall rate of 83.33% and a Specificity rate of 91.05%. The second test confirmed the possibility of determining the insurance fraud level based on decision rule, which includes the condition of claims being connected to each other. The rule is able to detect claim groups with a high concentration of fraud, in which every second participant is a fraudster.

Fusing image and physical data for fatigue life prediction of nickel-based single crystal superalloys

This paper introduces a method for predicting the fatigue life of single crystals, which fuses image data and physical data by exploiting machine learning techniques. The proposed approach establishes a comprehensive study by progressively contributing a new database to the field, enabling the inference of service conditions, including temperature and σmax, as well as inherent material properties from fracture images, such as crystal orientation. We evaluate the effectiveness of the proposed method via its application to the fatigue life prediction of DD6 alloy, which achieves promising performance across various crystal orientations. The extensive experimental analysis and evaluation suggest that our proposed method has high predictive accuracy and computational efficiency of our proposed method. These results show that our approach holds promise for enhancing our understanding of material fatigue and improving predictive precision in failure analysis.

Survey on Yogic Posture Recognition using Deep Learning

The abstract emphasizes the growing importance of yoga pose detection, especially in integrating technology into yoga practice. It highlights the various benefits of yoga, such as physical health, mental well-being, and stress reduction. The paper discusses the significance of automated yoga pose detection in providing real-time feedback and enhancing self-correction. The research explores different approaches to yoga pose classification, focusing on PoseNet and KNN classifier. Using deep learning algorithms and computer vision techniques like Open Pose, human pose estimation is employed to identify yoga postures. A combination of CNN and LSTM is utilized for real-time yoga pose recognition from monitored videos. Existing techniques for yoga pose recognition often fail in real-world situations, prompting the need for more efficient methods. The paper presents a computationally efficient approach using deep learning for yoga pose recognition in complex environments. Artificial intelligence is proposed as a substitute for yoga instructors, assisting individuals with their poses. The paper introduces algorithms for creating skeletons of yoga poses and hand mudras, extracting joint angles as features for machine learning and deep learning models. Furthermore, deep learning techniques are developed to detect incorrect yoga postures, enabling users to upload recorded videos for analysis. The system advises users on improving their poses by identifying abnormal angles between the actual and desired poses

A machine learning based method for parametric environmental impact model for electric vehicles

Environmental managers attempt to increasingly incorporate precautionary principles into decision making. The literature lacks Machine Learning -based approaches for forewarning lifecycle environmental impacts. This paper proposes a method to support electric vehicle design. The main innovation of the work lays in merging Life Cycle Assessment (LCA) and Machine Learning foundations to provide support and awareness to designers. The present approach overcomes the present literature because it provides a method for the design phase, is based on a well-established methodology (LCA) and provides quantitative results from little inputs. The approach exploits machine Learning Methods to develop models with the design features of a generic electric vehicle (such as vehicle mass and distance traveled) in six phases (Problem definition; Data collection; Data Preparation; Modeling; Model evaluation; Model interpretation). Differently from existing environmental analyses, all stages of the product life cycle have been considered in building the database; moreover, the model provides quantitative results. Regression models and supervised algorithms were used. The obtained model can be used by product engineers, as well as those not experts on LCA. Moreover, the model guarantees the database and hypothesis's uniqueness, ensuring the results coherence and comparability. The level of accuracy obtained in the case study (error or 17%) is comparable with studies handling full environmental analysis (that should be more accurate), and outstanding, as the present case is for the design phase. Future works will focus on additional significative indicators, similar electric vehicle design and integration with prospective LCA approaches.

Radiomic- and dosiomic-based clustering development for radio-induced neurotoxicity in pediatric medulloblastoma

BackgroundTexture analysis extracts many quantitative image features, offering a valuable, cost-effective, and non-invasive approach for individual medicine. Furthermore, multimodal machine learning could have a large impact for precision medicine, as texture biomarkers can underlie tissue microstructure. This study aims to investigate imaging-based biomarkers of radio-induced neurotoxicity in pediatric patients with metastatic medulloblastoma, using radiomic and dosiomic analysis.MethodsThis single-center study retrospectively enrolled children diagnosed with metastatic medulloblastoma (MB) and treated with hyperfractionated craniospinal irradiation (CSI). Histological confirmation of medulloblastoma and baseline follow-up magnetic resonance imaging (MRI) were mandatory. Treatment involved helical tomotherapy (HT) delivering a dose of 39 Gray (Gy) to brain and spinal axis and a posterior fossa boost up to 60 Gy. Clinical outcomes, such as local and distant brain control and neurotoxicity, were recorded. Radiomic and dosiomic features were extracted from tumor regions on T1, T2, FLAIR (fluid-attenuated inversion recovery) MRI-maps, and radiotherapy dose distribution. Different machine learning feature selection and reduction approaches were performed for supervised and unsupervised clustering.ResultsForty-eight metastatic medulloblastoma patients (29 males and 19 females) with a mean age of 12 ± 6 years were enrolled. For each patient, 332 features were extracted. Greater level of abstraction of input data by combining selection of most performing features and dimensionality reduction returns the best performance. The resulting one-component radiomic signature yielded an accuracy of 0.73 with sensitivity, specificity, and precision of 0.83, 0.64, and 0.68, respectively.ConclusionsMachine learning radiomic-dosiomic approach effectively stratified pediatric medulloblastoma patients who experienced radio-induced neurotoxicity. Strategy needs further validation in external dataset for its potential clinical use in ab initio management paradigms of medulloblastoma.

A Framework for Detection of Malicious Code by Exploiting Machine Learning Techniques on Portable Executables

Executable files coming from the internet bring along with them many potential hazards and vul- nerabilities in the form of malware to computer systems. The executables can be of form raw binaries, mnemonics, libraries, and function calls/APIs. They can misguide many of the conventional malware detection techniques. This paper explores the potential of Machine Learning- based methods for malware detection problems. The scope of the work here is currently limited to Static Anal- ysis of Executable files. Various feature selection tech- niques are implemented to reduce the size of the training data. Machine learning algorithms like K-Nearest Neigh- bors and Random Forest Classifier were trained on the curated feature sets. The outperforming experiment re- sult was shown by the Random Forest Classifier having an accuracy of 99.5%. We have developed a framework as a two-step module; in the first step, a list of features are extracted from a given executable file, and then for the next step, trained algorithm is integrated into the framework which will classify whether the given executa- ble file is malicious or not. This framework is demon- strated in the form of a Webapp developed in Python. Furthermore, this framework is evaluated based on its performance on a small dataset containing 35 portable executables (.exe) files and it is observed to be retaining the accuracy of the trained algorithm.

Microenvironmental reorganization in brain tumors following radiotherapy and recurrence revealed by hyperplexed immunofluorescence imaging

The tumor microenvironment plays a crucial role in determining response to treatment. This involves a series of interconnected changes in the cellular landscape, spatial organization, and extracellular matrix composition. However, assessing these alterations simultaneously is challenging from a spatial perspective, due to the limitations of current high-dimensional imaging techniques and the extent of intratumoral heterogeneity over large lesion areas. In this study, we introduce a spatial proteomic workflow termed Hyperplexed Immunofluorescence Imaging (HIFI) that overcomes these limitations. HIFI allows for the simultaneous analysis of > 45 markers in fragile tissue sections at high magnification, using a cost-effective high-throughput workflow. We integrate HIFI with machine learning feature detection, graph-based network analysis, and cluster-based neighborhood analysis to analyze the microenvironment response to radiation therapy in a preclinical model of glioblastoma, and compare this response to a mouse model of breast-to-brain metastasis. Here we show that glioblastomas undergo extensive spatial reorganization of immune cell populations and structural architecture in response to treatment, while brain metastases show no comparable reorganization. Our integrated spatial analyses reveal highly divergent responses to radiation therapy between brain tumor models, despite equivalent radiotherapy benefit.

Sensor-Based Indoor Fire Forecasting Using Transformer Encoder.

Indoor fires may cause casualties and property damage, so it is important to develop a system that predicts fires in advance. There have been studies to predict potential fires using sensor values, and they mostly exploited machine learning models or recurrent neural networks. In this paper, we propose a stack of Transformer encoders for fire prediction using multiple sensors. Our model takes the time-series values collected from the sensors as input, and predicts the potential fire based on the sequential patterns underlying the time-series data. We compared our model with traditional machine learning models and recurrent neural networks on two datasets. For a simple dataset, we found that the machine learning models are better than ours, whereas our model gave better performance for a complex dataset. This implies that our model has a greater potential for real-world applications that probably have complex patterns and scenarios.

Predictive modeling for breast cancer based on machine learning algorithms and features selection methods

Breast cancer is one of the leading causes of death among women worldwide. However, early prediction of breast cancer plays a crucial role. Therefore, strong needs exist for automatic accurate early prediction of breast cancer. In this paper, machine learning (ML) classifiers combined with features selection methods are used to build an intelligent tool for breast cancer prediction. The Wisconsin diagnostic breast cancer (WDBC) dataset is used to train and test the model. Classification algorithms, including support vector machine (SVM), light gradient boosting machine (LightGBM), random forest (RF), logistic regression (LR), k-nearest neighbors (k-NN), and naïve Bayes, were employed. Performance measures for each of them were obtained, namely: accuracy, precision, recall, F-score, Kappa, Matthews correlation coefficient (MCC), and time. The results indicate that without feature selection, LightGBM achieves the highest accuracy at 95%. With minimum redundancy maximum relevance (mRMR) feature selection (15 features), LightGBM outperforms other classifiers, achieving an accuracy of 98%. For Pearson correlation coefficient feature selection (15 features), LightGBM also excels with a 95% accuracy rate. Lasso feature selection (5 features) produces varied results across classifiers, with logistic regression achieving the highest accuracy at 96%. These findings underscore the importance of feature selection in refining model performance and in improving detection for breast cancer.

Merger-tree-based Galaxy Matching: A Comparative Study across Different Resolutions

We introduce a novel halo/galaxy matching technique between two cosmological simulations with different resolutions, which utilizes the positions and masses of halos along their subhalo merger tree. With this tool, we conduct a study of resolution biases through the galaxy-by-galaxy inspection of a pair of simulations that have the same simulation configuration but different mass resolutions, utilizing a suite of IllustrisTNG simulations to assess the impact on galaxy properties. We find that, with the subgrid physics model calibrated for TNG100-1, subhalos in TNG100-1 (high resolution) have ≲0.5 dex higher stellar masses than their counterparts in the TNG100-2 (low resolution). It is also discovered that the subhalos with M gas ∼ 108.5 M ⊙ in TNG100-1 have ∼0.5 dex higher gas mass than those in TNG100-2. The mass profiles of the subhalos reveal that the dark matter masses of subhalos in TNG100-2 converge well with those from TNG100-1, except within 4 kpc of the resolution limit. The differences in stellar mass and hot gas mass are most pronounced in the central region. We exploit machine learning to build a correction mapping for the physical quantities of subhalos from low- to high-resolution simulations (TNG300-1 and TNG100-1), which enables us to find an efficient way to compile a high-resolution galaxy catalog even from a low-resolution simulation. Our tools can easily be applied to other large cosmological simulations, testing and mitigating the resolution biases of their numerical codes and subgrid physics models.

Digitally Controlled Printing of Bioink Barriers for Paper-Based Analytical Devices: An Environmentally Friendly One-Step Approach.

The patterning of hydrophilic paper with hydrophobic materials has emerged as an interesting method for the fabrication of paper-based devices (PADs). Herein, we demonstrate a digitally automated, easy, low-cost, eco-friendly, and readily available method to create highly hydrophobic barriers on paper that can be promptly employed with PADs by simply using a bioink made with rosin, a commercially available natural resin obtained from conifer trees. The bioink can be easily delivered with the use of a ballpoint pen to produce water- and organic solvent-resistant barriers, showing superior properties when compared to other methods such as wax-printing or permanent markers. The approach enables the pen to be attached to a commercially available cutting printer to perform the semiautomated fabrication of hydrophobic barriers for PADs. With the aid of digitally controlled optimization, together with features of machine learning and design of experiments, we show a thorough investigation on the barrier strength that can be further adjusted to the desired application's needs. Then, we explored the barrier sturdiness across various uses, such as wide range aqueous pH sensing and the harsh acidic/organic conditions needed for the colorimetric detection of cholecalciferol.