Feature Machine Learning Research Articles

Hybrid machine learning for stock price prediction in the Moroccan banking sector

Analyzing historical stock market data using machine-learning techniques is crucial for data scientists and researchers to optimize stock price prediction models. This study uses machine learning regression algorithms and feature selection methods to optimize a simulated stock price prediction model using real historical data from Bank of Africa, a Moroccan bank. The approach compares multiple supervised regression algorithms, such as linear regression, extreme gradient boosting, ordinary least squared, random forest regressor, a linear least-squares L2-regularized, epsilon-support vector regression, and linear support vector regression. Each of these algorithms is associated with different feature selection algorithms to improve the performance of the prediction model. The analysis results revealed that hybridizing algorithms between the highest score percentiles, univariate linear regression, and linear support vector regression perform better according to the root mean squared error and R2-Score measures. This approach overcomes the problems associated with high-dimensional data by reducing the number of features and improving prediction accuracy.

Prediction of CCL2 in breast cancer based on enhanced MRI radiomics model with integrated bioinformatics analysis and machine learning.

e12509 Background: Breast cancer is the most common malignant tumor in women. According to the 2023 Cancer Statistics Report, breast cancer has surpassed lung cancer to become the most common malignant tumor. Its incidence and mortality both rank first among female malignant tumors. This study proposes non-invasive prediction of CCL2 mRNA expression in breast cancer tissue through enhanced MRI radiomics, while integrating bioinformatics analysis to explore the underlying molecular mechanisms and their association with the immune microenvironment. Methods: Excluding criteria were applied to select primary solid tumor samples with RNA-seq data (n = 840) from the TCGA-BRCA database. Enhanced MRI imaging data (n = 98) intersecting with the TCGA-BRCA data were obtained from TCIA-BRCA. The imaging data were randomly divided into training (n = 70) and testing sets (n = 28) in a 7:3 ratio. A feature subset was selected, and a model was constructed using the support vector machine (SVM) algorithm in the training set. The predictive performance of the model was evaluated using receiver operating characteristic (ROC) curves, calibration curves, and the Hosmer-Lemeshow goodness-of-fit test. The calibration of the radiomics prediction model was assessed using the Brier score, and the clinical utility of the radiomics prediction model was demonstrated by drawing decision curves (DCA). External validation of the model was performed using a validation set. The radiomics score (RS) was calculated using the LR radiomics model and categorized as a binary variable (Low/High RS). Subsequently, KEGG pathway, immune-related genes, and gene mutation analysis were conducted based on the CCL2 high and low expression levels. Results: A total of 1810 radiomic features were obtained with ICC values ≥ 0.75. Subsequently, the mRMR (Maximum relevance, minimum redundancy) method, and the optimal 7 feature subsets were further identified using the RFE (Recursive feature elimination) algorithm. The SVM model demonstrated favorable predictive performance, with an AUC value of 0.822 for the training set and 0.779 for the validation set, as indicated by the ROC curve. Calibration curve and Hosmer-Lemeshow goodness-of-fit test revealed good consistency between the radiomic predictive model for the probability of high gene expression and the actual values (P > 0.05). Additionally, the decision curve analysis (DCA) demonstrated high clinical utility of the model. The TNF-α signaling pathway was found to influence the occurrence and development of breast cancer. Furthermore, the degree of Plasma Cell infiltration was observed to be highest in breast cancer. Conclusions: The CCL2 model constructed using a combination of machine learning and radiomic features, can non-invasively predict the prognosis of breast cancer treatment and exhibits good generalizability.

NCI MATCH (EAY131) arm Z1D: Retrospective evaluation of attributions of adverse events experienced on nivolumab in patients with MSI-high tumors.

e14711 Background: Machine learning algorithms for prediction of Adverse Events (AEs) due to immune checkpoint inhibitors (ICIs) rely on clinically annotated data to serve as gold standards. Clinical trials are the best sources of such data, but audits to assess validity are rare, compromising future machine learning efforts. We reviewed AEs reported on the National Cancer Institute Molecular Analysis for Therapy Choice (NCI-MATCH) EAY-131 Arm Z1D. Patients matched to Arm Z1D due to MSI-high status were treated with nivolumab. Routine and serious AEs were reported to NCI. Attribution data from AEs on 29 patients with tumor genomics suitable for machine learning approaches were obtained and reviewed retrospectively. Methods: Documented AEs were sorted by grade via the Common Terminology for Adverse Events (CTCAE) system and categorized as related or unrelated to ICIs. Primary source data reported to NCI were reviewed. If any ambiguity was found, trial sites were queried to provide additional information and documentation. Decisions were made by an experienced NCI medical officer with the benefit of additional data and site responses. All serious AEs and a subset of ambiguous routine AEs were assessed to determine if attribution conformed to expected attribution per the nivolumab package insert. Uncertain attribution required additional queries, including email or phone correspondence. Grade 1 or 2 events were only evaluated to determine if any attribution to ICIs was possible. If a higher grade event was attributed to ICIs, there was no further query for lower grade events on the same patient. Results: A total of 639 Grade 1, 152 Grade 2, 75 Grade 3, 10 Grade 4, and 5 Grade 5 events were reported among the 29 patients. One Grade 5 event, 5 Grade 4 events, 13 Grade 3 Events, 9 Grade 2 events, and 6 Grade 1 events were considered less likely to be attributed to ICIs compared with the initial sponsor or site investigator assessment of possibly, probably, or definitely related. Two Grade 4 events, 1 Grade 2 event, and 8 Grade 1 events were considered to be possibly, probably, or definitely due to ICIs compared with the initial sponsor or site assessment of less likely or unrelated. Site responsiveness to retrospective queries from the sponsor varied. Additional lab values, primary source documentation, and communication with site investigators were necessary to make modifications. A total of 11 AEs required additional queries to site investigators. Conclusions: Retrospective reviews of real-time site and sponsor assessments of attribution exhibit significant modification of initial attribution to ICIs in this review of 29 patients on Arm Z1D of the NCI MATCH trial. Machine learning approaches for prediction of AEs on patients treated with ICIs may require careful assessments of primary source data to develop adequate training sets, given the paucity of gold standard data available.

Machine Learning Approach to Detect Depression through Social Media Posts

Abstract: Depression is one of the biggest issues which is increasing rapidly day by day. It has always been a topic for various researchers to discover new methods and techniques to overcome it. It is basically a kind of disorder in which people do not feel happy and joy. Since with the increase of the social networking sites people are expressing their feelings on the social media platforms which are helping the various researchers to analyze these posts and find an effective solution. Now a days there are various studies that aim to exploit machine learning techniques for detecting a probable depressed Twitter (Now X) user tweets. Textual data can be collected from the comments on the post, exchanging information etc. Using Facebook, Twitter, and Instagram. There is a huge amount of data which is present on these platforms which can be used for research. The aim of this study is that whether machine learning can be able to effectively detect the various signs of depression by analyzing the user post. This paper is basically on the sentimental analysis using NLP that identifies that the post is positive, negative or neutral.

User Behavior Analysis Using Web-based Machine Learning Features: New Solutions for IT Business

User Behavior Analysis Using Web-based Machine Learning Features: New Solutions for IT Business

Tree-based Machine Learning Ensembles and Feature Importance Approach for the Identification of Intrusions in UNR-IDD Dataset

Detection of intrusions from network data with the use of machine learning techniques has gained great attention in the past decades. One of the key problems in the network security domain is the availability of representative datasets for testing and evaluation purposes. Despite several efforts by researchers to release datasets that can be used for benchmarking attack detection models, some of the released datasets still suffer from one limitation or the other. Thus, some researchers at the University of Nevada released a dataset named UNR-IDD dataset which was argued to be free from some of the limitations of the past datasets. This study proposed Tree-based ensemble approaches for building binary intrusion identification models from the UNR-IDD dataset. Decision Tree algorithms are used as base classifiers in the Extra Trees, Random Forest and AdaBoost-based intrusion detection models. The results of the experimental analyses carried out indicated that the three ensembles performed excellently when feature selection was used compared to when all features were applied. For instance, Extra Trees model achieved an accuracy of 0.97, precision of 0.98, recall of 0.98 and f1-score of 0.98. Similarly, Random Forest model achieved an accuracy of 0.98, precision of 0.98, recall of 0.99 and f1-score of 0.98. Adaboost-based model had an accuracy of 0.96, precision of 0.96, recall of 0.99 and f1-score of 0.98. It was deduced that Random Forest intrusion classification model achieved slight overall best results when compared to the other models built. It is concluded that the three homogeneous ensemble models achieved very promising results while feature importance was used as attribute selection method.

CAIMP: Cross-Architecture IoT Malware Detection and Prediction Based On Static Feature

Abstract IoT malware and cross-platform malware are currently the top threats to information systems. This paper proposes a robust cross-architecture IoT malware detection and prediction model based on machine learning and opcode features using a novel approach. In our method, a feature opcode transformation model between chip architecture platforms is proposed to facilitate the process of building a detection model for cross-architecture malware on IoT devices. The feature transformation model is capable of converting opcodes between different architecture platforms using an unsupervised machine learning approach. In our approach, a machine learning model is used for the detection of cross-platform malware based on the proposed opcode features. Experiments have demonstrated that our method is effective in detecting and predicting cross-platform malware with an accuracy of up to 99.4% and an F1-score of 99.3%. The method is capable of learning on one architecture platform and detecting malware on a different architecture platform. Therefore, the method can be used to develop cross-architecture detection and zero-day malware prediction solutions on IoT devices.

Improved pediatric ICU mortality prediction for respiratory diseases: machine learning and data subdivision insights

The growing concern of pediatric mortality demands heightened preparedness in clinical settings, especially within intensive care units (ICUs). As respiratory-related admissions account for a substantial portion of pediatric illnesses, there is a pressing need to predict ICU mortality in these cases. This study based on data from 1188 patients, addresses this imperative using machine learning techniques and investigating different class balancing methods for pediatric ICU mortality prediction. This study employs the publicly accessible “Paediatric Intensive Care database” to train, validate, and test a machine learning model for predicting pediatric patient mortality. Features were ranked using three machine learning feature selection techniques, namely Random Forest, Extra Trees, and XGBoost, resulting in the selection of 16 critical features from a total of 105 features. Ten machine learning models and ensemble techniques are used to make accurate mortality predictions. To tackle the inherent class imbalance in the dataset, we applied a unique data partitioning technique to enhance the model's alignment with the data distribution. The CatBoost machine learning model achieved an area under the curve (AUC) of 72.22%, while the stacking ensemble model yielded an AUC of 60.59% for mortality prediction. The proposed subdivision technique, on the other hand, provides a significant improvement in performance metrics, with an AUC of 85.2% and an accuracy of 89.32%. These findings emphasize the potential of machine learning in enhancing pediatric mortality prediction and inform strategies for improved ICU readiness.

How much can ChatGPT really help computational biologists in programming?

ChatGPT, a recently developed product by openAI, is successfully leaving its mark as a multi-purpose natural language based chatbot. In this paper, we are more interested in analyzing its potential in the field of computational biology. A major share of work done by computational biologists these days involve coding up bioinformatics algorithms, analyzing data, creating pipelining scripts and even machine learning modeling and feature extraction. This paper focuses on the potential influence (both positive and negative) of ChatGPT in the mentioned aspects with illustrative examples from different perspectives. Compared to other fields of computer science, computational biology has (1) less coding resources, (2) more sensitivity and bias issues (deals with medical data), and (3) more necessity of coding assistance (people from diverse background come to this field). Keeping such issues in mind, we cover use cases such as code writing, reviewing, debugging, converting, refactoring, and pipelining using ChatGPT from the perspective of computational biologists in this paper.

Exploiting Machine Learning Bias: Predicting Medical Denials

For a large healthcare system, ignoring costs associated with managing the patient encounter denial process (staffing, contracts, etc.), total denial-related amounts can be more than $1B annually in gross charges. Being able to predict a denial before it occurs has the potential for tremendous savings. Using machine learning to predict denial has the potential to allow denial-preventing interventions. However, challenges of data imbalance make creating a single generalized model difficult. We employ two biased models in a hybrid voting scheme to achieve results that exceed the state-of-the art and allow for incremental predictions as the encounter progresses. The model had the added benefit of monitoring the human-driven denial process that affect the underlying distribution, on which the models’ bias is based.

Machine learning feature importance selection for predicting aboveground biomass in African savannah with landsat 8 and ALOS PALSAR data

In remote sensing, multiple input bands are derived from various sensors covering different regions of the electromagnetic spectrum. Each spectral band plays a unique role in land use/land cover characterization. For example, while integrating multiple sensors for predicting aboveground biomass (AGB) is important for achieving high accuracy, reducing the dataset size by eliminating redundant and irrelevant spectral features is essential for enhancing the performance of machine learning algorithms. This accelerates the learning process, thereby developing simpler and more efficient models. Our results indicate that compared individual sensor datasets, the random forest (RF) classification approach using recursive feature elimination (RFE) increased the accuracy based on F score by 82.86 % and 26.19 respectively. The mutual information regression (MIR) method shows a slight increase in accuracy when considering individual sensor datasets, but its accuracy decreases when all features are taken into account for all models. Overall, the combination of features from the Landsat 8, ALOS PALSAR backscatter, and elevation data selected based on RFE provided the best AGB estimation for the RF and XGBoost models. In contrast to the k-nearest neighbors (KNN) and support vector machines (SVM), no significant improvement in AGB estimation was detected even when RFE and MIR were used. The effect of parameter optimization was found to be more significant for RF than for all the other methods. The AGB maps show patterns of AGB estimates consistent with those of the reference dataset. This study shows how prediction errors can be minimized based on feature selection using different ML classifiers.

Precision healthcare: A deep dive into machine learning algorithms and feature selection strategies for accurate heart disease prediction

This paper presents a comprehensive exploration of machine learning algorithms (MLAs) and feature selection techniques for accurate heart disease prediction (HDP) in modern healthcare. By focusing on diverse datasets encompassing various challenges, the research sheds light on optimal strategies for early detection. MLAs such as Decision Trees (DT), Random Forests (RF), Support Vector Machines (SVM), Gaussian Naive Bayes (NB), and others were studied, with precision and recall metrics emphasized for robust predictions. Our study addresses challenges in real-world data through data cleaning and one-hot encoding, enhancing the integrity of our predictive models. Feature extraction techniques—Recursive Feature Extraction (RFE), Principal Component Analysis (PCA), and univariate feature selection—play a crucial role in identifying relevant features and reducing data dimensionality. Our findings showcase the impact of these techniques on improving prediction accuracy. Optimized models for each dataset have been achieved through grid search hyperparameter tuning, with configurations meticulously outlined. Notably, a remarkable 99.12 % accuracy was achieved on the first Kaggle dataset, showcasing the potential for accurate HDP. Model robustness across diverse datasets was highlighted, with caution against overfitting. The study emphasizes the need for validation of unseen data and encourages ongoing research for generalizability. Serving as a practical guide, this research aids researchers and practitioners in HDP model development, influencing clinical decisions and healthcare resource allocation. By providing insights into effective algorithms and techniques, the paper contributes to reducing heart disease-related morbidity and mortality, supporting the healthcare community's ongoing efforts.

Comparison of Different Fusion Radiomics for Predicting Benign and Malignant Sacral Tumors: A Pilot Study.

Differentiating between benign and malignant sacral tumors is crucial for determining appropriate treatment options. This study aims to develop two benchmark fusion models and a deep learning radiomic nomogram (DLRN) capable of distinguishing between benign and malignant sacral tumors using multiple imaging modalities. We reviewed axial T2-weighted imaging (T2WI) and non-contrast computed tomography (NCCT) of 134 patients pathologically confirmed as sacral tumors. The two benchmark fusion models were developed using fusion deep learning (DL) features and fusion classical machine learning (CML) features from multiple imaging modalities, employing logistic regression, K-nearest neighbor classification, and extremely randomized trees. The two benchmark models exhibiting the most robust predictive performance were merged with clinical data to formulate the DLRN. Performance assessment involved computing the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, accuracy, negative predictive value (NPV), and positive predictive value (PPV). The DL benchmark fusion model demonstrated superior performance compared to the CML fusion model. The DLRN, identified as the optimal model, exhibited the highest predictive performance, achieving an accuracy of 0.889 and an AUC of 0.961 in the test sets. Calibration curves were utilized to evaluate the predictive capability of the models, and decision curve analysis (DCA) was conducted to assess the clinical net benefit of the DLR model. The DLRN could serve as a practical predictive tool, capable of distinguishing between benign and malignant sacral tumors, offering valuable information for risk counseling, and aiding in clinical treatment decisions.

Approach to Detecting Malicious Bots in the Vkontakte Social Network and Assessing Their Parameters

The emergence of new varieties of bots in social networks and the improvement of their capabilities to imitate the natural behavior of real users represent a significant problem in the field of protection of social networks and online communities. This paper proposes a new approach to detecting and assessing the parameters of bots within the social network «VKontakte». The basis of the proposed approach is the creation of datasets using the method of «controlled purchase» of bots, which allows one to assess bots’ characteristics such as price, quality, and speed of action of bots, and using the Turing Test to assess how much users trust bots. In combination with traditional machine learning methods and features extracted from interaction graphs, text messages, and statistical distributions, it becomes possible to not only detect bots accurately but also predict their characteristics. This paper demonstrates that the trained machine learning model, based on the proposed approach, is robust to imbalanced data and can identify most types of bots as it has only a minor correlation with their main characteristics. The proposed approach can be used within the choice of countermeasures for the protection of social networks and for historical analysis, which allows not only to confirm the presence of bots but also to characterize the specifics of the attack.

Building datasets to support information extraction and structure parsing from electronic theses and dissertations

Despite the millions of electronic theses and dissertations (ETDs) publicly available online, digital library services for ETDs have not evolved past simple search and browse at the metadata level. We need better digital library services that allow users to discover and explore the content buried in these long documents. Recent advances in machine learning have shown promising results for decomposing documents into their constituent parts, but these models and techniques require data for training and evaluation. In this article, we present high-quality datasets to train, evaluate, and compare machine learning methods in tasks that are specifically suited to identify and extract key elements of ETD documents. We explain how we construct the datasets by manual labeling the data or by deriving labeled data through synthetic processes. We demonstrate how our datasets can be used to develop downstream applications and to evaluate, retrain, or fine-tune pre-trained machine learning models. We describe our ongoing work to compile benchmark datasets and exploit machine learning techniques to build intelligent digital libraries for ETDs.

Physics-Informed Design of Hybrid Pulse Power Characterization Tests for Rechargeable Batteries

Industry-standard diagnostic methods for rechargeable batteries, such as hybrid pulse power characterization (HPPC) tests for hybrid electric vehicles, provide some indications of state of health (SoH), but lack a physical basis to guide protocol design and identify degradation mechanisms. We develop a physics-based theoretical framework for HPPC tests, which are able to accurately determine specific mechanisms for battery degradation in porous electrode simulations. We show that voltage pulses are generally preferable to current pulses, since voltage-resolved linearization more rapidly quantifies degradation without sacrificing accuracy or allowing significant state changes during the measurement. In addition, asymmetric amounts of information gain between charge /discharge pulses are found from differences in electrode kinetic scales. We demonstrate our approach of physics-informed HPPC on simulated Li-ion batteries with nickel-rich cathodes and graphite anodes. Multivariable optimization by physics-informed HPPC rapidly determines kinetic parameters that correlate with degradation phenomena at the anode, such as solid-electrolyte interphase (SEI) growth and lithium plating, as well as at the cathode, such as oxidation-induced cation disorder. If validated experimentally, standardized voltage protocols for HPPC tests could play a pivotal role in expediting battery SoH assessment and accelerating materials design by providing new electrochemical features for interpretable machine learning of battery degradation.

Automated Classification of Indian Mango Varieties Using Machine Learning and MobileNet-v2 Deep Features

Automated Classification of Indian Mango Varieties Using Machine Learning and MobileNet-v2 Deep Features

Calibrating Deep Learning Classifiers for Patient-Independent Electroencephalogram Seizure Forecasting.

The recent scientific literature abounds in proposals of seizure forecasting methods that exploit machine learning to automatically analyze electroencephalogram (EEG) signals. Deep learning algorithms seem to achieve a particularly remarkable performance, suggesting that the implementation of clinical devices for seizure prediction might be within reach. However, most of the research evaluated the robustness of automatic forecasting methods through randomized cross-validation techniques, while clinical applications require much more stringent validation based on patient-independent testing. In this study, we show that automatic seizure forecasting can be performed, to some extent, even on independent patients who have never been seen during the training phase, thanks to the implementation of a simple calibration pipeline that can fine-tune deep learning models, even on a single epileptic event recorded from a new patient. We evaluate our calibration procedure using two datasets containing EEG signals recorded from a large cohort of epileptic subjects, demonstrating that the forecast accuracy of deep learning methods can increase on average by more than 20%, and that performance improves systematically in all independent patients. We further show that our calibration procedure works best for deep learning models, but can also be successfully applied to machine learning algorithms based on engineered signal features. Although our method still requires at least one epileptic event per patient to calibrate the forecasting model, we conclude that focusing on realistic validation methods allows to more reliably compare different machine learning approaches for seizure prediction, enabling the implementation of robust and effective forecasting systems that can be used in daily healthcare practice.

Machine Learning-Based Prediction of Reduction Potentials for PtIV Complexes.

Some of the well-known drawbacks of clinically approved PtII complexes can be overcome using six-coordinate PtIV complexes as inert prodrugs, which release the corresponding four-coordinate active PtII species upon reduction by cellular reducing agents. Therefore, the key factor of PtIV prodrug mechanism of action is their tendency to be reduced which, when the involved mechanism is of outer-sphere type, is measured by the value of the reduction potential. Machine learning (ML) models can be used to effectively capture intricate relationships within PtIV complex data, leading to highly accurate predictions of reduction potentials and other properties, and offering significant insights into their electrochemical behavior and potential applications. In this study, a machine learning-based approach for predicting the reduction potentials of PtIV complexes based on relevant molecular descriptors is presented. Leveraging a data set of experimentally determined reduction potentials and a diverse range of molecular descriptors, the proposed model demonstrates remarkable predictive accuracy (MSE = 0.016 V2, RMSE = 0.13 V, R2 = 0.92). Ab initio calculations and a set of different machine learning algorithms and feature engineering techniques have been employed to systematically explore the relationship between molecular structure and similarity and reduction potential. Specifically, it has been investigated whether the reduction potential of these compounds can be described by combining ML models across different combinations of constitutional, topological, and electronic molecular descriptors. Our results not only provide insights into the crucial factors influencing reduction potentials but also offer a rapid and effective tool for the rational design of PtIV complexes with tailored electrochemical properties for pharmaceutical applications. This approach has the potential to significantly expedite the development and screening of novel PtIV prodrug candidates. The analysis of principal components and key features extracted from the model highlights the significance of structural descriptors of the 2D Atom Pairs type and the lowest unoccupied molecular orbital energy. Specifically, with just 20 appropriately selected descriptors, a notable separation of complexes based on their reduction potential value is achieved.

Machine learning-based wood anatomy identification: towards anatomical feature recognition

Summary Computer vision-based wood identification has been successfully applied to recognize tree species using digital images of wood sections or surfaces. However, this image-to-species approach can only recognize a limited number of species due to two main reasons: 1) the lack of a good reference database requiring high-quality standardized images from multiple individuals of hundreds or even thousands of traded timber species, and 2) species not included in the reference database cannot be identified without expert knowledge. Another bottleneck is that the feature extraction process used by these species recognition approaches is a black box, thereby creating a discrepancy between machine learning features and wood anatomical features. This discrepancy prevents wood anatomists from understanding how these machine-learning algorithms work. Here, we survey currently existing methods used in feature extraction, classification, and deep learning methods applied in wood identification along with their pitfalls and opportunities. As an example of how the field could move forward, we launch the idea of building an image-to-features-to-species identification approach based on microscopic wood images as well as text files comprising wood anatomical descriptions. If we can manage machine learning-based algorithms to recognize the main wood anatomical traits that experts use to identify species in a (semi-)automated way, this would boost wood identification in two ways: (1) extensive reference databases for each species would become less crucial as the databases are ordered at the trait level, (2) timber identification would become more feasible for species that have not yet been included in the reference database as long as wood anatomical descriptions are available.