Static Machine Learning Research Articles

Integrating Feature Selection with Machine Learning for Accurate Reservoir Landslide Displacement Prediction

Accurate prediction of reservoir landslide displacements is crucial for early warning and hazard prevention. Current machine learning (ML) paradigms for predicting landslide displacement demonstrate superior performance, while often relying on various feature engineering techniques, such as decomposing into different temporal lags and feature selection. This study investigates the impact of various feature selection techniques on the performance of ML algorithms for landslide displacement prediction. The Shuping and Baishuihe landslides in China’s Three Gorges Reservoir Area are used to comprehensively benchmark four prevalent ML algorithms. Both static ML models, including backpropagation neural network (BPNN), support vector machine (SVM), and dynamic models, such as long short-term memory (LSTM), and gated recurrent unit (GRU), are included. Each ML model is evaluated under three feature engineering techniques: raw multivariate time series, and feature selection under maximal information coefficient-partial autocorrelation function (MIC-PACF), or grey relational analysis-PACF (GRA-PACF). The results demonstrate that appropriate feature selection methods could significantly improve the performance of static ML models. In contrast, dynamic models effectively leverage inherent capabilities in capturing temporal dynamics within raw multivariate time series, seeing marginal gains with extensive feature engineering compared to no feature selection strategy. The optimal feature selection approach varies based on the ML model and specific landslide, highlighting the importance of case-specific assessments. The findings in this study offer guidance on integrating feature selection techniques with different machine learning models to maximize the robustness and generalizability of data-driven landslide displacement prediction frameworks.

LungEcho - Resource Constrained Lung Ultrasound Video Analysis Tool for Faster Triaging and Active Learning

Lung ultrasound (LUS) is possibly the only medical imaging modality which could be used for continuous and periodic monitoring of the lung. This is extremely useful in tracking the lung manifestations either during the onset of lung infection or to track the effect of vaccination on lung as in pandemics such as COVID-19. There have been many attempts in automating the classification of the severity of lung involvement into various classes or automatic segmentation of various LUS landmarks and manifestations. However, all these approaches are based on training static machine learning models which require a significantly large clinically annotated dataset and are computationally heavy and are most of the time non-real time. In this work, a real-time light weight active learning-based approach is presented for faster triaging in COVID-19 subjects in resource constrained settings. The tool, based on the you look only once (YOLO) network, has the capability of providing the quality of images based on the identification of various LUS landmarks, artefacts and manifestations. This tool also predict the severity of lung infection and make use of the possibility of active learning based on the feedback from clinicians or on the image quality. The capability of this tool to summarize the significant frames which are having high severity of infection and high image quality will be helpful for clinicians to discern things more easily. The results show that the proposed object detection tool has a mean average precision (mAP) of 66% at an Intersection over Union (IoU) threshold of 0.5 for the prediction of LUS landmarks with initial training on less than 1000 images. The 14MB lightweight YOLOv5s network achieves 123 FPS while running on a Quadro P4000 GPU. The tool is available for usage and analysis upon request from the authors and details can be found online.

Financial Time Series Forecasting: A Data Stream Mining-Based System

Data stream mining (DSM) represents a promising process to forecast financial time series exchange rate. Financial historical data generate several types of cyclical patterns that evolve, grow, decrease, and end up dying. Within historical data, we can notice long-term, seasonal, and irregular trends. All these changes make traditional static machine learning models not relevant to those study cases. The statistically unstable evolution of financial market behavior yields a progressive deterioration in any trained static model. Those models do not provide the required characteristics to evolve continuously and sustain good forecasting performance as the data distribution changes. Online learning without DSM mechanisms can also miss sudden or quick changes. In this paper, we propose a possible DSM methodology, trying to cope with that instability by implementing an incremental and adaptive strategy. The proposed algorithm includes the online Stochastic Gradient Descent algorithm (SGD), whose weights are optimized using the Particle Swarm Optimization Metaheuristic (PSO) to identify repetitive chart patterns in the FOREX historical data by forecasting the EUR/USD pair’s future values. The data trend change is detected using a statistical technique that studies if the received time series instances are stationary or not. Therefore, the sliding window size is minimized as changes are detected and maximized as the distribution becomes more stable. Results, though preliminary, show that the model prediction is better using flexible sliding windows that adapt according to the detected distribution changes using stationarity compared to learning using a fixed window size that does not incorporate any techniques for detecting and responding to pattern shifts.

Enhanced Intrusion Detection with Data Stream Classification and Concept Drift Guided by the Incremental Learning Genetic Programming Combiner

Concept drift (CD) in data streaming scenarios such as networking intrusion detection systems (IDS) refers to the change in the statistical distribution of the data over time. There are five principal variants related to CD: incremental, gradual, recurrent, sudden, and blip. Genetic programming combiner (GPC) classification is an effective core candidate for data stream classification for IDS. However, its basic structure relies on the usage of traditional static machine learning models that receive onetime training, limiting its ability to handle CD. To address this issue, we propose an extended variant of the GPC using three main components. First, we replace existing classifiers with alternatives: online sequential extreme learning machine (OSELM), feature adaptive OSELM (FA-OSELM), and knowledge preservation OSELM (KP-OSELM). Second, we add two new components to the GPC, specifically, a data balancing and a classifier update. Third, the coordination between the sub-models produces three novel variants of the GPC: GPC-KOS for KA-OSELM; GPC-FOS for FA-OSELM; and GPC-OS for OSELM. This article presents the first data stream-based classification framework that provides novel strategies for handling CD variants. The experimental results demonstrate that both GPC-KOS and GPC-FOS outperform the traditional GPC and other state-of-the-art methods, and the transfer learning and memory features contribute to the effective handling of most types of CD. Moreover, the application of our incremental variants on real-world datasets (KDD Cup '99, CICIDS-2017, CSE-CIC-IDS-2018, and ISCX '12) demonstrate improved performance (GPC-FOS in connection with CSE-CIC-IDS-2018 and CICIDS-2017; GPC-KOS in connection with ISCX2012 and KDD Cup '99), with maximum accuracy rates of 100% and 98% by GPC-KOS and GPC-FOS, respectively. Additionally, our GPC variants do not show superior performance in handling blip drift.

A Static Machine Learning Based Evaluation Method for Usability and Security Analysis in E-Commerce Website

A Static Machine Learning Based Evaluation Method for Usability and Security Analysis in E-Commerce Website

Personalized model to predict seizures based on dynamic and static continuous EEG monitoring data

Background/ObjectiveEarly recognition of patients who may be at risk of developing acute symptomatic seizures would be useful. We aimed to determine whether continuous electroencephalography (cEEG) data using machine learning techniques such as neural networks and decision trees could predict seizure occurrence in hospitalized patients. MethodsThis was a single center retrospective cohort analysis of cEEG data in patients aged 18–90 years who were admitted and underwent cEEG monitoring between 2010 and 2019 limited to 72 h excluding those who were seizing at the onset of recording. A total of 41,491 patients were reviewed; of these, 3874 were used to develop the static model and 1687 to develop the dynamic model (half with seizure and half without seizure in each cohort). Of these, 80% were randomly selected as derivation cohorts for each model and 20% were randomly selected as validation cohorts. Dynamic and static machine learning models (long short term memory (LSTM) and Extreme Gradient Boosting algorithm (XGBoost)) based on day-to-day dynamic EEG changes and binary static EEG features over the prior 72 h or until seizure, which ever was earlier, were used. ResultsThe static model was able to predict seizure occurrence based on cEEG data with sensitivity and specificity of 0.81 and 0.59, respectively, with an AUC of 0.70. The dynamic model was able to predict seizure occurrence with sensitivity and specificity of 0.72 and 0.80, respectively, and AUC of 0.81. ConclusionsMachine learning models could be applied to cEEG data to predict seizure occurrence based on available cEEG data. Dynamic day-to-day EEG data are more useful in predicting seizures than binary static EEG data. These models could potentially be used to determine the need for ongoing cEEG monitoring and to prioritize resources.

KF-Loc: A Kalman Filter and Machine Learning Integrated Localization System Using Consumer-Grade Millimeter-Wave Hardware

With the ever-increasing demands of e-commerce, the need for smarter warehousing is increasing exponentially. Such warehouses require industry automation beyond Industry 4.0. In this work, we use consumer-grade millimeter-wave (mmWave) equipment to enable fast, and low-cost implementation of our localization system. However, the consumer-grade mmWave routers suffer from coarse-grained channel state information due to cost-effective antenna array design limiting the accuracy of localization systems. To address these challenges, we present a machine learning (ML) and Kalman filter (KF) integrated localization system (KF-Loc). The ML model learns the complex wireless features for predicting the static position of the robot. When in dynamic motion, the static ML estimates suffer from position mispredictions, resulting in loss of accuracy. To overcome the loss in accuracy, we design and integrate a KF that learns the dynamics of the robot motion to provide highly accurate tracking. Our system achieves centimeter-level accuracy for the two aisles with RMSE of 0.35 m and 0.37 m, respectively. Further, compared with ML only localization systems, we achieve a significant reduction in RMSE by 28.5% and 54.3% within the two aisles.

Abstract 476: Predicting response to treatment in early breast cancer using dynamic integrative multi-omic profiling

Abstract Aim: We recently published the first machine learning framework that integrates multi-omic data derived from the pre-therapy breast tumor ecosystem to accurately predict response to neoadjuvant systemic therapy (Sammut et al, Nature 2021). We aimed to extend this framework to incorporate serially acquired multi-omic data to further improve response predictions and model tumor biology as it is perturbed by treatment. Methods: Breast tumor core biopsies were acquired at diagnosis from 168 women that went on to receive pre-operative chemotherapy (or chemotherapy plus anti-HER2 targeted therapy). Serial tumor core biopsies were obtained midway (n=78) and on completion (n=69) of neoadjuvant systemic therapy. Response was assessed at surgery using the Residual Cancer Burden score. Core biopsies were molecularly profiled by shallow whole genome, exome and RNA sequencing and their histological architecture characterized using digital pathology. Baseline clinical, molecular, and digital pathology imaging features associated with response were identified and their dynamics modelled throughout therapy. Results: At baseline, a total of 34 features derived from multi-omic data were associated with response to neoadjuvant therapy. These included: tumor mutation and neoantigen burden, subclonal diversity, HRD and chromosomal instability. A suppressed immune response, typified by the presence of T-cell dysfunction and immune exclusion, was associated with extensive chemoresistance. The changes in abundance of these features across the serially sampled on-therapy tumors were then mapped to response outcomes. An early increase in adaptive and innate immune infiltration and activation was associated with a linear decrease in expressed neoantigens, tumor proliferation and loss of subclonal mutational and copy number diversity, indicating early response to therapy. Conversely, a stable tumor and microenvironment transcriptional landscape throughout treatment, corresponding to a minimal change in tumor clonal architecture and microenvironment composition, was associated with a poor response to therapy. Notably, tumors with LOH HLA failed to engage a productive immune response during treatment and this was associated with resistance. A dynamic framework that models the change of these on-therapy features and extends the functionality of the published static machine learning model is being developed. Conclusion: Response to neoadjuvant therapy is determined by the baseline characteristics of the tumor ecosystem. During therapy, both the tumor and its microenvironment follow distinct evolutionary trajectories that can be mapped to outcome. The change in enrichment of features derived from the tumor and its microenvironment can be integrated within machine learning frameworks that leverage dynamic data from the entire therapy time course for more accurate response prediction. Citation Format: Stephen-John Sammut, Mireia Crispin-Ortuzar, Suet-Feung Chin, Elena Provenzano, Wei Cope, Ali Dariush, Sarah-Jane Dawson, Paul D. Pharoah, Florian Markowetz, Oscar M. Rueda, Helena M. Earl, Carlos Caldas. Predicting response to treatment in early breast cancer using dynamic integrative multi-omic profiling [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 476.

Auditing static machine learning anti-Malware tools against metamorphic attacks

Malicious software is one of the most serious cyber threats on the Internet today. Traditional malware detection has proven unable to keep pace with the sheer number of malware because of their growing complexity, new attacks and variants. Most malware implement various metamorphic techniques in order to disguise themselves, therefore preventing successful analysis and thwarting the detection by signature-based anti-malware engines. During the past decade, there has been an increase in the research and deployment of anti-malware engines powered by machine learning, and in particular deep learning, due to their ability to handle huge volumes of malware and generalize to never-before-seen samples. However, there is little research about the vulnerability of these models to adversarial examples. To fill this gap, this paper presents an exhaustive evaluation of the state-of-the-art approaches for malware classification against common metamorphic attacks. Given the limitations found in deep learning approaches, we present a simple architecture that increases 14.95% the classification performance with respect to MalConv’s architecture. Furthermore, the use of the metamorphic techniques to augment the training set is investigated and results show that it significantly improves the classification of malware belonging to families with few samples.

A novel approach to define the local region of dynamic selection techniques in imbalanced credit scoring problems

Lenders, such as banks and credit card companies, use credit scoring models to evaluate the potential risk posed by lending money to customers, and therefore to mitigate losses due to bad credit. The profitability of the banks thus highly depends on the models used to decide on the customer’s loans. State-of-the-art credit scoring models are based on machine learning and statistical methods. One of the major problems of this field is that lenders often deal with imbalanced datasets that usually contain many paid loans but very few not paid ones (called defaults). Recently, dynamic selection methods combined with ensemble methods and preprocessing techniques have been evaluated to improve classification models in imbalanced datasets presenting advantages over the static machine learning methods. In a dynamic selection technique, samples in the neighborhood of each query sample are used to compute the local competence of each base classifier. Then, the technique selects only competent classifiers to predict the query sample. In this paper, we evaluate the suitability of dynamic selection techniques for credit scoring problem, and we present Reduced Minority k-Nearest Neighbors (RMkNN), an approach that enhances state of the art in defining the local region of dynamic selection techniques for imbalanced credit scoring datasets. This proposed technique has a superior prediction performance in imbalanced credit scoring datasets compared to state of the art. Furthermore, RMkNN does not need any preprocessing or sampling method to generate the dynamic selection dataset (called DSEL). Additionally, we observe an equivalence between dynamic selection and static selection classification. We conduct a comprehensive evaluation of the proposed technique against state-of-the-art competitors on six real-world public datasets and one private one. Experiments show that RMkNN improves the classification performance of the evaluated datasets regarding AUC, balanced accuracy, H-measure, G-mean, F-measure, and Recall.

Classification Of Association Item Sets From Large Data Sets Based On User Awareness Using Hybrid

In business intelligence, large number of data to be generated because of increasing data in business applications. Analysis and prediction of data is very aggressive concept to evaluate the results present in data based on decision making analysis. To provide effective analysis of data traditionally some of the machine learning related methods like Clustering, Classification, Neural network based approaches and association rule based approaches were used to explore and analysis of business data. Because of increasing depth analysis of data in business intelligence related applications then above static machine learning approaches were not satisfied to form association between different attributes in real time data sets. So that in this paper, we propose Advanced & Hybrid Machine Learning Approach (AHMLA) for effective data analysis of different associated attributes of high dimensional data. Our proposed approach increase customer service, report generations based on user awareness in business intelligence applications. An experimental result of proposed approach gives better high performance with respect to different parameters with respect to existing approaches.

Incremental supervised learning: algorithms and applications in pattern recognition

The most effective well-known methods in the context of static machine learning offer no alternative to evolution and dynamic adaptation to integrate new data or to restructure problems already partially learned. In this area, the incremental learning represents an interesting alternative and constitutes an open research field, becoming one of the major concerns of the machine learning and classification community. In this paper, we study incremental supervised learning techniques and their applications, especially in the field of pattern recognition. This article presents an overview of the main concepts and supervised algorithms of incremental learning, including a synthesis of research studies done in this field and focusing on neural networks, decision trees and support vector machines.

Activity Recognition for Incomplete Spinal Cord Injury Subjects Using Hidden Markov Models

Successful activity recognition in patients with motor disabilities can improve patient care by informing researchers and clinicians about changes in patient mobility both in the clinic and at home. Standard machine learning approaches can improve activity recognition in patient populations by tailoring recognition models to specific populations. However, many approaches use only static machine learning classifiers, which classify each data sample individually, ignoring the temporal relationship of successive samples over time. Static classification can be augmented by integrating the output of static classifiers with a dynamic state estimation model. Here, we use a hidden Markov model (HMM) and apply the static supervised machine learning classifier results as observations. We experimentally validate the effectiveness of our model by recognizing six activities from 13 ambulatory incomplete spinal cord injury subjects who were instructed to perform a standardized set of activities while wearing a waist-worn accelerometer in a clinical setting. Activities included lying, sitting, standing, walking, wheeling, and stair climbing. Using within-subject cross validation, the highest classification accuracy from static classifiers alone was 86.3% (85.5%–87.0% and 95% confidence). By augmenting the classification model with an HMM, we were able to improve the accuracy to 88.9% (88.2%–89.6%). The additional 2.6% demonstrated a significant improvement of the classification accuracy using a hybrid static/dynamic classifier compared to the use of static classifiers alone. Such improved activity recognition can provide better outcome measures, aiding clinicians to select or refine the right physical or drug therapies to improve patient mobility.

An adaptive strategy for the classification of g-protein coupled receptors

One of the major problems in computational biology is the inability of existing classification models to incorporate expanding and new domain knowledge. The problem of static classification models is addressed in this paper by the introduction of incremental learning for problems in boinformatics. Many machine learning tools have been applied to this problem using static machine learning structures such as neural networks or support vector machines that are unable to accommodate new information into their existing models. We utilize the fuzzy ARTMAP as an alternate machine learning system that has the ability of incrementally learning new data as it becomes available. The fuzzy ARTMAP is found to be comparable to many of the widespread machine learning systems. The use of an evolutionary strategy in the selection and combination of individual classifiers into an ensemble system, coupled with the incremental learning ability of the fuzzy ARTMAP is proven to be suitable as a pattern classifier. The algorithm presented is tested using data from the G-Coupled Protein Receptors Database and shows good accuracy of 83%. The system presented is also generally applicable, and can be used in problems in genomics and proteomics.