Hybrid Machine Learning System Research Articles

Hybrid machine learning system based on multivariate data decomposition and feature selection for improved multitemporal evapotranspiration forecasting

Evapotranspiration is an essential component of the hydrological cycle. Forecasting the reference crop evapotranspiration (ETo) using a reliable and generalized framework is crucial for agricultural operations, especially irrigation. This study was aimed at evaluating the performance of a hybrid system including the K-Best selection (KBest), multivariate variational mode decomposition (MVMD), and Machine learning (ML) models for 1-, 3-, 7-, and 10-day-ahead forecasting of the daily ETo in twelve stations of California. The analysis covered a span of 20 years, from 2003 to 2022. Three stand-alone ML models, namely Cascade Forward Neural Network (CFNN), Extreme Learning Machine (ELM), and Bagging Regression Tree (BRT) are used and were integrated with various preprocessing techniques to construct three hybrid models, i.e., MVMD-KBest-CFNN, MVMD-KBest-ELM, and MVMD-KBest-BRT. According to the results obtained in the testing phase, averaged across all stations, all three stand-alone models (CFNN, ELM, and BRT) yielded similar outcomes. In contrast, the hybrid models exhibited significantly enhanced performances compared with the standalone models, and MVMD-KBest-CFNN and MVMD-KBest-ELM models outperformed MVMD-KBest-BRT model. The BRT-based models were vulnerable to overfitting. The performance of the best models is superior compared to similar existing studies. Examining the variations across stations, it was found that the stations located further from the coast and in arid regions could be susceptible to prediction errors and necessitate more attention.

A Machine Learning Model for Determination of Gender Utilizing Hybrid Classifiers

One part of forensic anthropology involves investigating skeletal remains to identify corpses, and many of these remains were found incomplete, burned, broken, or destroyed, making investigation challenging. This study aims to use the pelvis and femur to identify the gender of skeletal remains. The pelvis and femur have previously been proven to be accurate indicators of a corpse's gender. The identification process is done through the measurement of the subpubic angle of the pelvis and the angle taken straight down from the top of the femur to the patella and then straight up. The two measurements were combined using the principal component analysis (PCA) method into two attributes on the x and y axes. These attributes were later used as data for the machine learning model design. The design process consisted of an Artificial Neutral Network (ANN) design model and Support Vector Machine (SVM) design model combined into a hybrid machine learning system. The ANN and SVM hybrid machine learning were tested with acquired data. The result of the test using the confusion matrix showed 83.33% accuracy, which is categorized as "good classification" based on Area Under the Curve (AUC).

CO2 concentration forecasting in smart cities using a hybrid ARIMA–TFT model on multivariate time series IoT data

Carbon Dioxide (CO_{2}) is a significant contributor to greenhouse gas emissions and one of the main drivers behind global warming and climate change. In spite of the global economic slowdown due to the COVID-19 pandemic, the global average atmospheric CO_{2} concentration reached a new record high in 2020 with its year-on-year increase being the fifth highest annual increase in 63 years, according to the National Oceanic and Atmospheric Administration. Furthermore, the years 2020 and 2019 were respectively the second and third warmest, while the decade 2010–2019 was the warmest decade ever recorded. In an attempt to curb this climate emergency, many countries and organizations globally have adopted ambitious goals and announced plans to help dramatically reduce CO_{2} emissions. As part of these plans, various innovative smart city projects are being developed, focusing on implementing Internet of Things (IoT) technologies. By collecting sensor-based data, such technologies aim towards automating data-driven decision-making around carbon emission management and reduction. In this work, a hybrid machine learning system, aimed at forecasting CO_2 concentration levels in a smart city environment was developed using a multivariate time series dataset containing IoT sensor measurements of CO_{2}, as well as various environmental factors, taken at every second. The proposed system demonstrated superior performance to similar methods, while also maintaining a high degree of interpretability. More specifically, the approach was empirically compared against other similar approaches in several scenarios and use cases, thus also offering more insight into the predictive capabilities of such state-of-the-art systems. For this comparison, both traditional time series and deep learning approaches were employed, including the current state-of-the-art architectures, such as attention-based, transformer networks. Results demonstrated that, when measured across various settings and metrics, including three different forecasting horizons, the hybrid solution achieved the best overall results, and in some cases, the difference in performance was statistically significant. At the same time, insights from the system’s inner workings were extracted, shedding light on the reasoning behind the model’s predictions and the factors that contribute to them, thus showcasing its transparency. Lastly, throughout the experiments, deep learning approaches illustrated their ability to better handle the multivariate nature of the dataset and in general tended to outperform the traditional time series methods, especially for longer forecasting horizons.

Deep Learning Based Hybrid Intrusion Detection Systems to Protect Satellite Networks

Despite the fact that satellite-terrestrial systems have advantages such as high throughput, low latency, and low energy consumption, as well as low exposure to physical threats and natural disasters and cost-effective global coverage, their integration exposes both of them to particular security challenges that can arise due to the migration of security challenges from one to another. Intrusion Detection Systems (IDS) can also be used to provide a high level of protection for modern network environments such as satellite-terrestrial integrated networks (STINs). To optimize the detection performance of malicious activities in network traffic, four hybrid intrusion detection systems for satellite-terrestrial communication systems (SAT-IDSs) are proposed in this paper. All the proposed systems exploit the sequential forward feature selection (SFS) method based on random forest (RF) to select important features from the dataset that increase relevance and reduce complexity and then combine them with a machine learning (ML) or deep learning (DL) model; Random Forest (RF), Long Short-Term memory (LSTM), Artificial Neural Networks (ANN), and Gated Recurrent Unit (GRU). Two datasets—STIN, which simulates satellite networks, and UNSW-NB15, which simulates terrestrial networks—were used to evaluate the performance of the proposed SAT-IDSs. The experimental results indicate that selecting significant and crucial features produced by RF-SFS vastly improves detection accuracy and computational efficiency. In the first dataset (STIN), the proposed hybrid ML system SFS-RF achieved an accuracy of 90.5% after using 10 selected features, compared to 85.41% when using the whole dataset. Furthermore, the RF-SFS-GRU model achieved the highest performance of the three proposed hybrid DL-based SAT-IDS with an accuracy of 87% after using 10 selected features, compared to 79% when using the entire dataset. In the second dataset (UNSW-NB15), the proposed hybrid ML system SFS-RF achieved an accuracy of 78.52% after using 10 selected features, compared to 75.4% when using the whole dataset. The model with the highest accuracy of the three proposed hybrid DL-based SAT-IDS was the RF-SFS-GRU model. It achieved an accuracy of 79% after using 10 selected features, compared to 74% when using the whole dataset.

Prediction of Parkinson’s disease pathogenic variants using hybrid Machine learning systems and radiomic features

PurposeIn Parkinson’s disease (PD), 5–10% of cases are of genetic origin with mutations identified in several genes such as leucine-rich repeat kinase 2 (LRRK2) and glucocerebrosidase (GBA). We aim to predict these two gene mutations using hybrid machine learning systems (HMLS), via imaging and non-imaging data, with the long-term goal to predict conversion to active disease. MethodsWe studied 264 and 129 patients with known LRRK2 and GBA mutations status from PPMI database. Each dataset includes 513 features such as clinical features (CFs), conventional imaging features (CIFs) and radiomic features (RFs) extracted from DAT-SPECT images. Features, normalized by Z-score, were univariately analyzed for statistical significance by the t-test and chi-square test, adjusted by Benjamini-Hochberg correction. Multiple HMLSs, including 11 features extraction (FEA) or 10 features selection algorithms (FSA) linked with 21 classifiers were utilized. We also employed Ensemble Voting (EV) to classify the genes. ResultsFor prediction of LRRK2 mutation status, a number of HMLSs resulted in accuracies of 0.98 ± 0.02 and 1.00 in 5-fold cross-validation (80% out of total data points) and external testing (remaining 20%), respectively. For predicting GBA mutation status, multiple HMLSs resulted in high accuracies of 0.90 ± 0.08 and 0.96 in 5-fold cross-validation and external testing, respectively. We additionally showed that SPECT-based RFs added value to the specific prediction of of GBA mutation status. ConclusionWe demonstrated that combining medical information with SPECT-based imaging features, and optimal utilization of HMLS can produce excellent prediction of the mutations status in PD patients.

Fusion-based tensor radiomics using reproducible features: Application to survival prediction in head and neck cancer

BackgroundNumerous features are commonly generated in radiomics applications as applied to medical imaging, and identification of robust radiomics features (RFs) can be an important step to derivation of reliable, reproducible solutions. In this work, we utilize a tensor radiomics (TR) framework, where numerous fusions are explored, to generate different flavours of RFs, and we aimed to identify RFs that are robust to fusion techniques in head and neck cancer. Overall, we aimed to predict progression-free survival (PFS) using Hybrid Machine Learning Systems (HMLS) and reproducible RFs. MethodsThe study was performed on 408 patients with head and neck cancer from The Cancer Imaging Archive. After image preprocessing, 15 fusion techniques were employed to combine Positron Emission Tomography (PET) and Computed Tomography (CT) images. Subsequently, 215 RFs were extracted through a standardized radiomics software, with 17 ‘flavours’ generated using PET-only, CT-only, and 15 fused PET&CT images. The variability of RFs across flavours was studied using the Intraclass Correlation Coefficient (ICC). Furthermore, the features were categorized into seven reliability groups, 106 reproducible RFs with ICC>0.75 were selected, highly correlated flavours were removed, Principal Component Analysis was used to convert 17 flavours to 1 attribute, the polynomial function was utilized to increase RFs, and Analysis of variance (ANOVA) was used to select the relevant attributes. Finally, 3 classifiers including Random Forest (RFC), Logistic regression (LR), and Multi-layer perceptron were applied to the preselected relevant attributes to predict binary PFS. In 5-fold cross-validation, 80% of 4 divisions were utilized to train the model, and the remaining 20% was utilized to evaluate the model. Further, the remaining fold was used for external nested testing. ResultsReliability analysis indicated that most morphological features belong to the high-reliability category. By contrast, local intensity and statistical features extracted from images belong to the low-reliability category. In the tensor framework, the highest 5-fold cross-validation accuracy of 76.7%±3.3% with an external nested testing of 70.6%±6.7% resulted from the reproducible TR+polynomial function+ANOVA+LR algorithm while the accuracy of 70.0%±4.2% with the external nested testing of 67.7%±4.9% was achieved through the PCA fusion+RFC (non-tensor paradigm). ConclusionsThis study demonstrated that using reproducible RFs as utilized within a tensor fusion radiomics framework, linked with ANOVA and LR, added value to prediction of progression-free survival outcome in head and neck cancer patients.

Prediction of Cognitive Decline in Parkinson's Disease Using Clinical and DAT SPECT Imaging Features, and Hybrid Machine Learning Systems.

We aimed to predict Montreal Cognitive Assessment (MoCA) scores in Parkinson's disease patients at year 4 using handcrafted radiomics (RF), deep (DF), and clinical (CF) features at year 0 (baseline) applied to hybrid machine learning systems (HMLSs). 297 patients were selected from the Parkinson's Progressive Marker Initiative (PPMI) database. The standardized SERA radiomics software and a 3D encoder were employed to extract RFs and DFs from single-photon emission computed tomography (DAT-SPECT) images, respectively. The patients with MoCA scores over 26 were indicated as normal; otherwise, scores under 26 were indicated as abnormal. Moreover, we applied different combinations of feature sets to HMLSs, including the Analysis of Variance (ANOVA) feature selection, which was linked with eight classifiers, including Multi-Layer Perceptron (MLP), K-Neighbors Classifier (KNN), Extra Trees Classifier (ETC), and others. We employed 80% of the patients to select the best model in a 5-fold cross-validation process, and the remaining 20% were employed for hold-out testing. For the sole usage of RFs and DFs, ANOVA and MLP resulted in averaged accuracies of 59 ± 3% and 65 ± 4% for 5-fold cross-validation, respectively, with hold-out testing accuracies of 59 ± 1% and 56 ± 2%, respectively. For sole CFs, a higher performance of 77 ± 8% for 5-fold cross-validation and a hold-out testing performance of 82 + 2% were obtained from ANOVA and ETC. RF+DF obtained a performance of 64 ± 7%, with a hold-out testing performance of 59 ± 2% through ANOVA and XGBC. Usage of CF+RF, CF+DF, and RF+DF+CF enabled the highest averaged accuracies of 78 ± 7%, 78 ± 9%, and 76 ± 8% for 5-fold cross-validation, and hold-out testing accuracies of 81 ± 2%, 82 ± 2%, and 83 ± 4%, respectively. We demonstrated that CFs vitally contribute to predictive performance, and combining them with appropriate imaging features and HMLSs can result in the best prediction performance.

Prediction of drug amount in Parkinson's disease using hybrid machine learning systems and radiomics features

AbstractParkinson's disease (PD) is progressive and heterogeneous. Levodopa is widely prescribed to control PD, and its long‐term‐treatment leads to dyskinesia in a dose‐dependent manner. Interpretation of clinical trials comparing different drug treatments for PD is complicated by different dose intensities employed: higher doses of levodopa produce better symptomatic control but more late complications. Thus, the dose must be recalibrated and reduced gradually. Since recommendations for gradually reducing Levodopa are currently lacking and estimation of Levodopa amount can help doctor to correctly prescribe drug amount, this study aims to predict Levodopa amount and incremental doses using Hybrid Machine Learning Systems (HMLS) and a mixture of radiomics and clinical features. We selected 264 patients from PPMI and obtained 950 features including imaging and nominating features. We generated seven datasets constructed from the dataset in years 0 and 1, which linked with outcomes, (O1) patients being on/off drug in year 1, (O2) dose amount in year 1, and (O3‐8) incremental dose from 1st to 2nd, 2nd to 3rd, 3rd to 4th, 4th to 5th, 1st to 4th, 1st to 5th year. HMLSs included 10 feature extraction/9 feature selection algorithms followed by 10 prediction algorithms. To predict O1, timeless dataset + Random Forest + ReliefA had the highest accuracy~88.5% ± 2.2%, and external testing~91.6%. Furthermore, to predict O2, timeless dataset + Minimum Redundancy Maximum Relevance Algorithm (MRMR) + K Nearest Neighbor Regressor (KNN‐R) achieved a mean absolute error (MAE) ~ 47.5 ± 13.6 ([30.3:850 milligram]) and external testing~31.9. To predict dose increments (O3‐8), HMLSs: Unsupervised Feature Selection with Ordinal Locality + KNNR, ReliefA + KNNR, ReliefA + KNNR, Local Learning‐based Clustering Feature Selection + KNNR, MRMR + KNNR, and MRMR + KNNR applied to timeless datasets resulted in MAEs ~ 0.42 ± 0.18, 0.10 ± 0.09, 0.04 ± 0.01, 0.24 ± 0.15, 0.25 ± 0.05, and 0.33 ± 0.26 ([0.23:29.7]), respectively. Moreover, their external testing confirmed our findings. We demonstrated that timeless datasets including a mixture of clinical and imaging features, linked with appropriate HMLSs, significantly improve prediction performances.

Application of novel hybrid machine learning systems and radiomics features for non-motor outcome prediction in Parkinson’s disease

Objectives. Parkinson’s disease (PD) is a complex neurodegenerative disorder, affecting 2%–3% of the elderly population. Montreal Cognitive Assessment (MoCA), a rapid nonmotor screening test, assesses different cognitive dysfunctionality aspects. Early MoCA prediction may facilitate better temporal therapy and disease control. Radiomics features (RF), in addition to clinical features (CF), are indicated to increase clinical diagnoses, etc, bridging between medical imaging procedures and personalized medicine. We investigate the effect of RFs, CFs, and conventional imaging features (CIF) to enhance prediction performance using hybrid machine learning systems (HMLS). Methods. We selected 210 patients with 981 features (CFs, CIFs, and RFs) from the Parkinson’s Progression-Markers-Initiative database. We generated 4 datasets, namely using (i), (ii) year-0 (D1) or year-1 (D2) features, (iii) longitudinal data (D3, putting datasets in years 0 and 1 longitudinally next to each other), and (iv) timeless data (D4, effectively doubling dataset size by listing both datasets from years 0 and 1 separately). First, we directly applied 23 predictor algorithms (PA) to the datasets to predict year-4 MoCA, which PD patients this year have a higher dementia risk. Subsequently, HMLSs, including 14 attribute extraction and 10 feature selection algorithms followed by PAs were employed to enhance prediction performances. 80% of all datapoints were utilized to select the best model based on minimum mean absolute error (MAE) resulting from 5-fold cross-validation. Subsequently, the remaining 20% was used for hold-out testing of the selected models. Results. When applying PAs without ASAs/FEAs to datasets (MoCA outcome range: [11,30]), Adaboost achieved an MAE of 1.74 ± 0.29 on D4 with a hold-out testing performance of 1.71. When employing HMLSs, D4 + Minimum_Redundancy_Maximum_Relevance (MRMR)+K_Nearest_Neighbor Regressor achieved the highest performance of 1.05 ± 0.25 with a hold-out testing performance of 0.57. Conclusion. Our study shows the importance of using larger datasets (timeless), and utilizing optimized HMLSs, for significantly improved prediction of MoCA in PD patients.

An Hybrid Ontology Matching Mechanism for Adaptive Educational eLearning Environments

Providing the same pedagogical and educational methods to all students is pedagogically ineffective. In contrast, the pedagogical strategies that adapt to the fundamental individual skills of the students have proved to be more effective. An important innovation in this direction is the adaptive educational systems (AESs) that adjust the teaching content on educational needs and students’ skills. Effective utilization of these approaches can be enhanced with artificial intelligence (AI) and semantic web technologies that can increase data generation, access, flow, integration, and comprehension using the same open standards driving the World Wide Web. This study proposes a novel adaptive educational eLearning system (AEeLS) that can gather and analyze data from learning repositories and adapt these to the educational curriculum according to the student’s skills and experience. It is an innovative hybrid machine learning system that combines a semi-supervised classification method for ontology matching and a recommendation mechanism that uses a sophisticated way from neighborhood-based collaborative and content-based filtering techniques to provide a personalized educational environment for each student.

Dynamic Forecasting of Solar Energy Microgrid Systems Using Feature Engineering

The intermittent and stochastic nature of solar energy generation systems, climate change, and the inefficiency of modern power systems due to zero inertia have created many challenges for on-grid operators. Solar forecasting systems based on machine learning algorithms are an emerging and effective solution that uses Big Data related to weather phenomena. However, the predictive ability of these algorithms is hampered by the sporadic nature of solar energy generation. In this article, a robust hybrid machine learning system that utilizes multiple linear regression (MLR) and a Pearson correlation coefficient (PCC) was tested on solar power plant sites of varying capacities in Germany (100–8500 kW). The volume of Big Data features can be reduced by focusing on the features that significantly improve the reliability of the mid-term forecasting system. In this way, drastic fluctuations in the prediction of photovoltaic (PV) power generation can be avoided. The results of our approach are evaluated regarding real-world data using the extreme gradient boosting (XGBoost) with feature engineering, and principal component analysis (PCA), in order to forecast PV energy, rank, and track the importance of feature engineering for different PV capacities. Furthermore, we found that the need for selectivity and reduction of performance error was supported by ridge regression. In addition, the proposed novel XGBoost forecast system decreased the root-mean-square error (RMSE) and mean absolute error (MAE) by 30% and 18%, respectively, compared to the Autoencoder and long short-term memory (LSTM) for same datasets. Furthermore, the CoD determination coefficient ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$R^{2}$</tex-math></inline-formula> ) increased by 85% compared to the statistical model's autoregressive integrated moving average (ARIMA).

Breast cancer prediction with transcriptome profiling using feature selection and machine learning methods

BackgroundWe used a hybrid machine learning systems (HMLS) strategy that includes the extensive search for the discovery of the most optimal HMLSs, including feature selection algorithms, a feature extraction algorithm, and classifiers for diagnosing breast cancer. Hence, this study aims to obtain a high-importance transcriptome profile linked with classification procedures that can facilitate the early detection of breast cancer.MethodsIn the present study, 762 breast cancer patients and 138 solid tissue normal subjects were included. Three groups of machine learning (ML) algorithms were employed: (i) four feature selection procedures are employed and compared to select the most valuable feature: (1) ANOVA; (2) Mutual Information; (3) Extra Trees Classifier; and (4) Logistic Regression (LGR), (ii) a feature extraction algorithm (Principal Component Analysis), iii) we utilized 13 classification algorithms accompanied with automated ML hyperparameter tuning, including (1) LGR; (2) Support Vector Machine; (3) Bagging; (4) Gaussian Naive Bayes; (5) Decision Tree; (6) Gradient Boosting Decision Tree; (7) K Nearest Neighborhood; (8) Bernoulli Naive Bayes; (9) Random Forest; (10) AdaBoost, (11) ExtraTrees; (12) Linear Discriminant Analysis; and (13) Multilayer Perceptron (MLP). For evaluating the proposed models' performance, balance accuracy and area under the curve (AUC) were used.ResultsFeature selection procedure LGR + MLP classifier achieved the highest prediction accuracy and AUC (balanced accuracy: 0.86, AUC = 0.94), followed by an LGR + LGR classifier (balanced accuracy: 0.84, AUC = 0.94). The results showed that achieved AUC for the LGR + LGR classifier belonged to the 20 biomarkers as follows: TMEM212, SNORD115-13, ATP1A4, FRG2, CFHR4, ZCCHC13, FLJ46361, LY6G6E, ZNF323, KRT28, KRT25, LPPR5, C10orf99, PRKACG, SULT2A1, GRIN2C, EN2, GBA2, CUX2, and SNORA66.ConclusionsThe best performance was achieved using the LGR feature selection procedure and MLP classifier. Results show that the 20 biomarkers had the highest score or ranking in breast cancer detection.

A hybrid machine learning system to impute and classify a component-based robot

AbstractIn the field of cybernetic systems and more specifically in robotics, one of the fundamental objectives is the detection of anomalies in order to minimize loss of time. Following this idea, this paper proposes the implementation of a Hybrid Intelligent System in four steps to impute the missing values, by combining clustering and regression techniques, followed by balancing and classification tasks. This system applies regression models to each one of the clusters built on the instances of data set. Subsequently, a variety of balancing techniques are applied to improve the classifier’s ability to discern whether it is in an error or a normal state. These techniques support to obtain better classification ratios in which a robot is close to error and allow us to bring the behavior back to a normal state. The experimentation is performed using a modern and public data set, which has been extracted from a component-based robotic system, in which different anomalies are induced by software in their components.

Longitudinal clustering analysis and prediction of Parkinson's disease progression using radiomics and hybrid machine learning.

We employed machine learning approaches to (I) determine distinct progression trajectories in Parkinson's disease (PD) (unsupervised clustering task), and (II) predict progression trajectories (supervised prediction task), from early (years 0 and 1) data, making use of clinical and imaging features. We studied PD-subjects derived from longitudinal datasets (years 0, 1, 2 & 4; Parkinson's Progressive Marker Initiative). We extracted and analyzed 981 features, including motor, non-motor, and radiomics features extracted for each region-of-interest (ROIs: left/right caudate and putamen) using our standardized standardized environment for radiomics analysis (SERA) radiomics software. Segmentation of ROIs on dopamine transposer - single photon emission computed tomography (DAT SPECT) images were performed via magnetic resonance images (MRI). After performing cross-sectional clustering on 885 subjects (original dataset) to identify disease subtypes, we identified optimal longitudinal trajectories using hybrid machine learning systems (HMLS), including principal component analysis (PCA) + K-Means algorithms (KMA) followed by Bayesian information criterion (BIC), Calinski-Harabatz criterion (CHC), and elbow criterion (EC). Subsequently, prediction of the identified trajectories from early year data was performed using multiple HMLSs including 16 Dimension Reduction Algorithms (DRA) and 10 classification algorithms. We identified 3 distinct progression trajectories. Hotelling's t squared test (HTST) showed that the identified trajectories were distinct. The trajectories included those with (I, II) disease escalation (2 trajectories, 27% and 38% of patients) and (III) stable disease (1 trajectory, 35% of patients). For trajectory prediction from early year data, HMLSs including the stochastic neighbor embedding algorithm (SNEA, as a DRA) as well as locally linear embedding algorithm (LLEA, as a DRA), linked with the new probabilistic neural network classifier (NPNNC, as a classifier), resulted in accuracies of 78.4% and 79.2% respectively, while other HMLSs such as SNEA + Lib_SVM (library for support vector machines) and t_SNE (t-distributed stochastic neighbor embedding) + NPNNC resulted in 76.5% and 76.1% respectively. This study moves beyond cross-sectional PD subtyping to clustering of longitudinal disease trajectories. We conclude that combining medical information with SPECT-based radiomics features, and optimal utilization of HMLSs, can identify distinct disease trajectories in PD patients, and enable effective prediction of disease trajectories from early year data.

Feature selection and machine learning methods for optimal identification and prediction of subtypes in Parkinson's disease

ObjectivesThe present work focuses on assessment of Parkinson's disease (PD), including both PD subtype identification (unsupervised task) and prediction (supervised task). We specifically investigate optimal feature selection and machine learning algorithms for these tasks. MethodsWe selected 885 PD subjects as derived from longitudinal datasets (years 0–4; Parkinson's Progressive Marker Initiative), and investigated 981 features including motor, non-motor, and imaging features (SPECT-based radiomics features extracted using our standardized SERA software). Two different hybrid machine learning systems (HMLS) were constructed and applied to the data in order to select optimal combinations in both tasks: (i) identification of subtypes in PD (unsupervised-clustering), and (ii) prediction of these subtypes in year 4 (supervised-classification). From the original data based on years 0 (baseline) and 1, we created new datasets as inputs to the prediction task: (i,ii) CSD0 and CSD01: cross-sectional datasets from year 0 only and both years 0 & 1, respectively; (iii) TD01: timeless dataset from both years 0 & 1. In addition, PD subtype in year 4 was considered as outcome. Finally, high score features were derived via ensemble voting based on their prioritizations from feature selector algorithms (FSAs). ResultsIn clustering task, the most optimal combinations (out of 981) were selected by individual FSAs to enable high correlation compared to using all features (arriving at 547). In prediction task, we were able to select optimal combinations, resulting in an accuracy >90% only for timeless dataset (TD01); there, we were able to select the most optimal combination using 77 features, directly selected by FSAs. In both tasks, however, using combination of only high score features from ensemble voting did not enable acceptable performances, showing optimal feature selection via individual FSAs to be more effective. ConclusionCombining non-imaging information with SPECT-based radiomics features, and optimal utilization of HMLSs, can enable robust identification of subtypes as well as appropriate prediction of these subtypes in PD patients. Moreover, use of timeless dataset, beyond cross-sectional datasets, enabled predictive accuracies over 90%. Overall, we showed that radiomics features extracted from SPECT images are important in clustering as well as prediction of PD subtypes.

An Improved Method for Extracting Photovoltaic Module I–V Characteristic Curve Using Hybrid Learning Machine System

Abstract This paper provides an improved method for predicting the I–V curve of the photovoltaic module using a hybrid machine learning system. The proposed method is based on a random forest algorithm and a cascade forward neural network. A random forest algorithm is used to predict a specific factor that is subsequently used as an input for the cascade neural network to remove the correlation between voltage and current. Then, the actual current is predicted using the cascade neural network. This procedure assures the ability of the proposed model to extract the I–V curve of any photovoltaic module regardless of its rating or type. A dataset that contains values for air temperature, solar radiation, voltage, and current of two polycrystalline photovoltaic modules is used in the training process of the proposed algorithm. The hybrid model has general inputs such as ambient temperature, solar radiation, and data from the photovoltaic module datasheet (Voc and Isc). The proposed model is trained, tested, and validated by 86% of the data. Meanwhile, 14% of the data are used for testing. Thus, the proposed model is tested using unknown data so as to avoid overfitting. Results show that the proposed model is very accurate in predicting I–V curves based on three types of errors which are mean absolute percentage error (0.68%), mean bias error (0.0191 A), and root-mean-squared error (0.04458 A). This hybrid model can be used to obtain the I–V curves for several types of photovoltaic modules.

Research on prediction of online purchasing behavior based on hybrid model

The research on the potential purchase behavior of users can help merchants develop better marketing strategies. At present, many research methods of online purchasing behavior are based on simple rule prediction, and the prediction results are not satisfactory. We design a hybrid model of Gradient Boosting Decision Tree and logistic regression to accurately predict the purchase behavior of users, which combines the association characteristics between users and commodities. Firstly, clustering algorithm and association rules are used to solve the problem of data imbalance and mine more potential related features. This scheme not only improves the processing efficiency of large data, but also solves the problem of user cold start. Secondly, we construct a scalable tree enhancement system (XGBoost) to train the initial feature set, which is a strong classifier composed of several weak classifiers. A new training set combines the new features with the original features through feature reconstruction, and a hybrid machine learning system is constructed by logistic regression (LR) model. Finally, the LR model is trained by the new training set. Compared with the existing schemes, the integrated decision tree model can train more sample sets with less resources. The experimental results show that the accuracy of the hybrid model is better than single model, and the F1_score is higher.

Robust identification of Parkinson's disease subtypes using radiomics and hybrid machine learning

ObjectivesIt is important to subdivide Parkinson's disease (PD) into subtypes, enabling potentially earlier disease recognition and tailored treatment strategies. We aimed to identify reproducible PD subtypes robust to variations in the number of patients and features. MethodsWe applied multiple feature-reduction and cluster-analysis methods to cross-sectional and timeless data, extracted from longitudinal datasets (years 0, 1, 2 & 4; Parkinson's Progressive Marker Initiative; 885 PD/163 healthy-control visits; 35 datasets with combinations of non-imaging, conventional-imaging, and radiomics features from DAT-SPECT images). Hybrid machine-learning systems were constructed invoking 16 feature-reduction algorithms, 8 clustering algorithms, and 16 classifiers (C-index clustering evaluation used on each trajectory). We subsequently performed: i) identification of optimal subtypes, ii) multiple independent tests to assess reproducibility, iii) further confirmation by a statistical approach, iv) test of reproducibility to the size of the samples. ResultsWhen using no radiomics features, the clusters were not robust to variations in features, whereas, utilizing radiomics information enabled consistent generation of clusters through ensemble analysis of trajectories. We arrived at 3 distinct subtypes, confirmed using the training and testing process of k-means, as well as Hotelling's T2 test. The 3 identified PD subtypes were 1) mild; 2) intermediate; and 3) severe, especially in terms of dopaminergic deficit (imaging), with some escalating motor and non-motor manifestations. ConclusionAppropriate hybrid systems and independent statistical tests enable robust identification of 3 distinct PD subtypes. This was assisted by utilizing radiomics features from SPECT images (segmented using MRI). The PD subtypes provided were robust to the number of the subjects, and features.

Anomaly Detection for Urban Vehicle GNSS Observation with a Hybrid Machine Learning System

In urban areas, the accuracy and reliability of global navigation satellite systems (GNSS) vehicle positioning decline due to substantial non-line-of-sight (NLOS) signals and multipath effects. Recently, positioning enhancement approaches with supervised GNSS signal type classification based on 3D building model-aided labelling have received widespread attention. Despite the reduced computing costs and improved real-time performance, the strict requirements of 3D building models on accuracy and timeliness limit the popularization of the technology to some extent. Meanwhile, the diversity of anomalous observation sources is beyond the reach of NLOS/multipath detection methods. This paper attempts to construct an alternative framework for quality identification of GNSS observations combining clustering-based anomaly detection and supervised classification, in which the hierarchical density-based spatial clustering of applications with noise (HDBSCAN) algorithm is used to label the offline dataset as normal and anomalous observations without the aid of 3D building models, and the supervised classifier in the online system learns the classification rule for real-time anomaly detection. The experimental results based on the measured vehicle GPS/BeiDou data show that after excluding anomalous observations the single point positioning accuracy of the offline dataset is improved by 87.0%, 45.9%, and 69.6% in the east, north, and up directions, respectively, and the positioning accuracy of two online datasets is improved by 48.4%/45.7%, 39.6%/63.3%, and 49.6%/49.1% in the three directions. Through a large number of comparative experiments and discussion on key issues, it is certified that the proposed method is highly feasible and has great potential in the practical application of urban GNSS vehicle positioning.

Hybrid Machine Learning System to Forecast Electricity Consumption of Smart Grid-Based Air Conditioners

This study develops a hybrid prediction system to forecast 1-day-ahead electricity consumption of air conditioners in office spaces. The hybrid system combines a linear autoregressive integrated moving average model and a nonlinear nature-inspired metaheuristic optimization-based prediction model. To evaluate the efficacy of the proposed system, a smart grid-based monitoring device was installed in an office space, which consists of smart meters, environmental monitoring sensors, infrared sensors, and fan adjustment systems. Data were retrieved to train and test the proposed system. Sensitivity analyses were performed to identify the optimal parameters of the model and inputs for future use. Evaluation results confirmed that the proposed hybrid system outperformed the conventional linear and nonlinear models, showing good agreement between predicted and actual electricity consumption of air conditioners. Particularly, the proposed system obtained the correlation coefficient R of 0.71 and total error rate of 4.8%. The hybrid system can facilitate facility managers in forecasting electricity consumption of air conditioners.