Robust Feature Selection Method Research Articles

FTDZOA: An Efficient and Robust FS Method with Multi-Strategy Assistance.

Feature selection (FS) is a pivotal technique in big data analytics, aimed at mitigating redundant information within datasets and optimizing computational resource utilization. This study introduces an enhanced zebra optimization algorithm (ZOA), termed FTDZOA, for superior feature dimensionality reduction. To address the challenges of ZOA, such as susceptibility to local optimal feature subsets, limited global search capabilities, and sluggish convergence when tackling FS problems, three strategies are integrated into the original ZOA to bolster its FS performance. Firstly, a fractional order search strategy is incorporated to preserve information from the preceding generations, thereby enhancing ZOA's exploitation capabilities. Secondly, a triple mean point guidance strategy is introduced, amalgamating information from the global optimal point, a random point, and the current point to effectively augment ZOA's exploration prowess. Lastly, the exploration capacity of ZOA is further elevated through the introduction of a differential strategy, which integrates information disparities among different individuals. Subsequently, the FTDZOA-based FS method was applied to solve 23 FS problems spanning low, medium, and high dimensions. A comparative analysis with nine advanced FS methods revealed that FTDZOA achieved higher classification accuracy on over 90% of the datasets and secured a winning rate exceeding 83% in terms of execution time. These findings confirm that FTDZOA is a reliable, high-performance, practical, and robust FS method.

Multi-level graph regularized robust multi-modal feature selection for Alzheimer’s disease classification

Multi-modal classification has demonstrated its superiority over conventional single-modal based methods on Alzheimer’s disease (AD) diagnosis, and multi-modal feature selection has attracted increasing attention. However, most previous approaches use a fixed affinity matrix to describe the local neighborhood relations among samples, and only consider the intra-modal similarity while ignoring the inter-modal similarity. Besides, they generally treat all samples equally and neglect the negative influence of noise and outliers. For solving these problems, this paper proposes a new multi-level Graph regularized Robust Multi-modal Feature Selection method called GRMFS that simultaneously performs noise-robust feature selection and adaptive multi-level similarity preservation. On the one hand, GRMFS introduces an ɛ-capped ℓ2-norm loss into regression framework to improve the robustness against outliers, which adaptively assigns a weight to each sample. On the other hand, to explore the intrinsic multi-modal local structures, GRMFS simultaneously learns intra-modal and inter-modal local similarities, and preserves them in subspace to guide feature selection. Experiments on real AD database illustrate the advantages of our proposed in identifying disease status compared with other approaches.

Continuous Patient-Independent Estimation of Respiratory Rate and Blood Pressure Using Robust Spectro-Temporal Features Derived From Photoplethysmogram Only.

Objective: A patient-independent approach for continuous estimation of vital signs using robust spectro-temporal features derived from only photoplethysmogram (PPG) signal. Methods: In the pre-processing stage, we remove baseline shifts and artifacts of the PPG signal using Incremental Merge Segmentation with adaptive thresholding. From the cleaned PPG, we extract multiple parameters independent of individual patient PPG morphology for both Respiration Rate (RR) and Blood Pressure (BP). In addition, we derived a set of novel spectral and statistical features strongly correlated to BP. We proposed robust correlation-based feature selection methods for accurate RR estimates. For fewer computations and accurate measurements of BP, the most significant features are selected using correlation and mutual information measures in the feature engineering part. Finally, RR and BP are estimated using breath counting and a neural network regression model, respectively. Results: The proposed approach outperforms the current state-of-the-art in both RR and BP. The RR algorithm results in mean absolute errors (median, 25th-75th percentiles) of 0.4 (0.1-0.7) for CapnoBase dataset and 0.5(0.3-2.8) for BIDMC dataset without discarding any data window. Similarly, BP approach has been validated on a large dataset derived from MIMIC-II ([Formula: see text]1700 records) which has errors (mean absolute, standard deviation) of 5.0(6.3) and 3.0(4.0) for systolic and diastolic BP, respectively. The results meet the American Association for the Advancement of Medical Instrumentation (AAMI) and British Hypertension Society (BHS) Class A criteria. Conclusion: By using robust features and feature selection methods, we alleviated patient dependency to have reliable estimates of vitals.

Performance of radiomics in the differential diagnosis of parotid tumors: a systematic review.

A systematic review and meta-analysis were conducted to evaluate the diagnostic precision of radiomics in the differential diagnosis of parotid tumors, considering the increasing utilization of radiomics in tumor diagnosis. Although some researchers have attempted to apply radiomics in this context, there is ongoing debate regarding its accuracy. Databases of PubMed, Cochrane, EMBASE, and Web of Science up to May 29, 2024 were systematically searched. The quality of included primary studies was assessed using the Radiomics Quality Score (RQS) checklist. The meta-analysis was performed utilizing a bivariate mixed-effects model. A total of 39 primary studies were incorporated. The machine learning model relying on MRI radiomics for diagnosis malignant tumors of the parotid gland, demonstrated a sensitivity of 0.80 [95% CI: 0.74, 0.86], SROC of 0.89 [95% CI: 0.27-0.99] in the validation set. The machine learning model based on MRI radiomics for diagnosis malignant tumors of the parotid gland, exhibited a sensitivity of 0.83[95% CI: 0.76, 0.88], SROC of 0.89 [95% CI: 0.17-1.00] in the validation set. The models also demonstrated high predictive accuracy for benign lesions. There is great potential for radiomics-based models to improve the accuracy of diagnosing benign and malignant tumors of the parotid gland. To further enhance this potential, future studies should consider implementing standardized radiomics-based features, adopting more robust feature selection methods, and utilizing advanced model development tools. These measures can significantly improve the diagnostic accuracy of artificial intelligence algorithms in distinguishing between benign and malignant tumors of the parotid gland. https://www.crd.york.ac.uk/prospero/, identifier CRD42023434931.

Active Antinoise Fuzzy Dominance Rough Feature Selection Using Adaptive K-Nearest Neighbors

Feature selection methods with anti-noise performance are effective dimensionality reduction methods for classification tasks with noise. However, there are few studies on robust feature selection methods for monotonic classification tasks. The fuzzy dominance rough set (FDRS) model is a nontrivial knowledge acquisition tool, which is widely used in feature selection of monotonic classification tasks. Nonetheless, this model has been proved in practice to be generally poorly fault-tolerance, and only one noisy sample can cause huge interference in acquiring knowledge. In view of these two issues, this paper firstly designs an adaptive <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -nearest neighbors strategy to calculate the density of samples. The noisy samples are identified according to their densities, and then an active anti-noise fuzzy dominance rough set model is proposed. Then, in the active anti-noise fuzzy dominance rough approximation space, the class-separability is evaluated by the approximation operators of the proposed model, and the feature-redundancy is evaluated by the fuzzy ranking conditional mutual information. On this basis, a feature evaluation index is designed comprehensively considering class-separability and feature-redundancy. Finally, a feature selection algorithm is designed to select the feature subset with the highest classification performance. The experimental results show that the proposed algorithm has better robustness and classification performance.

Robust Feature Selection Method Based on Joint L2,1 Norm Minimization for Sparse Regression

Feature selection methods are widely used in machine learning tasks to reduce the dimensionality and improve the performance of the models. However, traditional feature selection methods based on regression often suffer from a lack of robustness and generalization ability and are easily affected by outliers in the data. To address this problem, we propose a robust feature selection method based on sparse regression. This method uses a non-square form of the L2,1 norm as both the loss function and regularization term, which can effectively enhance the model’s resistance to outliers and achieve feature selection simultaneously. Furthermore, to improve the model’s robustness and prevent overfitting, we add an elastic variable to the loss function. We design two efficient convergent iterative processes to solve the optimization problem based on the L2,1 norm and propose a robust joint sparse regression algorithm. Extensive experimental results on three public datasets show that our feature selection method outperforms other comparison methods.

Netrank: network-based approach for biomarker discovery

BackgroundIntegrating multi-omics data is fast becoming a powerful approach for predicting disease progression and treatment outcomes. In light of that, we introduce a modified version of the NetRank algorithm, a network-based algorithm for biomarker discovery that incorporates the protein associations, co-expressions, and functions with its phenotypic association to differentiate different types of cancer. NetRank is introduced here as a robust feature selection method for biomarker selection in cancer prediction. We assess the robustness and suitability of the RNA gene expression data through scanning genomic data for 19 cancer types with more than 3000 patients from The Cancer Genome Atlas (TCGA).ResultsThe results of evaluating different cancer type profiles from the TCGA data demonstrate the strength of our approach to identifying interpretable biomarker signatures for cancer outcome prediction. NetRank’s biomarkers segregate most cancer types with an area under the curve (AUC) above 90% using compact signatures.ConclusionIn this paper we provide a fast and efficient implementation of NetRank, with a case study from The Cancer Genome Atlas, to assess the performance. We incorporated complete functionality for pre and post-processing for RNA-seq gene expression data with functions for building protein-protein interaction networks. The source code of NetRank is freely available (at github.com/Alfatlawi/Omics-NetRank) with an installable R library. We also deliver a comprehensive practical user manual with examples and data attached to this paper.

The optimal method for water quality parameters retrieval of urban river based on machine learning algorithms using remote sensing images

ABSTRACT Water eutrophication has become one of the prominent problems of environmental protection in inland watersheds. Turbidity, total phosphorus (TP) and total nitrogen (TN) concentrations are key water quality parameters (WQPs) that reflect the level of water eutrophication in inland waters. Due to the complex interaction effects between different water quality in urban rivers, the water quality retrieval models still have the problem of single input features and poor applicability. This paper proposed a robust feature selection method based on machine learning and utilized Sentinel-2 remote sensing images for water quality retrieval of Chan and Ba rivers in Xi’an City. The ReliefF and global sensitivity analysis (GSA) methods (ReliefF-GSA) were used to select the optimal feature combination from the potential feature dataset. Based on the optimal feature combination, Random Forest regression (RFR), LightGBM and XGboost models were constructed for the three WQPs retrieval, respectively. The optimal models were then used to invert the three WQPs and the spatial-temporal variation of WQPs from January 2021 to January 2022 was analysed. The results show that (1) The RelieF-GSA method is suitable for high-dimensional feature filtration and enables optimal feature selection for specific WQPs retrieval. It is revealed that the BOI index (black odour water index) is the key feature for the retrieval of turbidity and TN concentration. (2) The RFR model was found to be better than other models and more appropriate for Chan and Ba rivers, with coefficients of determination (R2) of 0.90, 0.89 and 0.81, respectively. (3) It was found that the water qualities in the Chan and Ba rivers have prominent seasonal characteristics. Turbidity and TP concentrations showed higher, while TN concentration showed relatively low in autumn. The method and conclusions of this paper can further provide a reference for WPQs retrieval in urban rivers.

Automatic Features Extraction Integrated With Exact Gaussian Process for Respiratory Rate and Uncertainty Estimations

Respiratory rate monitoring has become necessary for people with respiratory diseases, especially those living alone. Sudden changes in respiratory rate (RR) in these individuals may indicate a severe illness. Hence, estimating the RR is essential for cardiopulmonary health conditions. Capnography is a well-known standard technique for measuring RR. This study presents a novel methodology that uses an automatic feature extraction algorithm integrated with a robust feature selection technique based on exact Gaussian process regression (EGPR) for RR and uncertainty estimations. As EGPR is vital to noisy data and is naturally normalized, it can generate uncertainty estimates. Uncertainty estimation is essential for estimating physiological parameters. In the proposed methodology, data dimension is obtained using a power spectral density feature based on an autocorrelation function; subsequently, the long-resampled wave signal is split to increase the number of input data samples. Selecting an appropriate high-dimensional feature is necessary to predict a response effectively. Herein, we use a robust feature selection method based on neighbor component analysis. The proposed EGPR with automatic feature extraction is more accurate than conventional algorithms for RR and confidence intervals (CIs) estimations. The proposed methodology proves superior to conventional algorithms in terms of saving computer resources. We confirm that the proposed methodology, 2.085 breath per minute (bpm), and EGPR with robust feature selection, 2.049 bpm, show the lowest mean absolute error compared to the conventional algorithms in the Beth Israel Deaconess Medical Center data set.

Class-Level Refactoring Prediction by Ensemble Learning with Various Feature Selection Techniques

Background: Refactoring is changing a software system without affecting the software functionality. The current researchers aim i to identify the appropriate method(s) or class(s) that needs to be refactored in object-oriented software. Ensemble learning helps to reduce prediction errors by amalgamating different classifiers and their respective performances over the original feature data. Other motives are added in this paper regarding several ensemble learners, errors, sampling techniques, and feature selection techniques for refactoring prediction at the class level. Objective: This work aims to develop an ensemble-based refactoring prediction model with structural identification of source code metrics using different feature selection techniques and data sampling techniques to distribute the data uniformly. Our model finds the best classifier after achieving fewer errors during refactoring prediction at the class level. Methodology: At first, our proposed model extracts a total of 125 software metrics computed from object-oriented software systems processed for a robust multi-phased feature selection method encompassing Wilcoxon significant text, Pearson correlation test, and principal component analysis (PCA). The proposed multi-phased feature selection method retains the optimal features characterizing inheritance, size, coupling, cohesion, and complexity. After obtaining the optimal set of software metrics, a novel heterogeneous ensemble classifier is developed using techniques such as ANN-Gradient Descent, ANN-Levenberg Marquardt, ANN-GDX, ANN-Radial Basis Function; support vector machine with different kernel functions such as LSSVM-Linear, LSSVM-Polynomial, LSSVM-RBF, Decision Tree algorithm, Logistic Regression algorithm and extreme learning machine (ELM) model are used as the base classifier. In our paper, we have calculated four different errors i.e., Mean Absolute Error (MAE), Mean magnitude of Relative Error (MORE), Root Mean Square Error (RMSE), and Standard Error of Mean (SEM). Result: In our proposed model, the maximum voting ensemble (MVE) achieves better accuracy, recall, precision, and F-measure values (99.76, 99.93, 98.96, 98.44) as compared to the base trained ensemble (BTE) and it experiences less errors (MAE = 0.0057, MORE = 0.0701, RMSE = 0.0068, and SEM = 0.0107) during its implementation to develop the refactoring model. Conclusions: Our experimental result recommends that MVE with upsampling can be implemented to improve the performance of the refactoring prediction model at the class level. Furthermore, the performance of our model with different data sampling techniques and feature selection techniques has been shown in the form boxplot diagram of accuracy, F-measure, precision, recall, and area under the curve (AUC) parameters.

Air quality monitoring based on chemical and meteorological drivers: Application of a novel data filtering-based hybridized deep learning model

Particulate matter (PM) or particle pollution include the tiny particles of dust and fly ash particles are expelled from coal-burning power plants. Coal combustion is an extremely prevalent source of air pollution, and resulting PM has substantial impacts on human health, especially in industrial zones. This paper aims to design a novel hybrid deep learning framework based on long short-term memory (LSTM) integrated with a two-stage data filtering technique to accurately predict the air quality indices (i.e., PM2.5 and PM10) in a chosen study region ‘Miles Airport, Queensland,’ that meets the needs of the coal seam gas industry in Australia. The data used to construct the novel hybrid two-stage data filtering technique based on LSTM comprising of six meteorological parameters (i.e., wind direction, wind speed, air temperature, relative humidity, solar radiation, and rainfall) and two environmental factors (i.e., ozone and total suspended particulate). In the first stage, two robust feature selection methods, namely, extreme gradient boosting (XGBoost) and the classification and regression tree (CART) approach, were adopted to explore the most significant predictors. Then in the second stage, the best subset regression (BSR) technique is utilized to determine the best subsets input combinations i.e., C1, C2, and C3 based on several particular metrics. The three BSR-based input combinations were employed in the LSTM model to estimate the PM2.5 and PM10. Furthermore, to validate the main hybrid framework, two advanced machine learning (ML) methods (i.e., LightGBM and ridge kernel regression (KRR)) and two traditional ML methods (i.e., Adaptive neuro-fuzzy inference system (ANFIS) and multilayer perceptron neural network (MLP)) were hybridized with the multi-level data filtering strategy by examining the optimal input combinations. Several statistical metrics, graphical tools, and diagnostic analyses evaluated the hybrid models. The outcomes of the PM2.5 simulation based on 2375 data samples showed that the LSTM-C3 containing all the selected predictors yielded the most promising accuracy, followed by the LightGBM-C3 and MLP-C3 models. On the other hand, the simulation of PM10 distribution demonstrated that the LSTM-C3 was superior to other models, followed by the KRR-C3 and LightGBM-C3 models.

A robust feature selection method based on meta-heuristic optimization for speech emotion recognition

A robust feature selection method based on meta-heuristic optimization for speech emotion recognition

Label correlations variation for robust multi-label feature selection

Numerous high-dimension multi-label data are produced, leading to the imperative need to design excellent multi-label feature selection methods. It is of paramount importance to exploit label correlations in previous methods. However, there exist two unsolved issues in most of existing methods. First, most of existing methods explore label correlations based on the original label space with redundant and irrelevant label information. Second, previous methods either consider second-order label correlations or high-order label correlations, in fact, both two types of label correlations are significant and complementary for capturing label information. To this end, this paper proposes a robust multi-label feature selection method with both two types of label correlations. To start with, we introduce the self-expression model to consider the high-order label correlations, additionally, the l2,1-norm is imposed onto the self-expression coefficient matrix to eliminate redundant and noisy information. Furthermore, we employ a label-level regularizer to achieve pairwise label correlations. Finally, an optimization scheme with convergence proof is designed to deal with the objective function. Multiple experimental analysis results on fourteen multi-label data sets manifest that the classification performance of the proposed method outperforms other baselines.

Optimized structure learning of Bayesian Network for investigating causation of vehicles’ on-road crashes

A vehicle's crash can be seen as a failure of microscopic road transportation system. The causal investigation of vehicles’ crashes has drawn much attention from academia and industry alike, which is of significance to road traffic safety. This study develops a structure learning method to construct Bayesian Network (BN). The BN as generated by the method can comprehensively illustrate the causal relationships between risk contributing features and vehicles’ on-road risky events (i.e. near-crash and crash). The proposed structure learning method has following three advantages: (1). considering multiple categories of features; (2). applying robust feature selection method to improve prediction performance and facilitate the explanation of causation; and (3). making a trade-off between the complexity and interpretability of BN structure. The method is applied on the Second Highway Research Program (SHRP2) Naturalistic Driving Study (NDS) database for case study. The results show that the generated optimal BN achieves satisfactory performances on both structure complexity and prediction accuracy. Besides, as compared to the BN built by the other state-of-the-art structure learning methods, the optimal BN presents superior performance on causal interpretability. Also, by performing causal inferences upon the optimal BN, this study examines and analyzes the contributions of several key features to the risky events. Several interesting findings about the features’ contributions are reported in this paper, which could provide valuable references for road safety engineering in the future.

Unsupervised feature selection with robust data reconstruction (UFS-RDR) and outlier detection

In unsupervised learning, the traditional feature selection methods are not always efficient and their feature selection performance can be severely affected in the presence of outliers and noise. To address this issue, we propose a novel robust unsupervised feature selection method, called Unsupervised Feature Selection with Robust Data Reconstruction (UFS-RDR), that minimizes the graph regularized weighted data reconstruction error function. For the detection of outliers, the well-known Mahalanobis distance is used and further determine the Huber-type weight function using these Mahalanobis distances. This weight function downweights the clustering observations that have large distance. Our experimental results on both synthetic and real-world datasets indicate that the proposed UFS-RDR approach has good feature selection performance and also outperforms the competitive non-robust unsupervised feature selection methods in the presence of contamination in the unlabeled data.

Orthogonally constrained matrix factorization for robust unsupervised feature selection with local preserving

Feature selection is a significant preprocessing technique that involves discarding redundant and irrelevant features, so as to reduce the data dimensionality and build a better understanding of data. Pruning superfluous features tend to build better generalization models while improving the computation efficiency extremely. In practices, obtaining the labels of data is time-consuming and labor-intensive, which brings great challenges for feature selection. In this paper, we present a novel robust unsupervised feature selection method for unlabeled dimensionality reduction, which obtains feature importance information by predicting cluster labels of data with matrix factorization. The orthogonal constraints on two decomposing matrices facilitate achieving more accurate class labels of clusters so as to select features with highly discrimination power. Then, the local preserving term is integrated into the projected data for selecting features with local retention ability. Independent of that, an alternative iterative algorithm is incorporated into the optimization of ℓ2,1-norm objective function for efficient and robust feature selection. Extensive comparative experiments are carried out on six benchmark datasets to evaluate the performance of the proposed method. The results show that our method outperforms several well-known unsupervised feature selection methods in terms of both clustering accuracy and normalized mutual information.

Assessing Parkinson's Disease at Scale Using Telephone-Recorded Speech: Insights from the Parkinson's Voice Initiative.

Numerous studies have reported on the high accuracy of using voice tasks for the remote detection and monitoring of Parkinson’s Disease (PD). Most of these studies, however, report findings on a small number of voice recordings, often collected under acoustically controlled conditions, and therefore cannot scale at large without specialized equipment. In this study, we aimed to evaluate the potential of using voice as a population-based PD screening tool in resource-constrained settings. Using the standard telephone network, we processed 11,942 sustained vowel /a/ phonations from a US-English cohort comprising 1078 PD and 5453 control participants. We characterized each phonation using 304 dysphonia measures to quantify a range of vocal impairments. Given that this is a highly unbalanced problem, we used the following strategy: we selected a balanced subset (n = 3000 samples) for training and testing using 10-fold cross-validation (CV), and the remaining (unbalanced held-out dataset, n = 8942) samples for further model validation. Using robust feature selection methods we selected 27 dysphonia measures to present into a radial-basis-function support vector machine and demonstrated differentiation of PD participants from controls with 67.43% sensitivity and 67.25% specificity. These findings could help pave the way forward toward the development of an inexpensive, remote, and reliable diagnostic support tool for PD using voice as a digital biomarker.

Stable Iterative Variable Selection.

MotivationThe emergence of datasets with tens of thousands of features, such as high-throughput omics biomedical data, highlights the importance of reducing the feature space into a distilled subset that can truly capture the signal for research and industry by aiding in finding more effective biomarkers for the question in hand. A good feature set also facilitates building robust predictive models with improved interpretability and convergence of the applied method due to the smaller feature space.ResultsHere, we present a robust feature selection method named Stable Iterative Variable Selection (SIVS) and assess its performance over both omics and clinical data types. As a performance assessment metric, we compared the number and goodness of the selected feature using SIVS to those selected by Least Absolute Shrinkage and Selection Operator regression. The results suggested that the feature space selected by SIVS was, on average, 41% smaller, without having a negative effect on the model performance. A similar result was observed for comparison with Boruta and caret RFE.Availability and implementationThe method is implemented as an R package under GNU General Public License v3.0 and is accessible via Comprehensive R Archive Network (CRAN) via https://cran.r-project.org/package=sivs.Supplementary information Supplementary data are available at Bioinformatics online.

Adaptive Pose Estimation for Gait Event Detection Using Context-Aware Model and Hierarchical Optimization

To understand daily events accurately, adaptive pose estimation (APE) systems require a robust context-aware model and optimal feature selection methods. In this paper, we propose a novel gait event detection (GED) system that consists of saliency silhouette detection, a robust body parts model and a 2D stick-model followed by a hierarchical optimization algorithm. Furthermore, the most prominent context-aware features such as energy, 0–180° intensity and distinct moveable features are proposed by focusing on invariant and localized characteristics of human postures in different event classes. Finally, we apply Grey Wolf optimization and a genetic algorithm to discriminate complex postures and to provide appropriate labels to each event. In order to evaluate the performance of proposed GED, two public benchmark datasets, UCF101 and YouTube, are examined via the n-fold cross validation method. For the two benchmark datasets, our proposed method detects the human body key points with 82.4% and 83.2% accuracy respectively. Also, it extracts the context-aware features and finally recognizes the gait events with 82.6% and 85.0% accuracy, respectively. Compared with other well-known statistical and state-of-the-art methods, our proposed method outperforms other similarly tasked methods in terms of posture detection and recognition accuracy.

Resample aggregating improves the generalizability of connectome predictive modeling

It is a longstanding goal of neuroimaging to produce reliable, generalizable models of brain behavior relationships. More recently, data driven predictive models have become popular. However, overfitting is a common problem with statistical models, which impedes model generalization. Cross validation (CV) is often used to estimate expected model performance within sample. Yet, the best way to generate brain behavior models, and apply them out-of-sample, on an unseen dataset, is unclear. As a solution, this study proposes an ensemble learning method, in this case resample aggregating, encompassing both model parameter estimation and feature selection. Here we investigate the use of resampled aggregated models when used to estimate fluid intelligence (fIQ) from fMRI based functional connectivity (FC) data. We take advantage of two large openly available datasets, the Human Connectome Project (HCP), and the Philadelphia Neurodevelopmental Cohort (PNC). We generate aggregated and non-aggregated models of fIQ in the HCP, using the Connectome Prediction Modelling (CPM) framework. Over various test-train splits, these models are evaluated in sample, on left-out HCP data, and out-of-sample, on PNC data. We find that a resample aggregated model performs best both within- and out-of-sample. We also find that feature selection can vary substantially within-sample. More robust feature selection methods, as detailed here, are needed to improve cross sample performance of CPM based brain behavior models.