Low-dimensional Representation Research Articles

Low-dimensional signal representations for massive black hole binary signals analysis from LISA data

The space-based gravitational wave observatory LISA will provide a wealth of information to analyze massive black hole binaries with high chirp masses, beyond $10^5$ solar masses. The large number of expected MBHBs (one event a day on average) increases the risk of overlapping between events. As well, the data will be contaminated with non-stationary artifacts, such as glitches and data gaps, which are expected to strongly impact the MBHB analysis, which mandates the development of dedicated detection and retrieval methods on long time intervals. Building upon a methodological approach we introduced for galactic binaries, in this article we investigate an original non-parametric recovery of MBHB signals from measurements with instrumental noise typical of LISA in order to tackle detection and signal reconstruction tasks on long time intervals. We investigated different approaches based on sparse signal modeling and machine learning. In this framework, we focused on recovering MBHB waveforms on long time intervals, which is a building block to further tackling more general signal recovery problems, from gap mitigation to unmixing overlapped signals. To that end, we introduced a hybrid method called SCARF (sparse chirp adaptive representation in Fourier), which combines a deep learning modeling of the merger of the MBHB with a specific adaptive time-frequency representation of the inspiral. Numerical experiments have been carried out on simulations of single MBHB events that account for the LISA response and with realistic realizations of noise. We checked the performances of the proposed hybrid method for the fast detection and recovery of the MBHB.

Deep hyperbolic convolutional model for knowledge graph embedding

Recent advancements in knowledge graph embedding have enabled the representation of entities and relations in continuous vector spaces. Performing link prediction on incomplete knowledge graphs using these embeddings has emerged as a challenging task, drawing considerable research interest. Knowledge graphs in the real world typically exhibit a natural hierarchical structure. Hyperbolic embedding methods have shown promising results in modeling hierarchical data, especially in low-dimensional representations. However, existing hyperbolic models for knowledge graph embedding are generally shallow, resulting in limited expressiveness. Additionally, some deep hyperbolic models exist but are often constrained to fixed curvature spaces, which may not optimally capture varying hierarchical structures. In this work, we propose DeER, a novel multi-layer hyperbolic model that leverages a trainable curvature space, marking an advancement in the depth and adaptability of hyperbolic modeling for knowledge graph embeddings. We specifically design a hyperbolic convolutional neural network to learn the heterogeneous interactions between entities and relations within this adaptable hyperbolic space. Additionally, we introduce a hyperbolic feedforward neural network to transform hyperbolic features across multiple layers, enhancing the model’s ability to capture complex hierarchical relationships. Experimental results on three benchmark datasets demonstrate that DeER significantly enhances expressiveness and hierarchical modeling performance when compared to both shallow and other deep baseline approaches.

PiRNA-disease association prediction based on multi-channel graph variational autoencoder.

Piwi-interacting RNA (piRNA) is a type of non-coding small RNA that is highly expressed in mammalian testis. PiRNA has been implicated in various human diseases, but the experimental validation of piRNA-disease associations is costly and time-consuming. In this article, a novel computational method for predicting piRNA-disease associations using a multi-channel graph variational autoencoder (MC-GVAE) is proposed. This method integrates four types of similarity networks for piRNAs and diseases, which are derived from piRNA sequences, disease semantics, piRNA Gaussian Interaction Profile (GIP) kernel, and disease GIP kernel, respectively. These networks are modeled by a graph VAE framework, which can learn low-dimensional and informative feature representations for piRNAs and diseases. Then, a multi-channel method is used to fuse the feature representations from different networks. Finally, a three-layer neural network classifier is applied to predict the potential associations between piRNAs and diseases. The method was evaluated on a benchmark dataset containing 5,002 experimentally validated associations with 4,350 piRNAs and 21 diseases, constructed from the piRDisease v1.0 database. It achieved state-of-the-art performance, with an average AUC value of 0.9310 and an AUPR value of 0.9247 under five-fold cross-validation. This demonstrates the method's effectiveness and superiority in piRNA-disease association prediction.

Semi-Supervised Learning for Multi-View Data Classification and Visualization

Data visualization has several advantages, such as representing vast amounts of data and visually demonstrating patterns within it. Manifold learning methods help us estimate lower-dimensional representations of data, thereby enabling more effective visualizations. In data analysis, relying on a single view can often lead to misleading conclusions due to its limited perspective. Hence, leveraging multiple views simultaneously and interactively can mitigate this risk and enhance performance by exploiting diverse information sources. Additionally, incorporating different views concurrently during the graph construction process using interactive visualization approach has improved overall performance. In this paper, we introduce a novel algorithm for joint consistent graph construction and label estimation. Our method simultaneously constructs a unified graph and predicts the labels of unlabeled samples. Furthermore, the proposed approach estimates a projection matrix that enables the prediction of labels for unseen samples. Moreover, it incorporates the information in the label space to further enhance the accuracy. In addition, it merges the information in different views along with the labels to construct a consensus graph. Experimental results conducted on various image databases demonstrate the superiority of our fusion approach compared to using a single view or other fusion algorithms. This highlights the effectiveness of leveraging multiple views and simultaneously constructing a unified graph for improved performance in data classification and visualization tasks in semi-supervised contexts.

Pre-frontal cortex guides dimension-reducing transformations in the occipito-ventral pathway for categorization behaviors

To interpret our surroundings, the brain uses a visual categorization process. Current theories and models suggest that this process comprises a hierarchy of different computations that transforms complex, high-dimensional inputs into lower-dimensional representations (i.e., manifolds) in support of multiple categorization behaviors. Here, we tested this hypothesis by analyzing these transformations reflected in dynamic MEG source activity while individual participants actively categorized the same stimuli according to different tasks: face expression, face gender, pedestrian gender, and vehicle type. Results reveal three transformation stages guided by the pre-frontal cortex. At stage 1 (high-dimensional, 50-120ms), occipital sources represent both task-relevant and task-irrelevant stimulus features; task-relevant features advance into higher ventral/dorsal regions, whereas task-irrelevant features halt at the occipital-temporal junction. At stage 2 (121-150ms), stimulus feature representations reduce to lower-dimensional manifolds, which then transform into the task-relevant features underlying categorization behavior over stage 3 (161-350ms). Our findings shed light on how the brain's network mechanisms transform high-dimensional inputs into specific feature manifolds that support multiple categorization behaviors.

Real-time Bayesian inversion in resin transfer moulding using neural surrogates

In Resin Transfer Moulding (RTM), local variations in reinforcement properties (porosity and permeability) and the formation of gaps along the reinforcement edges result in non-uniform resin flow patterns, which may cause defects in the produced composite component. The ensemble Kalman inversion (EKI) algorithm has previously been used to invert in-process data to estimate local reinforcement properties. However, implementation of this algorithm in some applications is limited by the requirement to run thousands of computationally expensive resin flow simulations. In this study, a machine learning approach is used to train a surrogate model which can emulate resin flow simulations near-instantaneously. A partition of the flow domain into a low-dimensional representation enables an artificial neural network (ANN) surrogate to make accurate predictions, with a simple architecture. When the ANN is integrated within the EKI algorithm, estimates for local reinforcement permeability and porosity can be achieved in real time, as was verified by virtual and lab experiments. Since EKI utilises the Bayesian framework, estimates are given within confidence intervals and statements can be made on-line regarding the probability of defects within sections of the reinforcement. The proposed framework has shown good predictive capabilities for the set of laboratory experiments and estimates for reinforcement properties were always computed within 1 s.

Cube is a good form: Hyperspectral band selection via multi-dimensional and high-order structure preserved clustering

As an effective strategy for reducing the noisy and redundant information for hyperspectral imagery (HSI), hyperspectral band selection intends to select a subset of original hyperspectral bands, which boosts the subsequent different tasks. In this paper, we introduce a multi-dimensional high-order structure preserved clustering method for hyperspectral band selection, referred to as MHSPC briefly. By regarding original hyperspectral images as a tensor cube, we apply the tensor CP (CANDECOMP/PARAFAC) decomposition on it to exploit the multi-dimensional structural information as well as generate a low-dimensional latent feature representation. In order to capture the local geometrical structure along the spectral dimension, a graph regularizer is imposed on the new feature representation in the lower dimensional space. In addition, since the low rankness of HSIs is an important global property, we utilize a nuclear norm constraint on the latent feature representation matrix to capture the global data structure information. Different to most of previous clustering based hyperspectral band selection methods which vectorize each band as a vector without considering the 2-D spatial information, the proposed MHSPC can effectively capture the spatial structure as well as the spectral correlation of original hyperspectral cube in both local and global perspectives. An efficient alternatively updating algorithm with theoretical convergence guarantee is designed to solve the resultant optimization problem, and extensive experimental results on four benchmark datasets validate the effectiveness of the proposed MHSPC over other state-of-the-arts.

Replacing non-biomedical concepts improves embedding of biomedical concepts.

Concept embeddings are low-dimensional vector representations of concepts such as MeSH:D009203 (Myocardial Infarction), whose similarity in the embedded vector space reflects their semantic similarity. Here, we test the hypothesis that non-biomedical concept synonym replacement can improve the quality of biomedical concepts embeddings. We developed an approach that leverages WordNet to replace sets of synonyms with the most common representative of the synonym set. We tested our approach on 1055 concept sets and found that, on average, the mean intra-cluster distance was reduced by 8% in the vector-space. Assuming that homophily of related concepts in the vector space is desirable, our approach tends to improve the quality of embeddings. This pilot study shows that non-biomedical synonym replacement tends to improve the quality of embeddings of biomedical concepts using the Word2Vec algorithm. We have implemented our approach in a freely available Python package available at https://github.com/TheJacksonLaboratory/wn2vec.

P-149 A visual interpretability method to unbox ‘black-box’ deep learning image-based classification of embryo properties

Abstract Study question Can we decipher the underlying visual properties that drive image-based AI embryo classification models to assist clinical decisions and biological discovery? Summary answer Our framework interpreted which annotated and non-explicitly-annotated phenotypes impact model predictions and rank their importance. These discoveries were aligned with known blastocyst quality criteria. What is known already Deep learning models have shown great promise for complex pattern recognition when applied to embryo images. The success of these models relies on their ability to perform non-linear optimization of feature extraction during model construction. However, this involves their entanglement of multiple classification-driving image properties, thereby producing ‘black-box’ systems that lack user confidence, trust and interpretability. Therefore, there is an urgent need for an interpretability method that can uncover the semantic image properties that contribute to ‘black box’ embryo image-based AI classification model predictions to assist in blastocyst selection. Study design, size, duration 11,211 time-lapse videos were retrospectively collected from three IVF centers. A deep convolutional neural network is first trained to discriminate high-versus-low quality blastocysts. We then developed DISCOVER, a general-purpose interpretability method designed to discover underlying visual properties driving the classifier. DISCOVER encodes an image to an interpretable lower dimensional representation which is correlated to the classifier and encapsulates a different distinct phenotype in each one of the dimensions. Participants/materials, setting, methods The encoding of embryo images to low dimensional representations enables interpretability globally and locally. Globally, the embryo images are synthetically altered by amplifying subtle properties that affect the classification decision. With our method this can be done one property at a time, therefore separating confounding properties. By evaluating the altered images, embryologists can decipher their meaning. Locally, each one of the discovered properties can be ranked by its importance for a specific embryo instance. Main results and the role of chance Using DISCOVER, we interpreted the classification model driving features. We quantitatively linked the top two classification features as blastocyst size (as proxy to degree of expansion and development) and trophectoderm quality, by embryologists evaluation and annotations. We then asked whether DISCOVER can identify non-explicitly annotated latent features that encode morphologic properties not defined by ASEBIR/Gardner criteria. Expert embryologist interpreted the third top classification feature to be the blastocoel. DISCOVER interpreted high quality embryos as having denser and more granular blastocoelic regions, suggesting that this change in the blastocoel appearance is one of the encoded classification-driving morphologic properties. This visualization indicates that there are additional parameters of the blastocoel beyond its volume expansion associated with its quality. We showed how embryo properties can be weighted differently by the classifier on a per embryo basis, giving clinical insight to which properties influence the classification of a specific instance. These results indicate that DISCOVER enables expert-in-the-loop interpretation of the classification model both globally, discovering the overall main properties driving the classifier, and locally, showing a per instance explanation. Limitations, reasons for caution DISCOVER failed to interpret the inner cell mass (ICM) as a classification-driving feature in its latent representation, though it was explicitly used to label the data for training the classification model. It is possible that other properties collectively contained the discriminative information encoded in the ICM. Wider implications of the findings This deep analysis demonstrates the feasibility of providing interpretability for biomedical image-based classification models for clinical use in the IVF clinic. Trial registration number not applicable

S CIRCLE-An interactive visual exploration tool for single cell RNA-Seq data.

sCIRCLE (single-Cell Interactive Real-time Computer visualization for Low-dimensional Exploration) is a tool for exploratory analysis of single cell RNA-seq (scRNA-seq) data sets, with a focus on bacterial scRNA-seq. The software takes an information design perspective to re-envision visually and interactively exploring low dimensional representations of scRNA-Seq data. Users can project cells in various 3D and 2D spaces and interactively query and paint cells using rich metadata sets reporting on cell cluster, gene function, and gene expression. As a standalone application it contains, among other features, options for dimensionality reduction, navigation and interaction with data in 3d and 2d space, gene filtering, fold change and metacell computation as well as various capabilities for visualization, data import and export.

Variational autoencoder-based dimension reduction of Ichimoku features for improved financial market analysis

Financial markets are complex and dynamic, and accurately predicting market trends is crucial for traders and financial analysts. Ichimoku-based features have gained significant attention in financial market analysis due to their ability to capture essential market signals and patterns. This significant compression retains essential patterns related to trends, support/resistance levels, and trading signals. The reduced dimensionality improves computational efficiency and could allow for more accurate predictive modeling by traders. However, real-world testing is needed because compressing data risks losing useful nuances. In this study, we utilize an autoencoder for the dimensionality reduction of Ichimoku-based features in financial market analysis. The autoencoder, a neural network architecture, compresses high-dimensional data into a lower-dimensional representation by learning important features and patterns. The experiments conducted on a Euro/Dollar market dataset spanning 1990, comprising 16 columns with Ichimoku features, reveal the remarkable reduction of dataset size from 2,269,500 to 756,375, equivalent to a decrease of 66.67 %. These results highlight the efficiency of the proposed approach in reducing the dimensionality of financial market data, suggesting its potential as a valuable tool for traders and financial analysts to predict market trends and make informed decisions in the financial markets.

How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model

Deep learning algorithms demonstrate a surprising ability to learn high-dimensional tasks from limited examples. This is commonly attributed to the depth of neural networks, enabling them to build a hierarchy of abstract, low-dimensional data representations. However, how many training examples are required to learn such representations remains unknown. To quantitatively study this question, we introduce the random hierarchy model: a family of synthetic tasks inspired by the hierarchical structure of language and images. The model is a classification task where each class corresponds to a group of high-level features, chosen among several equivalent groups associated with the same class. In turn, each feature corresponds to a group of subfeatures chosen among several equivalent groups and so on, following a hierarchy of composition rules. We find that deep networks learn the task by developing internal representations invariant to exchanging equivalent groups. Moreover, the number of data required corresponds to the point where correlations between low-level features and classes become detectable. Overall, our results indicate how deep networks overcome the curse of dimensionality by building invariant representations and provide an estimate of the number of data required to learn a hierarchical task. Published by the American Physical Society 2024

Reduced-order modeling of unsteady fluid flow using neural network ensembles

The use of deep learning has become increasingly popular in reduced-order models (ROMs) to obtain low-dimensional representations of full-order models. Convolutional autoencoders (CAEs) are often used to this end as they are adept at handling data that are spatially distributed, including solutions to partial differential equations. When applied to unsteady physics problems, ROMs also require a model for time-series prediction of the low-dimensional latent variables. Long short-term memory (LSTM) networks, a type of recurrent neural network useful for modeling sequential data, are frequently employed in data-driven ROMs for autoregressive time-series prediction. When making predictions at unseen design points over long time horizons, error propagation is a frequently encountered issue, where errors made early on can compound over time and lead to large inaccuracies. In this work, we propose using bagging, a commonly used ensemble learning technique, to develop a fully data-driven ROM framework referred to as the CAE-eLSTM ROM that uses CAEs for spatial reconstruction of the full-order model and LSTM ensembles for time-series prediction. When applied to two unsteady fluid dynamics problems, our results show that the presented framework effectively reduces error propagation and leads to more accurate time-series prediction of latent variables at unseen points.

Multi-sample non-negative spatial factorization.

It is important to model biological variation when analyzing spatial transcriptomics data from multiple samples. One approach to multi-sample analysis is to spatially align samples, but this is a challenging problem. Here, we provide an alignment-free framework for generalizing a one-sample spatial factorization model to multi-sample data. Using this framework, we develop a method, called multi-sample non-negative spatial factorization (mNSF) that extends the one-sample non-negative spatial factorization (NSF) framework to a multi-sample dataset. Our model allows for a sample-specific model for the spatial correlation structure and extracts a low-dimensional representation of the data. We illustrate the performance of mNSF by simulation studies and real data. mNSF identifies true factors in simulated data, identifies shared anatomical regions across samples in real data and reveals region-specific biological functions. mNSFs performance is similar to alignment based methods when alignment is possible, but extends analysis to situations where spatial alignment is impossible. We expect multi-sample factorization methods to be a powerful class of methods for analyzing spatially resolved transcriptomics data.

Incremental minimum distortion embedding for online structural damage classification.

The damage detection problem in structures diminishes the maintenance cost and extends the life of the structures. Online structural damage classification is a relevant topic in structural health monitoring since monitoring the structure using sensors can help decision-makers can rely on empirical evidence for making informed choices regarding the maintenance, repair, or replacement of structural components. Sensors generate high-dimensional data. To address this challenge, employing dimensionality reduction methods becomes essential. These methods help obtain a low-dimensional representation of the data. Traditional dimensionality reduction methods are trained on a fixed data set, and incorporating new data may require retraining the entire model. This study employed the Minimum Distortion Embedding (MDE) method incrementally, ensuring the maintenance of a consistent embedding as new data becomes available. This incremental embedding took advantage of an anchor constraint to pin the existing embedding in place, then embed the new points. An online structural classification methodology was developed using the incremental MDE and a stream based hoeffding tree classifier. Stream data processing differs from batch processing, as it involves the real-time consumption of each incoming data. The developed methodology was tested with data obtained by accelerometers in a laboratory scaled jacket-type wind turbine foundation. The structural damage classification problems had 5 different classes including the undamaged class and 4 different damage classes. The damage corresponded to a 5mm crack in four different structural elements of the wind turbine foundation. The results indicate the good behavior of the damage classification methodology supported by the high values of the classification metrics obtained.

Generating extremely low-dimensional representation of subsurface earth models using vector quantization and deep Autoencoder

Geological model compression is crucial for making large and complex models more manageable. By reducing the size of these models, compression techniques enable efficient storage, enhance computational efficiency, making it feasible to perform complex simulations and analyses in a shorter time. This is particularly important in applications such as reservoir management, groundwater hydrology, and geological carbon storage, where large geomodels with millions of grid cells are common. This study presents a comprehensive overview of previous work on geomodel compression and introduces several autoencoder-based deep-learning architectures for low-dimensional representation of modified Brugge-field geomodels. The compression and reconstruction efficiencies of autoencoders (AE), variational autoencoders (VAE), vector-quantized variational autoencoders (VQ-VAE), and vector-quantized variational autoencoders 2 (VQ-VAE2) were tested and compared to the traditional singular value decomposition (SVD) method. Results show that the deep-learning-based approaches significantly outperform SVD, achieving higher compression ratios while maintaining or even exceeding the reconstruction quality. Notably, VQ-VAE2 achieves the highest compression ratio of 667:1 with a structural similarity index metric (SSIM) of 0.92, far surpassing the 10:1 compression ratio of SVD with a SSIM of 0.9. The result of this work shows that, unlike traditional approaches, which often rely on linear transformations and can struggle to capture complex, non-linear relationships within geological data, VQ-VAE's use of vector quantization helps in preserving high-resolution details and enhances the model's ability to generalize across varying geological complexities.

Reduced‐Order Modeling for Linearized Representations of Microphysical Process Rates

AbstractRepresenting cloud microphysical processes in large scale atmospheric models is challenging because many processes depend on the details of the droplet size distribution (DSD, the spectrum of droplets with different sizes in a cloud). While full or partial statistical moments of droplet size distributions are the typical variables used in bulk models, prognostic moments are limited in their ability to represent microphysical processes across the range of conditions experienced in the atmosphere. Microphysical parameterizations employing prognostic moments are known to suffer from structural uncertainty in their representations of inherently higher dimensional cloud processes, which limit model fidelity and lead to forecasting errors. Here we investigate how data‐driven reduced‐order modeling can be used to learn predictors for microphysical process rates in bulk microphysics schemes in an unsupervised manner from higher dimensional bin distributions. Using simulations characteristic of marine stratiform clouds, we simultaneously learn lower dimensional representations of droplet size distributions and predict the evolution of the microphysical state of the system. Droplet collision‐coalescence, the main process for generating warm rain, is estimated to have an intrinsic dimension of three. This intrinsic dimension provides a lower limit on the number of degrees of freedom needed to accurately represent collision‐coalescence in models. We demonstrate how deep learning based reduced‐order modeling can be used to discover intrinsic coordinates describing the microphysical state of the system, where process rates such as collision‐coalescence are globally linearized. These implicitly learned representations of the DSD retain more information about the DSD than typical moment‐based representations.

ST-CellSeg: Cell segmentation for imaging-based spatial transcriptomics using multi-scale manifold learning.

Spatial transcriptomics has gained popularity over the past decade due to its ability to evaluate transcriptome data while preserving spatial information. Cell segmentation is a crucial step in spatial transcriptomic analysis, as it enables the avoidance of unpredictable tissue disentanglement steps. Although high-quality cell segmentation algorithms can aid in the extraction of valuable data, traditional methods are frequently non-spatial, do not account for spatial information efficiently, and perform poorly when confronted with the problem of spatial transcriptome cell segmentation with varying shapes. In this study, we propose ST-CellSeg, an image-based machine learning method for spatial transcriptomics that uses manifold for cell segmentation and is novel in its consideration of multi-scale information. We first construct a fully connected graph which acts as a spatial transcriptomic manifold. Using multi-scale data, we then determine the low-dimensional spatial probability distribution representation for cell segmentation. Using the adjusted Rand index (ARI), normalized mutual information (NMI), and Silhouette coefficient (SC) as model performance measures, the proposed algorithm significantly outperforms baseline models in selected datasets and is efficient in computational complexity.

A Stochastic Landscape Approach for Protein Folding State Classification.

Protein folding is a critical process that determines the functional state of proteins. Proper folding is essential for proteins to acquire their functional three-dimensional structures and execute their biological role, whereas misfolded proteins can lead to various diseases, including neurodegenerative disorders like Alzheimer's and Parkinson's. Therefore, a deeper understanding of protein folding is vital for understanding disease mechanisms and developing therapeutic strategies. This study introduces the Stochastic Landscape Classification (SLC), an innovative, automated, nonlearning algorithm that quantitatively analyzes protein folding dynamics. Focusing on collective variables (CVs) - low-dimensional representations of complex dynamical systems like molecular dynamics (MD) of macromolecules - the SLC approach segments the CVs into distinct macrostates, revealing the protein folding pathway explored by MD simulations. The segmentation is achieved by analyzing changes in CV trends and clustering these segments using a standard density-based spatial clustering of applications with noise (DBSCAN) scheme. Applied to the MD-based CV trajectories of Chignolin and Trp-Cage proteins, the SLC demonstrates apposite accuracy, validated by comparing standard classification metrics against ground-truth data. These metrics affirm the efficacy of the SLC in capturing intricate protein dynamics and offer a method to evaluate and select the most informative CVs. The practical application of this technique lies in its ability to provide a detailed, quantitative description of protein folding processes, with significant implications for understanding and manipulating protein behavior in industrial and pharmaceutical contexts.

Improved understanding of calibration efficiency, difficulty and parameter uniqueness of conceptual rainfall runoff models using fitness landscape metrics

The ease and efficiency with which conceptual rainfall runoff (CRR) models can be calibrated, as well as issues related to the uniqueness of their parameters, has received significant attention in literature. While several studies have tried to gain a better understanding of the underlying factors affecting these issues by examining the features of model error surfaces, this has generally been done in an ad-hoc fashion using lower-dimensional representations of higher-dimensional surfaces. In this paper, it is suggested that exploratory landscape analysis (ELA) metrics can be used to quantify key features of the error surfaces of CRR models, including their roughness and flatness, as well as their degree of optima dispersion throughout the surface. This enables key error surface features of CRR models to be compared in a consistent, efficient and easily communicable fashion for models with different combinations of attributes (e.g. model structure, catchment climate conditions, error metrics, and calibration data set lengths). Results from the application of ELA metrics to the error surfaces of 420 CRR models with different combinations of the above attributes indicate that increasing model complexity results in an increase in relative error surface roughness and relative optima dispersion and that, while increasing catchment wetness increases the relative roughness of error surfaces, it also decreases optima dispersion. This suggests that for the models considered in this study, optimisation efficiency is likely to decrease with increasing model complexity and catchment wetness, while optimisation difficulty is likely to increase and parameter uniqueness likely to decrease with model complexity and catchment dryness. While implications for choice of model complexity will need further work, this study highlights the potential value of the proposed approach to understanding the calibration efficiency, difficulty and parameter uniqueness of conceptual rainfall runoff models.