Synthetic Ground Truth Research Articles

Stimulation artefact removal: review and evaluation of applications in evoked responses

Objective.This study investigated software methods for removing stimulation artefacts in recordings undertaken during deep brain stimulation (DBS). We aimed to evaluate artefact attenuation using sample recordings of evoked resonant neural activity (ERNA), as well as a synthetic ground-truth waveform that emulated observed ERNA characteristics.Approach.The synthetic waveform and eight raw DBS recordings were processed by fourteen algorithms spanning the following categories: signal modification, signal decomposition, and template subtraction. For the synthetic waveform, performance was quantified by comparing each reconstructed signal against the ground-truth waveform. For DBS recordings, performance was contrasted amongst each other. The stimulation artefact was quantified by its amplitude and subsequent decay to baseline by the time to first zero-crossing. Each reconstructed ERNA signal was characterised by peak-to-peak-amplitude, root-mean-square amplitude, latency, and number of zero-crossings.Main results.None of the methods performed overall as well as the Backward Filter. Signal decomposition techniques were able to attenuate stimulation artefact albeit with unacceptable ERNA distortion.Significance.Upon evaluation of common software methods for DBS artefact attenuation, we advocate the use of the Backward Filter for reducing such artefacts while reconstructing ERNA.

Investigating the relative accuracy of GPS, GSM and CDR data for inferring spatiotemporal travel trajectories

AbstractThe potential of passively generated big data sources in transport modelling is well‐recognised. However, assessing their accuracy and suitability for policymaking remains challenging due to the lack of ground‐truth (GT) data for validation. This study evaluates the accuracy of inferring human mobility patterns from global positioning system (GPS), call detail records (CDR), and global system for mobile communication (GSM) data. Using outputs from an agent‐based simulation platform (MATSim) as ‘synthetic GT’ (SGT), synthetic GPS, CDR, and GSM data were generated, considering their positional disturbances and conventional spatiotemporal resolutions. Mobility information, including activity location, departure time, and trajectory distance, derived from the synthetic data, was compared with SGT to evaluate the accuracy of passive trajectory data at both disaggregate and aggregate levels. The results indicated a higher accuracy of GPS data in identifying stay locations at high resolution. But, GSM data at a lower resolution effectively accounted for over 80% of the variability in stay locations. Comparisons of departure time distribution and travel distance revealed higher measurement errors in GSM and CDR data than in GPS data. The proposed simulation‐based accuracy assessment framework will aid transport planners select the most suitable data for specific analyses and understand the potential margin of error involved.

LUIE: Learnable physical model-guided underwater image enhancement with bi-directional unsupervised domain adaptation

Recently, learning-based underwater enhancement (UIE) methods have made considerable progress, significantly benefiting downstream tasks such as underwater semantic segmentation and underwater depth estimation. Most existing unsupervised UIE methods utilize the atmospheric image formation model to decompose underwater images into background color, transmission map, and scene radiance. However, they rely on simplified physical models for estimating the transmission map, over-simplifying its complex formation, which results in imprecise modeling of underwater scattering effects. Additionally, supervised UIE methods heavily depend on synthetic data or ground truth, leading to limited generalization capabilities due to the substantial domain gap presented in different underwater scenarios. To tackle these challenges, we propose a Learnable physical model-guided unsupervised domain adaptation framework for Underwater Image Enhancement, dubbed LUIE. LUIE learns to predict background light, depth, and scene radiance from an underwater image. We incorporate a learnable network to estimate the transmission map based on the predicted depth map. To minimize the inter-domain gap between synthetic and real underwater images, we introduce a bi-directional domain adaptation method that alternates the background light from each domain. Experimental results demonstrate the effectiveness of our proposed method compared to existing approaches, and high-level experiment results validate that our enhanced underwater results. Experiments in real-world settings on underwater ROVs platform with NVIDIA Jetson AGX Xavier further confirm the effectiveness and efficiency of our work.

Ensemble learning and ground-truth validation of synaptic connectivity inferred from spike trains.

Probing the architecture of neuronal circuits and the principles that underlie their functional organization remains an important challenge of modern neurosciences. This holds true, in particular, for the inference of neuronal connectivity from large-scale extracellular recordings. Despite the popularity of this approach and a number of elaborate methods to reconstruct networks, the degree to which synaptic connections can be reconstructed from spike-train recordings alone remains controversial. Here, we provide a framework to probe and compare connectivity inference algorithms, using a combination of synthetic ground-truth and in vitro data sets, where the connectivity labels were obtained from simultaneous high-density microelectrode array (HD-MEA) and patch-clamp recordings. We find that reconstruction performance critically depends on the regularity of the recorded spontaneous activity, i.e., their dynamical regime, the type of connectivity, and the amount of available spike-train data. We therefore introduce an ensemble artificial neural network (eANN) to improve connectivity inference. We train the eANN on the validated outputs of six established inference algorithms and show how it improves network reconstruction accuracy and robustness. Overall, the eANN demonstrated strong performance across different dynamical regimes, worked well on smaller datasets, and improved the detection of synaptic connectivity, especially inhibitory connections. Results indicated that the eANN also improved the topological characterization of neuronal networks. The presented methodology contributes to advancing the performance of inference algorithms and facilitates our understanding of how neuronal activity relates to synaptic connectivity.

Generation and applications of synthetic computed tomography images for neurosurgical planning.

CT and MRI are synergistic in the information provided for neurosurgical planning. While obtaining both types of images lends unique data from each, doing so adds to cost and exposes patients to additional ionizing radiation after MRI has been performed. Cross-modal synthesis of high-resolution CT images from MRI sequences offers an appealing solution. The authors therefore sought to develop a deep learning conditional generative adversarial network (cGAN) which performs this synthesis. Preoperative paired CT and contrast-enhanced MR images were collected for patients with meningioma, pituitary tumor, vestibular schwannoma, and cerebrovascular disease. CT and MR images were denoised, field corrected, and coregistered. MR images were fed to a cGAN that exported a "synthetic" CT scan. The accuracy of synthetic CT images was assessed objectively using the quantitative similarity metrics as well as by clinical features such as sella and internal auditory canal (IAC) dimensions and mastoid/clinoid/sphenoid aeration. A total of 92,981 paired CT/MR images obtained in 80 patients were used for training/testing, and 10,068 paired images from 10 patients were used for external validation. Synthetic CT images reconstructed the bony skull base and convexity with relatively high accuracy. Measurements of the sella and IAC showed a median relative error between synthetic CT scans and ground truth images of 6%, with greater variability in IAC reconstruction compared with the sella. Aerations in the mastoid, clinoid, and sphenoid regions were generally captured, although there was heterogeneity in finer air cell septations. Performance varied based on pathology studied, with the highest limitation observed in evaluating meningiomas with intratumoral calcifications or calvarial invasion. The generation of high-resolution CT scans from MR images through cGAN offers promise for a wide range of applications in cranial and spinal neurosurgery, especially as an adjunct for preoperative evaluation. Optimizing cGAN performance on specific anatomical regions may increase its clinical viability.

Anomaly detection in Smart Traffic Light system using blockchain: Securing through proof of stake and machine learning

<p>The Smart Traffic Light system plays a crucial role in smart cities, contributing significantly to enhancing the overall urban living standards, supporting sustainable practices, assuring public safety, and optimizing operational efficiency. Smart lighting systems leverage advanced technology such as the Internet of Things (IoT), data analytics, and networking to construct a complex and flexible lighting infrastructure. Anomalies within the context of a Smart Light system’s data pertain to patterns, events, or behaviors that are unexpected or uncommon, displaying a considerable deviation from the system’s typical operational state. The distributed and tamper-resistant ledger of blockchain technology renders it well-suited for maintaining an open and immutable record of data and events pertaining to smart lighting systems. The integration of blockchain technology and anomaly detection techniques enables the establishment of a resilient and reliable smart lighting system. The primary objective of this study is to investigate the application of the Isolation Forest algorithm for anomaly detection and its enhancement through the implementation of Proof of Stake for enhanced security measures. The Isolation Forest algorithm is utilized for anomaly prediction and evaluation of accuracy and precision. This is achieved by employing synthetic ground truth labels. Subsequently, the non-anomalous data is incorporated into the system as blocks using blockchain technology. The experimentation involves the use of synthetic numerical data derived from a dataset accessible on Kaggle. This process is conducted in two distinct phases. The first phase focuses on the analysis of synthetic numerical data, specifically targeting the identification of anomalies, prior to the implementation of blockchain technology. In the second phase, the same technique is applied to the synthetic numerical data following the integration of blockchain technology. Various machine learning (ML) models were utilized to analyze the data, resulting in improved accuracy from 48% to 94%, as evidenced by the validation of the obtained results. The empirical results indicate that the utilization of blockchain technology in data processing leads to an improvement in accuracy. Furthermore, the random forest algorithm exhibits robust performance when integrated with blockchain technology.</p>

Channel-wise attention enhanced and structural similarity constrained cycleGAN for effective synthetic CT generation from head and neck MRI images

BackgroundMagnetic resonance imaging (MRI) plays an increasingly important role in radiotherapy, enhancing the accuracy of target and organs at risk delineation, but the absence of electron density information limits its further clinical application. Therefore, the aim of this study is to develop and evaluate a novel unsupervised network (cycleSimulationGAN) for unpaired MR-to-CT synthesis.MethodsThe proposed cycleSimulationGAN in this work integrates contour consistency loss function and channel-wise attention mechanism to synthesize high-quality CT-like images. Specially, the proposed cycleSimulationGAN constrains the structural similarity between the synthetic and input images for better structural retention characteristics. Additionally, we propose to equip a novel channel-wise attention mechanism based on the traditional generator of GAN to enhance the feature representation capability of deep network and extract more effective features. The mean absolute error (MAE) of Hounsfield Units (HU), peak signal-to-noise ratio (PSNR), root-mean-square error (RMSE) and structural similarity index (SSIM) were calculated between synthetic CT (sCT) and ground truth (GT) CT images to quantify the overall sCT performance.ResultsOne hundred and sixty nasopharyngeal carcinoma (NPC) patients who underwent volumetric-modulated arc radiotherapy (VMAT) were enrolled in this study. The generated sCT of our method were more consistent with the GT compared with other methods in terms of visual inspection. The average MAE, RMSE, PSNR, and SSIM calculated over twenty patients were 61.88 ± 1.42, 116.85 ± 3.42, 36.23 ± 0.52 and 0.985 ± 0.002 for the proposed method. The four image quality assessment metrics were significantly improved by our approach compared to conventional cycleGAN, the proposed cycleSimulationGAN produces significantly better synthetic results except for SSIM in bone.ConclusionsWe developed a novel cycleSimulationGAN model that can effectively create sCT images, making them comparable to GT images, which could potentially benefit the MRI-based treatment planning.

Cycle Consistent Generative Motion Artifact Correction in Coronary Computed Tomography Angiography

In coronary computed tomography angiography (CCTA), motion artifacts due to heartbeats can obscure coronary artery diagnoses. In this study, we introduce a cycle-consistent adversarial-network-based method for motion artifact correction in CCTA. Our methodology involves extracting image patches and using style transfer for synthetic ground truth creation, followed by CycleGAN network training for motion compensation. We employ Dynamic Time Warping (DTW) to align extracted image patches along the artery centerline with their corresponding motion-free phase patches, ensuring matched pixel correspondences and similar anatomical features for accuracy in subsequent processing steps. Our quantitative analysis, using metrics like the Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD), demonstrates CycleGAN’s superior performance in reducing motion artifacts, with improvements in image quality and clarity. An observer study using a 5-point Likert scale further validates the reduction of motion artifacts and improved visibility of coronary arteries. Additionally, we present a quantitative analysis on clinical data, affirming the correction of motion artifacts through metric-based evaluations.

Cross2SynNet: cross-device-cross-modal synthesis of routine brain MRI sequences from CT with brain lesion.

CT and MR are often needed to determine the location and extent of brain lesions collectively to improve diagnosis. However, patients with acute brain diseases cannot complete the MRI examination within a short time. The aim of the study is to devise a cross-device and cross-modal medical image synthesis (MIS) method Cross2SynNet for synthesizing routine brain MRI sequences of T1WI, T2WI, FLAIR, and DWI from CT with stroke and brain tumors. For the retrospective study, the participants covered four different diseases of cerebral ischemic stroke (CIS-cohort), cerebral hemorrhage (CH-cohort), meningioma (M-cohort), glioma (G-cohort). The MIS model Cross2SynNet was established on the basic architecture of conditional generative adversarial network (CGAN), of which, the fully convolutional Transformer (FCT) module was adopted into generator to capture the short- and long-range dependencies between healthy and pathological tissues, and the edge loss function was to minimize the difference in gradient magnitude between synthetic image and ground truth. Three metrics of mean square error (MSE), peak signal-to-noise ratio (PSNR), and structure similarity index measure (SSIM) were used for evaluation. A total of 230 participants (mean patient age, 59.77years ± 13.63 [standard deviation]; 163 men [71%] and 67 women [29%]) were included, including CIS-cohort (95 participants between Dec 2019 and Feb 2022), CH-cohort (69 participants between Jan 2020 and Dec 2021), M-cohort (40 participants between Sep 2018 and Dec 2021), and G-cohort (26 participants between Sep 2019 and Dec 2021). The Cross2SynNet achieved averaged values of MSE = 0.008, PSNR = 21.728, and SSIM = 0.758 when synthesizing MRIs from CT, outperforming the CycleGAN, pix2pix, RegGAN, Pix2PixHD, and ResViT. The Cross2SynNet could synthesize the brain lesion on pseudo DWI even if the CT image did not exhibit clear signal in the acute ischemic stroke patients. Cross2SynNet could achieve routine brain MRI synthesis of T1WI, T2WI, FLAIR, and DWI from CT with promising performance given the brain lesion of stroke and brain tumor.

Oblique view imaging analysis of a hypothetical earth observation satellite enabled with metamaterial-based image sensors

This study presents the image quality estimates of a hypothetical earth observation satellite (EOS) equipped with metaxel-based image sensors for the oblique viewing angle of the pitch manoeuvre scenario. Oblique-view image estimation methods are proposed and demonstrated for the optical and detector images. Owing to the small detector period (p) of the metaxel supercells ( p = 1.32 μ m ), the optical system image is optics limited. For oblique angle of θ p = 15 ∘ , 30 ∘ , 45 ∘ , 60 ∘ , image quality degradation owing to the aberration remained minor and average signal to noise ratio (SNR) for nadir view was SNR = 64 because of the local field enhancement of the metaxels. However, changes in the viewing angle reduced the image quality. Correlation coefficient between synthetic ground truth and oblique viewing cases was corr = 0.833, 0.818, 0.793 and 0.745 for the viewing angles of interest listed above.

Efficient estimation of propagator anisotropy and non-Gaussianity in multishell diffusion MRI with micro-structure adaptive convolution kernels and dual Fourier integral transforms.

We seek to reformulate the so-called Propagator Anisotropy (PA) and Non-Gaussianity (NG), originally conceived for the Mean Apparent Propagator diffusion MRI (MAP-MRI), to the Micro-Structure adaptive convolution kernels and dual Fourier Integral Transforms (MiSFIT). These measures describe relevant normalized features of the Ensemble Average Propagator (EAP). First, the indices, which are defined as the EAP's dissimilarity from an isotropic (PA) or a Gaussian (NG) one, are analytically reformulated within the MiSFIT framework. Then a comparison between the resulting maps is drawn by means of a visual analysis, a quantitative assessment via numerical simulations, a test-retest study across the MICRA dataset (6 subjects scanned five times) and, finally, a computational time evaluation. Findings illustrate the visual similarity between the indices computed with either technique. Evaluation against synthetic ground truth data, however, demonstrates MiSFIT's improved accuracy. In addition, the test-retest study reveals MiSFIT's higher degree of reliability in most of white matter regions. Finally, the computational time evaluation shows MiSFIT's time reduction up to two orders of magnitude. Despite being a direct development on the MAP-MRI representation, the PA and the NG can be reliably and efficiently computed within MiSFIT's framework. This, together with the previous findings in the original MiSFIT's article, could mean the difference that definitely qualifies diffusion MRI to be incorporated into regular clinical settings.

Nonparametric Bayesian inference for meta-stable conformational dynamics

Analyses of structural dynamics of biomolecules hold great promise to deepen the understanding of and ability to construct complex molecular systems. To this end, both experimental and computational means are available, such as fluorescence quenching experiments or molecular dynamics simulations, respectively. We argue that while seemingly disparate, both fields of study have to deal with the same type of data about the same underlying phenomenon of conformational switching. Two central challenges typically arise in both contexts: (i) the amount of obtained data is large, and (ii) it is often unknown how many distinct molecular states underlie these data. In this study, we build on the established idea of Markov state modeling and propose a generative, Bayesian nonparametric hidden Markov state model that addresses these challenges. Utilizing hierarchical Dirichlet processes, we treat different meta-stable molecule conformations as distinct Markov states, the number of which we then do not have to set a priori. In contrast to existing approaches to both experimental as well as simulation data that are based on the same idea, we leverage a mean-field variational inference approach, enabling scalable inference on large amounts of data. Furthermore, we specify the model also for the important case of angular data, which however proves to be computationally intractable. Addressing this issue, we propose a computationally tractable approximation to the angular model. We demonstrate the method on synthetic ground truth data and apply it to known benchmark problems as well as electrophysiological experimental data from a conformation-switching ion channel to highlight its practical utility.

Underwater enhancement based on a self-learning strategy and attention mechanism for high-intensity regions

Images acquired during underwater activities suffer from environmental properties of the water, such as turbidity and light attenuation. These phenomena cause color distortion, blurring, and contrast reduction. In addition, irregular ambient light distribution causes color channel unbalance and regions with high-intensity pixels. Recent works related to underwater image enhancement, and based on deep learning approaches, tackle the lack of paired datasets generating synthetic ground-truth. In this paper, we present a self-supervised learning methodology for underwater image enhancement based on deep learning that requires no paired datasets. The proposed method estimates the degradation present in underwater images. Besides, an autoencoder reconstructs this image, and its output image is degraded using the estimated degradation information. Therefore, the strategy replaces the output image with the degraded version in the loss function during the training phase. This procedure misleads the neural network that learns to compensate the additional degradation. As a result, the reconstructed image is an enhanced version of the input image. Also, the algorithm presents an attention module to reduce high-intensity areas generated in enhanced images by color channel unbalances and outlier regions. Furthermore, the proposed methodology requires no ground-truth. Besides, only real underwater images were used to train the neural network, and the results indicate the effectiveness of the method in terms of color preservation, color cast reduction, and contrast improvement.

Color image distortion assessment based on synthetic ground truth recovery

This paper proposes a quantitative method to measure the color distortion that can occur in color images. There are two main types of color distortion, false color and decolorization. Traditionally, the demosaic process of converting bayer to RGB might results in color distortion, but up-to-date complex color filter array(CFA) can cause even more severe color distortion. Since the conventional method of measuring color distortion requires a reference image, it is difficult to measure color distortion in a situation where the reference image is not secured. We have developed a comprehensive method based on recovering the undistorted color components corresponding to ground truth. Our method uses a chart designed for this purpose and evaluates the color distortion based on this chart.

Seismic Volumetric Dip Estimation via Multichannel Deep Learning Model

Although there are plenty of approaches proposed for addressing seismic volumetric dip estimation, it still suffers from several limitations, for example, the expensive computation cost, the perturbations from sequence stratigraphic anomalies, and the difficulty for handling the complicated geologic structures. Recently, deep learning (DL) based models have been proposed for seismic dip estimation, which utilize seismic dips calculated by using the traditional methods as the training labels. Apparently, these DL based models can effectively improve the computational efficiency, however, it still subjects to the limitations of the traditional algorithms. We propose a multi-channel deep learning (MCDL) model for implementing seismic volumetric dip estimation, mainly including share module (SM), particular module (PM), and fused module (FM). First, we calculate seismic dips by using several traditional methods based on 3D real seismic data as the training labels, which are used to pre-train SM and PM. Then, we propose a workflow to create synthetic seismic data and ground truth dip labels, which are utilized to fine-tune SM/PM and train FM. In this way, we can obtain a DL model by considering both the features of synthetic ground truth dips and the calculated dips from real data. Moreover, we can effectively enhance the generalization ability of the MCDL by pre-training with the estimated dip volumes from real data. To demonstrate its validity and availability, we apply the MCDL to synthetic data and two 3D real seismic volumes. The qualitative and quantitative comparisons illustrate the superiority of the proposed model over the traditional methods.

A Superpixel-Guided Unsupervised Fast Semantic Segmentation Method of Remote Sensing Images

Semantic segmentation is one of the fundamental tasks of pixel-level remote sensing image analysis. Currently, most high-performance semantic segmentation methods are trained in a supervised learning manner. These methods require a large number of image labels as support, but manual annotations are difficult to obtain. To address the problem, we propose an efficient unsupervised remote sensing image segmentation method based on superpixel segmentation and fully convolutional networks (FCNs) in this letter. Our method can achieve pixel-level images segmentation of various scales rapidly without any manual labels or prior knowledge. We use the superpixel segmentation results as synthetic ground truth to guide the gradient descent direction during FCN training. In experiments, our method achieved high performance compared to current unsupervised image segmentation methods on three public datasets. Specifically, our method achieves an Adjusted Mutual Information (AMI) score of 0.2955 on the Gaofen Image Dataset (GID) dataset, while processing each image of size 7200 × 6800 pixels in just 30 seconds.

Motion trajectory estimation of salmon using stereo vision

A main concern for the aquaculture industry is the fish behaviour and welfare. Motion trajectory analysis of salmon at aquaculture farming sites with respect to certain aquaculture operations aims to provide information about the behaviour and possibly stress level of the farmed salmon and may help to generate a general welfare indicator index. Towards this aim we present an innovative computer vision and machine learning based approach for motion trajectory estimation of salmon. Video footage was recorded with a stereo camera setup. Deep learning based object detection was performed to detect particular features. We focused on tracking the fish eyes and heads as a reliable indicators of the fish's position. Feature matching and subsequent 3D reconstruction was performed to calculate the 3D position of the fish from which trajectories of the fish movement were estimated. Related experiments were conducted at an aquaculture research facility under natural lighting conditions and extracted trajectories allowed a qualitative verification. The developed method was verified using synthetic ground truth data produced with an open source computer graphics software for quantifiable performance metrics.

Automatic Detection of Aquatic Weeds: A Case Study in the Guadiana River, Spain

The spread of aquatic invasive plants is a major concern in several zones of the world's geography. These plants, which are not part of the natural ecosystem, cause a negative impact to the environment, as well as to economy and society. In Spain, large areas of Guadiana (the second longest river in Spain) have been invaded by such plants. Among the strategies to address this problem, monitoring and detection play an important role to control the spatio-temporal distribution of the invasive plants. The main objective of this work is to develop a methodology able to automatically detect the geo-location of aquatic invasive plants using remote sensing and machine learning techniques. To this end, several classification algorithms have been applied to freely available multispectral satellite imagery, collected by ESA's Sentinel-2 satellite. A quantitative and comparative assessment is conducted using different machine and deep learning algorithms, from classical methods such as unsupervised K-means to supervised random forests (RFs) and convolutional neural networks (CNNs). This study also proposes a methodology for validating the obtained classification results, generating synthetic ground truth images based on available high spatial resolution imagery. The obtained results demonstrate the suitability of some of the considered algorithms for automatic detection of aquatic weeds in satellite images with medium spatial resolution.

Deep learning-based single image face depth data enhancement

Face recognition can benefit from the utilization of depth data captured using low-cost cameras, in particular for presentation attack detection purposes. Depth video output from these capture devices can however contain defects such as holes or general depth inaccuracies. This work proposes a deep learning face depth enhancement method in this context of facial biometrics, which adds a security aspect to the topic. U-Net-like architectures are utilized, and the networks are compared against hand-crafted enhancer types, as well as a similar depth enhancer network from related work trained for an adjacent application scenario. All tested enhancer types exclusively use depth data as input, which differs from methods that enhance depth based on additional input data such as visible light color images. Synthetic face depth ground truth images and degraded forms thereof are created with help of PRNet, to train multiple deep learning enhancer models with different network sizes and training configurations. Evaluations are carried out on the synthetic data, on Kinect v1 images from the KinectFaceDB, and on in-house RealSense D435 images. These evaluations include an assessment of the falsification for occluded face depth input, which is relevant to biometric security. The proposed deep learning enhancers yield noticeably better results than the tested preexisting enhancers, without overly falsifying depth data when non-face input is provided, and are shown to reduce the error of a simple landmark-based PAD method.

Clustering assessment in weighted networks.

We provide a systematic approach to validate the results of clustering methods on weighted networks, in particular for the cases where the existence of a community structure is unknown. Our validation of clustering comprises a set of criteria for assessing their significance and stability. To test for cluster significance, we introduce a set of community scoring functions adapted to weighted networks, and systematically compare their values to those of a suitable null model. For this we propose a switching model to produce randomized graphs with weighted edges while maintaining the degree distribution constant. To test for cluster stability, we introduce a non parametric bootstrap method combined with similarity metrics derived from information theory and combinatorics. In order to assess the effectiveness of our clustering quality evaluation methods, we test them on synthetically generated weighted networks with a ground truth community structure of varying strength based on the stochastic block model construction. When applying the proposed methods to these synthetic ground truth networks’ clusters, as well as to other weighted networks with known community structure, these correctly identify the best performing algorithms, which suggests their adequacy for cases where the clustering structure is not known. We test our clustering validation methods on a varied collection of well known clustering algorithms applied to the synthetically generated networks and to several real world weighted networks. All our clustering validation methods are implemented in R, and will be released in the upcoming package clustAnalytics.