Agreement Metrics Research Articles

Randomized Trial Replication Using Observational Data for Comparative Effectiveness of Secukinumab and Ustekinumab in Psoriasis

Treatments for psoriasis may be less effective in everyday practice than in clinical trials. Emulating a target trial using data from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR) can provide treatment effect estimates that are robust and can inform both clinicians and regulatory bodies. To assess the comparative effectiveness of ustekinumab and secukinumab in patients with psoriasis, and to test whether the relative effectiveness estimate of the CLEAR trial, a randomized clinical trial that compared secukinumab with ustekinumab for psoriasis, can be replicated. This comparative effectiveness research study used a target trial emulation approach and was performed between November 2007 and August 2019. Data were obtained from BADBIR, a multicenter longitudinal pharmacovigilance register of patients with moderate to severe psoriasis in the United Kingdom and Republic of Ireland. Participants had chronic plaque psoriasis, were 18 years or older, and had at least 1 record of a Psoriasis Area and Severity Index (PASI) of 12 or higher before their initiation to secukinumab or ustekinumab. Propensity score (PS) 1:1 matched analysis and inverse probability treatment weighted analysis were performed. The primary outcomes were the risk ratio (RR) and the risk difference (RD) for achieving PASI of 2 or lower after 12 months of therapy for secukinumab compared with ustekinumab. Methods to account for missing outcome data were complete case analysis, nonresponder imputation, last observation carried forward, inverse probability of censoring weighting, and multiple imputation. Regulatory and estimate agreement metrics were used to benchmark the effect estimates in this study against those in the CLEAR trial. A total of 1231 patients were included in the analysis, with 917 receiving ustekinumab and 314 receiving secukinumab. Secukinumab was superior to ustekinumab in all analyses, except under the nonresponder imputation method, in the proportion of participants achieving a PASI of 2 or lower (PS-weighted complete case analysis: RR, 1.28 [95% CI, 1.06-1.55]; RD, 11.9% [1.6-22.1]). All analyses, except for nonresponder imputation, reached regulatory agreement in both PS-matching and PS-weighted analyses. This comparative effectiveness study found that secukinumab resulted in more patients achieving a PASI of 2 or lower after 12 months of therapy compared with ustekinumab in patients with psoriasis. Target trial emulation in this study resulted in regulatory and estimate agreement with the CLEAR randomized clinical trial; further such studies may help fill the evidence gap when comparing other systemic therapies for psoriasis.

Green Cloud Software Engineering for Big Data Processing

Internet of Things (IoT) coupled with big data analytics is emerging as the core of smart and sustainable systems which bolsters economic, environmental and social sustainability. Cloud-based data centers provide high performance computing power to analyze voluminous IoT data to provide invaluable insights to support decision making. However, multifarious servers in data centers appear to be the black hole of superfluous energy consumption that contributes to 23% of the global carbon dioxide (CO2) emissions in ICT (Information and Communication Technology) industry. IoT-related energy research focuses on low-power sensors and enhanced machine-to-machine communication performance. To date, cloud-based data centers still face energy–related challenges which are detrimental to the environment. Virtual machine (VM) consolidation is a well-known approach to affect energy-efficient cloud infrastructures. Although several research works demonstrate positive results for VM consolidation in simulated environments, there is a gap for investigations on real, physical cloud infrastructure for big data workloads. This research work addresses the gap of conducting real physical cloud infrastructure-based experiments. The primary goal of setting up a real physical cloud infrastructure is for the evaluation of dynamic VM consolidation approaches which include integrated algorithms from existing relevant research. An open source VM consolidation framework, Openstack NEAT is adopted and experiments are conducted on a Multi-node Openstack Cloud with Apache Spark as the big data platform. Open sourced Openstack has been deployed because it enables rapid innovation, and boosts scalability as well as resource utilization. Additionally, this research work investigates the performance based on service level agreement (SLA) metrics and energy usage of compute hosts. Relevant results concerning the best performing combination of algorithms are presented and discussed.

Central Reading of Ulcerative Colitis Clinical Trial Videos Using Neural Networks

Endoscopic disease activity scoring in ulcerative colitis (UC) is useful in clinical practice but done infrequently. It is required in clinical trials, where it is expensive and slow because human central readers are needed. A machine learning algorithm automating the process could elevate clinical care and facilitate clinical research. Prior work using single-institution databases and endoscopic still images has been promising. Seven hundred and ninety-five full-length endoscopy videos were prospectively collected from a phase 2 trial of mirikizumab with 249 patients from 14 countries, totaling 19.5 million image frames. Expert central readers assigned each full-length endoscopy videos 1 endoscopic Mayo score (eMS) and 1 Ulcerative Colitis Endoscopic Index of Severity (UCEIS) score. Initially, video data were cleaned and abnormality features extracted using convolutional neural networks. Subsequently, a recurrent neural network was trained on the features to predict eMS and UCEIS from individual full-length endoscopy videos. The primary metric to assess the performance of the recurrent neural network model was quadratic weighted kappa (QWK) comparing the agreement of the machine-read endoscopy score with the human central reader score. QWK progressively penalizes disagreements that exceed 1 level. The model's agreement metric was excellent, with a QWK of 0.844 (95% confidence interval, 0.787-0.901) for eMS and 0.855 (95% confidence interval, 0.80-0.91) for UCEIS. We found that a deep learning algorithm can be trained to predict levels of UC severity from full-length endoscopy videos. Our data set was prospectively collected in a multinational clinical trial, videos rather than still images were used, UCEIS and eMS were reported, and machine learning algorithm performance metrics met or exceeded those previously published for UC severity scores.

Agreement Study Using Gesture Description Analysis.

Choosing adequate gestures for touchless interfaces is a challenging task that has a direct impact on human-computer interaction. Such gestures are commonly determined by the designer, ad-hoc, rule-based or agreement-based methods. Previous approaches to assess agreement grouped the gestures into equivalence classes and ignored the integral properties that are shared between them. In this work, we propose a generalized framework that inherently incorporates the gesture descriptors into the agreement analysis (GDA). In contrast to previous approaches, we represent gestures using binary description vectors and allow them to be partially similar. In this context, we introduce a new metric referred to as Soft Agreement Rate ( ) to measure the level of agreement and provide a mathematical justification for this metric. Further, we performed computational experiments to study the behavior of and demonstrate that existing agreement metrics are a special case of our approach. Our method was evaluated and tested through a guessability study conducted with a group of neurosurgeons. Nevertheless, our formulation can be applied to any other user-elicitation study. Results show that the level of agreement obtained by is 2.64 times higher than the previous metrics. Finally, we show that our approach complements the existing agreement techniques by generating an artificial lexicon based on the most agreed properties.

Comparison performance of machine learning and geostatistical methods for the interpolation of monthly air temperature over Costa Rica

The performance of three machine learning (ML) methods; cubist regression (CR), random forest (RF) and generalized additive model using splines (GAM) in generating monthly air temperature grids over Costa Rica was evaluated against two heavily used geostatistical methods; ordinary kriging (OK) and kriging with external drift (KED). The skill of the interpolation methods was evaluated using a 10-fold cross-validation technique; selecting the root-mean square error (RMSE), the mean absolute error (MAE) and the Pearson correlation-coefficient (R) as agreement metrics. To this purpose, data from an irregularly-distributed observational-network comprised by 73 weather-stations were selected for the period 1950-1987. Several spatial fields derived from a high-resolution digital elevation model (DEM) were tested as covariants. Results from the 10-fold cross-validation test show that CR yielded the best individual performance followed by KED, whereas GAM performed worst. Elevation on the other hand, was the only covariant ultimately incorporated in the interpolation process, since the remaining spatial fields exhibited poor correlation with temperature or resulted in data redundancy. While the quantitative and qualitative evaluation of CR and KED can be said to be comparable, CR is considered the best approach since the method is unaffected by assumptions on data normality and homoscedasticity.

Generation of Monthly Precipitation Climatologies for Costa Rica Using Irregular Rain-Gauge Observational Networks

Precipitation climatologies for the period 1961–1990 were generated for all climatic regions of Costa Rica using an irregular rain-gauge observational network comprised by 416 rain-gauge stations. Two sub-networks were defined: a high temporal resolution sub-network (HTR), including stations having at least 20 years of continuous records during the study period (157 in total); and a high spatial resolution sub-network (HSR), which includes all HTR-stations plus those stations with less than 20 years of continuous records (416 in total). Results from the kriging variance reduction efficiency (KRE) objective function between the two sub-networks, show that ordinary kriging (OK) is unable to fully explain the spatio-temporal variability of precipitation within most climatic regions if only stations from the HTR sub-network are used. Results also suggests that in most cases, it is beneficial to increase the density of the rain-gauge observational network at the expense of temporal fidelity, by including more stations even though their records may not represent the same time step. Thereafter, precipitation climatologies were generated using seven deterministic (IDW, TS2, TS2PARA, TS2LINEAR, TPS, MQS and NN) and two geostatistical (OK and KED) interpolation methods. Performance of the various interpolation methods was evaluated using cross validation technique, selecting the mean absolute error (MAE) and the root-mean square error (RMSE) as agreement metrics. Results suggest that IDW is marginally superior to OK and KED for most climatic regions. The remaining deterministic methods however, considerably deviate from IDW, which suggests that these methods are incapable of properly capturing the true-nature of spatial precipitation patterns over the considered climatic regions. The final generated IDW climatology was then validated against the Global Precipitation Climatology Centre (GPCC), Climate Research Unit (CRU) and WorldClim datasets, in which overall spatial and temporal coherence is considered satisfactory, giving assurance about the use this new climatology in the development of local climate impact studies.

Gestures for Picture Archiving and Communication Systems (PACS) operation in the operating room: Is there any standard?

ObjectiveGestural interfaces allow accessing and manipulating Electronic Medical Records (EMR) in hospitals while keeping a complete sterile environment. Particularly, in the Operating Room (OR), these interfaces enable surgeons to browse Picture Archiving and Communication System (PACS) without the need of delegating functions to the surgical staff. Existing gesture based medical interfaces rely on a suboptimal and an arbitrary small set of gestures that are mapped to a few commands available in PACS software. The objective of this work is to discuss a method to determine the most suitable set of gestures based on surgeon’s acceptability. To achieve this goal, the paper introduces two key innovations: (a) a novel methodology to incorporate gestures’ semantic properties into the agreement analysis, and (b) a new agreement metric to determine the most suitable gesture set for a PACS.Materials and methodsThree neurosurgical diagnostic tasks were conducted by nine neurosurgeons. The set of commands and gesture lexicons were determined using a Wizard of Oz paradigm. The gestures were decomposed into a set of 55 semantic properties based on the motion trajectory, orientation and pose of the surgeons’ hands and their ground truth values were manually annotated. Finally, a new agreement metric was developed, using the known Jaccard similarity to measure consensus between users over a gesture set.ResultsA set of 34 PACS commands were found to be a sufficient number of actions for PACS manipulation. In addition, it was found that there is a level of agreement of 0.29 among the surgeons over the gestures found. Two statistical tests including paired t-test and Mann Whitney Wilcoxon test were conducted between the proposed metric and the traditional agreement metric. It was found that the agreement values computed using the former metric are significantly higher (p < 0.001) for both tests.ConclusionsThis study reveals that the level of agreement among surgeons over the best gestures for PACS operation is higher than the previously reported metric (0.29 vs 0.13). This observation is based on the fact that the agreement focuses on main features of the gestures rather than the gestures themselves. The level of agreement is not very high, yet indicates a majority preference, and is better than using gestures based on authoritarian or arbitrary approaches. The methods described in this paper provide a guiding framework for the design of future gesture based PACS systems for the OR.

Comparing Traffic Speed Deflectometer and Falling Weight Deflectometer Data

In recent years the pavement engineering community has shown increasing interest in shifting from a stationary falling weight deflectometer (FWD) to moving testing platforms such as the traffic speed deflectometer (TSD). This paper dealt with comparing TSD measurements against FWD measurements; it focused on the comparison methodology, utilizing experimental data for demonstration. To better account for differences in loading conditions between the two devices a new FWD deflection index was formulated first. This index served as reference/benchmark for assessing the corresponding TSD measurements. Next, a Taylor diagram was proposed for visualizing several comparison statistics. Finally, a modern agreement metric was identified and applied for ranking comparison results across different datasets. Overall, the suggested methodology is deemed generic and highly applicable to future situations, especially for assessing the worth of emerging device upgrades or improved interpretation schemes (or both).

Benefit of using motion compensated reconstructions for reducing inter-observer and intra-observer contouring variation for organs at risk in lung cancer patients

Background and purposeIn lung cancer patients, accuracy in contouring is hampered by image artefacts introduced by respiratory motion. With the widespread introduction of 4DCT there is additional uncertainty caused by the use of different reconstruction techniques which will influence contour definition. This work aims to assess both inter- and intra-observer contour variation on average and motion compensated (mid-position) reconstructions. Material and methodsEight early stage non-small cell lung cancer patients that received 4DCT were selected and these scans were reconstructed as average and motion compensated datasets. 5 observers contoured the organs at risk (trachea, oesophagus, proximal bronchial tree, heart and brachial plexus) for each patient and each reconstruction. Contours were compared against a STAPLE volume with distance to agreement metrics. Intra-observer variation was assessed by redelineation after 4 months. ResultsThe inter-observer variation was significantly smaller using the motion compensated datasets for the trachea (p = 0.006) and proximal bronchial tree (p = 0.004). For intra-observer variation, a reduction in contour variation was seen across all organs at risk in using motion compensated reconstructions. ConclusionsThis work shows that there is benefit in using motion compensated reconstructions for reducing both inter-observer and intra-observer contouring variations for organs at risk in lung cancer patients.

The interplay of various sources of noise on reliability of species distribution models hinges on ecological specialisation.

Digitized species occurrence data provide an unprecedented source of information for ecologists and conservationists. Species distribution model (SDM) has become a popular method to utilise these data for understanding the spatial and temporal distribution of species, and for modelling biodiversity patterns. Our objective is to study the impact of noise in species occurrence data (namely sample size and positional accuracy) on the performance and reliability of SDM, considering the multiplicative impact of SDM algorithms, species specialisation, and grid resolution. We created a set of four ‘virtual’ species characterized by different specialisation levels. For each of these species, we built the suitable habitat models using five algorithms at two grid resolutions, with varying sample sizes and different levels of positional accuracy. We assessed the performance and reliability of the SDM according to classic model evaluation metrics (Area Under the Curve and True Skill Statistic) and model agreement metrics (Overall Concordance Correlation Coefficient and geographic niche overlap) respectively. Our study revealed that species specialisation had by far the most dominant impact on the SDM. In contrast to previous studies, we found that for widespread species, low sample size and low positional accuracy were acceptable, and useful distribution ranges could be predicted with as few as 10 species occurrences. Range predictions for narrow-ranged species, however, were sensitive to sample size and positional accuracy, such that useful distribution ranges required at least 20 species occurrences. Against expectations, the MAXENT algorithm poorly predicted the distribution of specialist species at low sample size.

A Simulation-based Approach to Measuring Team Situational Awareness in Emergency Medicine: A Multicenter, Observational Study.

Team situational awareness (TSA) is critical for effective teamwork and supports dynamic decision making in unpredictable, time-pressured situations. Simulation provides a platform for developing and assessing TSA, but these efforts are limited by suboptimal measurement approaches. The objective of this study was to develop and evaluate a novel approach to TSA measurement in interprofessional emergency medicine (EM) teams. We performed a multicenter, prospective, simulation-based observational study to evaluate an approach to TSA measurement. Interprofessional emergency medical teams, consisting of EM resident physicians, nurses, and medical students, were recruited from the University of Washington (Seattle, WA) and Wayne State University (Detroit, MI). Each team completed a simulated emergency resuscitation scenario. Immediately following the simulation, team members completed a TSA measure, a team perception of shared understanding measure, and a team leader effectiveness measure. Subject matter expert reviews and pilot testing of the TSA measure provided evidence of content and response process validity. Simulations were recorded and independently coded for team performance using a previously validated measure. The relationships between the TSA measure and other variables (team clinical performance, team perception of shared understanding, team leader effectiveness, and team experience) were explored. The TSA agreement metric was indexed by averaging the pairwise agreement for each dyad on a team and then averaging across dyads to yield agreement at the team level. For the team perception of shared understanding and team leadership effectiveness measures, individual team member scores were aggregated within a team to create a single team score. We computed descriptive statistics for all outcomes. We calculated Pearson's product-moment correlations to determine bivariate correlations between outcome variables with two-tailed significance testing (p < 0.05). A total of 123 participants were recruited and formed three-person teams (n = 41 teams). All teams completed the assessment scenario and postsimulation measures. TSA agreement ranged from 0.19 to 0.9 and had a mean (±SD) of 0.61 (±0.17). TSA correlated with team clinical performance (p < 0.05) but did not correlate with team perception of shared understanding, team leader effectiveness, or team experience. Team situational awareness supports adaptive teams and is critical for high reliability organizations such as healthcare systems. Simulation can provide a platform for research aimed at understanding and measuring TSA. This study provides a feasible method for simulation-based assessment of TSA in interdisciplinary teams that addresses prior measure limitations and is appropriate for use in highly dynamic, uncertain situations commonly encountered in emergency department systems. Future research is needed to understand the development of and interactions between individual-, team-, and system (distributed)-level cognitive processes.

Abstract 428: Supporting neoantigen identification for personalized cancer vaccines through analytical validation of an augmented content enhanced (ACE) exome

Abstract The identification of neoantigens has become a critical step in the development of neoantigen-based personalized cancer vaccines and other immunotherapy applications. Since neoantigens can be generated from tumor specific mutations in any expressed gene, the first step in identification of neoantigens typically involves deep exome and transcriptome sequencing on the tumor and exome sequencing of the matched normal. As personalized vaccines enter clinical trials with the potential for clinical use, there is a growing need for strong analytical validation of these platforms. To address this we have developed our ACE Exome (~200X) and Transcriptome platforms for neoantigen identification which utilitize an augmented exome approach designed to increase sensitivity for neoantigens in low complexity, traditionally hard to sequencing regions. To enable this platform for neoantigen based personalized cancer vaccines, we have performed a validation of both our ACE Exome (tumor and normal) and ACE transcriptome platforms for detecting DNA-based SNVs and Indels, as well as for RNA based small variant and fusion calls. These are variant types are especially important for neoantigen identification. In this abstract we describe the ACE Exome validation. We used 11 cancer cell lines and their matched normals to assess analytical sensitivity and limits of detection (LOD) for small variant (SNV and Indel) detection using our ACE exome and Tumor Normal bioinformatics pipeline. We identified a gold set of variants, 875 SNVs and 19 Indels that were previously validated in these 11 cell lines (COSMIC, CCLE and Sanger Sequencing confirmed variants). These gold set variants were used to calculate our analytical sensitivity (percent of gold variants detected across the 11 cell line pairs using our assay). To determine our LOD, we chose 3 of the 11 cancer cell lines and created 6 dilutions (5%, 10%, 20%, 30%, 50% and 80% tumor purity) with their matched normal. We then determined Positive Predictive Agreement (PPA, percent of pure cell line variants detected in a diluted samples) and False Discovery Rate (FDR, percent of erroneously detected variants in the diluted sample that were not detected in the pure cell lines) metrics for variants across different minor allelic frequencies (MAF) in the diluted samples. The ACE “Tumor Normal” Exome assay had a high sensitivity of 98% for SNVs and 95% for Indels. The assay also showed robust PPA (sensitivity) of 97% and FDR (specificity) of 2% for SNVs with MAF &amp;gt;= 10% and PPA of 87% and FDR of 3% for Indels with MAF &amp;gt;= 10%. We demonstrate that the ACE “Tumor Normal” Exome assay is highly accurate for identification of SNVs and Indels in cancer exomes. With high analytical sensitivity, PPA and low FDR we believe this assay provides augmented ability to detect cancer driver and potential neoantigen generating mutations across various tumor types. Citation Format: Ravi K. Alla, Jennifer Yen, Sean M. Boyle, Richard Chen. Supporting neoantigen identification for personalized cancer vaccines through analytical validation of an augmented content enhanced (ACE) exome [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 428. doi:10.1158/1538-7445.AM2017-428

An evergreen cloud: Optimizing energy efficiency in heterogeneous cloud computing architectures

In a heterogeneous Cloud network scenario where a Cloud computing data center serves mobile Cloud computing requests, Cloud providers are expected to implement more innovative and effective solutions for a list of long standing challenges. Energy efficiency in the Cloud data center is one of the more pressing issues near the top of that list. Cloud providers are in constant pursuit of a system that satisfies client demands for resources, maximizes availability and other service level agreement metrics while minimizing energy consumption and, in turn, minimizing Cloud providers' cost.In this work, we introduce a novel mathematical optimization model to solve the problem of energy efficiency in a cloud data center. Next, we offer a solution based on VM migration that tackles this problem and minimizes energy efficiency in comparison to other common solutions. This solution includes a novel proposed technique to be integrated in any consolidation-based energy efficiency solution. This technique depends on dynamic idleness prediction (DIP) using machine learning classifiers. Moreover, we offer a robust energy efficiency scheduling solution that does not depend on live migration. This technique, termed Smart VM Over Provision (SVOP), offers a major advantage to cloud providers in the cases when live migration of VMs is not preferred due to its effects on performance. We evaluate the aforementioned solutions in terms of a number of critical metrics, namely, energy used per server, energy used per served request, acceptance rate, and the number of migrations performed.

Observer performance in characterization of carotid plaque texture and surface characteristics with 3D versus 2D ultrasound

ObjectiveTo determine the reproducibility of three-dimensional (3D) ultrasound (US) over two-dimensional (2D) US in characterizing atherosclerotic carotid plaques using inter- and intra-observer agreement metrics. MethodsA Total of 51 patients with 105 carotid artery plaques were screened using 3D and 2D US probes attached to the same US scanner. Two independent observers characterized the plaques based on the morphological features namely echotexture, echogenicity and surface characteristics. The scores assigned to each morphological feature were used to determine intra- and inter-observer performance. The level of agreement was measured using Kappa coefficient. ResultsThe first observer with 2D US showed fair (k=0.4–0.59) and very strong (k>0.8) with 3D US intra-observer agreements using three morphological features. The second observer indicated moderate strong (k=0.6–0.79) with 2D US and very strong with 3D US (k>0.8) intra-observer performances. Moderate strong (k=0.6–0.79) and very strong (k>0.8) inter-observer agreements were reported with 2D US and 3D US respectively. The results with 2D and 3D US were correlated 62% using only echotexture and 56% using surface morphology coupled with echogenicity. 3D US gave a lower score than 2D 71% of the time (p=0.005) in disagreement cases. ConclusionHigh reproducibility in carotid plaque characterization was obtained using 3D US rather than 2D US. Hence, it can be a preferred imaging modality in routine or follow up plaque screening of patients with carotid artery disease.

Quantifying uncertainty in crop model predictions due to the uncertainty in the observations used for calibration

Despite modellers are paying increasing attention to analyse and manage the different sources of uncertainty affecting model predictions, the impact of the uncertainty in the observations used for calibration has been ignored. This study proposes a methodology for its quantification and provides an illustrative case study with data collected in two field experiments where rice was grown under flooded conditions in northern Italy in 2002 and 2004. Latin hypercube sampling was used to generate virtual series of observations from the mean and standard deviation of aboveground biomass values collected during the season in the two experiments. Each of the generated series was then used to calibrate the parameters maximum radiation use efficiency and optimum temperature for growth of the WARM model by means of the simplex optimization algorithm. The analysis of the distribution of key outputs (aboveground and panicle biomass at harvest) and of agreement metrics revealed that the impact of uncertainty in the observations used for calibration (explored here running calibration experiments for each of the generated series) can be large. The difference between maximum and minimum aboveground biomass at maturity was 2.79tha−1 and 3.78tha−1 for the datasets collected in 2004 and 2002, respectively. Corresponding values for panicle biomass were 0.97tha−1 and 2.36tha−1. In all cases, model outputs were normally distributed. Large differences were achieved also in the values of the agreement metrics, with RRMSE ranging from 13.64% to 36.22% and from 8.04% to 29.97% for the 2004 and 2002 datasets. The methodology proposed – although applicable to a variety of models and domains – deals only with the uncertainty due to random errors, which could derive, e.g. from non-representative sampling or from the repeatability of the method used to determine the variable of interest. Other sources of uncertainty, like those involved with systematic errors, need to be addressed in further studies. This study highlighted the need for conceptual and mathematical frameworks where the different sources of uncertainty affecting model predictions can be analysed in an integrated way.

Experiencing the interestingness concept within and between pictures

Interestingness is the quantification of the ability of an image to induce interest in a user. Because defining and interpreting interestingness remain unclear in the literature, we introduce in this paper two new notions, intra- and inter-interestingness, and investigate a novel set of dedicated experiments. More specifically, we propose four experimental protocols: 1/ object ranking with a pre-defined word list, 2/ pair-wise comparison, 3/ image ranking and 4/ eye-tracking. We take advantage of experimenting on the same dataset to draw potential links between the collected data and to state on the agreement between subjects. While we do not evidence a relationship between the local (intra) and global (inter) notions of interestingness, we do observe correlated outputs throughout the different protocols. Beyond the low or moderate values obtained from inter-rater agreement metrics, we point out the experimental reproducibility to argue about the universal nature of the interestingness notions. In addition, we bring deep insights on the relationships between interestingness and 7 other criteria, some of them already pointed out in the literature as being linked with interestingness. Unusualness and emotion seem to be the strongest enablers for interestingness. These insights are highly relevant for future work on modeling.

QoS Design Consideration for Enterprise and Provider’s Network at Ingress and Egress Router for VoIP protocols

Compliance with the Service Level Agreement (SLA) metric is a major challenge in a Multiprotocol Label Switching Virtual Private Network (MPLS VPN) because mandatory models must be maintained on both sides of the MPLS VPN in order to achieve end-to-end service levels. The end-to-end service of an MPLS VPN can be degraded owing to various issues such as distributed denial of service (DDoS), and Random Early Detection (RED) that prevents congestion and differentiates between legitimate and illegitimate user traffic. In this study, we propose a centralized solution that uses a SLA Violation Detector (SLAVD) and intrusion detection to prevent SLA violation.

Fusion set selection with surrogate metric in multi-atlas based image segmentation

Multi-atlas based image segmentation sees unprecedented opportunities but also demanding challenges in the big data era. Relevant atlas selection before label fusion plays a crucial role in reducing potential performance loss from heterogeneous data quality and high computation cost from extensive data. This paper starts with investigating the image similarity metric (termed ‘surrogate’), an alternative to the inaccessible geometric agreement metric (termed ‘oracle’) in atlas relevance assessment, and probes into the problem of how to select the ‘most-relevant’ atlases and how many such atlases to incorporate. We propose an inference model to relate the surrogates and the oracle geometric agreement metrics. Based on this model, we quantify the behavior of the surrogates in mimicking oracle metrics for atlas relevance ordering. Finally, analytical insights on the choice of fusion set size are presented from a probabilistic perspective, with the integrated goal of including the most relevant atlases and excluding the irrelevant ones. Empirical evidence and performance assessment are provided based on prostate and corpus callosum segmentation.

Selvester scoring in patients with strict LBBB using the QUARESS software

BackgroundEstimation of the infarct size from body-surface ECGs in post-myocardial infarction patients has become possible using the Selvester scoring method. Automation of this scoring has been proposed in order to speed-up the measurement of the score and improving the inter-observer variability in computing a score that requires strong expertise in electrocardiography. In this work, we evaluated the quality of the QuAReSS software for delivering correct Selvester scoring in a set of standard 12-lead ECGs. MethodStandard 12-lead ECGs were recorded in 105 post-MI patients prescribed implantation of an implantable cardiodefibrillator (ICD). Amongst the 105 patients with standard clinical left bundle branch block (LBBB) patterns, 67 had a LBBB pattern meeting the strict criteria. The QuAReSS software was applied to these 67 tracings by two independent groups of cardiologists (from a clinical group and an ECG core laboratory) to measure the Selvester score semi-automatically. Using various level of agreement metrics, we compared the scores between groups and when automatically measured by the software. ResultsThe average of the absolute difference in Selvester scores measured by the two independent groups was 1.4±1.5 score points, whereas the difference between automatic method and the two manual adjudications were 1.2±1.2 and 1.3±1.2 points. Eighty-two percent score agreement was observed between the two independent measurements when the difference of score was within two point ranges, while 90% and 84% score agreements were reached using the automatic method compared to the two manual adjudications. ConclusionThe study confirms that the QuAReSS software provides valid measurements of the Selvester score in patients with strict LBBB with minimal correction from cardiologists.

A symmetric probabilistic γ-index for Monte Carlo dose comparisons

The γ-index is a widely used tool to compare two dose distributions, which combines both the dose difference and distance-to-agreement criteria into a single metric. The γ-index passing rate, defined as the percentage of dose points with γ-index value less than one, is often used as an agreement metric. However, the γ-index is not symmetric with respect to the choice of the reference and evaluation distributions. Moreover, the statistical fluctuations present in the dose distributions may have non-negligible effects on γ-passing rates. Fluctuations have opposite effects on the γ-passing rates depending whether they are present in the evaluation or the reference dose distributions. Those discrepancies are analyzed in the case of realistic clinical proton dose distributions. The concept of a probabilistic and symmetric γ-index is introduced to make more robust versus statistical fluctuations.