Reliable Judgments Research Articles

Reliability-based optimal design of linear structures subjected to stochastic excitations

In this paper the problem of reliability-based optimal design of linear structures subjected to stochastic excitations is considered. A global optimization method based on Transitional Markov chain Monte Carlo (TMCMC) is used to address the problem, where the optimization problem is converted into the task of generating sample points (designs) according to a probability density function (PDF) suitably constructed on the feasible space of designs satisfying all the constraints. TMCMC is used for generating sample points, in order to get higher convergence rate of the stationary distribution of the Markov chain states to the constructed PDF. The generation of sample points uniformly distributed in the feasible space, which is required at the initial stage of TMCMC, is achieved by using Subset Simulation. To apply Subset Simulation and TMCMC in the concerned reliability-based optimization problem, Domain Decomposition Method (DDM) is used to examine the reliability constraint, that is, whether the failure probability at a given design exceeds a specified threshold. Based on the statistical properties of the failure probability estimator given by DDM, a ‘minimum’ computational effort, in terms of providing a reliable judgment on the reliability constraint, is defined so that a further reduction in the computational cost can be achieved in the proposed reliability-based optimization (RBO) algorithm. Illustrative examples are presented to show the application and the advantages of the proposed global RBO algorithm.

A novel decoding algorithm based on the hierarchical reliable strategy for SCG-LDPC codes in optical communications

An effective hierarchical reliable belief propagation (HRBP) decoding algorithm is proposed according to the structural characteristics of systematically constructed Gallager low-density parity-check (SCG-LDPC) codes. The novel decoding algorithm combines the layered iteration with the reliability judgment, and can greatly reduce the number of the variable nodes involved in the subsequent iteration process and accelerate the convergence rate. The result of simulation for SCG-LDPC(3969,3720) code shows that the novel HRBP decoding algorithm can greatly reduce the computing amount at the condition of ensuring the performance compared with the traditional belief propagation (BP) algorithm. The bit error rate (BER) of the HRBP algorithm is considerable at the threshold value of 15, but in the subsequent iteration process, the number of the variable nodes for the HRBP algorithm can be reduced by about 70% at the high signal-to-noise ratio (SNR) compared with the BP algorithm. When the threshold value is further increased, the HRBP algorithm will gradually degenerate into the layered-BP algorithm, but at the BER of 10−7 and the maximal iteration number of 30, the net coding gain (NCG) of the HRBP algorithm is 0.2 dB more than that of the BP algorithm, and the average iteration times can be reduced by about 40% at the high SNR. Therefore, the novel HRBP decoding algorithm is more suitable for optical communication systems.

A SATISFACTORY MINIMUM CONCEPTION OF JUSTICE: RECONSIDERING RAWLS'S MAXIMIN ARGUMENT

John Rawls argues that it is possible to describe a suitably defined initial situation from which to form reliable judgements about justice. In this initial situation, rational persons are deprived of information that is ‘irrelevant from the standpoint of justice’. It is rational, Rawls argues, for persons choosing principles of justice from this standpoint to be guided by the maximin rule. Critics, however, argue that (i) the maximin rule is not the appropriate decision rule for Rawls's choice position; (ii) the maximin argument relies upon an imprecise account of the satisfactory minimum to be secured under the maximin rule; or that (iii) Rawls relies upon unrealistic assumptions about diminishing marginal value. These critics, I will suggest, argue from a number of assumptions that are confused or false. The satisfactory minimum that choosers in the original position – employing the maximin rule – seek to achieve isnota minimum level of primary goods, nor is the satisfactory minimum sought under the maximin rule supplied by the difference principle. I will argue that the maximin argument is more robust than has generally been recognized and that this argument performs a number of important functions in clarifying the nature and implications of Rawls's argument for justice as fairness.

A RELIABILISM BUILT ON COGNITIVE CONVERGENCE: AN EMPIRICALLY GROUNDED SOLUTION TO THE GENERALITY PROBLEM

AbstractProcess-reliabilist analyses of justification and knowledge face the generality problem. Recent discussion of this problem turns on certain untested empirical assumptions that this paper investigates. Three experiments are reported: two are free-naming studies that support the existence of a basic level in the previously unexplored domain of names for belief-forming processes; the third demonstrates that reliability judgments for the basic-level belief-forming process types are very strongly correlated with the corresponding justification and knowledge judgments. I argue that these results lend support to process-reliabilism; I then go on to explore several different ways to make use of them in solutions to the generality problem.

A comparative study on different assessment procedures applied to loudspeaker sound quality

Paradoxically, one of the hardest to measure characteristics of a sound reproduction device such as a loudspeaker is its sound quality. The perception of this subjective character is linked to numerous parameters (stimulus type, listening environment, etc.) that must be drastically controlled to lead to reliable and repeatable judgments. Industrial and academic researchers are still focusing on the design of standard assessment procedures. The conditions under which a sound reproduction system is assessed in laboratory tests is often very far from those under which it is designed to be used. As a result, the assessment task might appear unnatural to test subjects, which could possibly bias the test results. The aim of this study is to compare, on the basis of sound quality ratings, three different test procedures based on paired comparison and exhibiting procedural differences. One of the procedures consisted in comparing loudspeakers by listening to short music excerpts (5s) at a preset level, which was assumed to be a very controllable method. In the two other procedures, the listener could compare the systems by listening to long music excerpts (30s), which was assumed to be more natural for loudspeaker assessment. The level was either preset by expert listeners or set by the subject himself in the two latter procedures. This paper shows that the test results were very stable over the different assessment procedures, but that some of them enabled, under certain conditions, to separate between systems obtaining very close quality ratings.

Interobserver variability of radiation therapists aligning to fiducial markers for prostate radiation therapy

As the use of fiducial markers (FMs) for the localisation of the prostate during external beam radiation therapy (EBRT) has become part of routine practice, radiation therapists (RTs) have become increasingly responsible for online image interpretation. The aim of this investigation was to quantify the limits of agreement (LoA) between RTs when localising to FMs with orthogonal kilovoltage (kV) imaging. Six patients receiving prostate EBRT utilising FMs were included in this study. Treatment localisation was performed using kV imaging prior to each fraction. Online stereoscopic assessment of FMs, performed by the treating RTs, was compared with the offline assessment by three RTs. Observer agreement was determined by pairwise Bland-Altman analysis. Stereoscopic analysis of 225 image pairs was performed online at the time of treatment, and offline by three RT observers. Eighteen pairwise Bland-Altman analyses were completed to assess the level of agreement between observers. Localisation by RTs was found to be within clinically acceptable 95% LoAs. Small differences between RTs, in both the online and offline setting, were found to be within clinically acceptable limits. RTs were able to make consistent and reliable judgements when matching FMs on planar kV imaging.

Energy Valuation Methods for Biofuels in South Florida: Introduction to Life Cycle Assessment and Emergy Approaches

This 5-page fact sheet gives an overview of two methods for evaluating energy transformations in biofuels production. The Life Cycle Assessment approach involves measurements affecting greenhouse gases, which can be linked to the energy considerations used in the Emergy Assessment. Although these two methods have their basis in energy or greenhouse gas emission evaluations, their approaches can lead to a reliable judgment regarding a biofuel process. We can use them to evaluate the economic environmental component of a biofuel process, and decide which biofuel processes favor sustainability. The intended audiences of this publication are growers, researchers, students, and any other readers interested in agriculture and ecology. Written by J. Van Treese II, E. A. Hanlon, N. Y. Amponsah, J. L. Izursa, and J. C. Capece, and published by the UF Department of Soil and Water Science, March 2013.
 http://edis.ifas.ufl.edu/ss579

Is comprehension of problem solutions resistant to misleading heuristic cues?

Previous studies in the domain of metacomprehension judgments have primarily used expository texts. When these texts include illustrations, even uninformative ones, people were found to judge that they understand their content better. The present study aimed to delineate the metacognitive processes involved in understanding problem solutions — a text type often perceived as allowing reliable judgments regarding understanding, and was not previously considered from a metacognitive perspective. Undergraduate students faced difficult problems. They then studied solution explanations with or without uninformative illustrations and provided judgments of comprehension (JCOMPs). Learning was assessed by application to near-transfer problems in an open-book test format. As expected, JCOMPs were polarized — they tended to reflect good or poor understanding. Yet, JCOMPs were higher for the illustrated solutions and even high certainty did not ensure resistance to this effect. Moreover, success in the transfer problems was lower in the presence of illustrations, demonstrating a bias stronger than that found with expository texts. Previous studies have suggested that weak learners are especially prone to being misled by superficial cues. In the present study, matching the difficulty of the task to the ability of the target population revealed that even highly able participants were not immune to misleading cues. The study extends previous findings regarding potential detrimental effects of illustrations and highlights aspects of the metacomprehension process that have not been considered before.

Composite reliability of a workplace-based assessment toolbox for postgraduate medical education

In recent years, postgraduate assessment programmes around the world have embraced workplace-based assessment (WBA) and its related tools. Despite their widespread use, results of studies on the validity and reliability of these tools have been variable. Although in many countries decisions about residents' continuation of training and certification as a specialist are based on the composite results of different WBAs collected in a portfolio, to our knowledge, the reliability of such a WBA toolbox has never been investigated. Using generalisability theory, we analysed the separate and composite reliability of three WBA tools [mini-Clinical Evaluation Exercise (mini-CEX), direct observation of procedural skills (DOPS), and multisource feedback (MSF)] included in a resident portfolio. G-studies and D-studies of 12,779 WBAs from a total of 953 residents showed that a reliability coefficient of 0.80 was obtained for eight mini-CEXs, nine DOPS, and nine MSF rounds, whilst the same reliability was found for seven mini-CEXs, eight DOPS, and one MSF when combined in a portfolio. At the end of the first year of residency a portfolio with five mini-CEXs, six DOPS, and one MSF afforded reliable judgement. The results support the conclusion that several WBA tools combined in a portfolio can be a feasible and reliable method for high-stakes judgements.

Sensitivity and uncertainty analysis of England's housing energy model

SAP/BREDEM-based models are commonly used for estimating national domestic energy consumption and for assessing the energy and environmental impacts of changes in the housing stock. However, policy-makers should recognize that such models are subject to multiple potential sources of uncertainty. A one-at-a-time sensitivity analysis is undertaken on the Cambridge Housing Model, a SAP 2009 model for estimating total domestic energy consumption in England and the UK. The analysis identifies several parameters to which the model is particularly sensitive, determines that the response to variations in individual parameters is typically non-linear, and concludes that changes to multiple parameters are not additive. A Monte Carlo uncertainty analysis is then undertaken to provide an indication of the impact of multiple uncertainties on model outputs. A concept map is developed to outline a number of potential sources of uncertainty, and in the absence of reliable information expert judgement is used to describe these uncertainties. Outputs suggest that any single estimate of total consumption may be subject to considerable inaccuracy compared with actual use. Similar levels of uncertainty and inaccuracy are likely for any SAP/BREDEM-based national domestic energy model, such as the models underpinning the Green Deal. Des modèles de type SAP/BREDEM sont couramment utilisés pour estimer la consommation domestique nationale d'énergie et pour évaluer les impacts énergétiques et environnementaux des changements intervenant dans le parc bâti. Cependant, les décideurs devraient reconnaître que ces modèles sont sujets à de multiples sources possibles d'incertitude. Une analyse de sensibilité par l'approche One-At-A-Time (OAT) est entreprise sur le Modèle Cambridge Housing, un modèle SAP 2009 permettant d'estimer la consommation domestique totale d'énergie en Angleterre et au Royaume-Uni. L'analyse identifie plusieurs paramètres auxquels le modèle est particulièrement sensible, détermine que la réponse aux variations des paramètres individuels est habituellement non linéaire et conclue que les modifications des paramètres multiples ne sont pas additives. Une analyse Monte-Carlo des incertitudes est ensuite entreprise pour fournir une indication de l'impact des incertitudes multiples sur les sorties du modèle. Une carte conceptuelle est élaborée pour exposer un certain nombre de sources possibles d'incertitude, et en l'absence d'informations fiables, un jugement d'expert est exercé pour décrire ces incertitudes. Les sorties suggèrent que chaque estimation de la consommation totale peut être sujette à des inexactitudes considérables par rapport à l'utilisation réelle. Des niveaux d'incertitude et d'inexactitude similaires sont probables pour tous les modèles de consommation domestique nationale d'énergie de type SAP/BREDEM, tels que les modèles sous-tendant le New Deal vert. Mots clés: énergie domestique, modèle énergétique, énergie des ménages, parc bâti, analyse de sensibilité, incertitude

Clustering numerical weather forecasts to obtain statistical prediction intervals

The numerical weather prediction (NWP) model outputs are point deterministic values arranged on a three-dimensional grid. However, there is always some level of uncertainty in the prediction. Many applications would benefit from provision of relevant uncertainty information along with the forecast. A common means of formulating and communicating forecast uncertainty are prediction intervals (PI). In this study, various methods for modelling the uncertainty of NWP forecasts are investigated and PIs provided for predictions accordingly. In particular, the interest is in analysing the historical performance of the system as a valuable source of information for uncertainty analysis. Various clustering algorithms are employed to group the performance records as the first step of the PI determination process. In the second step, a range of methods are used to fit appropriate probability distributions to errors of each cluster. As a result, PIs can be computed dynamically depending on the forecast context. The clustering algorithms are applied over different feature sets and derived and generated features. All presented PI computation methods are empirically evaluated using a comprehensive verification framework in a set of experiments involving two real-world data sets of NWP forecasts and observations. The proposed evaluation provides a considerably fairer and more reliable judgement compared to existing methods. Results show that incorporating trained uncertainty model outputs into the NWP point predictions provides PI forecasts with higher reliability and skill. This can lead to improvement of decision processes for many applications that rely on these forecasts. Copyright © 2013 Royal Meteorological Society

Progress of the earthquake early warning system in Fujian, China

In this article, we systematically introduce the latest progress of the earthquake early warning (EEW) system in Fujian, China. We focus on the following key technologies and methods: continuous earthquake location and its error evaluation; magnitude estimation; reliability judgment of EEW system information; use of double-parameter principle in EEW system information release threshold; real-time estimation of seismic intensity and available time for target areas; seismic-monitoring network and data sharing platform; EEW system information release and receiving platform; software test platform; and test results statistical analysis. Based on strong ground motion data received in the mainshock of the Wenchuan earthquake, the EEW system developed by the above algorithm is simulated online, and the results show that the system can reduce earthquake hazards effectively. In addition, we analyzed four earthquake cases with magnitude greater than 5.5 processed by our EEW system since the online-testing that was started one year ago, and results indicate that our system can effectively reduce earthquake hazards and have high practical significance.

BIM기반 공동주택 마감 물량 산출 정확도 연구

건설 산업 생애주기 전반에 걸친 정보를 BIM을 활용하여 관리할 수 있게 되었다. 그 중에서도 BIM기반 물량산출은 공사비와 직결되는 중요한 관리 요소로서 사업 타당성 및 경제성 분석을 위한 필수적인 관리 요소 중에 하나이다. 또한, BIM기반 물량산출 Data는 작업자마다 물량산출 결과가 상이하게 나타날 수 있으므로 BIM모델 구축에 대한 정확한 기준이 필요하다. 이러한 BIM기반 물량 산출은 정보의 신뢰성과 정확도 측면에서 매우 중요하다. 따라서 본 연구에서는 BIM기반 물량산출 결과의 정확도 및 신뢰성을 판단하고, BIM 모델링의 현실적이고 바람직한 방법을 제시하고자 단위세대 내부 마감재를 개별부재와 복합부재로 모델링하여 산출되는 BIM기반 물량산출 결과를 정량적으로 비교 검증하였다. 연구의 결과, 정확한 BIM기반 물량산출을 위해서는 다양한 마감재를 개별부재로 모델링하는 것이 바람직하지만, 작업의 생산성 측면에서 보다 현실적인 BIM기반 물량산출을 위해서는 개별부재와 복합부재를 병용하여 모델링 작업을 실시하는 것이 현실적인 BIM 모델링 방법임이 검증되었다. Construction industry throughout the construction projects life cycle using BIM is able to manage information. Among them, BIM-based quantity take-off is directly connected to the data of the construction business as an important management element that is essential for one of the management factors of feasibility study and economic analysis. In addition, the result of BIM-based quantity take-off can be varied depending on modeler, so the exact guideline for BIM modeling is necessary. The BIM-based quantity take-off in terms of reliability and accuracy of the information is very important. Therefore, this study verified the BIM modeling of the apartment interior materials using both separate object and composite object was validated the quantitative comparison of the difference in BIM-based quantity take-off for accuracy & reliability judgment of BIM-based quantity take-off and realistic & desirable suggestion of BIM modeling way. As the result of this study, it is preferable to model various interior materials to use separate object for the purpose of correct BIM-based quantity take-off, but the realistic way of the BIM modeling using both separate object and composite object jointly was verified in order to judge BIM-based quantity take-off in terms of work productivity.

Presentation Modality and Source Monitoring During Management Inquiry: Effects on Auditors’ Reliability Judgments

An experiment was performed to determine whether the presentation modality of evidence affected auditors’ accuracy in identifying the sources of that evidence (i.e., source monitoring) and their subsequent judgments of evidence reliability. Participants were provided with conflicting explanations for a material increase in a fictitious client firm’s gross margin. The explanations were presented in either the same (both visual) or mixed (one visual and one auditory) modalities. The sources of the explanations had either the same (both high) or mixed (one high and one low) levels of credibility. Results indicate that auditors who received evidence in mixed presentation modalities were more accurate in identifying the sources of that evidence. Also, auditors with more accurate source monitoring judged evidence obtained from a less-credible source to be of significantly lower reliability than did auditors who were not as accurate in identifying evidence sources. These findings suggest that auditors’ judgments of evidence reliability may be influenced by the sensory modality by which evidence is perceived. Also, post-hoc analysis indicates that taking notes may improve auditors’ source monitoring accuracy when evidence is perceived via an auditory modality.

Theory-laden experimentation

The thesis of theory-ladenness of observations, in its various guises, is widely considered as either ill-conceived or harmless to the rationality of science. The latter view rests partly on the work of the proponents of New Experimentalism who have argued, among other things, that experimental practices are efficient in guarding against any epistemological threat posed by theory-ladenness. In this paper I show that one can generate a thesis of theory-ladenness for experimental practices from an influential New Experimentalist account. The notion I introduce for this purpose is the concept of ‘theory-driven data reliability judgments’ (TDR), according to which theories which are sought to be tested with a particular set of data guide reliability judgments about those very same data. I provide various prominent historical examples (among others, the confirmation of Einstein’s prediction of star light bending in 1919 by Eddington and others) to show that TDRs are used by scientists to resolve data conflicts. I argue that the rationality of the practices which employ TDRs can be saved if the independent support of the theories driving TDRs is construed in a particular way.

Haptic distal spatial perception mediated by strings: Size at a distance and egocentric localization based on ellipse geometry

Participants explored ellipse perimeters defined by fixed-length strings, held taut by their moving finger, with ends attached to two fixed hooks (foci). Participants haptically judged ellipse interfocal distance (IFD; in effect, exocentric separation, or size at a distance) or location of ellipse major axes (i.e., egocentric localization relative to the moving finger). In Experiment 1A, perceivers made reliable and accurate multialternative forced choice IFD judgments. Experiment 1B showed similar reliability for direct estimations but showed greater scaling error. In Experiment 2, perceivers reliably localized ellipse major axes. Both experiments derived from a priori geometrical analysis, consistent with the Gibsonian perceptual research program and previous string-mediated haptic distal spatial studies. Results are discussed with respect to haptic perception as a telemodality and to dynamic touch.

Credibility of Asylum Claims: Consistency and Accuracy of Autobiographical Memory Reports Following Trauma

SummaryHerlihy and colleagues (current issue) review the literature on the characteristics of autobiographical memory in asylum seekers who are presenting evidence of their traumatic experiences in the immigration courts with a view to finding a safe haven. In this commentary, I briefly discuss how the quality of the memory report may influence reliability and credibility judgements in individuals whose memories may be subject to post‐traumatic stress disorder. Copyright © 2012 John Wiley & Sons, Ltd.

Anaesthetic training: not better, not worse, just different

Anaesthetic training: not better, not worse, just different

First impressions: Gait cues drive reliable trait judgements

Personality trait attribution can underpin important social decisions and yet requires little effort; even a brief exposure to a photograph can generate lasting impressions. Body movement is a channel readily available to observers and allows judgements to be made when facial and body appearances are less visible; e.g., from great distances. Across three studies, we assessed the reliability of trait judgements of point-light walkers and identified motion-related visual cues driving observers’ judgements. The findings confirm that observers make reliable, albeit inaccurate, trait judgements, and these were linked to a small number of motion components derived from a Principal Component Analysis of the motion data. Parametric manipulation of the motion components linearly affected trait ratings, providing strong evidence that the visual cues captured by these components drive observers’ trait judgements. Subsequent analyses suggest that reliability of trait ratings was driven by impressions of emotion, attractiveness and masculinity.

Conceptualising creativity in the creative arts: Seeking common ground

This paper reports on a research study that investigated how university academics in the creative arts conceptualise creativity, both as a general concept and also as a set of situated disciplinary beliefs and practices. The study, located primarily in the UK and Australia, reveals that the problems of defining and articulating creativity lead to a series of ongoing dilemmas for teaching and assessment in the creative arts. These dilemmas include the ability to explain and foster the creative endeavour with students; to provide transparency to students about expectations and standards; and to make demonstrably fair and reliable judgements about student work in a relatively subjective domain. It argues that if tertiary-level teachers are to devise creative curricula, facilitate creative environments and make judgements about the quality of creative work, they need to both share an understanding about the creative endeavour and the ability to communicate this to students effectively.