Between-unit Variation Research Articles

Evaluation of CRM homogeneity in cases of insufficient method repeatability: Comparison of Bayesian analysis with substitutes for ANOVA based estimates

Insufficient method repeatability is a problem characterising the evaluation of certified reference materials (CRMs). In investigating the homogeneity studies of 216 certified parameters from 36 CRMs released by the European Commission’s Joint Research Centre (JRC) over the last four years, it was found that in 1/3 of the cases, the method repeatability (sr) was too high to calculate the standard deviation between units (sbb) by classical analysis of variance (ANOVA). It was also found that the application of the repeatability requirement stated in the ISO Guide 35:2017 is not feasible since it would require unrealistically low repeatability standard deviations or an impossibly high number of replicates per unit.Evaluation of the uncertainty of homogeneity (ubb) as evaluated by ANOVA using both the maximum of sbb and 0, the maximum of sbb and u∗bb, the uncertainty hidden by method repeatability, the maximum of sbb and sbb/√n and Bayesian analysis, using both informative and diffuse priors, as well as the standard deviation of the unit means, were compared using simulated homogeneity studies with repeatabilities of 1–8% and sbb between 0.2 and 2.8%. It was found that using the maximum of sbb and sbb/√n as an estimate of ubb guards against severe underestimation but usually results in a severe overestimation of the between-unit variation. Using the maximum of (sbb, 0) shows the least average bias but results in a severe underestimation of ubb in a high fraction of cases. Using the maximum of (sbb, u∗bb) limits, but does not completely eliminate cases of a severe underestimation. Also, it leads to average results biased towards high values. For the range of sbb and sr investigated, Bayesian analysis performed worse than max (sbb, u∗bb) in limiting severe underestimation of ubb, but limited the average bias towards high results.A risk-based approach to cases of insufficient method repeatability is proposed where, after evaluating the other contributions to the uncertainty of certified values, the effect of severe over- and underestimation of ubb is evaluated, and an appropriate approach is chosen based on this analysis.

Beyond police culture

Purpose Combining insights from the traditional literature on police culture with insights from the broader literature on organisational culture and on grid-group cultural theory (Douglas, 1970), the purpose of this paper is to introduce a new 15-dimensional framework of “organisational culture in the police” and test this framework via a survey instrument. This new conceptualisation is broader than the traditional police culture concept and allows for comparisons of the police with other organisations. Design/methodology/approach A newly developed instrument to measure the 15-dimensional framework, called the “Leuven Organisational Culture Questionnaire (LOCQ)”, was tested in 64 local police forces in Belgium (n=3,847). Findings The hypothesised 15-dimensional model is largely confirmed by confirmatory factor analysis. Assessments of between-unit variation show that the LOCQ is sufficiently sensitive to identify differences between work units in police organisations. The authors also find that traditional police culture characteristics tend to vary slightly less between units than the other characteristics. Also, there is less variation for characteristics related to police work (e.g. law enforcement orientation and citizen orientation) than for characteristics associated with the unit level (e.g. weak supervisory support and internal solidarity) or the organisational level (e.g. rule orientation and results orientation). Originality/value This paper expands the traditional “police culture” concept to a more generic and theory-driven conceptualisation of “organisational culture in the police”. The survey instrument offers a standardised way to map and compare culture within police organisations, and to compare it with the culture of other organisations both within and outside law enforcement.

The Treatment Group and Recidivism: A Multilevel Analysis of Prison-Based Substance Abuse Treatment

This study contributes to the substance abuse treatment literature by examining recidivism across treatment groups, the level of aggregation between the individual and the program. The sample consisted of 618 drug-involved offenders who participated in 12 different treatment groups within a single prison. Reincarceration significantly varied across groups, controlling for time at risk and treatment modality. This between-unit variation was explained by individual level treatment response measures and control variables. Results demonstrated that treatment effectiveness is impacted by the group in which the individual participates. These findings have substantial implications for correctional treatment theory, research, and policy.

Is organizational justice climate at the workplace associated with individual-level quality of care and organizational affective commitment? A multi-level, cross-sectional study on dentistry in Sweden

PurposeThe aim of this study is to investigate whether organizational justice climate at the workplace level is associated with individual staff members’ perceptions of care quality and affective commitment to the workplace.MethodsThe study adopts a cross-sectional multi-level design. Data were collected using an electronic survey and a response rate of 75% was obtained. Organizational justice climate and affective commitment to the workplace were measured by items from Copenhagen Psychosocial Questionnaire and quality of care by three self-developed items. Non-managerial staff working at dental clinics with at least five respondents (n = 900 from 68 units) was included in analyses. A set of Level-2 random intercept models were built to predict individual-level organizational affective commitment and perceived quality of care from unit-level organizational justice climate, controlling for potential confounding by group size, gender, age, and occupation.ResultsThe results of the empty model showed substantial between-unit variation for both affective commitment (ICC-1 = 0.17) and quality of care (ICC-1 = 0.12). The overall results showed that the shared perception of organizational justice climate at the clinical unit level was significantly associated with perceived quality of care and affective commitment to the organization (p < 0.001).ConclusionsOrganizational justice climate at work unit level explained all variation in affective commitment among dental clinics and was associated with both the individual staff members’ affective commitment and perceived quality of care. These findings suggest a potential for that addressing organizational justice climate may be a way to promote quality of care and enhancing affective commitment. However, longitudinal studies are needed to support causality in the examined relationships. Intervention research is also recommended to probe the effectiveness of actions increasing unit-level organizational justice climate and test their impact on quality of care and affective commitment.

Preparation and certification of two freshwater sediments certified reference materials for polycyclic aromatic hydrocarbons

Two freshwater sediments certified reference materials (CRMs) for 16 polycyclic aromatic hydrocarbons (PAHs) have been developed by the Institute for Environmental Reference Materials (IERM) of Ministry of Environmental Protection (MEP) in China. The methodology for preparing the CRMs of PAHs in sediments is described in this paper. The collected natural sediment samples were air-dried, ground, homogenised, packed, sterilised and tested on stability and homogeneity. Homogeneity results showed that the between-unit variation was confirmed to be below 4.5% for each compound. Stability was assessed after storage of samples for 16 months at temperature less than 30°C and in shade. The certification of the natural sediment matrix CRMs for PAHs was based on the agreement of results using different analytical techniques including gas chromatography/mass spectrometry (GC/MS) and reversed-phase liquid chromatography (LC) by no less than eight collaborating laboratories including IERM. Results of the homogeneity showed that the calculated ubb′ was 0.9–2.5% for environmental river standard-4 (ERS-4) and 9–2.3% for environmental lake standard-1 (ELS-1), whereas stability results of total 16 PAHs indicated that the calculated urel,lts was 4.2% for ERS-4 and 2.2% for ELS-1. Certified values of 16 PAHs in ERS-4 varied from 8.5 to 167 μg/kg and ranged from 0.036 to 2.8 mg/kg in ELS-1.The good comparability, together with the independent confirmation of the assigned mass fraction by using different methods, confirmed that the CRMs are suitable for the method validation and quality control in soil or sediments analysis.

Treatment escalation in the intensive care unit among patients with preexisting treatment limitations: best-laid plans gone awry?

In this issue of JAMA Internal Medicine, Hart and colleagues 1 provide, to our knowledge, the first known epidemiologic data on treatment patterns among patients admitted to the intensive care unit (ICU) with preexisting treatment limitations. Treatment limitations areexplicit statementsby thepatient,documented in themedical record, refusing certain treatments, as is their right according to the Patient Self-determination Act of 1990.2 In a hospital context, treatment limitationsmostoften includedonot resuscitate (DNR) or do not intubate orders. UsingProject IMPACT, anonrepresentative sampleof ICUs across the United States with detailed clinical data on all ICU admissions, Hart et al1 document that 5% of all ICU admissionswere amongpatientswith preexisting treatment limitations. Most treatment limitations were DNR orders. Many of thesepatients experiencedanescalationof treatment, including some form of life-supporting treatment, while in the ICU, and 1 in 4 of these patients received cardiopulmonary resuscitation (CPR), seemingly in violation of their wishes. Overall, 65% survived to discharge, but 35% died, corresponding to theobservation that 1 in 6patients admitted to the ICUwith a preexisting DNR order had received CPR. The authors further document substantial between-unit variation inmanagementof ICUpatientswithpreexisting treatment limitations. Although the 95%CIs arewide and unmeasured confounding cannot be ruled out, this variation nonetheless leads us to ask: What are the “right” rates? First,what is the right proportionof admissions to the ICU that should be made up of patients with preexisting treatment limitations?There isa lotof controversyabout thisamong clinicians, but guidelines promulgated by the Society of Critical Care Medicine3 regarding ICU admission criteria indicate that apreexistingDNRorder is not a criterion for refusal of ICU admission. Patients with DNR orders frequently desire curativemeasures shortofCPR, including treatmentsavailableonly in an ICU setting, such as hemodynamic support and noninvasivemechanical ventilation,4 andpatientsmaybenefit from such treatment. If there is a potential for meaningful clinical benefit from an escalation of treatment (suggested by a 65% survival rate) andpatientswould accept treatment escalation short of CPR, then the right rate of ICU admission will be directly proportional to the prevalence of preexisting treatment limitations in the critically ill population. The overall prevalence of treatment-limiting advance directives among Medicare decedents is 39%, and this varies significantly byUS region.5 Given a lack of information about the prevalence of preexisting treatment limitations inpatients admitted to each of the study hospitals, we cannot know whether the observed rates are too high, too low, or just right. But what about CPR, which most patients with DNR orders say theywish to forgo?What is the right rateofCPRamong ICUpatientswithpreexisting treatment limitations?Receiptof CPR in this population requires a reversal (ie, revocation) or reflects a violation (ie, ignoring) of the patient’s DNRorder. A reversalmight occur in 2ways: first, the patient and/or their surrogatemightreversetheDNRorderafterreasonedconversations with themedical care teambased on expected treatment benefit and thepatient’sunderlyinggoals for treatment.This could be because the patient’s treatment preference changed. Studies show that patients’ treatment preferences are unstable. In one study, competentpatientswillingly acceptedburdensome Related article page 1019 Research Original Investigation ICU Care and Preexisting Limits on Sustaining Life

Development of a patient safety climate survey for Chinese hospitals: cross-national adaptation and psychometric evaluation

BackgroundExisting patient safety climate instruments, most of which have been developed in the USA, may not accurately reflect the conditions in the healthcare systems of other countries.ObjectivesTo develop and evaluate...

Assessment of variation in the alberta context tool: the contribution of unit level contextual factors and specialty in Canadian pediatric acute care settings

BackgroundThere are few validated measures of organizational context and none that we located are parsimonious and address modifiable characteristics of context. The Alberta Context Tool (ACT) was developed to meet this need. The instrument assesses 8 dimensions of context, which comprise 10 concepts. The purpose of this paper is to report evidence to further the validity argument for ACT. The specific objectives of this paper are to: (1) examine the extent to which the 10 ACT concepts discriminate between patient care units and (2) identify variables that significantly contribute to between-unit variation for each of the 10 concepts.Methods859 professional nurses (844 valid responses) working in medical, surgical and critical care units of 8 Canadian pediatric hospitals completed the ACT. A random intercept, fixed effects hierarchical linear modeling (HLM) strategy was used to quantify and explain variance in the 10 ACT concepts to establish the ACT's ability to discriminate between units. We ran 40 models (a series of 4 models for each of the 10 concepts) in which we systematically assessed the unique contribution (i.e., error variance reduction) of different variables to between-unit variation. First, we constructed a null model in which we quantified the variance overall, in each of the concepts. Then we controlled for the contribution of individual level variables (Model 1). In Model 2, we assessed the contribution of practice specialty (medical, surgical, critical care) to variation since it was central to construction of the sampling frame for the study. Finally, we assessed the contribution of additional unit level variables (Model 3).ResultsThe null model (unadjusted baseline HLM model) established that there was significant variation between units in each of the 10 ACT concepts (i.e., discrimination between units). When we controlled for individual characteristics, significant variation in the 10 concepts remained. Assessment of the contribution of specialty to between-unit variation enabled us to explain more variance (1.19% to 16.73%) in 6 of the 10 ACT concepts. Finally, when we assessed the unique contribution of the unit level variables available to us, we were able to explain additional variance (15.91% to 73.25%) in 7 of the 10 ACT concepts.ConclusionThe findings reported here represent the third published argument for validity of the ACT and adds to the evidence supporting its use to discriminate patient care units by all 10 contextual factors. We found evidence of relationships between a variety of individual and unit-level variables that explained much of this between-unit variation for each of the 10 ACT concepts. Future research will include examination of the relationships between the ACT's contextual factors and research utilization by nurses and ultimately the relationships between context, research utilization, and outcomes for patients.

Development of the First Certified Reference Materials for Several Brominated Flame Retardants in Polymers

The first reference materials certified for several polybrominated flame retardants in polymers were developed. Commercially available polyethylene and polypropylene were fortified with technical mixtures of Pentabrominated diphenylether (Penta-BDE), Octa-BDE, Deca-BDE, and Decabrominated biphenyl (BB) (where the capitalized forms refer to the technical mixtures). Homogeneity was tested on 20 units of each material, and between-unit variation was confirmed to be below 4% for all congeners. Stability was assessed after storage of samples for 1 year at 4, 18, and 60 degrees C. Uncertainty of degradation during transport was found negligible for all congeners, whereas uncertainty of degradation for storage of 24 months at 4 degrees C was estimated between 2% and 11%. A characterization intercomparison involving 16 laboratories was organized. After exclusion of technically doubtful results, between-laboratory standard deviations ranged from 3% to 12%, making this intercomparison the best for this field of analysis so far. Statistical analysis revealed that the use of isotopically labeled internal standards did not improve analytical precision in this study. The good comparability, together with the independent confirmation of the assigned mass fractions via the total bromine content as well as by using non-GC/MS-based methods, allowed for the first time the certification of polymer materials for several brominated flame retardants.

Children and young people with diabetes in Yorkshire: a population‐based clinical audit of patient data 2005/2006

To provide a population-based clinical audit of children and young people with diabetes, reporting outcomes, including glycaemic control, for named individual units. Clinical audit data on care processes and glycated haemoglobin (HbA(1c)) were collected for 1742 children and young people treated in 16 paediatric units in Yorkshire, from January 2005 to March 2006. The Yorkshire Register of Diabetes in Children and Young People provided information technology support and validation that enhanced data quality. Multi-level linear regression modelling investigated factors affecting glycaemic control. An HbA(1c) measure was recorded for 91.6% of patients. The National Institute for Clinical Excellence-recommended target level for HbA(1c) of < 7.5% was achieved for 14.7% of patients. HbA(1c) was positively associated with duration of diabetes and later age at diagnosis. Patients living in deprived areas had significantly poorer control compared with those from affluent areas. Significant between-unit variation in HbA(1c) was not reflected by any association with unit size. Our population-based clinical audit of children with diabetes is the product of an effective collaboration between those who deliver care and health services researchers. High levels of recording the key care process measuring diabetes control, compared with national figures, suggests collaboration has translated into improved services. The interesting association between poor diabetes control and higher deprivation is noteworthy and requires further investigation. Future audits require recording of clinical management and clinic structures, in addition to resources to record, assemble and analyse data.

Integrated Late Quaternary chronostratigraphy for San Salvador Island, Bahamas: Patterns and trends of morphological change in the land snail Cerion

Reconstructing the phylogeny and biogeography of the Caribbean land snail Cerion requires a robust stratigraphic and chronological framework. To this end, we have determined the stratigraphic succession on San Salvador, a Bahamian island with a rich fossil and modern Cerion fauna. A primary purpose of this paper is to independently verify this succession through whole-rock and Cerion aminostratigraphies and AMS 14C-based age models. Over 150 individual Cerion shells were age-ranked from 140 ka to modern using stratigraphic position and reverse-phase HPLC (RPC) amino acid racemization, which was sufficiently sensitive to resolve stratigraphic subunits within the Holocene and late Pleistocene. A secondary purpose of this paper is to assess broad changes in the gross morphology (height, width) of supersets of Cerion from age-ranked lots spanning this ~ 140 kyr chronostratigraphy. Through each of the three interglacial sequences (i.e., marine isotope stages 5e, 5a, and 1), between-sample trends in mean gross morphology often greatly exceed within-sample variances (± 1 σ). Live-collected Cerion exhibit a range in gross morphology that nearly encompasses that of the entire fossil sequence. A trend of increasing gross shell size characterizes each of the interglacial phases, with a major step-decrease between marine isotope stages 5a and 1. While between-unit variation is often great in Cerion from SSI, within-unit variation appears unimodal through the record.

The DEBIT trial: an intervention to reduce antipsychotic polypharmacy prescribing in adult psychiatry wards – a cluster randomized controlled trial

Clinical guidelines advise against prescribing more than one antipsychotic with limited exceptions. Despite this, surveys continue to report high antipsychotic polypharmacy rates. The aim of the study was to investigate the effectiveness of a multi-faceted intervention in reducing prescribing of antipsychotic polypharmacy on general adult psychiatry wards, compared with guidelines alone. A pragmatic cluster randomized controlled trial recruited 19 adult psychiatric units (clusters) from the South West of England. Participants were all ward doctors and nurses. The multi-faceted intervention comprised: an educational/CBT workbook; an educational visit to consultants; and a reminder system on medication charts. The odds of being prescribed antipsychotic polypharmacy in those patients prescribed antipsychotic medication was significantly lower in the intervention than control group when adjusted for confounders (OR 0.43, 95% CI 0.21-0.90, p=0.028). There was considerable between-unit variation in polypharmacy rates and in the change in rates between baseline and follow-up (5 months after baseline). The intervention reduced levels of polypharmacy prescribing compared to guidelines alone although the effect size was relatively modest. Further work is needed to elicit the factors that were active in changing prescribing behaviour.

Deterministic chaos and historical geomorphology: A review and look forward

The study of nonlinear dynamics in geomorphology has been dominated by analytical techniques derived largely from other fields. Ultimately, however, confronting nonlinear complexity requires problematizing from within a geomorphological context. This paper focuses on deterministic chaos. The major classes of techniques for detecting chaos in geomorphic systems are reviewed. While the long time series most amenable to traditional methods of chaos detection are only occasionally available to geomorphologists, there are methods for detecting chaos in spatial data, including qualitative and categorical data. Many of these are similar to, or are adaptations of, familiar techniques for partitioning within- and between-unit variation, and for identifying elementary uniform landscape units. The manifestations of chaos in landforms and landscapes take the form of divergent vs. convergent evolution (increasing vs. decreasing irregularity), disproportionality vs. proportionality of response to perturbations or initial variations, and the (lack of) geographical consistency or commonality of response. These also serve as criteria for distinguishing between achievement of a new steady-state and unstable divergence in interpreting geomorphic changes. Study of nonlinear dynamics in earth science need not be dominated by theoreticians and modellers, as complex nonlinear behaviors can be identified and diagnosed using the types of data, observations, and methods typically available to geomorphologists. Even more important is the need to further inform the study of nonlinear dynamics on the basis of field-based geomorphology.

Statistical Issues in the Prospective Monitoring of Health Outcomes Across Multiple Units

Summary Following several recent inquiries in the UK into medical malpractice and failures to deliver appropriate standards of health care, there is pressure to introduce formal monitoring of performance outcomes routinely throughout the National Health Service. Statistical process control (SPC) charts have been widely used to monitor medical outcomes in a variety of contexts and have been specifically advocated for use in clinical governance. However, previous applications of SPC charts in medical monitoring have focused on surveillance of a single process over time. We consider some of the methodological and practical aspects that surround the routine surveillance of health outcomes and, in particular, we focus on two important methodological issues that arise when attempting to extend SPC charts to monitor outcomes at more than one unit simultaneously (where a unit could be, for example, a surgeon, general practitioner or hospital): the need to acknowledge the inevitable between-unit variation in ‘acceptable’ performance outcomes due to the net effect of many small unmeasured sources of variation (e.g. unmeasured case mix and data errors) and the problem of multiple testing over units as well as time. We address the former by using quasi-likelihood estimates of overdispersion, and the latter by using recently developed methods based on estimation of false discovery rates. We present an application of our approach to annual monitoring ‘all-cause’ mortality data between 1995 and 2000 from 169 National Health Service hospital trusts in England and Wales.

Selecting end point variables for a community intervention trial

Purpose: Community interventions, while aimed at affecting individual health behavior, are typically applied at a level of organization higher than the individual, such as schools or worksites. Evaluation of a community intervention trial may likewise involve either direct measurement of the environment or aggregation of individual-level end points. We review some statistical considerations that enter into the choice of end points for a community trial. Methods: Statistical precision is what ultimately determines the power of a trial to detect an effect of intervention. Random-effects analysis of variance provides an apt model for assessing statistical precision in the presence of several levels of between-unit and within-unit variation, as are typically present in a community trial. Results: Three examples of multilevel variation are provided, one based on a simple measurement problem and the other two on data from the Child and Adolescent Trial for Cardiovascular Health. The examples demonstrate that choosing to conduct measurements at a higher or lower level of organization can profoundly influence the power, cost, and interpretability of a trial. Cconclusions: The best end point for a community trial is one that is sensitive and specific to the intervention. The level of measurement need not match the level of intervention exactly; however, too great a discrepancy may be deleterious, lowering statistical power and blurring interpretation. Careful assessment of variance at each level can help an investigator choose the best level for evaluation and the optimal sample size at each level.

Petrography and Chemistry of High-Carbon Fly Ash from the Shawnee Power Station, Kentucky

Abstract The Shawnee power station in western Kentucky consists of ten 150-MW units, eight of which bum low-sulfur ( 1 wt %) coal in an atmospheric fluidized-bed combustion unit and in a research unit. The eight low-sulfur coal units were sampled in a 1992 survey of Kentucky utilities. Little between-unit variation is seen in the ash-basis major oxide and minor element chemistry. The carbon content of the fly ashes varies from 5 to 25 wt %. Similarly, the isotropic and anisotropic coke in the fly ash varies from 6% to 42% (volume basis). Much of the anisotropic coke is a thin-walled macroporous variety, but there is a portion that is a thick-walled variety similar to a petroleum coke.