Reliability of Aggregated Organizational Data: An Evaluation of Five Empirical Indices

Abstract

Nursing systems research has included many levels of measurement such as individuals, work groups, organizations, and systems of organizations. Variables important to organizational survival often are measured at the individual level, with inferences made to the work group or the organization. This study demonstrated a process for determining the reliability of individual level data aggregated to the work group and organization levels. Data were analyzed on four variables: job enjoyment; manager's leadership style in terms of structuring expectations and consideration; and control over nursing practice. Job enjoyment was assessed over time. Registered staff nurses (n = 632), representing 54 nursing units from 4 large acute care hospitals, comprised the sample. The results indicated the assessment of reliability and validity is important at the level of inference and at each time interval. Aggregated data were deemed reliable when the following criteria were met: Cronbach's alpha > .60; intraclass correlation (1,k) > .60; and a significant F ratio. The interpretation of omega 2 as an indicator of effect size suggested that the validity of inference at the aggregated level is in question when effect size is negligible or small.