Card Sorting Exercise Research Articles

Pre-clerkship medical students\u2019 perceptions of medical professionalism

BackgroundProfessionalism instruction and assessment is a core component of medical education, and essential for professional identity formation (PIF). Thus, understanding the socialization of medical students to the values of the profession (i.e., medical professionalism), and how these may evolve, warrants continued understanding.MethodsThe purpose of this study was to examine and compare pre-clerkship (first and second year) medical students’ perceptions of professionalism. First and second year medical students participate in this study. This was a two-phase mixed-methods cohort study conducted across two academic years (2014–2015 and 2015–2016). In Phase I, first and second year medical students participated in a nominal group technique (NGT) session. NGT data was analyzed qualitatively to generate a card-sorting exercise of professionalism attributes for Phase II. In Phase II, data from the sorting task was analyzed using Principle Component Analysis (PCA).ResultsThe PCA for first year students derived a 7-factor solution. Factors (i.e., professionalism domains) identified were: Self-management and patient-centeredness, ethics and professional reputation, dependability, self-awareness and self-improvement, image, proficiency and lifelong learning and integrity. The PCA for second year students derived a 5-factor solution; factors identified were: “Good Doctor” attributes, responsibility, ethics, innovation and self-improvement and unbiased.ConclusionsIdentification and organization of attributes into an overarching professionalism mental model provide a window into the active reconstruction of students’ professional identity during the nascent stages of medical education. M1 professionalism domains were more consistent with the conventional professional image of the physician (e.g. Ethics and Professional reputation, Dependability, Integrity), whereas, M2 domains reflected a more global view (e.g., “Good Doctor” attributes, Responsibility, Ethics). This study provides a lens into the dynamic nature of students’ PIF and encourages educators to evaluate PIF pedagogy at their own institutions.

Communication through design sketches: implications for stakeholder interpretation during concept design

We investigate how stakeholder expertise influences design interpretation derived from intent communication through conceptual design representations. Four types of design sketch are used as stimuli to gather responses from three stakeholder groups: Designers, Managers, Engineers. Physical response derived from an eye-tracking study, together with self-report responses gathered through a card-sorting exercise using semantic differentials, is subjected to qualitative and statistical analysis. Results indicate how expertise implicated response to design intent expressed through the ambiguity of sketch representations, significant differences towards graphic elements of the sketch stimuli, and how stakeholder expertise (i.e. Design, Manager, Engineer) implicated comprehension. Findings are discussed in terms implications for understanding the role and use of sketches in the communication of intent during conceptual design ideation.

Abstract P5-14-06: Prioritization of patient reported outcomes by women with metastatic breast cancer

Abstract Introduction: An emphasis on patient-centered care has led to a growing interest in collecting patient-reported outcomes (PROs) in the setting of cancer care. Routine collection of actionable PROs has been shown to improve patient satisfaction with care and even prolong survival. However, completion rates of PROs outside of the research setting are low, which may be due to an incomplete understanding of the outcomes patients value most. Prior work has focused primarily on symptom burden, but patients are also affected by disease and treatment across multiple domains (e.g. physical, psychological, social, and financial). To address this knowledge gap, we conducted a qualitative study among women with metastatic breast cancer (MBC) to identify the optimal patient-centered approach to collecting PRO data. Methods: We conducted 1-on-1 interviews with patients who had started a treatment regimen for MBC within the past 6 weeks at the Breast Center at Smilow Cancer Hospital of Yale New Haven Hospital to determine which PROs were most personally relevant. We assessed heterogeneity across patients in their prioritization. Patients were asked which of a list of six PRO domains they would like their provider to have information about and then ranked the domains by order of importance (from most to least important). The following domains were created from the NCCN Distress Thermometer: physical well-being, emotional well-being, treatment burden, functional status, financial concerns, and social well-being. For each ranked domain, patients were asked to rank items within the domain using a card sorting exercise where the number of items ranged from 5 to 15. Patients were then asked where and how often they preferred to report PROs. Results: Ten women with MBC completed the card sorting exercise: mean age was 58 years (+/- 12), 7 were white, 2 African American and 1 Asian; 1 identified as Hispanic. After 10 interviews, it was apparent that no single set of domain rankings was common across patients. Patient prioritization of PRO domains was unique and varied. Selection and prioritization of PRO domains and items within each domain were unique and varied. Five women reported “physical well-being” as the most important domain; treatment burden and emotional well-being were also selected as most important or ranked as highly important. Participants preferred reporting MBC PROs while in the waiting room for all domains except emotional well-being (from home was the preference). However, participants were willing to complete PRO assessment in the waiting room for about ten minutes and at home for twenty minutes. Conclusion: Substantial variation exists in how women with MBC rate the importance of specific PRO domains and items within each domain. Importantly, “physical symptoms” was not the top concern for half of the interviewed patients. This is an important finding, given that previous published studies of patient-reported outcomes have focused on one domain, such as symptoms and side effects or the financial burden of treatment. Our findings support the development of multi-dimensional tools for the collection of PROs. Although toxicity and physical symptoms are of utmost concern, clinicians should not neglect other dimensions of quality of life in women with MBC. Citation Format: Mougalian SS, Aminawung JA, Presley C, Canavan ME, Holland ML, Hu X, Gross CP. Prioritization of patient reported outcomes by women with metastatic breast cancer [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P5-14-06.

Doing Design Thinking: Conceptual Review, Synthesis, and Research Agenda

Design thinking has attracted considerable interest from practitioners and academics alike, as it offers a novel approach to innovation and problem‐solving. However, there appear to be substantial differences between promoters and critics about its essential attributes, applicability, and outcomes. To shed light on current knowledge and conceptualizations of design thinking we undertook a multiphase study. First, a systematic review of the design thinking literature enabled us to identify 10 principal attributes and 8 tools and methods. To validate and refine our findings, we then employed a card sorting exercise with professional designers. Finally, we undertook a cluster analysis to reveal structural patterns within the design thinking literature. Our research makes three principal contributions to design and innovation management theory and practice. First, in rigorously deriving 10 attributes and 8 essential tools and methods that support them from a broad and multidisciplinary assortment of articles, we bring much needed clarity and validity to a construct plagued by polysemy and thus threatened by “construct collapse.” Second, aided by the identification of perspectives of scholars writing about design thinking, we provide detailed recommendations for relevant topics warranting further study in order to advance theoretical understanding of design thinking and test its applications. Third, we identify the enduring, yet essential, questions that remain unresolved across the extant design thinking literature and that may impede its practical implementation. We also provide suggestions for the theoretic frames, which may help address them, and thus advance the ability of scholars and managers alike to benefit from design thinking’s apparent advantages.

Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods

Many electronic health records fail to support information uptake because they impose low-level information organization tasks on users. Clinical concept-oriented views have shown information processing improvements, but the specifics of this organization for critical care are unclear. To determine high-level cognitive processes and patient information organization schema in critical care. We conducted an open card sort of 29 patient data elements and a modified Delphi card sort of 65 patient data elements. Study participants were 39 clinicians with varied critical care training and experience. We analyzed the open sort with a hierarchical cluster analysis (HCA) and factor analysis (FA). The Delphi sort was split into three initiating groups that resulted in three unique solutions. We compared results between open sort analyses (HCA and FA), between card sorting exercises (open and Delphi), and across the Delphi solutions. Between the HCA and FA, we observed common constructs including cardiovascular and hemodynamics, infectious disease, medications, neurology, patient overview, respiratory, and vital signs. The more comprehensive Delphi sort solutions also included gastrointestinal, renal, and imaging constructs. We identified primarily system-based groupings (e.g., cardiovascular, respiratory). Source-based (e.g., medications, laboratory) groups became apparent when participants were asked to sort a longer list of concepts. These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians' mental models. These results can contribute to the design of information displays to better support clinicians' access and interpretation of information for critical care decisions.

Q Methodology: A Method for Understanding Complex Viewpoints in Communities Served by Extension

This article introduces Q methodology, an idea-sorting activity that can help Extension improve outreach and education on new and contentious issues. Q methodology is a helpful tool when Extension professionals are confronted with controversial or complex resource management challenges. Through the analysis of a simple card-sorting exercise, researchers can determine quantitatively and qualitatively how different issues combine to result in (a) an individual's viewpoint on an issue and (b) groupings of different viewpoints within a community. We describe the basic approach to implementing Q methodology and suggest circumstances in which it can help facilitate Extension outreach and education.

Development of novel emoji scale to measure patient-reported outcomes in cancer patients.

174 Background: Patient-reported outcomes (PROs), including physical function and quality of life (QOL), are important outcomes in cancer patients. Emoji are a near-universal, popular form of communication, understandable by diverse populations, including those with low health literacy. We aimed to explore whether emoji could be a valid, reliable scale for assessing PROs in cancer patients. Methods: To develop the emoji scale, 5 emoji faces were selected from the Apple emoji dictionary by study authors. 20 cancer patients participated in a card-sorting exercise to rank the emoji from negative to positive; 19 out of 20 selected the same order, which exceeded Coomb’s criteria for being a valid ordinal scale. Patients were recruited from Mayo Clinic practices. Eligibility included: adult patients with a diagnosis of lymphoma, multiple myeloma, brain, pancreatic, breast, and ovarian cancer within the past 5 years; life expectancy of > 6 months; and own an iPhone version ≥ 5.0. All patients were provided with an Apple Watch and downloaded the study app. Baseline PRO data was collected on the iPhone, including single-item linear analog self-assessments (LASA) and emoji scales for fatigue, physical function, emotional well-being, and overall QOL. Results: From 2/2017-8/2017, 294 eligible patients were recruited. 25% were male, and median age was 53 years (range 20-79). The median time since diagnosis was 8.4 months (range 0-60); 63% were on active therapy, 27% were in observation post-treatment, and 10% had not been treated. All patients were “extremely” confident filling out forms, 99% had used a computer in the past 12 months, and 27% had ever used a smart watch. Emoji responses were significantly associated with validated measures of PROs; the Spearman correlations between the ordinal emoji scales and LASAs were -0.83 for fatigue, 0.72 for physical well-being, 0.72 for emotional well-being, and 0.77 for overall QOL (all p < 0.0001). Of the 75 patients who have completed the study thus far, 92% reported they would use emoji scales again and 89% would recommend that others use emoji scales. Conclusions: A novel ordinal emoji scale is strongly associated with accepted measures of PROs in cancer patients and should be studied further.

Online scholarly tools: Use and value

ABSTRACTIn this paper, we describe our preliminary findings of our multi‐site study on researchers' use of online scholarly tools. Using surveys and in‐depth interviews, our study will provide information professionals and scholars a 360‐degree panorama of the ways social media is impacting the dissemination and evaluation of scholarly research. At the visual presentation session, we will present the statistical results and analysis of our data, as well as individual case stories exemplifying the main findings. Additionally, we will have an activity where our audience can participate in the same card sorting exercise that study subjects did, and compare their results with the study participants, to better understand the value of various scholarly activities.

Experience of Symptoms and Disease Impact in Patients with Adenomyosis.

BackgroundAdenomyosis is a poorly understood, benign disease of the uterus.ObjectiveIn this study, patient interviews were conducted to characterize the symptoms and impact of adenomyosis.MethodsThis was a cross-sectional study in which women with adenomyosis were recruited from five US clinics and a health-related social network forum. Participants (aged 18–55 years) were pre-menopausal with a history of regular menstrual cycles. Participants were interviewed about their experiences with adenomyosis, symptoms and impacts on day-to-day activities (concept elicitation), and subsequently about the occurrence, relative severity, and impact of symptoms (card-sorting exercise).ResultsIn total, 31 women were interviewed. Mean duration since onset of first adenomyosis symptom was 5.7 years; 41.9% reported severe/very severe adenomyosis. Over 50 symptoms and 30 impacts of adenomyosis were reported in the concept elicitation; 87% of symptoms were reported after 7 interviews and 78% of impacts after 5 interviews, indicating a condition with a significant symptom burden and a consistent presentation. The most common symptoms were heavy menstrual bleeding (87%), cramps (84%), and blood clots during menstrual bleeding (84%). The most common impacts were burdensome self-care hygiene (71%), and fatigue/low energy (71%). In the card-sorting exercise, the most commonly endorsed symptoms were pain during menstruation/menstrual cramps and heavy menstrual bleeding (both frequently rated as severe). The symptom with the highest impact was heavy menstrual bleeding.ConclusionInitiatives to understand women’s experiences with adenomyosis may improve management of the condition. This study provides a first step in understanding their experience and new information on the symptom profile of adenomyosis.Electronic supplementary materialThe online version of this article (10.1007/s40271-017-0284-2) contains supplementary material, which is available to authorized users.

Understanding Adolescents' Categorisation of Animal Species.

Simple SummaryWhen people try to make sense of the world they often use categorisations, which are seen as a basic function of human cognition. People use specific attributes to categorise animals with young children using mostly visual cues like number of legs, whereas adults use more comprehensive attributes such as the habitat that the animal lives in. The aim of the present study was to investigate how adolescents categorise different types of animals. A card sorting exercise in combination with a survey questionnaire was implemented. Adolescents were asked to group images of a variety of common British farm, pet, and wild animals that were printed on cards. Furthermore, adolescents were asked to rate a number of animals regarding their utility, likability, and fear, which served as affective responses. Results show that adolescents primarily use an animal’s perceived utility as a means for their categorisation along with their affective feelings towards those animals. In other words, adolescents group animals into farm, pet, and wild animals with one exception, birds. Birds, regardless of their role in society (pet, farm, or wild animal), were mostly grouped together. The results are important to understand adolescents’ perception of animals, which may explain the different attitudes and behaviours towards animals.Categorisations are a means of investigating cognitive maps. The present study, for the first time, investigates adolescents’ spontaneous categorisation of 34 animal species. Furthermore, explicit evaluations of 16 selected animals in terms of their perceived utility and likeability were analysed. 105 British adolescents, 54% female, mean age 14.5 (SD = 1.6) participated in the study. Results of multidimensional scaling (MDS) techniques indicate 3-dimensional data representation regardless of gender or age. Property fittings show that affect and perceived utility of animals explain two of the MDS dimensions, and hence partly explain adolescents’ categorisation. Additionally, hierarchical cluster analyses show a differentiation between farm animals, birds, pet animals, and wild animals possibly explaining MDS dimension 3. The results suggest that utility perceptions predominantly underlie adolescents’ categorisations and become even more dominant in older adolescents, which potentially has an influence on attitudes to animals with implications for animal welfare, conservation, and education.

Deconstructing Cancer Patient Information Seeking in a Consumer Health Library Toward Developing a Virtual Information Consult for Cancer Patients and Their Caregivers: A Qualitative, Instrumental Case Study.

BackgroundCancer patients and their caregivers want information about their disease and are interested in finding health information online. Despite the abundance of cancer information online, it is often fragmented, its quality is highly variable, and it can be difficult to navigate without expert-level knowledge of the cancer system. The Patient & Family Library at the Princess Margaret Cancer Centre offers a broad collection of high-quality cancer health information and staff are available to help patrons refine their questions and explore information needs that they may not have considered.ObjectiveThe purpose of this research study was to deconstruct patrons’ information-seeking behaviors in the library to assess the feasibility of replicating the services provided in the library through a Web app, extending the service beyond the walls of the cancer centre. The specific aims of this research were to understand (1) how patrons approach information seeking in the library (interface design), (2) how patrons communicate their informational needs (information categorization and metadata requirements), and (3) what resources are provided to address the patrons’ information needs (collection development).MethodsWe employed a qualitative, instrumental case study to deconstruct patrons’ health information-seeking behavior. The study population included patients, the librarian, and library volunteers. Ethnographic observation was conducted at the library over 3 days and key informant interviews with library staff were conducted to address the first aim. A closed card-sorting activity was conducted to address the second aim and the library shift logs and Search Request Forms (SRFs) were reviewed to address the third aim.ResultsA total of 55 interactions were recorded during the ethnographic observation and nine semistructured interviews were conducted during the key informant interviews. Seven library patron personas were identified: (1) Newbie, (2) Seasoned, (3) Direct, (4) Window Shopper, (5) Collector, (6) Information Seeker, and (7) Distressed. A total of 83 participants completed the closed card-sorting exercise. The participants’ conceptual clusters within the similarity matrix overlapped with the groupings created by the librarian, with a few differences. A total of 161 entries in the library shift log and 65 SRFs were analyzed to determine what resources were given to patrons. Most resources that patrons received were available online (61%), although almost half of these required special access (47%).ConclusionsThe study findings suggest it is possible to replicate library functions in a Web app with a few exceptions that cannot be replicated online. These elements include access to journal articles or other content behind paywalls and the librarian’s ability to encourage further discussion through empathy and active listening. Discussion with the librarian could serve to refine and predict needs through observing information seekers and to provide immediate connection to spiritual care and psychosocial support for patrons in distress.

Assessing the health information source perceptions of tweens using card-sorting exercises

As young people are increasingly turning to the Internet to meet their information needs, it is imperative to investigate their perceptions regarding various potential sources of health information. A series of card-sorting exercises were administered to new participants in an after-school programme (HackHealth) to find out which sources of health information these greater Washington DC metro area middle school students would turn to, which they would not and their reasons behind these judgements. The findings revealed that participants were very aware of the importance of trustworthiness when looking for health information and they valued both professional expertise based on formal education and expertise born of personal experience with a particular health condition. However, they also valued convenience, ease and speed, and sometimes sacrificed information quality. Some important implications of these findings for healthcare and information professionals are identified and suggestions for future research in this area are offered.

Children's perception on healthy snacks using a card sorting exercise in Aberdeenshire nursery and primary schools

Children's perception on healthy snacks using a card sorting exercise in Aberdeenshire nursery and primary schools

Towards a Taxonomy for Classifying Intuitive Usability Prediction Strategies in Non-Experts

In order to compare the strategies consumers use to make predictions about the ease-of-use of products, the authors conducted a card sorting exercise with subject-matter experts (SMEs) to refine a coding scheme from current, widely-used usability definitions and measurement tools. Nine SMEs sorted 83 cards containing aspects found in usability definitions, guidelines, measurement tools and descriptions. The resulting fifteen clusters define a coding scheme that is both comprehensive and flexible. The refined taxonomy does not seek to redefine usability, but to provide a mechanism for researchers wanting to use a coding scheme that includes methods for classifying strategies such as usability attribution to system features, metacognitive behaviors (planning and evaluating), and exemplar-based strategies (memory for performing similar tasks).

From everyday communicative figurations to rigorous audience news repertoires: A mixed method approach to cross-media news consumption

In the last couple of decades there has been an unprecedented explosion of news media platforms and formats, as a succession of digital and social media have joined the ranks of legacy media. We live in a ‘hybrid media system’ (Chadwick, 2013), in which people build their cross-media news repertoires from the ensemble of old and new media available. This article presents an innovative mixed-method approach with considerable explanatory power to the exploration of patterns of news media consumption. This approach tailors Q-methodology in the direction of a qualitative study of news consumption, in which a card sorting exercise serves to translate the participants’ news media preferences into a form that enables the researcher to undertake a rigorous factor-analytical construction of their news consumption repertoires. This interpretive, factor-analytical procedure, which results in the building of six audience news repertoires in Denmark, also preserves the qualitative thickness of the participants’ verbal accounts of the communicative figurations of their day-in-the-life with the news media.

Development of a global hospital self-assessment tool and prioritization tier system based on FIP's Basel Statements.

The objective of this study was to develop and pilot a hospital self-assessment tool and prioritization system for hospitals to evaluate their pharmacy practice performance against the International Pharmaceutical Federation (FIP)'s Basel Statements. To develop the self-assessment tool, we reviewed the Basel statements to identify characteristics that could be measured at the individual hospital level and finalized these based on feedback from five cognitive interviews. The survey instrument was pilot tested in four countries; two high-, one lower middle- and one low-income country. Participants were electronically sent the survey tool and enrolled through the snowball sampling method. Tiers were developed by investigators to assist hospitals in prioritizing the achievement of Basel Statement practices. Tiers were validated by the Hospital Section chairs of FIP through a card sorting exercise. Simple agreement and Cohen's kappa was used to characterize inter-rater reliability. Descriptive statistics were used to characterize the responses. Forty-four survey responses were collected from 36 hospitals in four countries. The survey response rate was 29% and took an average of 26 minutes to complete. The overall average agreement of constructs was 83%. The survey characterized how far a hospital was to achieving the Basel Statement practices with an average achievement rate of 57% (ranging from 30% to 90%). The survey highlighted medication safety challenges facing the pharmacy profession. The results produced a benchmarking report for each respondent. The results support the validity and reliability of the survey tool. This comprehensive survey tool can be used by institutions to track their progress towards achieving the Basel Statement practices.

Is there a preference for PET or SPECT brain imaging in diagnosing dementia? The views of people with dementia, carers, and healthy controls

Positron emission tomography (PET) and single photon emission computed tomography (SPECT) brain imaging are widely used as diagnostic tools for suspected dementia but no studies have directly compared participant views of the two procedures. We used a range of methods to explore preferences for PET and SPECT. Patients and controls (and accompanying carers) completed questionnaires immediately after undergoing PET and SPECT brain scans. Pulse rate data were collected during each scan. Scan attributes were prioritized using a card sorting exercise; carers and controls additionally answered willingness to pay (WTP) questions. Few differences were found either between the scans or groups of participants, although carers marginally preferred SPECT. Diagnostic accuracy was prioritized over other scan characteristics. Mean heart rate during both scans was lower than baseline heart rate measured at home (p < 0.001). Most participants viewed PET and SPECT scans as roughly equivalent and did not have a preference for either scan. Carer preference for SPECT is likely to reflect their desire to be with the patient (routine practice for SPECT but not for PET), suggesting that they should be able to accompany vulnerable patients throughout imaging procedures wherever possible. Pulse rate data indicated that brain imaging was no more stressful than a home visit (HV) from a researcher. The data do not support the anecdotal view that PET is a more burdensome procedure and the use of PET or SPECT scans in dementia should be based on diagnostic accuracy of the technique.

From Crud to Cream: Imagining a Rich Scholarly Repository Interface

This article addresses the design of a dynamic repository interface to support numerous scholarly activities. Starting with the four fundamental functions associated with persistent storage — create, read, update, and delete (CRUD) — we tested, as an organizing rubric for the interface, the acronym CREAM: Create (represent, illustrate); Read (sample, read); Enhance (refer, annotate, process); Analyze (search, select, visualize, mine, cluster); and Manage (track, label, transform). Based on a card-sorting exercise conducted with researchers, we conclude that a slightly modified rubric of CREAMS offers a useful starting point that emphasizes the enriched functionality a scholarly repository or similarly complex digital environment requires, as well as the immense challenge of designing conceptually clear interfaces, even for a relatively homogenous community of researchers.

Designing the Information Architecture of Governmental One-Stop Portals

The latest paradigm shift in government, termed Transformational Government, puts the citizen in the centre of attention. Including citizens in the design of online one-stop portals can help governmental organisations to become more customer focussed. This study describes the initial efforts of an Australian state government to develop an information architecture to structure the content of their future one-stop portal. Hereby, card sorting exercises have been conducted and analysed, utilising contemporary approaches found in academic and non-scientific literature. This paper describes the findings of the card sorting exercises in this particular case and discusses the suitability of the applied approaches in general. These are distinguished into non-statistical, statistical, and hybrid approaches. Thus, on the one hand, this paper contributes to academia by describing the application of different card sorting approaches and discussing their strengths and weaknesses. On the other hand, this paper contributes to practice by explaining the approach that has been taken by the authors’ research partner in order to develop a customer-focussed governmental one-stop portal. Thus, they provide decision support for practitioners with regard to different analysis methods that can be used to complement recent approaches in Transformational Government.

Homicide Detectives' Intuition

AbstractLittle is known about the decision‐making processes of homicide detectives; this study is a first step towards understanding the inferential processes they engage in during the initial stages of an investigation. A card sorting exercise consisting of 20 crime scene photographs showing homicide victims in situ was designed to examine how detectives categorise and conceptualise homicide crime scenes. Forty homicide detectives were asked to ‘think aloud’ whilst categorising the crime scenes. Qualitative content analysis of the ‘think aloud’ accounts revealed detectives' intuitive ability to automatically make detailed inferences regarding the circumstances surrounding each homicide on the basis of available crime scene information (victim's sex, location, and method of death). A cycle of cognition was evident whereby detectives drew upon the contextual information available to generate hypotheses regarding homicide type (domestic, male brawl, and crime‐related) and derive inferences about victim–offender relationship, offender behaviour, motive, and whether the offence was spontaneous or planned. The detectives made 594 inferences of which 67% (N = 398) were accurate. The sorting exercise proved to be ecologically valid, with detectives reporting that they assessed and interpreted the homicide crime scenes, as they would in an actual investigation. The implications for police training and ‘offender profiling’ research are discussed. Copyright © 2013 John Wiley &amp; Sons, Ltd.