A Research Agenda for Playful Systems Design

Purpose Commercial aviation is feasible thanks to the complex socio-technical air transportation system, which involves interactions between human operators, technical systems, and procedures. In view of the expected growth in commercial aviation, significant changes in this socio-technical system are in development both in the USA and Europe. Such a complex socio-technical system may generate various types of emergent behavior, which may range from simple emergence, through weak emergence, up to strong emergence. The purpose of this paper is to demonstrate that agent-based modeling and simulation allows identifying changed and novel rare emergent behavior in this complex socio-technical system. Methods An agent based model of a specific operation at an airport has been developed. The specific operation considered is the controlled crossing by a taxiing aircraft of a runway that is in use for controlled departures. The agent-based model includes all relevant human and technical agents, such as the aircraft, the pilots, the controllers and the decision support systems involved. This agent-based model is used to conduct rare event Monte Carlo (MC) simulations. Results The MC simulation results obtained confirm that agent based modeling and simulation of a socio-technical air transportation system allows to identify rare emergent behavior that was not identified through earlier, non-agent-based simulations, including human-in-the-loop simulations of the same operation. A typical example of such emergent behavior is the finding that alerting systems do not really reduce the safety risk. Conclusions Agent based MC simulations of commercial aviation operations has been demonstrated as a viable way to be evaluated regarding rare emergent behaviour. This rare emergent behaviour could not have been found through the more traditional simulation approaches.

Due to its tightly coupled structure, intractable and unpredictable workflow and complex interaction of its elements, the construction industry’s work environment (i.e., vertical, horizontal or underground construction projects) is acknowledged as a complex sociotechnical system (STS). STS theory studies the interrelationship of components within a complex system rather than inquiring about its integral components in isolation. To expand knowledge and application of the STS perspective in construction safety management, this study generates a new (STS thinking based) conceptual model through systematic review and analysis of extant literature on construction safety management. An overarching epistemological design embraced interpretivist, critical realism and inductive reasoning to analyse bibliometric data of 2,227 articles retrieved from the Scopus database. Thematic analysis of bibliometric data that was conducted through manual screening of ‘titles’, ‘abstracts’ and ‘keywords’ revealed eleven distinctive thematical clusters of research activity within the prevailing body of knowledge, viz.: ‘contractor relationship’, ‘lean management’, ‘policy and regulations’, ‘prevention through design’, ‘safety culture’, ‘safety management strategies’, ‘safety of machinery’, ‘safety of sites’, ‘safety of workers’, ‘sustainability and waste management’ and ‘technology’. From these aforementioned clusters, ‘safety of workers’, ‘safety of machinery’ and ‘safety of sites’ thematic clusters were selected for further content analysis using the NVivo qualitative data analysis tool. The results from both thematic and content analysis form the basis of a novel theoretical conceptual model (entitled ‘safety-in-cohesion’) that provides palpable insights and guidance for academics and industry practitioners to reflect upon the design, development and analysis of safety management systems based on a holistic perspective of both social and technical aspects. Specifically, the ‘safety-in-cohesion’ model: combines pertinent elements of Safety-I and Safety-II principles that apply to construction safety management. It also promotes continuous learning (through relational loop between design and construction stages) based on integration of different project stakeholders, different data types (i.e., lagging and leading indicators) and different work modes (i.e., reactive and proactive). Emergent findings and concomitant polemic discussion present an unconventional perspective on construction safety within a complex (and often bespoke) STS system on site and generate new avenues of research direction for scholars.

Statement of a problem. One of the main tasks today is to prevent accidents in complex systems, which requires determining their cause. In this regard, several theories and models of the causality of accidents are being developed. Traditional approaches to accident modeling are not sufficient for the analysis of accidents occurring in complex environments such as socio-technical systems, since an accident is not the result of individual component failure or human error. Therefore, we need more systematic methods for the investigation and modeling of accidents. Purpose. Conduct a comparative analysis of accident models in complex systems, identify the strengths and weaknesses of each of these models, and study the feasibility of their use in risk management in socio-technical systems. The paper analyzes the main approaches of accident modeling and their limitations in determining the cause-and-effect relationships and dynamics of modern complex systems. the methodologies to safety and accident models in sociotechnical systems based on systems theory are discussed. The complexity of sociotechnical systems requires new methodologies for modeling the development of emergency management. At the same time, it is necessary to take into account the socio-technical system as a whole and to focus on the simultaneous consideration of the social and technical aspects of the systems. When modeling accidents, it is necessary to take into account the social structures and processes of social interaction, the cultural environment, individual characteristics of a person, such as their abilities and motivation, as well as the engineering design and technical aspects of systems. Practical importance. Based on analyzing various techniques for modeling accidents, as well as studying the examples used in modeling several previous accidents and review the results of this modeling, it is concluded that it is necessary to improve the modeling techniques. The result was the appearance of hybrid models of risk management in socio-technical systems, which we will consider in detail in our next work.

Sociotechnical systems consisting of infrastructures, businesses, and households are constantly expanding and evolving due to, among others, rapidly progressing economic development, urbanization, and globalization. These complex interdependent cybernetic systems become increasingly prone to both natural and man-made disruptions. Such complex systems exhibit emergent effects in response to any disruption which propagates throughout those systems. Consequently, using traditional methods of risk assessment of individual systems is insufficient to predict the emergent systemic impacts of disruptive events of the future. Hence, there is an urgent need to develop models assessing systemic risk of such complex interdependent sociotechnical systems. The aim of the presented study was to (1) present and apply a framework for modeling interdependencies between complex sociotechnical systems at different levels of detail, with a particular focus on urban areas; (2) develop disruption generators and devise the method of assessing impact of disruptions on the systems; and (3) demonstrate how the modeling framework can be applied to assess systemic risks. Our study resulted in the development of a simulation combining socioeconomic agents’ models, such as households and businesses, with physical infrastructure systems models to assess systemic risks, reliability and safety associated with interdependencies in such sociotechnical systems. The disruption cost served as a measure of performance of the system used to assess the risk. Our model was shown to capture the emergent behavior of complex sociotechnical systems in response to disruptions. Based on our analysis, the most risk was associated with low-reconfigurability infrastructure systems, particularly disruptions to the water supply system, resulted in largest impacts. We also identified areas for future research that focus on including a wider range of systems, more accurate disruption generators, and on applying the presented modeling framework to other complex networks and sociotechnical systems.

This paper introduces two novel applications of systemic design to facilitate a comparison of alternative methodologies that integrate systems thinking and design. In the first case study, systemic design helped the Procurement Department at the University of Toronto re-envision how public policy is implemented and how value is created in the broader university purchasing ecosystem. This resulted in an estimated $1.5 million in savings in the first year, and a rise in user retention rates from 40% to 99%. In the second case study, systemic design helped the clean energy and natural resources group within the Government of Alberta to design a more efficient and effective resource management system and shift the way that natural resource departments work together. This resulted in the formation of a standing systemic design team and contributed to the creation of an integrated resource management system. A comparative analysis of the two projects identifies a shared set of core principles for systemic design as well as areas of differentiation that reveal potential for learning across methodologies. Together, these case studies demonstrate the complementarity of systems thinking and design thinking, and show how they may be integrated to guide positive change within complex sociotechnical systems.

Many have conceptualized caregiver robots as consumer products and studied elders’ perceived needs afor and preferences about such products. For reviews, please see (Broadbent, Stafford, & MacDonald, 2009; Jones & Schmidlin, 2011). That approach, though, could create robots that cannot satisfy elders’ actual caregiving needs. Alternatively, one can conceptualize caregiver robots as workers in complex socio-technical systems. To do so, one would need a detailed account of the caregiving that takes place in elders’ homes. Unfortunately, as noted in a National Research Council (2011) report, such a detailed account of caregiving does not exist. Accordingly, we sought to develop such an account. There are many ways to analyze work (for a discussion of general approaches, see Vicente, 1999). They can be categorized into 3 general types: normative, descriptive, and formative approaches (Vicente, 1999). We adopted a formative approach because formative approaches are tailored to the analysis of complex socio-technical systems (Vicente, 1999). They capture work requirements without specifying how that work must be done or who must do it. For example, the constraint “must not lose track of time” captures a work requirement but allows the associated work to be accomplished in a number of different ways (e.g., by checking a clock, setting an alarm) and by a number of different entities (e.g., family member, caregiver robot). To conduct our analysis, researchers observed caregiving in elders’ homes, and interviewed caregivers about their work activities. Researchers then organized their findings into an Abstraction Hierarchy (AH; Vicente, 1999), that is, a detailed account of the aging in place socio-technical system. Our primary aim was to create an AH that describes means-ends relations between the complex socio-technical caregiving system’s overall objectives, work tasks, and physical resources. Such a description provides a detailed account of the caregiving work domain, and serves as the foundation for subsequent formative analyses of caregiving. To create the AH, research team members completed 4 steps: 1) analyzing existing caregiving documentation, 2) observing caregiving and interviewing caregivers, 3) drafting and/or refining the AH, and 4) validating the AH. Steps 2 and 3 were iterative. This process is consistent with Naikar, Hopcraft, and Moylan’s (2005) recommendations regarding formative analyses. The AH made clear that caregiving for those who age in place is a complex and nuanced activity. More specifically, our analysis confirmed existing research regarding categories of caregiving tasks and revealed aspects of caregiving that have not been detailed so far. The existing literature indicates that caregivers assist older adults with self-maintenance activities of daily life (ADLs), such as eating, toileting, and dressing (Lawton, 1990), instrumental activities of daily life (IADLs), such as cooking, cleaning, and shopping (Lawton, 1990), and enhanced activities of daily life (EADLs), such as participating in social activities and pursuing hobbies (Rogers, et al., 1998). Our analysis confirmed those findings, and our AH provides a more detailed account of those tasks than was previously available. Our analysis also revealed aspects of caregiving for those who are aging in place that have not been detailed thus far in the research literature. For example, our AH contains a purpose-related function called Counseling, which concerns ensuring that the elder does not experience psychological distress. To perform this function, the caregiver must understand the elder’s situation (e.g., a family conflict), use information about that situation (e.g., experience with relevant family members and/or past conflicts; the elders’ past choices), and offer the elder advice about how to proceed (e.g., which family member’s advice to follow). The main implication of our AH for the design of caregiver robots is that such robots cannot be designed to perform purpose-related functions in a one-size-fits-all way; rather, caregiver robots must exhibit context-conditioned variability (Vicente, 1999). Our AH has many other important implications for the design of caregiver robots, which unfortunately cannot be detailed here due to space constraints.

Towards a Meta-model for the Description of the Sociotechnical Perspective on Product-service Systems

Purpose: In order to improve the safety, capacity, economy, and sustainability of air transportation, revolutionary changes are required. These changes might range from the introduction of new technology and operational procedures to unprecedented roles of human operators and the way they interact. Implementing such changes can introduce both negative and positive emergent behaviour. i.e. behaviour that arises from the interactions between system entities as proposed in innovative concepts. Currently, the inability to understand and control such behaviour prevents us from avoiding undesired negative emergent behaviours and promoting positive ones. In order to address this problem, this thesis aims to understand emergent behaviour in the complex socio-technical air transportation system. Methods: The thesis proposes Agent-Based Modelling and Simulation (ABMS) as a method for capturing emergent behaviour of the socio-technical air transportation system, and evaluating novel system designs. The popularity of ABMS is driven by its capability of handling the increasing complexity of real world socio-technical systems that exhibit emergent behaviour. This thesis focuses on two main applications namely: 1) the identification of emergent safety risk of an active runway crossing operation; and 2) the evaluation of the role of coordination in Airline Operations Control (AOC) resilience. In both applications, ABMS has emerged as a key method because it is widely used in complexity science to understand how interactions give rise to emergent behavior. The agent-based models include all relevant human and technical agents, such as pilots and controllers and the decision support systems involved. Simulation of these agents interacting together is conducted to predict the impact of both existing and future concepts of operation. Results: The applications in this thesis highlight that ABMS has the capability to reveal unexpected emergent behaviour and provide novel insights in air transportation. For the airport safety application, various types of emergent behaviour have been revealed due to the development and simulation of the agent-based model that covers the totality of interactions of components and their variability in performance over time. The Monte Carlo simulations make it possible to understand the potential of agents in restricting the risk in off-nominal scenarios, through capturing their stochastic nature and accounting for uncertainty. For the airline resilience application, novel insights were gained about the role of coordination in airline resilience. Capitalizing on established airline practice and research about human coordination from the psychology domain, the agent-based simulations evaluated the operational effects of AOC coordination policies on a challenging disruption scenario. Conclusions & possible applications and implications: This thesis demonstrates that ABMS of air transport operations is a viable approach in gaining knowledge about emergent behaviour which was unknown before. This knowledge includes both bottlenecks of system designs and identified opportunities, and hence can be used to control and further optimize the socio-technical air transportation system. This also implies that ABMS can be a cost-effective method for evaluating new concepts during the early design phase of air transport operations.

The impacts of lean production on the complexity of socio-technical systems

Traditional approaches to cyber-security resilience, assuring the overall socio-technical system is secure from immediate known attacks and routes to potential future attacks, have relied on three pillars of people, process, and technology.In any complex socio-technical system, human behaviour can disrupt the secure and efficient running of the system with risk accumulating through individual and system-wide errors and compromised security behaviours that may be exploited by actors with malicious intent.Practitioners’ experience and use of different assessment methods and approaches to establish cyber-security vulnerabilities and risk are evaluated. Qualitative and quantitative methods and data are used for different stages of investigations in order to derive risk assessments and access contextual experience for further analyses. Organisational security culture and development approaches along with safety assessment methods are discussed in this case study to understand how well the people, the system, and the organisation interact.Cyber-security Human Factors practice draws on other application areas such as safety, usability, behaviours and culture to progressively assess security posture; the benefits of each approach are discussed.This study identifies the most effective methods for vulnerability identification and risk assessment, with focus on modelling large, dynamic and complex socio-technical systems, to be those which identify cultural factors with impact on human-system interactions.KeywordsHuman factorsCyber-securityBehavioural scienceOrganisational cultureSecurity cultureCyber- resilienceSocio-technicalSafety assessmentClimateBehavioralOrganizational cultureSocio-behaviouralSociotechnical

Health care increasingly depends on information and communication technology. This offers both opportunities and challenges when designing connected health systems. While individual studies examined particular cases, there is a limited synthesis of insights across projects. The objective of this paper is to explore these opportunities and challenges by examining 6 diverse connected health projects and synthesizing lessons from an expert workshop. To achieve this, we conducted a full-day workshop that brought together 6 connected health projects. The workshop used an iterative and participatory process which included paper submissions and presentations and facilitated discussions, and a gallery walk to enable cross-case comparison and collaborative reflection. Thematic analysis of workshop outputs was then used to synthesize key opportunities and challenges in designing connected health systems. The 6 projects represented a variety of design methods and approaches to connected health, and their discussion surfaced both opportunities and challenges in this domain. Key opportunities include improving data integration and usability, enhancing collaboration across stakeholders, using a user-centered and iterative design process, addressing complexity in sociotechnical systems, sustainability, and adopting digital infrastructures for seamless communication. Participants also identified important challenges, namely exchange of information, interoperability, and communication; ethical considerations, rules, and regulations; understanding design, evaluation, and standards; actionable data, reliability, quality, and trust in data; and stakeholder involvement. The contribution of this paper lies in the synthesis of insights across multiple projects and perspectives to provide practical guidance for researchers, designers, and policymakers. By highlighting opportunities and challenges in designing connected health systems, the findings emphasize the importance of patient-centered, sustainable, and collaborative design approaches while also pointing to the need to address persistent barriers. Advancing connected health will require adopting iterative and inclusive design processes that prioritize patient-centeredness, sustainability, and collaboration across health care systems.

A Qualitative-Quantitative Resilience Assessment Approach for Socio-technical Systems

Pedestrian safety at rail level crossings (RLXs) is a concern for government, the transport industry and the community. Collisions between trains and pedestrians are traditionally viewed as the result of errors or violations committed by pedestrians. However, as RLXs are complex sociotechnical systems, collisions are better understood as emergent properties of interactions amongst and between human and technical components within the system. Cognitive work analysis (CWA) is a powerful analytical framework that offers a sophisticated understanding of the functioning of RLXs as relevant to pedestrian safety, through identifying the constraints on pedestrian behaviour in this context. It does not, however, provide its users with guidance about how the findings of the analysis can be used to improve sociotechnical system functioning. Accordingly, the aim of this research was the development and evaluation of a CWA-based approach to support the design of complex sociotechnical systems, and the application of this approach to provide recommendations for RLX design to improve pedestrian safety. A secondary aim of the research was to investigate pedestrian behaviour within the RLX system using CWA. Drawing on systems theory, and more specifically, sociotechnical systems theory, the outcome of this research is a CWA design toolkit (the CWA-DT). The development and refinement of the CWA-DT is illustrated through a proof of concept application in the domain of public transport ticketing. Positive evaluation results were obtained and necessary refinements to the toolkit were implemented in the second version. A full evaluation of the toolkit was then undertaken within the complex, safety critical domain of RLXs. This evaluation found that while the CWA-DT could be considered a useful method, its application did not lead to the creation of design concepts that fully aligned with sociotechnical systems theory. The application did, however, lead to designs that were rated by human factors experts as more effective than the existing system design. These findings suggest that the sociotechnical systems theory approach may not provide an appropriate design philosophy in a public safety context. However, merging ideas and concepts from sociotechnical systems theory with existing paradigms such as safety management can lead to innovation and has the potential to improve safety performance. Awards: Winner of the Mollie Holman Doctoral Medal for Excellence, Monash Injury Research Institute, 2015.

Complex sociotechnical systems, e.g. in process, nuclear power plant, shipping, are required to perform challenging operations, and comprise a combination of interdependent technical and human elements. The performance assessment of such systems calls for a systematic design to gather and analyse the retrieved data. This involves using precise indicators to ensure that essential system elements are measured accurately. The current work reviews the last decade use of indicators in psychology and engineering research, which are essential scientific domains of sociotechnical systems. The paper introduces a conceptual framework of indicators featuring four categories: (i) individual and abstract, (ii) individual and concrete, (iii) team and abstract, and (iv) team and concrete contents. The framework provides a way of systemising the indicators to provide a consistent foundation for future research towards a systematic performance assessment of sociotechnical systems. It may increase the precision of the performance indicators that is used to assess complex sociotechnical systems.

The complexity of socio-technical systems (SS) requires tools that facilitate the understanding of their behaviour and daily performance under the perspective of safety management, therefore Resilience Engineering (RE) and its tools can help with this purpose. In this study, a literature review about the application of the Resilience Assessment Grid (RAG) in different sectors in the last decade is performed. The information was selected from the following databases: Scopus, Wos, ScienceDirect and PubMed, guided by the PRISMA methodology using search criteria by title, abstract and keywords. The most used basic tool is the RAG, which applied individually or in combination helps to understand the behaviour of these complex socio-technical systems (CSS) under the perspective of safety management. The tools used in the last decade in RE studies with the RAG in different sectors are presented chronologically. Safety-II could be considered as a suitable management system at present, where the RE helps to measure resilient performance potential in the CSS, but not resilience per se, using the RAG individually or in combination, with the basic number of questions or structured in a customised way.KeywordsSafety ISafety IIComplexityResilience engineeringTools