End-user Perspective Research Articles

School and childcare facility air quality decision-makers’ perspectives on using low-cost sensors for wildfire smoke response

BackgroundDuring wildfire smoke episodes, school and childcare facility staff and those who support them rely upon air quality data to inform activity decisions. Where ambient regulatory monitor data is sparse, low-cost sensors can help inform local outdoor activity decisions, and provide indoor air quality data. However, there is no established protocol for air quality decision-makers to use sensor data for schools and childcare facilities. To develop practical, effective toolkits to guide the use of sensors in school and childcare settings, it is essential to understand the perspectives of the potential end-users of such toolkit materials.MethodsWe conducted 15 semi-structured interviews with school, childcare, local health jurisdiction, air quality, and school district personnel regarding sensor use for wildfire smoke response. Interviews included sharing PM2.5 data collected at schools during wildfire smoke. Interviews were transcribed and transcripts were coded using a codebook developed both a priori and amended as additional themes emerged.ResultsThree major themes were identified by organizing complementary codes together: (1) Low-cost sensors are useful despite data quality limitations, (2) Low-cost sensor data can inform decision-making to protect children in school and childcare settings, and (3) There are feasibility and public perception-related barriers to using low-cost sensors.ConclusionsInterview responses provided practical implications for toolkit development, including demonstrating a need for toolkits that allow a variety of sensor preferences. In addition, participants expected to have a wide range of available time for monitoring, budget for sensors, and decision-making types. Finally, interview responses revealed a need for toolkits to address sensor uses outside of activity decisions, especially assessment of ventilation and filtration.

Developer and End-User Perspectives on Addressing Human Aspects in Mobile eHealth Apps

Context:eHealth apps are mobile apps that help in self-management of critical illnesses, provide home-based disease management, and help with personalized care. Users of eHealth apps are naturally very diverse in terms of their human aspects, e.g., their age, gender, emotional reactions to the apps, cognitive style, physical and mental challenges. Unfortunately, many eHealth apps do not take these user differences sufficiently into account, making them ineffective or even unusable. Objective:This paper reports a study from eHealth app stakeholders’ – developers and end-users – perspectives on critical challenges and benefits of better incorporating human aspects into eHealth app development and usage. We also investigate how different human aspects are being addressed by developers, which ones are the most important for different user groups, and which ones are currently missing/poorly handled. Method:A mixed-method approach that integrates qualitative and quantitative research was used for this study. We gathered and analyzed data from 240 online survey responses and 25 detailed interviews within the same study and validated the results. Results:We report key issues encountered in eHealth app design, difficulty in addressing different human aspects, areas requiring further research and practical assistance, and recommend our findings to best address these challenges. We found addressing human aspects throughout the app development life-cycle is beneficial for more effective eHealth apps. Our findings also suggest the need for improved standards and guidelines, better developer-user collaborative culture, and better human aspects education to produce more effective eHealth apps. Conclusion:This paper investigates current approaches used in the eHealth app domain that take into account the human aspects of app users. The paper guides eHealth app stakeholders, future researchers, academia and industry partners be aware of human aspects related challenges and improve produce apps.

Who Prefers Legal Wood: Consumers with Utilitarian or Hedonic Shopping Values?

Although certification is perceived to be beneficial for enhancing forest sustainability and open access to green markets, certification practices in Indonesia face controversy, particularly in its wood-based industry. We aim to approach this issue from the end-user perspective. Drawing on the theories of value-belief-norm and planned behavior, we examine the psychological aspects of consumers toward legal wood consumption. A survey of 515 consumers showed that individuals with hedonic values tended to have a high perception of green values toward legal wood. Also, when consumers’ hedonic values dominated over their utilitarian consumption, their perception of green values toward legal wood tends to be higher. Based on these results, wood marketers could benefit from directing their communication efforts toward emphasizing the hedonic worth of the product, as opposed to its utilitarian values. It is imperative for distributors and promoters of wood products to carefully deliberate on strategies to effectively elicit the hedonic shopping values that are inherently linked to the utilization of such green products. An illustration can be represented by the case of IKEA in Indonesia. Consumers are probably attracted to IKEA’s neuromarketing strategy, such as their attractive display and labyrinth effect, without realizing that IKEA also applies green marketing and supports green products.

Unraveling behavioral factors influencing the adoption of urban air mobility from the end user's perspective in Tehran – A developing country outlook

The integration of shared, autonomous, electric, and on-demand mobility services introduces urban air mobility (UAM) as an emerging intra-city transportation service operated by vertical take-off and landing aircraft. Despite the potential benefits of UAM limited literature exists on the end user's perception and the challenges to its societal acceptance, especially in developing economies. This paper addresses this gap in the literature by investigating the factors influencing UAM usage intention in Tehran, a developing country capital. Within the framework of the extended technology acceptance model, the study collected 828 valid questionnaires through a face-to-face survey. Structural equation modeling is employed to highlight the effects of a wide range of factors associated with individual differences, system characteristics, and social influence. The findings indicate that attitude, subjective norms, trust, perceived usefulness, perceived enjoyment, personal innovativeness, ecological awareness, and price evaluation are positively associated with UAM usage intention, while perceived safety correlates with the attitude toward UAM negatively. These findings provide valuable insights for policymakers and industry stakeholders, fostering a deeper understanding of end-user perspectives in developing countries, and prudently promising the potential for its targeted implementation. The study suggests that although its widespread adoption in developing countries is unlikely to occur in the near future, UAM should be regarded as a viable early and timely solution to persistent issues such as chronic congestion in metropolitan areas of the global south.

Helios.TALK: A decentralised messaging framework that preserves the privacy of users

Communication via digital means, such as mobile messaging applications (apps), plays an increasingly important role in everyday life. However, most messaging apps employ centralized computing principles that relinquish control of their users’ personal data to social network platform providers. Decentralization has been proposed as an alternative that provides trustworthiness and data confidentiality, but this comes at expense of fewer provided features and non-intuitive user experience. To address this issue, we hereby present two interconnected decentralized messaging tools, developed in the scope of the HELIOS platform, which can support new decentralized social networks. The first tool is a framework that supports the development of context-aware decentralized messaging apps in mobile Android devices by organically tying together many of the platform’s standalone decentralized operations. The second is a decentralized messaging app, called helios.TALK, that builds on the framework but accommodates additional design considerations from the perspective of end-users.

Elicitation and Empathy with AI-enhanced Adaptive Assistive Technologies (AATs)

Efforts to include people with disabilities in design education are difficult to scale, and dynamics of participation need to be carefully planned to avoid putting unnecessary burdens on users. However, given the scale of emerging AI-enhanced technologies and their potential for creating new vulnerabilities for marginalized populations, new methods for generating empathy and self-reflection in technology design students (as the future creators of such technologies) are needed. We report on a study with Information Systems graduate students where they used a participatory elicitation toolkit to reflect on two cases of end-user privacy perspectives towards AI-enhanced tools in the age of surveillance capitalism: their own when using tools to support learning, and those of older adults using AI-enhanced adaptive assistive technologies (AATs) that help with pointing and typing difficulties. In drawing on the experiences of students with intersectional identities, our exploratory study aimed to incorporate intersectional thinking in privacy elicitation and further understand its role in enabling sustainable, inclusive design practice and education. While aware of the risks to their own privacy and the role of identity and power in shaping experiences of bias, students who used the toolkit were more sanguine about risks faced by AAT users—assuming more data equates to better technology. Our tool proved valuable for eliciting reflection but not empathy.

Practices Supporting Electronic Health Record Transitions: Lessons from Four US Healthcare Systems

BackgroundElectronic health record (EHR) transitions are common and complex organizational changes, yet limited published literature is available to guide health systems that are changing from one EHR to another. Clinicians and staff end users at sites that have undergone EHR transitions may have critical insights that could inform future transitions.ObjectiveTo assess end user perspectives on organizational practices that support successful EHR transitions.DesignMulti-site qualitative study of end users at healthcare systems that transitioned to a new EHR (either Epic or Cerner) within the prior 3 years.ParticipantsForty-two participants, including providers, clinical leaders, and informaticists at four geographically and organizationally diverse US healthcare systems.ApproachWe conducted semi-structured telephone interviews, which were audio-recorded and transcribed. We used content analysis to identify key practices that influenced EHR transition success.Key ResultsParticipants described specific organizational practices that they found most helpful in supporting EHR transitions, and these practices transcended individual sites and EHR systems. We categorized practices based on how they were described relative to the stage of implementation. During pre-go-live, recommended practices included communicate rationale and anticipated outcomes of the EHR change; understand baseline workflows; and plan for appropriate customization. During go-live, recommended practices included personalize training and support; invest in robust internal support; reduce workload expectations; and proactively address challenges. The recommended post-go-live practice was to continue to invest in the change.ConclusionsOur findings may act as a roadmap for future EHR transitions by identifying specific and actionable organizational practices across stages of implementation. These recommendations highlight the role of health system leaders in preparing for the organizational change, working with and supporting end users, and addressing challenges that arise.

The Perception of Health Care Practitioners Regarding Telemedicine During COVID-19 in Saudi Arabia: Mixed Methods Study.

Telemedicine is a rapidly evolving field that uses information and communication technology to provide remote health care services, such as diagnosis, treatment, consultation, patient monitoring, and medication delivery. With advancements in technology, telemedicine has become increasingly popular during the COVID-19 lockdown and has expanded beyond remote consultations via telephone or video to include comprehensive and reliable services. The integration of telemedicine platforms can enable patients and health care providers to communicate more efficiently and effectively. This study aims to investigate the awareness, knowledge, requirements, and perceptions of health care practitioners in Saudi Arabia during the pandemic health crisis from the end-user perspective. The findings of this study will inform policy makers regarding the sustainability of telemedicine and how it affects the process of provision of health care and improves the patients' journey. This study adopted a mixed methods design with a quantitative-based cross-sectional design and qualitative interviews to assess the perceptions of various health care professionals working in outpatient departments that have a telemedicine system that was used during the COVID-19 pandemic. For both approaches, ethics approval was obtained, and informed consent forms were signed. In total, 81 completed questionnaires were used in this study. In the second phase, general interviews were conducted with managerial staff and health care professionals to obtain their view of telemedicine services in their hospitals. The study revealed that most participants (67/81, 83%) were familiar with telemedicine technology, and the study proved to be statistically significant at P<.05 with a proportion of the participants (52/81, 64%) believing that continuous training was essential for its effective use. The study also found that consultations (55/153, 35.9%) and monitoring patients (35/153, 22.9%) were the major components of telemedicine used by health care professionals, with telephones being the most commonly used mode of interaction with patients (74/117, 63.2%). In addition, 54% (44/81) of the respondents expressed concerns about patient privacy and confidentiality, highlighting this as a major issue. Furthermore, the majority of participants (58/81, 72%) reported the necessity of implementing national standards essential for telemedicine technology in Saudi Arabia. The interviews conducted as part of the study revealed 5 major themes: culture, barriers and difficulties, communication, implementation, and evaluation. These themes highlighted the importance of a culture of acceptance and flexibility, effective communication, and ongoing evaluation of telemedicine technologies in health care systems. This study provides a crucial message with insights into the perceptions and experiences of health care professionals with telemedicine during the COVID-19 pandemic in Saudi Arabia.

Automated detection, categorisation and developers’ experience with the violations of honesty in mobile apps

Human values such as honesty, social responsibility, fairness, privacy, and the like are things considered important by individuals and society. Software systems, including mobile software applications (apps), may ignore or violate such values, leading to negative effects in various ways for individuals and society. While some works have investigated different aspects of human values in software engineering, this mixed-methods study focuses on honesty as a critical human value. In particular, we studied (i) how to detect honesty violations in mobile apps, (ii) the types of honesty violations in mobile apps, and (iii) the perspectives of app developers on these detected honesty violations. We first develop and evaluate 7 machine learning (ML) models to automatically detect violations of the value of honesty in app reviews from an end-user perspective. The most promising was a Deep Neural Network model with F1 score of 0.921. We then conducted a manual analysis of 401 reviews containing honesty violations and characterised honesty violations in mobile apps into 10 categories: unfair cancellation and refund policies; false advertisements; delusive subscriptions; cheating systems; inaccurate information; unfair fees; no service; deletion of reviews; impersonation; and fraudulent-looking apps. A developer survey and interview study with mobile developers then identified 7 key causes behind honesty violations in mobile apps and 8 strategies to avoid or fix such violations. The findings of our developer study also articulate the negative consequences that honesty violations might bring for businesses, developers, and users. Finally, the app developers’ feedback shows that our prototype ML-based models can have promising benefits in practice.

End-User Perspectives of the Importance of Rehabilitation Technologies and Related Values for Implementation.

Despite the increased availability of rehabilitation technologies (such as robotics) and evidence to support their use, their adoption is still limited in practice. To begin to address this issue, we sought to better understand the perspectives of end-users regarding the implementation of rehabilitation technologies through an online survey. Fifty unique responses across Client (e.g., people with disability and carers), Allied Health (e.g., occupational and physical therapists, podiatrists, psychologists, dieticians, audiologists, exercise physiologists), and Medical (i.e., physicians and nurses) groups were analyzed. Respondents considered rehabilitation technologies to be important (median across groups of 9 out of 10). Across groups, value statements related to the implementation of rehabilitation technologies ranked from most to least important related to: benefits of technology to its intended end-user with disability, practical use of technologies, the technologies themselves (e.g., innovative, established, stand-alone), and their impact on health professionals or the health system as a whole. There were similarities and differences in ranking within and across end-user groups. The prioritization of statements indicates where development efforts can be focused to address end-user goals, and disparity in results across groups supports rich stakeholder engagement during rehabilitation technology development.

Conceptualization and Survey Instrument Development for Website Usability

The aim of this study is to conceptualize website usability and develop a survey instrument to measure related concepts from the perspective of end users. We designed a three-stage methodology. First, concepts related to website usability were derived using content analysis technique. A total of 16 constructs measuring website usability were defined with their explanations and corresponding open codes. Second, a survey instrument was developed according to the defined open codes and the literature. The instrument was first validated using face validity, pilot testing (n = 30), and content validity (n = 40). Third, the survey instrument was validated using explanatory and confirmatory analyses. In the explanatory analysis, 785 questionnaires were collected from e-commerce website users to validate the factor structure of website usability. For confirmatory factor analysis, a new sample collected from 1086 users of e-commerce websites was used to confirm the measurement model. In addition, nomological validation was conducted by analyzing the effect of website usability concepts on three key factors: “continued intention to use”, “satisfaction”, and “brand loyalty”.

Using evidence-based research approach and exploratory studies before conducting a new acupuncture randomized controlled trial

Adequate preliminary research is a prerequisite for ensuring the quality of randomized controlled trials (RCTs) on acupuncture. This paper proposes that prior to conducting a new RCT of acupuncture, an evidence-based research (EBR) approach and exploratory studies should be adopted to ensure value and avoid waste of research. EBR, defined as using systematic reviews of relevant clinical studies and obtain the perspectives of end users in a systematic and transparent way to justify research questions and inform trial design, is recommended as the first step before embarking on an acupuncture RCT to ensure its value and rationality. The subsequent exploratory studies can aid researchers in pre-evaluating the effect of acupuncture treatment protocols, identifying key factors that influence the outcome, and test trial feasibility for protocol optimization. This article elucidates the significance and implementation of the EBR approach and exploratory studies, with the aim of providing practical methods to enhance the quality of acupuncture RCTs.

Role of domestic renewable energy plants in combating energy deficiency in developing countries. End-user perspective

Energy crises and shattered economies are common issues in developing countries. The World is looking for green energy sources that help achieve a green economy and also help to fight against global warming, a common issue faced by the globe. Domestic renewable energy plants are a potential green energy source that can help combat energy crises, dwindling economies, and global warming. We have pointed out that the adoption of renewable energy plants at a domestic level is low. Hence, we have collected a cross-sectional dataset from one of the developing countries. 756 useable responses were used to test the proposed hypotheses and rank them according to their normalized importance by using the SEM-ANN hybrid approach. Results revealed that awareness of renewable energy plants, facilitating conditions, environmental knowledge, social influence, and perceived expectancy of renewable energy plants have significant positive. In contrast, cost value significantly negatively influences the behavioural​ intentions of developing countries’ consumers. Results also revealed a significant moderation effect of gender in the adoption of renewable energy plants at a domestic level. Awareness about renewable energy plants is crucial in adopting renewable energy plants domestically. Study findings have significant theoretical contributions and policy suggestions for governments and other stakeholders that will help them combat the energy crisis and achieve a sustainable green economy.

Student Competition (Knowledge Generation) ID 1978805

Background Individuals with neurological injury or disease are at increased risk of experiencing falls, highlighting the need for effective balance interventions. A novel system integrating functional electrical stimulation and visual feedback balance training (FES+VFBT) is a promising balance intervention; however, this lab-based system requires a redesign to increase clinical utility. Objective To identify possible challenges and solutions to implementing the FES+VFBT system as a balance intervention, from the perspective of end users. Design/Methods This qualitative study involved three semi-structured focus group meetings to explore participants’ perspectives on the feasibility and potential challenges of implementing FES+VFBT in neurorehabilitation. Two individuals with spinal cord injury (SCI), one individual with stroke, two physiotherapists and one hospital administrator participated. Interview transcripts were analyzed using a deductive-inductive content analysis. The levels of the Social Ecological Model (SEM) were used as themes for the deductive analysis. Categories and codes were detected using an inductive approach. Results The themes spanned the four levels of the SEM: intrapersonal, interpersonal, organizational/training environment and society/policy. Identified categories consisted of possible challenges mapped to the intrapersonal (e.g., lack of knowledge, tolerance of user) and organizational/training environment (e.g., cost, need for space and time, technical challenges) levels. Categories also reflected possible solutions mapped to all SEM levels: intrapersonal (e.g., reading and research), interpersonal (e.g., practicing together), organizational/training environment (e.g., tailoring system parameters, social support), and society/policy (e.g., create guidelines, provide cost options). Conclusion The findings will be used to improve the FES+VFBT system design and facilitate implementation of the intervention in neurorehabilitation.

Leveraging autonomous mobile robots for Industry 4.0 warehouses: a multiple case study analysis

PurposeDespite its potential, warehouse managers still struggle to successfully assimilate autonomous mobile robots (AMRs) in their operations. This paper means to identify the moderating factors of AMR assimilation for production warehouses that influence the digital transformation of their intralogistics via AMRs.Design/methodology/approachDrawing on innovation of assimilation theory (IAT), this study followed an explorative approach using the principles of the case study method in business research. The cases comprised of four AMR end users and six AMR service providers. Data were collected through semi-structured interviews.FindingsFour clusters of moderators that affect each stage of AMR assimilation were identified. These clusters include organizational attributes of end users (i.e. production warehouses), service attributes of service providers, technology attributes of AMRs and relational attributes between the AMR service providers and the AMR end users.Originality/valueThe authors extend the IAT framework by identifying various moderating factors between different stages of the AMR assimilation process. To the authors' knowledge, this is the first study to introduce the perspective of AMR end users in conjunction with AMR service providers to the “Industry 4.0” technology assimilation literature. The study propositions regarding these factors guide future intralogistics and AMR research.

Assistive Technology and Dementia Caregiving: A Few Social Considerations and Policy Implications

With older persons increasingly wanting to age in familiar places, among community and family, their natural habitat, technological innovations greatly facilitate that. IATs are designed to ease the burden on families and healthcare systems while enhancing the quality of life and independent living of persons with dementia. This article questions the readiness of societies to accept the use of IAT with their older loved ones. It outlines some of the key social considerations that should be better understood in order to mitigate the barriers to AT use from an end-user and community perspective. It proposes policy interventions that could enhance the acceptance and use of IAT among older persons, taking into account sociocultural and prevailing attitudes, especially in Arab Muslim societies, while benefiting from global experiences.

“This is where public transport falls down”: Place based perspectives of multimodal travel

No single transport mode can replace the private car in terms of its support for flexible mobility. Combinations of multiple transport modes are required. If we are to contribute to reduced car dependency it is crucial that we understand what makes such multi-modal journeys difficult. Despite large bodies of literature on mode choice and on perceptions and experiences of various travel modes, few scholars have looked specifically at journeys that combine more than one transport mode. This research fills that gap, taking a qualitative approach to explore end-user perspectives of the challenges therein. Specifically, it focusses on the barriers people in urban, peri-urban, and rural areas perceive when thinking about combining two or more transport modes in a single journey. Thematic analysis of the responses of 146 participants of a series of asynchronous online focus groups revealed an overall picture of challenges shared, with some differences in the relative importance of different barriers for those in different locations. The time incurred, the difficulties in synchronising timetables, and the criticality of each leg’s reliability are core barriers for all, while those in rural areas are more affected by basic service provision and the physical linking of different modes. Of greater significance for urban and peri-urban residents were the complexities around planning a multi-modal journey where there are multiple options available, as well as the facilities available at stations and stops. Results are discussed in terms of interventions that could help people in different residential settings use their car less.

Development and Usability Evaluation of a Web-Based Health Information Technology Dashboard of Quality and Economic Indicators

Introduction: Strategic dashboards, including hospital economic monitoring systems, play a major role in analyzing data and making decisions. As the health system is a multidimensional ecosystem, decision-makers and healthcare officials must gather and integrate information from numerous health information systems to monitor and direct healthcare centers. This study aims to develop and usability evaluation of a health information technology dashboard in Iran that utilizes qualitative and economic indicators.Material and Methods: This study was conducted in four phases. It included extracting the requirements of the system through the focus group technique. Based on these results, a comprehensive economic dashboard was developed. Then, the system’s usability is evaluated from the perspectives of experts and end-users by two scales of Nielsen and USE, respectively.Results: The dashboard was developed on the web and different access levels were defined for users according to their roles. This dashboard provides the ability to integrate information from different systems at the national level for decision-makers. The results of usability evaluation from users' point of view showed that it has a good level of usability. Furthermore, evaluation results revealed that aesthetic aspects and simple design and clarity of system status (0%), privacy (1.49%), "visibility and clarity of the system" and "adaptation between the system and the real world" (2.98%), have the fewest design errors. With 14 problems (20.89%), "recognition rather than remembering" and "compliance with uniformity and standards" have the highest frequency of problems.Conclusion: The development of an extensive integrated economic health dashboard, based on usability principles that are suitable for its stakeholders regardless of their specialty and granted access level, is welcomed by the health economist, hospital managers, and the ones who have an active role in monitoring and coordinating hospitals or even in greater scales such as national wide decision makers.

End user perspectives on design and conduct of health-related disaster research

Individuals and communities are the intended beneficiaries of disaster research, and are ideally positioned to inform research aims, methods, desirable outcomes and implementation strategies. However, it can be challenging for researchers more familiar with quantitative methods to engage end users in study design and conduct. In an attempt to break down barriers involving end users in health-related disaster research, we outline how our research team conducted community consultations in the early stages of a bushfire-focussed health study, what we found and how this influenced our subsequent research. Participants (n = 76) were recruited by convenience sampling from five communities- one rural community which had direct experience of bushfire 5–10 years ago (n = 9), three communities located in high bushfire risk areas (n = 47) and one community in an urban centre (n = 20). Participants provided their perspectives on previous experiences of research participation and/or priorities/concerns about future health-related disaster research in two custom survey instruments and a focus group. Data analysis using conventional content analysis revealed four themes- 1) the importance of a holistic view of health that encompasses mental, physical, and social wellbeing; 2) appropriate study design/conduct with emphasis on inclusivity, scientific rigor, sensitivity to participants’ needs/circumstances, minimised participant burden and responsiveness to participant feedback; 3) clear and regular communication; and 4) practical, participant-focussed study outcomes. We describe how these findings shaped the design of our subsequent prospective, longitudinal health-focussed disaster research study (the ISLAND Resilience Initiative). This work may support participant engagement in future research studies focused on end user needs following disaster experiences.

Leveraging web-based prediction calculators to set patient expectations for elective spine surgery: a qualitative study to inform implementation

BackgroundPrediction calculators can help set outcomes expectations following orthopaedic surgery, however effective implementation strategies for these tools are unknown. This study evaluated provider and patient perspectives on clinical implementation of web-based prediction calculators developed using national prospective spine surgery registry data from the Quality Outcomes Database.MethodsWe conducted semi-structured interviews in two health systems, Vanderbilt University Medical Center (VUMC) and Duke University Health System (DUHS) of orthopedic and neurosurgery health care providers (VUMC: n = 19; DUHS: n = 6), health care administrators (VUMC: n = 9; DUHS: n = 9), and patients undergoing elective spine surgery (VUMC: n = 16). Qualitative template analysis was used to analyze interview data, with a focus on end-user perspectives regarding clinical implementation of web-based prediction tools.ResultsHealth care providers, administrators and patients overwhelmingly supported the use of the calculators to help set realistic expectations for surgical outcomes. Some clinicians had questions about the validity and applicability of the calculators in their patient population. A consensus was that the calculators needed seamless integration into clinical workflows, but there was little agreement on best methods for selecting which patients to complete the calculators, timing, and mode of completion. Many interviewees expressed concerns that calculator results could influence payers, or expose risk of liability. Few patients expressed concerns over additional survey burden if they understood that the information would directly inform their care.ConclusionsInterviewees had a largely positive opinion of the calculators, believing they could aid in discussions about expectations for pain and functional recovery after spine surgery. No single implementation strategy is likely to be successful, and strategies vary, even within the same healthcare system. Patients should be well-informed of how responses will be used to deliver better care, and concerns over how the calculators could impact payment and liability should be addressed prior to use. Future research is necessary to determine whether use of calculators improves management and outcomes for people seeking a surgical consult for spine pain.