Expert Workshop Research Articles

The role of quantitative bias analysis for nonrandomized comparisons in health technology assessment: recommendations from an expert workshop.

The use of treatment effects derived from nonrandomized studies (NRS) in health technology assessment (HTA) is growing. NRS carry an inherently greater risk of bias than randomized controlled trials (RCTs). Although bias can be mitigated to some extent through appropriate approaches to study design and analysis, concerns around data availability and quality and the absence of randomization mean residual biases typically render the interpretation of NRS challenging. Quantitative bias analysis (QBA) methods are a range of methods that use additional, typically external, data to understand the potential impact that unmeasured confounding and other biases including selection bias and time biases can have on the results (i.e., treatment effects) from an NRS. QBA has the potential to support HTA bodies in using NRS to support decision-making by quantifying the magnitude, direction, and uncertainty of biases. However, there are a number of key aspects of the use of QBA in HTA which have received limited discussion. This paper presents recommendations for the use of QBA in HTA developed using a multi-stakeholder workshop of experts in HTA with a focus on QBA for unmeasured confounding.

Development of a Maturity Assessment Model for Digital Twins in Battery Cell Industry

Due to the increasing demand for batteries, fueled by the transition to sustainable energy sources and electric vehicles, battery factories are currently being built all around the world. To meet the resulting challenges, digital technologies are a key component in the context of competitive battery cell manufacturing. For a requirements-oriented implementation of an appropriate Industry 4.0 environment, three Digital Twins are implemented to meet the digital needs in the battery cell manufacturing environment: The Product Twin, the Plant Twin, and the Building Twin. Addressing the challenges of battery cell manufacturing requires holistic Digital Twins that encompass the entire lifecycle. Existing Maturity Models do not provide a sufficient basis for a systematic assessment and holistic development of the Product-, the Plant-, and the Building Twin in battery cell manufacturing. Therefore, a novel Maturity Model considering the specific requirements has been developed. The Maturity Model is developed using an action model based on the Design Science Research methodology. The content components are carried out with qualitative empiricism - literature review, expert workshops, expert interviews - within five iterations. This is followed by expert evaluation of the model using a case study. The result of the empiricism is a holistic Maturity Assessment Model for Digital Twins in Battery Cell Industry. The Maturity Model includes five maturity levels, twelve maturity dimensions, 50 maturity sub-dimensions and 250 maturity indicators. Fuzzy Logic is used to perform the quantitative maturity assessments of the Digital Twins by means of the maturity indicators. The evaluation results show that the model can offer consistent and reproducible maturity assessments. Once the model has been tested and verified with further case studies it is plausible that the Maturity Model can be applied to a wide range of Digital Twins in the manufacturing industry.

Framework for predictive sales and demand planning in customer-oriented manufacturing systems using data enrichment and machine learning

Companies with Make-to-Order (MTO) manufacturing have always faced the conflict of meeting a large volume of individual customer orders on time while remaining as flexible as possible. Unlike Make-to-Stock (MTS) manufacturing, planning the production requirements for MTO is a challenging task since incoming orders may vary in time and quantity, while also being subject to a number of variables. In some cases, the delivery time allocated by the customer can be less than the required Order Lead Time to fulfil the order. Manufacturers can respond to this with either with approached from production management or from data science. This paper presents a framework to leverage the benefit of both domains. We conduct a literature review and present the results of an expert workshop. We propose criteria for a suitable data enrichment and the application of Machine Learning (ML) methods in sales and demand forecasting. In conclusion, the proposed framework helps to equip manufacturing companies with a structured strategy for data management and to utilize the benefit of ML for sales and demand forecasting.

Caught in the Lifelong Learning Maze: Helping People with Learning Analytics and Chatbots to Find Personal Career Paths

Current lifelong learning platforms offer users a query option to select a wide variety of courses. However, finding a suitable course among the seemingly endless catalogs of options presented by the platforms is not straightforward. We argue that digital counseling can enhance this process. In this paper, we present a set of three formative studies where we explored the main aspects that can provide the counseling needed. The methods comprise an analysis of user profile characteristics and learning analytics indicators (e.g., learning progress/self-regulation) by means of an expert workshop, evaluating the feasibility of current technologies (e.g., natural language processing) for automatically assessing users' competencies, and a survey on the use of Chatbots as the interaction interface between the users and the lifelong learning portals. The analysis resulted in the extraction of basic requirements for digital counseling. We conclude the paper by presenting a system design derived from these studies.

Design Principles for Personalized Assistance Systems That Respect Privacy

Personalized assistance systems (PAS) provide real-time assistance tailored to individual users to improve efficiency in the workplace. PAS communicate dynamically with users through wearable computing devices. To deliver such personalized assistance, PAS need personal data from the individuals who wear them. However, concerns over data protection and security can negatively influence the extent to which users accept personalized assistance systems. The key aspects in this regard that the literature currently lacks include data protection law and the employee perspective. Hence, we develop seven design principles for PAS that respect user privacy through employee-determined approaches to data collection and use. We developed the principles based on a systematic literature review, user personas, privacy control, and European Union legal requirements for privacy by design and privacy by default. Our design principles, which we evaluated in a focus group and an expert workshop, provide a framework to help practitioners and software developers mitigate adoption barriers due to privacy concerns. Our study also contributes to the theoretical discussion of current developments in personalized assistance in the workplace by providing a new perspective on ensuring employees accept the required data collection and use.

Social Impact Assessment of Circular Construction: Case of Living Lab Ghent

The construction industry is considered to have a high potential in achieving the sustainable development goals. The circular economy is a promising framework that supports the shift from a linear-construction industry to an environmental-friendly and efficient sector. On the other hand, there is a lack of effort in measuring the impact of construction-related activities on users and society. The gap is greater when the context of social impacts is related to circular and bio-based construction. For this purpose, a social impact assessment framework was developed in the Interreg 2 seas CBCI project and tested on a residential prototype: Living Lab (LL) Ghent. Under 13 impact categories relevant to 4 stakeholder categories, circular and bio-based construction materials and methods were assessed for production and construction phases. Qualitative and quantitative data were collected through expert workshops and questionnaires. The results include identification of new indicators (urban mining, social economy, and post-intervention manuals) for several circular construction methods. The social impacts of the LL were discussed depending on each stakeholder category. It was seen that there are several positive impacts related to workers and the local community. Certain recommendations were also provided specifically on a construction-sector basis which may be integrated into existing social impact assessment guidelines.

Identifying Future Study Designs for Mental Health and Social Wellbeing Associated with Diets of a Cohort Living in Eco-Regions: Findings from the INSUM Expert Workshop.

Diets influence our mental health and social wellbeing (MHSW) in multiple ways. A rising community concept, Eco-Regions, has gained interest. The research project "Indicators for assessment of health effects of consumption of sustainable, organic school meals in Ecoregions" (INSUM) aims to develop future-oriented research approaches to measure the potential health effects of more sustainable and healthy diets. This first part of the project focuses on MHSW with the goal to identify suitable study designs and indicators. The methodology is based on a 2-day workshop with an interdisciplinary group of experts. This paper describes commonly applied research methods on the nexus between diet and MHSW as presented by the experts and summarises key points from the discussions. The results show that the dominating tool to investigate MSHW is questionnaires. Questionnaires vary largely depending on the research design, such as participants or distribution channels. Cohort studies addressing families and including in-depth interventional and/or experimental studies may be suitable for an Eco-Region investigation. Those MHSW studies can be conducted and combined with measurements of somatic health effects. We conclude that indicators should be seen as complementary rather than independent. Explorative research designs are required to investigate complex Eco-Regions.

User-Centered Requirements for Augmented Reality as a Cognitive Assistant for Safety-Critical Services

Augmented reality (AR) is widely acknowledged to be beneficial for services with exceptionally high requirements regarding knowledge and simultaneous tasks to be performed and are safety-critical. This study explores the user-centered requirements for an AR cognitive assistant in the operations of a large European maritime logistics hub. Specifically, it deals with the safety-critical service process of soil sounding. Based on fourteen think-aloud sessions during service delivery, two expert interviews, and two expert workshops, five core requirements for AR cognitive assistants in soil sounding are derived, namely (1) real-time overlay, (2) variety in displaying information, (3) multi-dimensional tracking, (4) collaboration, and (5) interaction. The study is the first one on the applicability and feasibility of AR in the maritime industry and identifies requirements that impact further research on AR use in safety-critical environments.

Development of a Psychological Intervention to Improve Depressive Symptoms and Enhance Adherence to Antiretroviral Therapy among Adolescents and Young People Living with HIV in Dar es Salaam Tanzania.

Background: Interventions that simultaneously target depression and antiretroviral therapy (ART) medication adherence are recommended for improving HIV treatment outcomes and quality of life for adolescents living with HIV. However, evidence is scarce on culturally feasible and acceptable interventions that can be implemented for HIV-positive adolescents in Tanzania. We, therefore, developed a manualized brief psychological intervention that utilizes evidence-based strategies to address depression and ART adherence in adolescents living with HIV in Tanzania. Methods: We used the Theory of Change Enhanced Medical Research Council framework (TOCMRC) for developing complex interventions in health care to develop the intervention in five phases. First, the literature was reviewed to identify potential intervention components. Second, we conducted a situational analysis using qualitative interviews with adolescents living with HIV, health care providers, and caregivers. Third, we conducted a mental health expert workshop; and fourth, theory of change workshops with representatives from the Ministry of Health, mental health professionals, HIV implementing partners, adolescents, and healthcare providers. Lastly, we synthesized results to finalize the intervention and a theory of change map showing the causal pathway for how we expect the developed intervention to achieve its impact. Results: Adolescents living with HIV in Tanzania experience several unmet mental health needs ranging from overwhelming depressive symptoms to not feeling understood by healthcare providers who lack mental health knowledge. Participants perceived psychological intervention that utilizes a task-shifting approach to be acceptable and beneficial to addressing those problems. The novel components of the NITUE intervention included incorporating evidence-based intervention components, namely, cognitive-behavioral therapy, motivational interviewing, and problem solving. In addition, caregiver inclusion in the treatment was essential to ensure access to care, compliance, and improved outcomes. Conclusions: A culturally appropriate brief psychological intervention that utilizes a task-shifting approach to address depression and medication adherence for adolescents living with HIV in Dar es Salaam, Tanzania, was developed. The intervention will be piloted for appropriateness, feasibility, and acceptability and will provide material for a future trial to determine its effectiveness.

Enhancing blue-green infrastructure in German cities with the involvement of urban society: insights from Frankfurt/Main and Stuttgart

AbstractEnhancing blue-green infrastructure (BGI) will help cities adapt to climate change. This study focused on urban society, specifically on residents, the housing and real estate sector, and municipal administration in the cities of Frankfurt and Stuttgart, and investigated ways to enhance BGI in line with their perceptions, expectations and requirements. A particular emphasis was placed on periods of hot, dry weather. During future workshops, actor analysis, expert interviews and two expert workshops, the positions, level of knowledge and attitudes of the various groups were examined and barriers to BGI identified, in addition to discussions about measures to overcome these barriers. The results showed that within urban society the greatest support for BGI comes from residents, followed by municipal employees, while actors in the housing and real estate sector are the most reluctant. It became evident that there is a need for action to overcome a range of barriers. Overall, political backing, alliances between stakeholders, encouragement of the housing and real estate sector to take action, and a focus on initiating action in selected urban areas are recommended for successful enhancement of BGI. To ensure maximum impact, there should also be a combination of bottom-up and top-down activities.

Enhancing the Measurement of Sentence Severity through Expert Knowledge Elicitation

Quantitative research on judicial decision-making faces the methodological challenge of analysing disposal types that are measured in different units (e.g. money for fines, days for custodial sentences). To overcome this problem a wide range of scales of sentence severity have been suggested in the literature. One particular group of severity scales that has achieved high validity and reliability are those based on Thurstone’s pairwise comparisons. However, this method invokes a series of simplifying assumptions, one of them being that the range of severity covered by different disposal types is constant. We undertook an expert elicitation workshop to assess the validity of that assumption. Responses from the six criminal law practitioners and researchers that participated in our workshop unanimously pointed at severity ranges being highly variable across disposal types (e.g. much wider severity ranges were identified for suspended custodial sentences than for fines). We used this information to re-specify Thurstone’s model allowing for unequal variances. As a result, we obtained a new, more robust, scale of sentence severity.

Criteria and Guidelines for Human-Centered Work Design in a Digitally Transformed World of Work: Findings from a Formal Consensus Process

With the increasing digital transformation, work tasks are changing-in some cases, significantly. Our study addresses the question of whether the established criteria for work design are still sufficient or if they should get updated and additional criteria become necessary in the context of digitalization. In a multistage consensus process involving interdisciplinary groups of experts, we have identified specific criteria for the humane design of work in a world increasingly permeated by digitalized work tools. Starting with an expert workshop using a combined nominal group/focus group technique, followed by a real-time Delphi study, a content analysis and a five-stage peer comment process, we detected 13 criteria and 38 design guidelines for human-centered work in digital transformation. Mapping these with established criteria, it became apparent that some established criteria have experienced a new dynamic because of the digital transformation. For other criteria, a need for digitization-sensitive design is discernible. In addition, criteria have emerged whose necessity is rooted in the digital transformation. A diffusion and stronger interconnection of the various levels of the work system in connection with the digital transformation of work is apparent.

Expanding the Focus of Occupational Safety and Health: Lessons from a Series of Linked Scientific Meetings.

There is widespread recognition that the world of work is changing, and agreement is growing that the occupational safety and health (OSH) field must change to contribute to the protection of workers now and in the future. Discourse on the evolution of OSH has been active for many decades, but formalized support of an expanded focus for OSH has greatly increased over the past 20 years. Development of approaches such as the National Institute for Occupational Safety and Health (NIOSH)'s Total Worker Health® concept and the World Health Organization (WHO)'s Healthy Workplace Framework are concrete examples of how OSH can incorporate a new focus with a wider view. In 2019, NIOSH initiated a multi-year effort to explore an expanded focus for OSH. This paper is a report on the outputs of a three-year cooperative agreement between NIOSH and The University of Texas School of Public Health, which led to subject matter expert workshops in 2020 and an international conference of global interest groups in 2021. This article traces the background of these meetings and identifies and assesses the lessons learned. It also reviews ten thematic topics that emerged from the meetings: worker health inequalities; training new OSH professionals; future OSH research and practice; tools to measure well-being of workers; psychosocial hazards and adverse mental health effects; skilling, upskilling and improving job quality; socioeconomic influences; climate change; COVID-19 pandemic influences; and strategic foresight. Cross-cutting these themes is the need for systems and transdisciplinary thinking and operationalization of the concept of well-being to prepare the OSH field for the work of the future.

Development of Practice Guidelines for Social Work in Medical Rehabilitation

Social work services are a major part of the treatment spectrum in medical rehabilitation. Given new demands and differences in the practical implementation of social work interventions, there is a need for good clinical practice standards of care. Therefore, practice guidelines (PG) for social work in medical rehabilitation facilities were developed. These are intended to support decision-making processes in interventions and give other professions working in rehabilitative care insight into the range of social work services. The PG were developed in a multi-stage process involving experts from social work practice. A national and international literature search was conducted on the current evidence on social work interventions in medical rehabilitation. In a survey of social work services in inpatient and outpatient medical rehabilitation facilities (all indications except addictive disorders, child/adolescent rehabilitation; Nmax=311), social workers were asked to provide information on current practice and their expectations towards the PG. On this basis, a preliminary version of the PG was modified and expanded in an expert workshop and evaluated in a second survey of social work services in rehabilitation facilities (Nmax=184). The results were discussed at a second workshop and on this basis the final PG version was prepared. The PG include information on framework conditions of social work in rehabilitation and on social diagnostics. At the center of the PG are frequent needs and problems of rehabilitation patients. These include work-related problems, social and financial problems, participation of persons with disability, and aftercare/follow-up. Each chapter provides information on needs assessment, the content and scope of suitable interventions, and additional materials and references. The PE are intended as a practical aid, based on both practical expertise and existing scientific evidence, to provide social work staff and other professional groups in rehabilitation with information on social work services and their implementation. The focus of the content reflects the expressed expectations of social services for PE.

Biosensors supporting healthcare in missions — expert consensus on the status of implementation in the military and future tasks

The monitoring of physiological parameters using wearable (bio-) sensors of military personnel is a progressing process within the military environment. It sets high demands on such devices, in order to support healthcare and performance of the personnel. To get an ovierview of the current status of the use, the evaluation and the implementation in the military, in May 2021, the Multinational Medical Coordination Centre / European Medical Command has organized an expert workshop about ‘Biosensors Supporting Healthcare in Missions’. Three thematic clusters were addressed: ‘Human Performance and Readiness’; ‘Health and Medical Management Applications’ and ‘Ethical and Legal Aspects of the Use of Biosensors’.

International Research and Philanthropy

International Research and Philanthropy

Setting the stage for Acute Febrile Illness Surveillance Network in West Africa: Report of a Regional Experts and Stakeholders Workshop in Accra, Ghana, April 2021

A three-day regional workshop of stakeholders, partners, and experts from the West African countries was held from April 14 -16, 2021 in Accra, Ghana to conduct a situational analysis on the existing knowledge, expertise, previous accomplishments, and current activities in laboratory and syndrome based Acute Febrile Illness (AFI) surveillance that will guide a regional strategy for AFI surveillance. Technical presentations were made by participants from ministries of health, national public health institutes, West African Health Organization, Food Agricultural Organization, and the United States Centers for Disease Control and Prevention (USCDC) on existing AFI surveillance activities and infrastructure. In groups, the participants conducted Strength, Weaknesses, Opportunities and Threat (SWOT) analysis on AFI surveillance followed by a prioritization of actionable steps. Thirty-eight participants from 9 West African countries attended the meeting. The strengths recognized from the SWOT analysis include the existence and implementation of Integrated Disease Surveillance and Response (IDSR) within the region for priority diseases and a regional collaborative platform. However, there was inadequate information and data sharing platforms, non-availability of accurate diagnostic tools, knowledge gap on AFI and lack of harmonized framework. The opportunities that can be tapped into include supporting partners and global bodies, existing laboratory networks in Africa and infrastructure built during the COVID-19 pandemic. The meeting unveiled opportunities and challenges associated with establishing a regional AFI surveillance framework. It provided opportunity for stakeholders and experts to discuss the way forward on laying a foundation for a regional AFI surveillance system in the region.

Watch out, pothole! featuring road damage detection in an end-to-end system for autonomous driving

While autonomous driving technology made significant progress in the last decade, road damage detection as a relevant challenge for ensuring safety and comfort is still under development. This paper addresses the lack of algorithms for detecting road damages that meet autonomous driving systems’ requirements. We investigate the environmental perception systems’ architecture and current algorithm designs for road damage detection. Based on the autonomous driving architecture, we develop an end-to-end concept that leverages data from low-cost pre-installed sensors for real-time road damage and damage severity detection as well as cloud- and crowd-based HD Feature Maps to share information across vehicles. In a design science research approach, we develop three artifacts in three iterations of expert workshops and design cycles: the end-to-end concept featuring road damages in the system architecture and two lightweight deep neural networks, one for detecting road damages and another for detecting their severity as the central components of the system. The research design draws on new self-labeled automotive-grade images from front-facing cameras in the vehicle and interdisciplinary literature regarding autonomous driving architecture and the design of deep neural networks. The road damage detection algorithm delivers cutting-edge performance while being lightweight compared to the winners of the IEEE Global Road Damage Detection Challenge 2020, which makes it applicable in autonomous vehicles. The road damage severity algorithm is a promising approach, delivering superior results compared to a baseline model. The end-to-end concept is developed and evaluated with experts of the autonomous driving application domain.

Ecotoxicity characterization of chemicals: Global recommendations and implementation in USEtox

Chemicals emitted to the environment affect ecosystem health from local to global scale, and reducing chemical impacts has become an important element of European and global sustainability efforts. The present work advances ecotoxicity characterization of chemicals in life cycle impact assessment by proposing recommendations resulting from international expert workshops and work conducted under the umbrella of the UNEP-SETAC Life Cycle Initiative in the GLAM project (Global guidance on environmental life cycle impact assessment indicators). We include specific recommendations for broadening the assessment scope through proposing to introduce additional environmental compartments beyond freshwater and related ecotoxicity indicators, as well as for adapting the ecotoxicity effect modelling approach to better reflect environmentally relevant exposure levels and including to a larger extent chronic test data. As result, we (1) propose a consistent mathematical framework for calculating freshwater ecotoxicity characterization factors and their underlying fate, exposure and effect parameters; (2) implement the framework into the USEtox scientific consensus model; (3) calculate characterization factors for chemicals reported in an inventory of a life cycle assessment case study on rice production and consumption; and (4) investigate the influence of effect data selection criteria on resulting indicator scores. Our results highlight the need for careful interpretation of life cycle assessment impact scores in light of robustness of underlying species sensitivity distributions. Next steps are to apply the recommended characterization framework in additional case studies, and to adapt it to soil, sediment and the marine environment. Our framework is applicable for evaluating chemicals in life cycle assessment, chemical and environmental footprinting, chemical substitution, risk screening, chemical prioritization, and comparison with environmental sustainability targets.

Business Models for Active Buildings

Active Buildings that allow users to adjust their demands on the grid to the needs of the energy system could greatly assist the transition to net zero, but will not be widely adopted unless the businesses involved can make money from doing so. We describe the construction, flexibility and information supply chains of activities needed to make these buildings work. Drawing on the results of an expert workshop, we set out four possible business models deserving further investigation. Developers may find it profitable to build or upgrade energy-efficient buildings with the monitoring and control equipment needed to adjust demand and energy storage as required, selling them soon after completion. Aggregators monitor the state of the building and communicate with the energy system to adjust the building’s demand while maintaining comfort levels, in return for suitable payments. Energy service companies may sell energy-as-a-service and own the equipment instead of a consumer who wishes to minimize their upfront costs, and the idea of an active, energy-efficient, building may be attractive to the tenants of the new group of all-inclusive rental companies, and hence to those companies. Our discussion shows that each is an evolution of an existing (successful) business model, but that further work will be needed to evaluate their profitability when applied to Active Buildings.