Knowledge Elicitation Research Articles

A smart modern construction enterprise maturity model for business scenarios leading to Industry 4.0

PurposeIndustry 4.0 is characterised by systemic transformations occurring exponentially, encompassing an array of dynamic processes and technologies. To move towards a more sustainable future, it is important to understand the nature of this transformation. However, construction enterprises are experiencing a capacity shortage in identifying the transitional management steps needed to navigate Industry 4.0 better. This paper presents a maturity model with the acronym “Smart Modern Construction Enterprise Maturity Model (SMCeMM)” that provides direction to construction enterprises.Design/methodology/approachIt adopts an iterative procedure to develop the maturity model. The attributes of Industry 4.0 maturity are obtained through a critical literature review. The model is further developed through knowledge elicitation using modified Delphi-based expert forums and subsequent analysis through qualitative techniques. The conceptual validity of the model is established through a validation expert forum.FindingsThe research defines maturity characteristics of construction enterprises across five levels namely ad-hoc, driven, transforming, integrated and innovative encompassing seven process categories; data management, people and culture, leadership and strategy, automation, collaboration and communication, change management and innovation. The maturity characteristics are then translated into assessment criteria which can be used to assess how mature a construction enterprise is in navigating Industry 4.0.Originality/valueThe results advance the field of Industry 4.0 strategy research in construction. The findings can be used to access Industry 4.0 maturity of general contractors of varying sizes and scales and generate a set of recommendations to support their macroscopic strategic planning.

Predicting the causative pathogen among children with pneumonia using a causal Bayesian network.

Pneumonia remains a leading cause of hospitalization and death among young children worldwide, and the diagnostic challenge of differentiating bacterial from non-bacterial pneumonia is the main driver of antibiotic use for treating pneumonia in children. Causal Bayesian networks (BNs) serve as powerful tools for this problem as they provide clear maps of probabilistic relationships between variables and produce results in an explainable way by incorporating both domain expert knowledge and numerical data. We used domain expert knowledge and data in combination and iteratively, to construct, parameterise and validate a causal BN to predict causative pathogens for childhood pneumonia. Expert knowledge elicitation occurred through a series of group workshops, surveys and one-on-one meetings involving 6-8 experts from diverse domain areas. The model performance was evaluated based on both quantitative metrics and qualitative expert validation. Sensitivity analyses were conducted to investigate how the target output is influenced by varying key assumptions of particular high degree of uncertainty around data or domain expert knowledge. Designed to apply to a cohort of children with X-ray confirmed pneumonia who presented to a tertiary paediatric hospital in Australia, the resulting BN offers explainable and quantitative predictions on a range of variables of interest, including the diagnosis of bacterial pneumonia, detection of respiratory pathogens in the nasopharynx, and the clinical phenotype of a pneumonia episode. Satisfactory numeric performance has been achieved including an area under the receiver-operating characteristics curve of 0.8 in predicting clinically-confirmed bacterial pneumonia with sensitivity 88% and specificity 66% given certain input scenarios (i.e., information that is available and entered into the model) and trade-off preferences (i.e., relative weightings of the consequences of false positive versus false negative predictions). We specifically highlight that a desirable model output threshold for practical use is very dependent upon different input scenarios and trade-off preferences. Three commonly encountered scenarios were presented to demonstrate the potential usefulness of the BN outputs in various clinical pictures. To our knowledge, this is the first causal model developed to help determine the causative pathogen for paediatric pneumonia. We have shown how the method works and how it would help decision making on the use of antibiotics, providing insight into how computational model predictions may be translated to actionable decisions in practice. We discussed key next steps including external validation, adaptation and implementation. Our model framework and the methodological approach can be adapted beyond our context to broad respiratory infections and geographical and healthcare settings.

Elicitation of Prior Knowledge: A Vital First Step for Engaging Pupils in Training

The paper emphasizes the importance of understanding the prior knowledge of trainees as a pre-requisite for designing an effective training programme. Data on prior knowledge of trainees and their attitude towards extension services were elicited from 153 extension service providers (ESPs) of varied disciplines using a semi-structured interview schedule. The interview focussed on the extension advisory services and the agriculture extension policy in India. The results show that ESPs carried misconceptions and stereotyped opinions towards various aspects of the extension system. An effort was done by the researcher to address the popular misconceptions, debunk myths with truth and persuading the trainees towards delivering effective extension services well within the recommended resource availability.

Fusing Conflicting Multisource Imprecise Information for Reliability Assessment of Multistate Systems: A Two-Stage Optimization Approach

Expert knowledge is an important information source for system reliability assessment, especially when historical data are limited. However, when elicited, expert knowledge is often imprecise with large uncertainty. Moreover, as experts usually own different expertise and knowledge, the elicited knowledge from different experts might be conflicting. In this article, a two-stage optimization model is put forth to fuse the imprecise and conflicting information from multiple sources to assess reliability of multistate systems. The degradation of the components in the multistate system is modeled via imprecise Markov models. Then, in the first-stage optimization, upper and lower bounds of the degradation model parameters are determined by minimizing the conflict between the prediction of the model and the multisource imprecise information elicited from experts. A particle swarm optimization algorithm is tailored to solve the computational problems brought by the presence of high-dimensional decision variables and resolve the optimization problem. The second-stage optimization is, then, conducted to identify the upper and lower bounds of the system reliability function given that the degradation model parameters are constrained in the bounds obtained from the first-stage optimization. A numerical example, along with a radar display and control system, is used to demonstrate the effectiveness and applicability of the proposed method.

Expert knowledge elicitation with SHELF: one distribution, one expert

Expert knowledge elicitation with SHELF: one distribution, one expert

Expert knowledge elicitation with SHELF: uncertainty and probability

Expert knowledge elicitation with SHELF: uncertainty and probability

Elicitation of domain knowledge for a machine learning model for paediatric critical illness in South Africa.

Delays in identification, resuscitation and referral have been identified as a preventable cause of avoidable severity of illness and mortality in South African children. To address this problem, a machine learning model to predict a compound outcome of death prior to discharge from hospital and/or admission to the PICU was developed. A key aspect of developing machine learning models is the integration of human knowledge in their development. The objective of this study is to describe how this domain knowledge was elicited, including the use of a documented literature search and Delphi procedure. A prospective mixed methodology development study was conducted that included qualitative aspects in the elicitation of domain knowledge, together with descriptive and analytical quantitative and machine learning methodologies. A single centre tertiary hospital providing acute paediatric services. Three paediatric intensivists, six specialist paediatricians and three specialist anaesthesiologists. None. The literature search identified 154 full-text articles reporting risk factors for mortality in hospitalised children. These factors were most commonly features of specific organ dysfunction. 89 of these publications studied children in lower- and middle-income countries. The Delphi procedure included 12 expert participants and was conducted over 3 rounds. Respondents identified a need to achieve a compromise between model performance, comprehensiveness and veracity and practicality of use. Participants achieved consensus on a range of clinical features associated with severe illness in children. No special investigations were considered for inclusion in the model except point-of-care capillary blood glucose testing. The results were integrated by the researcher and a final list of features was compiled. The elicitation of domain knowledge is important in effective machine learning applications. The documentation of this process enhances rigour in such models and should be reported in publications. A documented literature search, Delphi procedure and the integration of the domain knowledge of the researchers contributed to problem specification and selection of features prior to feature engineering, pre-processing and model development.

Development of adverse outcome pathways relevant for the identification of substances having endocrine disruption properties Uterine adenocarcinoma as adverse outcome.

Development of adverse outcome pathways (AOPs) for uterine adenocarcinoma can provide a practical tool to implement the EFSA-ECHA Guidance (2018) for the identification of endocrine disruptors in the context of Regulations (EU) No 528/2012 and (EC) No 1107/2009. AOPs can give indications about the strength of the relationship between an adverse outcome (intended as a human health outcome) and chemicals (pesticides but not only) affecting the pathways. In this scientific opinion, the PPR Panelexplored the development of AOPs for uterine adenocarcinoma. An evidence-based approach methodology was applied, and literature reviews were produced using a structured framework assuring transparency, objectivity, and comprehensiveness. Several AOPs were developed; these converged to a common critical node, that is increased estradiol availability in the uterus followed by estrogen receptor activation in the endometrium; therefore, a putative AOP network was considered. An uncertainty analysis and a probabilistic quantification of the weight of evidence have been carried out via expert knowledge elicitation for each set of MIEs/KEs/KERs included in individual AOPs. The collected data on the AOP network were evaluated qualitatively, whereas a quantitative uncertainty analysis for weight of the AOP network certainty has not been performed. Recommendations are provided, including exploring further the uncertainties identified in the AOPs and putative AOP network; further methodological developments for quantifying the certainty of the KERs and of the overall AOPs and AOP network; and investigating of NAMs applications in the context of some of the MIEs/KEs currently part of the putative AOP network developed.

Travel scenario workshops for geographical accessibility modeling of health services: A transdisciplinary evaluation study.

Limited geographical access to quality Emergency Obstetric and Newborn Care (EmONC) is a major driver of high maternal mortality. Geographic access to EmONC facilities is identified by the global community as a critical issue for reducing maternal mortality and is proposed as a global indicator by the Ending Preventable Maternal Mortality (EPMM) initiative. Geographic accessibility models can provide insight into the population that lacks adequate access and on the optimal distribution of facilities and resources. Travel scenarios (i.e., modes and speed of transport) used to compute geographical access to healthcare are a key input to these models and should approximate reality as much as possible. This study explores strategies to optimize and harmonize knowledge elicitation practices for developing travel scenarios. Knowledge elicitation practices for travel scenario workshops (TSW) were studied in 14 African and South-Asian countries where the United Nations Population Fund supported ministries of health and governments in strengthening networks of EmONC facilities. This was done through a mixed methods evaluation study following a transdisciplinary approach, applying the four phases of the Interactive Learning and Action methodology: exploration, in-depth, integration, and prioritization and action planning. Data was collected in November 2020-June 2021 and involved scoping activities, stakeholder identification, semi-structured interviews (N = 9), an evaluation survey (N = 31), and two co-creating focus group discussions (N = 8). Estimating realistic travel speeds and limited time for the workshop were considered as the largest barriers. The identified opportunities were inclusively prioritized, whereby preparation; a favorable composition of attendees; validation practices; and evaluation were anticipated to be the most promising improvement strategies, explaining their central place on the co-developed initial standard operating procedure (SOP) for future TSWs. Mostly extensive preparation-both on the side of the organization and the attendees-was anticipated to address nearly all of the identified TSW challenges. This study showed that the different identified stakeholders had contradicting, complementing and overlapping ideas about strategies to optimize and harmonize TSWs. Yet, an initial SOP was inclusively developed, emphasizing practices for before, during and after each TSW. This SOP is not only relevant in the context of the UNFPA EmONC development approach, but also for monitoring the newly launched EPMM indicator and even in the broader field of geographic accessibility modeling.

Addressing Human and Organizational Factors in Nuclear Industry Modernization: A Sociotechnically Based Strategic Framework

The modernization of nuclear power plants will require an advanced concept of operations, involving an integrated set of tightly coupled systems in which all stakeholders act in a coordinated manner. For this modernization effort to be enabled, we developed a human and organizational factors approach based on a broad sociotechnical framework. Starting from core human factors principles, we conducted a literature review of the methods and approaches relevant to the modernization problem. These included not only core disciplines such as cognitive systems engineering, systems theoretic accident modeling and processes, human systems integration, resilience engineering, and macroergonomics but also related topics of safety culture and organizational change. From this literature, we developed a conceptual framework centered around the work system with its four interacting components: people, technology, process, and governance. In an effective work system, these four components are jointly optimized according to three systems criteria: efficiency, effectiveness, and safety. System failure may result from excessive emphasis on any one criterion. The actual work of attaining joint optimization in a given work system can be accomplished by utilizing three high-level functions: knowledge elicitation, knowledge representation, and cross-functional integration. We illustrated the utility of this approach by applying it to practical problems and case studies.

Development of a Knowledge-Based Expert System for Diagnosing Post-Harvest Diseases of Apple

Post-harvest diseases are one of the main causes of economical losses in the apple fruit production sector. Therefore, this paper presents an application of a knowledge-based expert system to diagnose post-harvest diseases of apple. Specifically, we detail the process of domain knowledge elicitation for constructing a Bayesian network reasoning system. We describe the developed expert system, dubbed BN-DSSApple, and the diagnostic mechanism given the evidence provided by the user, as well as a likelihood evidence method, learned from the estimated consensus of users’ and expert’s interactions, to effectively transfer the performance of the model to different cohorts of users. Finally, we detail a novel technique for explaining the provided diagnosis, thus increasing the trust in the system. We evaluate BN-DSSApple with three different types of user studies, involving real diseased apples, where the ground truth of the target instances was established by microbiological and DNA analysis. The experiments demonstrate the performance differences in the knowledge-based reasoning mechanism due to heterogeneous users interacting with the system under various conditions and the capability of the likelihood-based method to improve the diagnostic performance in different environments.

Start-up collaboration units as knowledge brokers in Corporate Innovation Ecosystems: A study in the automotive industry

Start-up collaboration units (SCUs) are organizational units specifically created by large companies to facilitate collaboration with start-ups within their corporate innovation ecosystems (CIEs). The purpose of this study is to analyze the role of SCUs with reference to the transfer of knowledge between start-ups and large companies in collaborative innovation projects. An embedded case study was undertaken with three large original equipment manufacturers (OEMs) in the automotive sector. Interviews were conducted with managers of the three OEMs, of the intermediaries they collaborate with, and of the start-ups involved. Data were systematically coded and analyzed. The main barriers to knowledge exchange in partnerships between large companies and start-ups were identified, i.e. mismatches in the interpretation of knowledge and mismatches in the expectations of the partners. In addition, strategies have been identified that can be used by SCUs to facilitate knowledge flows, namely building networks, integrating communication, knowledge elicitation, orchestrating dialog, thinking outside the box, and increasing agility. This study is one of the first to focus on SCUs. It contributes to understanding collaborative innovation processes involving start-ups by studying the knowledge-broker role played by SCUs in the complex network of actors (large companies, start-ups, innovation intermediaries, public institutions) that make up a CIE.

Gathering Expert Knowledge in Process Industry

Knowledge intensive processes are often driven and constrained by mental models of experts. Especially in industrial processes, the knowledge is not explicit but tacit in many cases, which makes it difficult to provide verbal descriptions. Modeling and prediction tasks are often not compatible or differ from the view of experts if no additional knowledge regarding the process itself or important parts of the process are identified and integrated. In industrial environments, the required knowledge is present in operators, which have to deal with the task or process on a frequent basis. Including operators during the development process may increase the quality and performance as well as trust in the resulting application. Data scientists face great difficulties in eliciting and representing the knowledge of the experts in these systems for solving the given tasks. In this paper, we will provide a review of strategies and approaches for knowledge elicitation in industrial environments. The approaches focus on applicability in industrial use-cases, whereat the operators are the main source for knowledge. The differences of the approaches are highlighted and use-cases for their application are presented. Furthermore, we will provide a discussion on using these approaches at different stages during a project as usually multiple knowledge elicitation phases are required to reach a certain goal.

Evaluating the performance of natural agriculture in Cameroon by the probabilistic elicitation of expert knowledge

Evaluating the performance of natural agriculture in Cameroon by the probabilistic elicitation of expert knowledge

Evaluating the performance of natural agriculture in Cameroon by the probabilistic elicitation of expert knowledge

Evaluating the performance of natural agriculture in Cameroon by the probabilistic elicitation of expert knowledge

Commodity risk assessment of Prunus persica and P. dulcis plants from Türkiye.

The European Commission requested the EFSA Panelon Plant Health to prepare and deliver risk assessments for commodities listed in Commission Implementing Regulation (EU) 2018/2019 as 'High risk plants, plant products and other objects'. This Scientific Opinion covers plant health risks posed by plants of Prunus persica and P. dulcis, as budwood/graftwood, rooted or grafted on rootstocks of either P.persica, P. dulcis, P. armeniaca, P. davidiana or their hybrids, imported from Türkiye, taking into account the available scientific information, including the technical information provided by Türkiye. All pests associated with the commodity were evaluated against specific criteria for their relevance for this opinion. Four quarantine pests (peach rosette mosaic virus, tomato ringspot virus, Anoplophora chinensis, Scirtothrips dorsalis) and 14 non-regulated pests (Hoplolaimus galeatus, Lasiodiplodia pseudotheobromae, Neoscytalidium dimidiatum, Neoscytalidium novaehollandiae, Didesmococcus unifasciatus, Euzophera semifuneralis, Lepidosaphes malicola, Lepidosaphes pistaciae, Maconellicoccus hirsutus, Malacosoma parallela, Nipaecoccus viridis, Phenacoccus solenopsis, Pochazia shantungensis, Russellaspis pustulans) that fulfilled all relevant criteria were selected for further evaluation. For these 18 pests, the risk mitigation measures proposed in the technical Dossier from Türkiye were evaluated taking into account the possible limiting factors. For the selected pests, an expert judgement is given on the likelihood of pest freedom taking into consideration the risk mitigation measures acting on the pest, including uncertainties associated with the assessment. The degree of pest freedom varies among the pests evaluated, with fungi from Botryosphaeriaceae family (L. pseudotheobromae, N. dimidiatum and N. novaehollandiae) being the pests most frequently expected on the imported plants. The Expert Knowledge Elicitation indicated with 95% certainty that between 9,813 and 10,000 bundles (consisting of 10 or 25 plants each) per 10,000 would be free from the above-mentioned fungi in the Botryosphaeriaceae family.

Prior Knowledge Elicitation: The Past, Present, and Future

Prior Knowledge Elicitation: The Past, Present, and Future

Decolonization of Digital Learning Spaces: It’s Not About Knowing More but Knowing Better

Working alongside members of communities who are remote and/or marginalized from the dominant socio-economic powers, the long-term goal of the Decolonisation of Digital Learning Spaces project is to empower communities in choosing, adopting, developing, and/or appropriating culturally appropriate and sustainable digital learning technologies. Before we can co-envision useful options, however, we must first know what questions to ask and how to ask. It is necessary, therefore, to find appropriate, efficient, and innovative approaches to better understand community needs and values. This paper describes the preliminary planning of the research project in creating an international network of community members, activists, and researchers, and in identifying and testing methods for eliciting needs, values, and ways of understanding the world. Selected methods must allow the researchers to step outside their own pre-conceived understandings to avoid dominating or imposing meaning upon the participants’ understandings. In this presentation, we describe: a) the goals and concerns that were the impetus for the project, b) the nascent network, c) potential knowledge elicitation methods, and d) the repeated single-criterion card sort method as the first method that will be piloted. This deceptively simple method allows research participants to use their own words to express their conceptualizations thereby reducing the influence of the researcher upon participants’ mental model and values.

Applying cognitive task analysis to health services research.

Designing practical decision support tools and other health care technology in health services research relies on a clear understanding of the cognitive processes that underlie the use of these tools. Unfortunately, methods to explore cognitive processes are rarely used in health services research. Thus, the objective of this manuscript is to introduce cognitive task analysis (CTA), a family of methods to study cognitive processes involved in completing a task, to a health services research audience. This methods article describes CTA procedures, proposes a framework for their use in health services research studies, and provides an example of its application in a pilot study. Observations and interviews of health care providers involved in discharge planning at six hospitals in the Veterans Health Administration. Qualitative study of discharge planning using CTA. Data were collected from structured observations and semi-structured interviews using the Critical Decision Method and analyzed using thematic analysis. We developed an adaptation of CTA that could be used in a clinical environment to describe clinical decision-making and other cognitive processes. The adapted CTA framework guides the user through four steps: (1) Planning, (2) Environmental Analysis, (3) Knowledge Elicitation, and (4) Analyses and Results. This adapted CTA framework provides an iterative and systematic approach to identifying and describing the knowledge, expertise, thought processes, procedures, actors, goals, and mental strategies that underlie completing a clinical task. A better understanding of the cognitive processes that underly clinical tasks is key to developing health care technology and decision-support tools that will have a meaningful impact on processes of care and patient outcomes. Our adapted framework offers a more rigorous and detailed method for identifying task-related cognitive processes in implementation studies and quality improvement. Our adaptation of this underutilized qualitative research method may be helpful to other researchers and inform future research in health services research.

A Novel Tool to Assess the Risk for African Swine Fever in Hunting Environments: The Balkan Experience.

In Europe, African swine fever (ASF) can be sustained within wild boar populations, thus representing a constant source of virus and a huge challenge in the management of the disease. Hunters are the key stakeholders for the prevention, detection and control of ASF in wild boar. Their behavior and the biosecurity standards applied in infected or at-risk hunting grounds have a huge impact on disease dynamics and management. The Food and Agriculture Organization (FAO) has developed a semi-quantitative survey-based novel tool to assess the risk of ASF in hunting grounds (namely the risks of introduction and spread into and between hunting grounds, and the risk of not detecting the infection) and how such risks could be reduced if mitigation or corrective measures were applied at low, medium and high effort. The weight of risk factors was determined through an expert knowledge elicitation (EKE). The surveys for each hunting ground were filled in by their respective managers. The tool's outputs allow users to visualize the different ASF risks of hunting grounds, whether as numerical values or color-coded maps, at sub-national, national and regional levels. These outputs can be used to guide policy makers, highlighting gaps or geographical areas to prioritize. The tool was used to assess hunting grounds in Kosovo1 (1 As per United Nations Security Council resolution 1244). Montenegro and Serbia, showing overall a high risk.