Knowledge Elicitation Research Articles

'It depends': Characterizing speech and language therapy for preschool children with developmental speech and language disorders.

BackgroundSeveral studies have suggested that practitioners hold speech and language therapy (SLT) practice as tacit and consequently it is difficult for the therapist to describe. The current study uses a range of knowledge elicitation (KE) approaches, a technique not used before in SLT, as a way of accessing this tacit knowledge. There is currently no agreed framework that establishes key factors underpinning practice for preschool children with speech and language disorders. This paper attempts to address that gap.AimsTo develop a framework of SLTs’ practice when working with preschool children with developmental speech and language disorders (DS&LD).Methods & ProceduresA mixed‐methods approach was adopted for this study. Data were collected iteratively, from 245 SLTs with experience of working with preschool children with DS&LD across sites in England, by means of focus groups and national events. There were three stages of data collection: local sites, specific‐interest groups and two national events. KE techniques were used to gather data, with initial data being collected in local site focus groups. Findings from groups were taken to subsequent larger groups where a combination of concept mapping, teach‐back and sorting exercises generated a more detailed description of practice, using discussion of consensus and disagreement to stimulate further exploration and definition and provide validatory evidence.Outcomes & ResultsThis paper provides a high‐level framework of therapy for preschool children with DS&LD that makes practice explicit in this area. The framework proposes that therapists’ aims for this group of children fall into three categories: addressing children's areas of impairment and skills; achieving functionally meaningful skills and carryover; and supporting adults to provide a supportive communication environment. The exact configuration is shaped by the child's context and needs.Conclusions & ImplicationsThe framework highlights themes that are well researched in the literature (e.g., speech) and others that have been little studied (e.g., adult understanding), indicating a disconnect between research evidence and practice. The research also highlights the complex nature of interventions for preschool children with DS&LD and the importance therapists attribute to tailoring therapy to individual needs. The framework provides a scaffold upon which SLTs can focus their clinical practice and encourages the profession to understand and explore better the gaps between research evidence and clinical practice for preschool children with DS&LD.

Knowledge elicitation based on genetic programming for non destructive testing of critical aerospace systems

In non-destructive testing of aerospace structures’ defects, the tests reliability is a crucial issue for guaranteeing security of both aircrafts and passengers. Most of the widely recognized approaches rely on precision and reliability of testing equipment, but also the methods and techniques used for processing measurement results, in order to detect defects, may heavily influence the overall quality of the testing process. The effectiveness of such methods strongly depends on specific field knowledge that is definitely not easy to be formalized and codified within the results processing practices. Although many studies have been conducted in this direction, such issue is yet an open-problem. This work describes the use of Genetic Programming for the diagnosis and modeling of aerospace structural defects. The resulting approach aims at extracting such knowledge by providing a mathematical model of the considered defects, which can be used for recognizing other similar ones. Eddy-Current Testing has been selected as a case study in order to assess both the performance and functionality of the whole framework, and a publicly available dataset of specific measures for aircraft structures has been considered. The experimental results put into evidence the effectiveness of the proposed approach in building reliable models of the aforementioned defects, so that it can be considered a successful option for building the knowledge needed by tools for controlling the quality of critical aerospace systems.

Connected Health Services: Framework for an Impact Assessment.

BackgroundConnected health (CH), as a new paradigm, manages individual and community health in a holistic manner by leveraging a variety of technologies and has the potential for the incorporation of telehealth and integrated care services, covering the whole spectrum of health-related services addressing healthy subjects and chronic patients. The reorganization of services around the person or citizen has been expected to bring high impact in the health care domain. There are a series of concerns (eg, contextual factors influencing the impact of care models, the cost savings associated with CH solutions, and the sustainability of the CH ecosystem) that should be better addressed for CH technologies to reach stakeholders more successfully. Overall, there is a need to effectively establish an understanding of the concepts of CH impact. As services based on CH technologies go beyond standard clinical interventions and assessments of medical devices or medical treatments, the need for standardization and for new ways of measurements and assessments emerges when studying CH impact.ObjectiveThis study aimed to introduce the CH impact framework (CHIF) that serves as an approach to assess the impact of CH services.MethodsThis study focused on the subset of CH comprising services that directly address patients and citizens on the management of disease or health and wellness. The CHIF was developed through a multistep procedure and various activities. These included, as initial steps, a literature review and workshop focusing on knowledge elicitation around CH concepts. Then followed the development of the initial version of the framework, refining of the framework with the experts as a result of the second workshop, and, finally, composition and deployment of a questionnaire for preliminary feedback from early-stage researchers in the relevant domains.ResultsThe framework contributes to a better understanding of what is CH impact and analyzes the factors toward achieving it. CHIF elaborates on how to assess impact in CH services. These aspects can contribute to an impact-aware design of CH services. It can also contribute to a comparison of CH services and further knowledge of the domain. The CHIF is based on 4 concepts, including CH system and service outline, CH system end users, CH outcomes, and factors toward achieving CH impact. The framework is visualized as an ontological model.ConclusionsThe CHIF is an initial step toward identifying methodologies to objectively measure CH impact while recognizing its multiple dimensions and scales.

Description of surveillance components related to classical swine fever, blue tongue and rabies in selected European countries: An experts’ knowledge elicitation

In the European Union, there are surveillance systems to detect early appearance of classical swine fever (CSF), bluetongue (BT) and rabies, all notifiable diseases. In this study, we conducted a survey in three European countries, member of SPARE (SPatial Assessments of Risk for Europe) consortium to assess the surveillance systems for CSF, BT and rabies. The elicitation process recruited experts among national institutes in the targeted countries and the questionnaire gathered descriptive information on the surveillance system and evaluation feedbacks against 12 attributes for each surveillance component: timeliness, sensitivity, representativeness, acceptability, flexibility, coverage, costs, effectiveness, efficiency, impact, benefit, communication and dissemination. To express the overall perception of national experts, we calculated the median of the evaluation scores for each evaluation attribute across all components. The implementation of a passive surveillance for domestic and wild pigs for CSF was shared by all countries, although the source of data collection differed for slaughterhouses and farms versus wild environments. For BT, countries adopt different approaches because no surveillance component is common among the targeted countries. For rabies, passive surveillance of wildlife is a common component. The study highlighted differences between the countries in the organisation and design of the surveillance systems even when objective are similar (e.g. early detection) and implemented under the same EU legislative framework. The choice of surveillance components was a multifactorial process influenced by the epidemiological situation of the hazard in the country, national contexts, political dynamics, interests and country sanitary priorities. The different levels of performance of surveillance components reported by the recruited experts are interpreted as a major concern for livestock population, the economy and the public health in EU.

A risk‐based inventory of invasive plant species of Queensland, Australia: Regional, ecological and floristic insights

AbstractInvasive alien plant species threaten agriculture and biodiversity globally and require ongoing management to minimise impacts. However, the large number of invasive species means that a risk‐based approach to prioritisation is needed, taking into account the spatial scale of management decisions and myriad of available information. Here, we developed a risk‐based inventory of invasive plants in Queensland, Australia, using both current species distribution/abundance and the severity of their impacts. Our assessment followed a comprehensive data collection process including a scoping of local government pest management plans, herbarium records, the published literature and structured elicitation of expert knowledge during a series of regional stakeholder workshops. From ~300 plant species that were identified as established and/or emerging invaders in the State, only one‐third were considered by practitioners to pose significant risks across regions to be considered management priorities. We aggregated regional species lists into a statewide priority list and analysed the data set (107 species) for historical, geographical, floristic and ecological patterns. Regions on the mainland eastern seaboard of the State share similar invasive plant communities, suggesting that these regions may form a single management unit, unlike the western/inland and the extreme far north (Torres Strait Islands) regions, which share fewer invasive plant species. Positive correlations were detected between invasiveness and time since introduction for some but not all plant life forms. Stakeholders identified research and management priorities for the invasive plant list, including biological control options, public awareness/education, effective herbicide use, ecology/taxonomy and risk analysis. In the course of the exercise, a statewide invasive plant priority list of high‐, medium‐ and low‐impact scores for policy, research and management was compiled. Finally, our approach to invasive plant species prioritisation highlighted that planning and policy documents are not necessarily reflected at the grass‐root level in terms of species identity and management priorities.

Youth and forecasting of sustainable development pillars: An adaptive neuro-fuzzy inference system approach.

Sustainable development goals are used as a guidance for strategies development on local, regional and national levels. The importance of including young people in this complex process is recognized in all relevant documents (i.e. Agenda 21), however it is not an easy task to elicit opinions and preferences from the youth. Furthermore, the assessment of the sustainable development goals itself presents a challenge for the noisy data and nonlinear relationships in data. Popular approach is fuzzy set models where expert knowledge is presented with comprehensible rules; however expert knowledge elicitation takes a long time too. Several studies proposed an adaptive neuro-fuzzy inference system approach that combines the fuzzy set theory to model expert knowledge with neural networks for inferring rules and membership functions from data to assess the sustainable development performance. We base our assumptions that ANFIS can be used to predict the importance of sustainable development pillars from the demographic data of young people. For this purpose, we have conducted an online survey on sustainable development goals opinions and importance of young people in Serbia. The sample of 386 respondents has been split into a training sample of 300 instances (to generate membership functions and fuzzy rules) and a testing sample of 86 instances to predict the importance of the three pillars. We have conducted a trace-driven simulation test to validate the results of the proposed ANFIS model. Results of the study provided insights into how the young people in Serbia assess the importance of sustainable development goals. Secondly, the results suggest that ANFIS can be applied to predict values of importance of the three sustainable development pillars with the relative error of Rel Err < 5%. It must be noted that the considered model could be further improved by using training samples with more data.

Stochastic impact evaluation of an irrigation development intervention in Northern Ethiopia

Irrigation plays a significant role in achieving food and nutrition security in dry regions. However, detailed ex-ante appraisals of irrigation development investments are required to efficiently allocate resources and optimize returns on investment. Due to the inherent system complexity and uncertain consequences of irrigation development interventions coupled with limited data availability, deterministic cost-benefit analysis can be ineffective in guiding formal decision-making. Stochastic Impact Evaluation (SIE) helps to overcome the challenges of evaluating investments in such contexts. In this paper, we applied SIE to assess the viability of an irrigation dam construction project in northern Ethiopia. We used expert knowledge elicitation to generate a causal model of the planned intervention's impact pathway, including all identified benefits, costs and risks. Estimates of the input variables were collected from ten subject matter experts. We then applied the SIE tools: Monte Carlo simulation, Partial Least Squares regression, and Value of Information analysis to project prospective impacts of the project and identify critical knowledge gaps. Model results indicate that the proposed irrigation dam project is highly likely to increase the overall benefits and improve food and nutrition status of local farmers. However, the overall value of these benefits is unlikely to exceed the sum of the investment costs and negative externalities involved in the intervention. Simulation results suggest that the planned irrigation dam may improve income, as well as food and nutrition security, but would generate negative environmental effects and high investment costs. The Stochastic Impact Evaluation approach proved effective in this study and is likely to have potential for evaluating other agricultural development interventions that face system complexity, data scarcity and uncertainty.

Report on the methodology applied by EFSA to provide a quantitative assessment of pest-related criteria required to rank candidate priority pests as defined by Regulation (EU) 2016/2031.

In agreement with Article 6(2) of the Regulation (EU) 2016/2031 on protective measures against pests of plants, the European Commission has been tasked by the Council and European Parliament to establish a list of Union quarantine pests which qualify as priority pests. The prioritisation is based on the severity of the economic, social and environmental impact that these pests can cause in the Union territory. The Commission's Joint Research Centre (JRC) is in charge of developing a methodology based on a multi‐criteria decision analysis and composite indicators. In this context, EFSA has provided technical and scientific data related to these pests, in particular: (i) the potential host range and distribution of each of these pests in the Union territory at the level of NUTS2 regions; (ii) parameters quantifying the potential consequences of these pests, e.g. crop losses in terms of yield and quality, rate of spread and time to detection. Expert knowledge elicitation methodology has been applied by EFSA in order to provide those parameters in a consistent and transparent manner.

Extension of the Monte‐Carlo WEB application and Expert Knowledge Elicitation WEB application

Extension of the Monte‐Carlo WEB application and Expert Knowledge Elicitation WEB application

Automatic Rule Generator via FP-Growth for Eye Diseases Diagnosis

The conventional approach in developing a rule-based expert system usually applies a tedious, lengthy and costly knowledge acquisition process. The acquisition process is known as the bottleneck in developing an expert system. Furthermore, manual knowledge acquisition can eventually lead to erroneous in decision-making and function ineffective when designing any expert system. Another dilemma among knowledge engineers are handing conflict of interest or high variance of inter and intrapersonal decisions among domain experts during knowledge elicitation stage. The aim of this research is to improve the acquisition of knowledge level using a data mining technique. This paper investigates the effectiveness of an association rule mining technique in generating new rules for an expert system. In this paper, FP-Growth is the machine learning technique that was used in acquiring rules from the eye disease diagnosis records collected from Sumatera Eye Center (SMEC) Hospital in Pekanbaru, Riau, Indonesia. The developed systems are tested with 17 cases. The ophthalmologists inspected the results from automatic rule generator for eye diseases diagnosis. We found that the introduction of FP-Growth association rules into the eye disease knowledge-based systems, able to produce acceptable and promising eye diagnosing results approximately 88% of average accuracy rate. Based on the test results, we can conclude that Conjunctivitis and Presbyopia disease are the most dominant suffering in Indonesia. In conclusion, FP-growth association rules are very potential and capable of becoming an adequate automatic rules generator, but still has plenty of room for improvement in the context of eye disease diagnosing.

Combining empirical indicators and expert knowledge for surveillance of data-limited sea trout stocks

Abstract Inland and diadromous fish stocks can support important ecological, social and cultural functions. However, many of these stocks are data-limited and do not have formal state assessments or management reference points. Empirical surveillance indicators of relative abundance and population size-structure may provide an accessible first-step for monitoring such stocks. These indicators could be informed and interpreted via expert knowledge of specific systems and stock histories. The current study focused on an international scientific working group that met in 2017–2019. The group collated long-term monitoring data for sea trout Salmo trutta from six salmonid ‘index rivers’ in France, Ireland, Wales, Northern Ireland and Sweden. Indicators were selected, and time series of abundance- and length-based metrics for each river were presented on a new visual indicator plot. Expert knowledge from each system was elicited through one-to-one interviews to provide a corresponding stock narrative, in which indicator trends reflected known historical pressure-state events. The accessible presentation of simple empirical indicators supports elicitation of local expert knowledge, and provides an accessible framework for surveillance and reference-direction assessment of data-limited sea trout stocks. This approach could be applied to inland or diadromous stocks where available data are restricted to estimates of relative abundance and size-structure.

Harnessing social media data for pharmacovigilance: a review of current state of the art, challenges and future directions

The ever-increasing supply of information combined with the growing knowledge elicitation capabilities of key emerging technologies presents pharmacovigilance with enormous opportunities. Currently, safety monitoring is expanding its evidence base, moving beyond traditional approaches towards sophisticated methods that can identify possible safety signals from multiple information sources, both structured and unstructured. In this context, health information posted online by patients represents a potentially valuable, yet currently left largely unexploited source of post-market safety data that could supplement data from traditional sources of drug safety information. As the use of social media data for pharmacovigilance is still in its infancy, the present paper explores the state of the art in the application of social data to adverse drug reaction detection; provides a thorough review of existing work in the field, highlighting important research efforts and achievements; and finally, discusses the current challenges and promising avenues for future work. Following a literature review methodology, a critical appraisal was conducted of carefully selected work on the use of social data in post-market surveillance, as presented in the recent scientific literature. Out of a sample of more than 1300 articles, which was the result of the literature search, the final selection of articles was made based on their relevance to the applications of social networking sites (SNS) to pharmacovigilance, and a thorough review of this corpus was completed with a total of 100 articles reviewed. The main contributions of this review include the mapping and systematisation of the current knowledge in the field by drawing comparisons of different approaches, types of social data and of relevant sources currently used in the field, and by developing new classifications of social data sources and taxonomies for social data for use in pharmacovigilance, as well as the identification of key challenges and the extraction of new insights in terms of potential for practical applications and future research directions in the area of pharmacovigilance.

Towards the development of an index for the holistic assessment of the health status of a honey bee colony

Honey bees play an important role in the maintenance of both, biodiversity and food security through pollination services and also represent an important source of income for rural areas. Despite several studies and monitoring projects gathering a large amount of data on the main factors/stressors influencing honey bee colonies, there is a lack of holistic and multidimensional statistical tools integrating different aspects which define honey bee colony health. Such tools are important to support the sustainable management of honey bees. In this study, we designed and tested a methodological framework based on Structural Equation Models for the development of a honey bee Health Status Index. The index accounts for the main abiotic (e.g. pesticide contamination, landscape characteristics) and biotic (e.g. parasitosis, virosis) factors influencing honey bee health, including the beekeepers role in managing the colony. The proposed methodology was validated against a dataset representing seven scenarios generated through Expert Knowledge Elicitation. The validation procedure showed that the Health Status Index integrates different sources of data and it can quantify the health status of a honey bee colony based on the characteristics of each scenario. Furthermore, it allows the investigation of the influence of different drivers/stressors on the health of the honey bee colony. The Health Status Index provides flexibility in the selection of variables, making it a valuable tool for holistic and harmonised assessment of honey bee health. Once validated, the index can support the evaluation at different spatial (from local- to area-wide management) and temporal (medium- to long-term management) scales to support stakeholders’ (e.g. beekeepers, risk assessors, risk managers) decision-making.

Modelling Subject Domain Causality for Learning Content Renewal

The paper deals with the causality driven modelling method applied for the domain deep knowledge elicitation. This method is suitable for discovering causal relationships in domains that are characterized by internal circular causality, e.g. control and management, regulatory processes, self-regulation and renewal. Such domains are organizational systems (i.e. enterprise) or cyber-social systems, also biological systems, ecological systems, and other complex systems. Subject domain may be of different nature: real-world activities or documented content. A causality driven approach is applied here for the learning content analysis and normalization of the knowledge structures. This method was used in the field of education, and a case study of learning content renewal is provided. The domain here is a real world area – a learning content is about. The paper is on how to align the existing learning content and current (new) knowledge of the domain using the same causality driven viewpoint and the described models (frameworks). Two levels of the domain causal modelling are obtained. The first level is the discovery of the causality of the domain using the Management Transaction (MT) framework. Secondly, a deep knowledge structure of MT is revealed through a more detailed framework called the Elementary Management Cycle (EMC). The algorithms for updating the LO content in two steps are presented. Traceability matrix indicates the mismatch of the LO content (old knowledge) and new domain knowledge. Classification of the content discrepancies and an example of the study program content analysis is presented. The main outcome of the causality driven modelling approach is the effectiveness of discovering the deep knowledge when the relevant domain causality frameworks are applicable.

Limbless Medical Data Analyzing using CRISP Model a Case Study of UK Limbless Patients

Data mining, usually known as knowledge elicitation in the field of computer science databases, is the procedure to find out an important relationship, useful patterns in a huge amount of raw data. Besides, many sectors have adapted and used data mining in their applications such as healthcare and industry sector. In the healthcare sector, data mining can help in determining the probability of particular health cases in medical issues which the related variables pre-known as well as predicting future events. The availability of medical data for data mining usually exist in a raw data format, therefore, it needs for making ready and exploration to be willing to use. In the context of this paper, an analyzing of medical data was introduced to support prosthetics service centers to analyze find out the significant information from limbless medical cases, besides, in providing a comprehensive understanding of amputation and its types as well as the level of amputation. To ensure extract meaningful information from the intended data sets as well as to follow a systematic approach, the CRISP-DM model was adopted. The findings show the important and meaningful of the analyzing data using data mining modes.

Misconceptions and persistence: resources for targeting student alternative conceptions in biotechnology.

Conceptual understanding and reasoning of nonscience major students enrolled in a course on biotechnology were evaluated before and after instruction. The instrument for analysis of student understanding was the Biotechnology Instrument for Knowledge Elicitation (BIKE). The BIKE targets 11 key concepts, as determined by experts in the field. A statistically significant score improvement was observed in each of the 11 concept areas after completion of the course, Biotechnology in Society (N = 117). Student responses to both the pretest and posttest were highly informative and revealed several common misconceptions that could have been overlooked in a closed form testing scenario. These alternative conceptions and recommended clarifications have been presented here. Our goal in documenting and distributing these is to better equip educators in biology and biochemistry to anticipate student preconceptions, design targeted interventions, and improve student conceptual reasoning in topics pertaining to biology and biotechnology. © 2018 International Union of Biochemistry and Molecular Biology, 46(6):602-611, 2018.

Writing Processes in Short Written Responses to Questions Probing Prior Knowledge

AbstractThe purpose of the report is to explore some of the mechanisms involved in the writing process. In particular, we examine students' process data (keystroke log analysis) to uncover how students approach a knowledge‐telling task using 2 different task types. In the first task, students were asked to list as many words as possible related to a particular topic (word listing). In a second task, students were asked to write to a specific prompt that was designed to elicit their background knowledge of a topic using connected text (knowledge elicitation). Using a matrix incomplete block design, 1,592 high school students completed the 2 writing tasks in addition to a multiple‐choice test of their background knowledge in 2 of 5 possible topics in the domain of U.S. history. An array of process data including students' typing and associated timing features was used to predict the writing scores on the 2 different types of tasks. The analyses revealed several distinct patterns that were associated with processing at the task knowledge productivity level, the editing effort level, and the keyboarding effort level. The robustness of the features was reflected in a set of hierarchal regressions that demonstrated that the process features were predictive of the writing score even when students' knowledge scores on the associated multiple‐choice test were considered. In sum, the results indicate that process data in the form of log file analysis are useful for both understanding the writing process and exploring potential differences between students with high and low knowledge.

Elicitation of Tacit Knowledge Using Soft Systems Methodology

Elicitation of tacit knowledge is of paramount importance in organizations for novice people’s use of experts’ knowledge. One of the shortcomings of knowledge elicitation techniques is lack of consensus among experts in eliciting tacit knowledge. In other words, experienced individuals have different viewpoints regarding the statement of complicated knowledge activities. Hence, using soft thinking approach, it is tried to establish agreement among experts and solve the problems of previous techniques. In the present study, a combination of concept mapping, scenario making, teach back, and card sorting techniques within the framework of soft systems methodology have been used. In this way, a methodology is presented for eliciting tacit knowledge.

Training courses on “Steering an Expert Knowledge Elicitation” and “Use of the Expert Knowledge Elicitation Guidance in Risk Assessments for EFSA Management” and “Conduct of the Sheffield protocol for an Expert Knowledge Elicitation”

Training courses on “Steering an Expert Knowledge Elicitation” and “Use of the Expert Knowledge Elicitation Guidance in Risk Assessments for EFSA Management” and “Conduct of the Sheffield protocol for an Expert Knowledge Elicitation”

Like a DNA String: Sequence-Based Player Profiling in &lt;i&gt;Tom Clancy’s The Division&lt;/i&gt;

In this paper we present an approach to using sequence analysis to model player behavior. This approach is designed to work in game development contexts, integrating production teams and delivering profiles that inform game design. We demonstrate the method via a case study of the game Tom Clancy’s The Division, which with its 20 million players represents a major current commercial title. The approach presented provides a mixed-methods framework, combining qualitative knowledge elicitation and workshops with large-scale telemetry analysis, using sequence mining and clustering to develop detailed player profiles showing the core game-play loops of The Division’s players.