Novice Modelers Research Articles

Facilitating the comprehension of business process models for unexperienced modelers using token-based animations

Process models are structured representations of workflows in an organization that provide a powerful tool for facilitating communication and process redesign or improvement. Model comprehension is challenging for beginning modelers. This study scrutinizes the effect of token-animated process models on novice modelers’ comprehension in an experiment with 229 participants. The study is grounded in the theory of distributed cognition as well as other cognition theories. The results confirm the significant impact of token-animated models on comprehension. Several individual characteristics were found to be important as well. Given that the animations were well accepted, token animation can be considered a resourceful technique for pragmatic and educational purposes.

Calibrating a process-based simulation model for the Acadian forest region

Climate change is projected to have profound impacts on Canada’s Acadian Forest Region (AFR). However, large uncertainties arising from climate change and increasing disturbance activity pose challenges for forest management decisions. Process-based (mechanistic) simulation models offer a means by which vulnerabilities and different management strategies can be tested under multiple climate and disturbance regimes. However, applying these complex models can be dauting, especially for novice modelers and forest practitioners; nonetheless, this complexity is increasingly necessary to more realistically project changes in forest growth and composition, ecosystem services, biodiversity, disturbance regimes, and the spread of forest pests. Here, we present a methodology for calibrating and validating iLand (v1.1.1), a landscape-scale, process-based forest model that offers a novel approach for assessing the feedback between individual trees and their environment (ecosystem processes, climate, and disturbance). For the first time, 18 tree species were parameterized and calibrated for the AFR and model performance was evaluated against independent field observations at the tree population and stand level. iLand was able to accurately emulate the dynamics of individual tree species populations as well as the succession of mixed-species forest stands across a range of soil conditions and is now ready to be used to simulate future forest dynamics of the AFR. We also discuss calibration method selection and the potential impacts of model and project structure in relation to our project. As the accessibility and usability of process-based forests models increases, our work provides a unique case study for forest managers looking to expand their toolbox.

Stochastic-Skill-Level-Based Shared Control for Human Training in Urban Air Mobility Scenario

This paper proposes a novel stochastic-skill-level-based shared control framework to assist human novices to emulate human experts in complex dynamic control tasks. The proposed framework aims to infer stochastic-skill-levels (SSLs) of the human novices and provide personalized assistance based on the inferred SSLs. SSL can be assessed as a stochastic variable which denotes the probability that the novice will behave similarly to experts. We propose a data-driven method which can characterize novice demonstrations as a novice model and expert demonstrations as an expert model, respectively. Then, our SSL inference approach utilizes the novice and expert models to assess the SSL of the novices in complex dynamic control tasks. The shared control scheme is designed to dynamically adjust the level of assistance based on the inferred SSL to prevent frustration or tedium during human training due to poorly imposed assistance. The proposed framework is demonstrated by a human subject experiment in a human training scenario for a remotely piloted urban air mobility (UAM) vehicle. The results show that the proposed framework can assess the SSL and tailor the assistance for an individual in real-time. The proposed framework is compared to practice-only training (no assistance) and a baseline shared control approach to test the human learning rates in the designed training scenario with human subjects. A subjective survey is also examined to monitor the user experience of the proposed framework.

Ten strategies towards successful calibration of environmental models

Model calibration is the procedure of finding model settings such that simulated model outputs best match the observed data. Model calibration is necessary when the model parameters cannot directly be measured as is the case with a wide range of environmental models where parameters are conceptually describing upscaled and effective physical processes. Model calibration is therefore an important step of environmental modeling as the model might otherwise provide random outputs if never compared to a ground truth. Model calibration itself is often referred to be an art due to its plenitude of intertwined steps and necessary decisions along the way before a calibration can be carried out or can be regarded successful.This work provides a general guide specifying which steps a modeler needs to undertake, how to diagnose the success of each step, and how to identify the right action to revise steps that were not successful. The procedure is formalized into ten iterative steps generally appearing in calibration experiments. Each step of this “calibration life cycle” is either illustrated with an exemplary calibration experiment or providing an explicit checklist the modeler can follow. These ten strategies are: (1) using sensitivity information to guide the calibration, (2) handling of parameters with constraints, (3) handling of data ranging orders of magnitude, (4) choosing the data to base the calibration on, (5) presenting various methods to sample model parameters, (6) finding appropriate parameter ranges, (7) choosing objective functions, (8) selecting a calibration algorithm, (9) determining the success and quality of a multi-objective calibration, and (10) providing a checklist to diagnose calibration performance using ideas introduced in the previous steps. The formal definition of strategies through the calibration process is providing an overview while shedding a light on connections between these main ingredients to calibrate an environmental model and will therefore enable especially novice modelers to succeed.

A flexible method for parameterizing ranked nodes in Bayesian networks using Beta distributions.

When novice modelers first attempt to build a Bayesian network, they are often impressed with the intuitive graphical structures that capture their causal understanding. This favorable impression evaporates on proceeding to parameterization. Conditional probability tables (CPT) require parameters for often hundreds of very similar scenarios and specifying them in the absence of data can be overwhelming. The problem is even more severe when eliciting parameters from experts with limited time. Often, there is local structure with fewer parameters that better describes the relationship. Such structures include the Noisy OR, decision trees, and equations. These work well for modelers, but can be an issue for experts and particularly groups of experts. An alternative approach is to elicit only a few CPT rows and interpolate the remainder. This is a promising approach, as it can handle unknown structures and multiple experts, but existing techniques can be limited. Here, we present a flexible approach called InterBeta for performing CPT interpolation with ordered nodes. In the simplest case, just two CPT rows are needed, but this can be easily augmented with further information. The basic approach assumes input independence, but allows dependencies to be reintroduced as required, and can also be combined with other local structures such as decision trees or equations, leaving the interpolator to fill in the gaps. We explain the InterBeta method, describe its capabilities and limitations and how it compares to similar approaches and show how it can trade-off elicitation effort against faithfully representing expertunderstanding.

RunBioSimulations: an extensible web application that simulates a wide range of computational modeling frameworks, algorithms, and formats.

Comprehensive, predictive computational models have significant potential for science, bioengineering, and medicine. One promising way to achieve more predictive models is to combine submodels of multiple subsystems. To capture the multiple scales of biology, these submodels will likely require multiple modeling frameworks and simulation algorithms. Several community resources are already available for working with many of these frameworks and algorithms. However, the variety and sheer number of these resources make it challenging to find and use appropriate tools for each model, especially for novice modelers and experimentalists. To make these resources easier to use, we developed RunBioSimulations (https://run.biosimulations.org), a single web application for executing a broad range of models. RunBioSimulations leverages community resources, including BioSimulators, a new open registry of simulation tools. These resources currently enable RunBioSimulations to execute nine frameworks and 44 algorithms, and they make RunBioSimulations extensible to additional frameworks and algorithms. RunBioSimulations also provides features for sharing simulations and interactively visualizing their results. We anticipate that RunBioSimulations will foster reproducibility, stimulate collaboration, and ultimately facilitate the creation of more predictive models.

Nutmeg: a MIP and CP Hybrid Solver Using Branch-and-Check

This paper describes the implementation of Nutmeg, a solver that hybridizes mixed integer linear programming and constraint programming using the branch-and-cut style of logic-based Benders decomposition known as branch-and-check. Given a high-level constraint programming model, Nutmeg automatically derives a mixed integer programming master problem that omits global constraints with weak linear relaxations, and a constraint programming subproblem identical to the original model. At every node in the branch-and-bound search tree, the linear relaxation computes dual bounds and proposes solutions, which are checked for feasibility of the omitted constraints in the constraint programming subproblem. In the case of infeasibility, conflict analysis generates Benders cuts, which are appended to the linear relaxation to cut off the candidate solution. Experimental results show that Nutmeg’s automatic decomposition outperforms pure constraint programming and pure mixed integer programming on problems known to have successful implementations of logic-based Benders decomposition, but performs poorly on general problems, which lack specific decomposable structure. Nonetheless, Nutmeg outperforms the standalone approaches on one problem with no known decomposable structure, providing preliminary indications that a hand-tailored decomposition for this problem could be worthwhile. On the whole, Nutmeg serves as a valuable tool for novice modelers to try hybrid solving and for expert modelers to quickly compare different logic-based Benders decompositions of their problems.

Supporting the Process of Learning and Teaching Process Models

The creation of a process model is primarily a formalization task that faces the challenge of constructing a syntactically correct entity, which accurately reflects the semantics of reality, and is understandable to the model reader. This article proposes a framework called Model Judge , focused toward the two main actors in the process of learning process model creation: novice modelers and instructors. For modelers, the platform enables the automatic validation of the process models created from a textual description, providing explanations about quality issues in the model. Model Judge can provide diagnostics regarding model structure, writing style, and semantics by aligning annotated textual descriptions to models. For instructors, the platform facilitates the creation of modeling exercises by providing an editor to annotate the main parts of a textual description, which is empowered with natural language processing capabilities so that the annotation effort is minimized. So far around 300 students in process modeling courses of five different universities around the world have used the platform. The feedback gathered from some of these courses shows good potential in helping students to improve their learning experience, which might, in turn, impact process model quality and understandability. Moreover, our results show that instructors can benefit from getting insights into the evolution of modeling processes, including arising quality issues of single students, but also discovering tendencies in groups of students. Although the framework has been applied to process model creation, it could be extrapolated to other contexts where the creation of models based on a textual description plays an important role.

Learning How to Throw Darts. Effects of Modeling Type and Reflection on Novices’ Dart-Throwing Skills

In this study, we investigated the effects of modeling type and reflection on the acquisition of dart-throwing skills, self-efficacy beliefs and self-reaction scores by conceptually replicating a study by Kitsantas, Zimmerman, and Cleary (2000). Participants observing a novice model were expected to surpass participants observing an expert model who in turn were expected to outperform participants who learned without a model. Reflection was hypothesized to have a positive effect. 156 High school and university students were tested three times: in a pretest, after a modeling intervention, and after a practice round. Contrary to what was expected, we found no main effects of modeling type and reflection. No interaction effects were found either. There was an effect of testing moment, indicating that participants improved dart-throwing skills, self-efficacy beliefs, and self-reaction scores over time. With these findings, we are not able to replicate Kitsantas et al. From our study, we conclude that observational learning, irrespective of the model’s skill level, combined with physical practice, yields similar results as mere physical practice.

Modeling and Pedagogy

The depth of knowledge a student will develop while building a model, the memories and skills they will gain that transcend content, and the teamwork they will develop during the process are just a few of the benefits of the modeling process. Ultimately, the modeling process will allow students and teachers to experience education in a way that matches the way work is done in scientific research, engineering, and other professional settings. Having students build models to solve problems is an enormous goal and enormous goals require enormous planning. The purpose of this guide is to outline the pedagogical approaches a teacher will need to lead their students on the journey from novice modelers to expert modelers

Toward a methodology for case modeling

Case management is increasingly used to capture and enact flexible, knowledge-intensive processes in organizations. None of the existing case management approaches provides a methodology for case model elicitation and modeling. In this contribution, three modeling methods for fragment-based case management are presented: one which focuses on the control-flow view, the process-first method, one which has a data-centric view, the object lifecycle-first method, and one which focuses on the goals of a case, the goals-first method. Following the design science process, each of the three methods was evaluated in two case modeling workshops with two different stakeholder groups (PhD students and secretaries), resulting in a total of six workshops. All participants were novices in case management and most of them as well in process modeling. The results indicate that the process-first method can be quickly learned by novices and it might be useful for scenarios where the focus is on the main process with some degree of flexibility. The object lifecycle-first method yields more flexible and consistent case models, but requires a higher initial modeling effort, as the lifecycle of the main case object has to be designed first. The goals-first method leads to a detailed and consistent case model and additionally provides, by means of the defined goals, a checklist what needs to be done for a case. This method requires in addition to the process modeling notation another model type, the goal hierarchy, and therefore is less suited for novice modelers, as found by the workshop results.

Impact of mentoring relationships on nursing professional socialization.

This study qualitatively explored the impact mentoring relationships had on the professional socialization of novice clinical nurse leader. Professional socialization entails acquisition of the skills, knowledge and values associated with nursing. Model C clinical nurse leaders have completed a bachelor's degree before graduate-level nursing programme acceptance. Thereby, the mentoring needs of model C clinical nurse leaders may differ from that of traditionally educated novice nurses. Focus groups were conducted with seven novice model C clinical nurse leaders during their first year of employment. Qualitative data were analysed via a grounded theory approach. The participants described an intense focus on patient care and how multiple mentoring relationships motivated them to become competent bedside clinicians. They described how the mentors' actions enabled them to deal with negative feelings, which increased their confidence, comfort and competence with clinical skills. Clinical skills improved when a novice model C clinical nurse leader worked with multiple mentors. The qualitative data did not show that the model C clinical nurse leaders needed different mentoring relationships than traditionally educated nurses. The results suggest multiple mentors should be used to develop the clinical competences of novice model C clinical nurse leaders.

Comparing traditional conceptual modeling with ontology-driven conceptual modeling: An empirical study

This paper conducts an empirical study that explores the differences between adopting a traditional conceptual modeling (TCM) technique and an ontology-driven conceptual modeling (ODCM) technique with the objective to understand and identify in which modeling situations an ODCM technique can prove beneficial compared to a TCM technique. More specifically, we asked ourselves if there exist any meaningful differences in the resulting conceptual model and the effort spent to create such model between novice modelers trained in an ontology-driven conceptual modeling technique and novice modelers trained in a traditional conceptual modeling technique. To answer this question, we discuss previous empirical research efforts and distill these efforts into two hypotheses. Next, these hypotheses are tested in a rigorously developed experiment, where a total of 100 students from two different Universities participated. The findings of our empirical study confirm that there do exist meaningful differences between adopting the two techniques. We observed that novice modelers applying the ODCM technique arrived at higher quality models compared to novice modelers applying the TCM technique. More specifically, the results of the empirical study demonstrated that it is advantageous to apply an ODCM technique over an TCM when having to model the more challenging and advanced facets of a certain domain or scenario. Moreover, we also did not find any significant difference in effort between applying these two techniques. Finally, we specified our results in three findings that aim to clarify the obtained results.

Mirror-hand selection is influenced by training perspective and model skill level in a motor-learning task.

This study examined mirror and non-mirror arm selection processes in an observational learning context. Observer groups watched either a novice (instruction or discovery) or skilled model performing a bimanual task with the right arm leading the left arm. The models were viewed from a third-person perspective. Observers of the skilled model more often selected a mirror-image (left-hand) hand-lead in post-observations tests, while observers of the novice models more often selected a non-mirror image (right hand) hand-lead in post-observation tests. This is a novel finding regarding arm selection processes in a learning context, yet it is consistent with imaging data that has revealed specific neural areas linked to the selection of mirror and non-mirror imitation processes for first- and third-person viewing perspectives. The skilled model also supported more accurate and stable performance of the bimanual task in observers compared to the instruction and novice models. It is concluded that a skilled model supports attention focus being directed at pattern analysis, while novice models support attention focus being allocated to strategy identification first, followed by pattern analysis.

Effect of model’s skill level and frequency of feedback on learning of complex serial aiming task

Background and Study Aim : Modelling in observational learning and feedback are most important sources of information for learning of a new task. This study aimed to investigate the effect of model’s skill level and feedback frequency on learning of complex serial aiming task. Material and Method: 48 female students aged 19 to 25 years old were selected as sample; based on pre-test scores, they were divided into four groups of 12 subjects: expert model and 100% feedback, expert model and 50% feedback, novice model and 100% feedback, and novice model and 50% feedback). In acquisition phase, the groups performed 80 times the serial aiming task according to specific instructions. The immediate retention test was conducted 15 minutes after completion of acquisition phase and the delayed retention test was conducted 24 hours later. Results: In acquisition stage, the results of analysis of variance with repeated measures showed that the expert model observation group had less spatial error and longer movement time. In immediate and delayed retention stages, the results of two-way analysis of variance showed that the expert model observation group had less spatial error and longer movement time. In delayed retention test, also, the main effect of feedback frequency on spatial error was significant. The 100% feedback group had less spatial error than 50% feedback group. Conclusion: According to Fitz’s speed-accuracy trade-off law, the results are justified as following: since the expert model observers focus on error reduction and increased accuracy in executing complex tasks, their movement time gets longer. Also, the 100% feedback frequency in complex tasks leads to stronger memory consolidation.

A Simulation Conceptual Modelling Procedure for SCM Applications

Purpose – The purpose of this paper is to outline a seven-phase simulation conceptual modelling procedure that incorporates existing practice and embeds a process reference model (i.e. SCOR). Design/methodology/approach – An extensive review of the simulation and SCM literature identifies a set of requirements for a domain-specific conceptual modelling procedure. The associated design issues for each requirement are discussed and the utility of SCOR in the process of conceptual modelling is demonstrated using two development cases. Ten key concepts are synthesised and aligned to a general process for conceptual modelling. Further work is outlined to detail, refine and test the procedure with different process reference models in different industrial contexts. Findings - Simulation conceptual modelling is often regarded as the most important yet least understood aspect of a simulation project (Robinson, 2008a). Even today, there has been little research development into guidelines to aid in the creation of a conceptual model. Design issues are discussed for building an ‘effective’ conceptual model and the domain-specific requirements for modelling supply chains are addressed. The ten key concepts are incorporated to aid in describing the supply chain problem (i.e. components and relationships that need to be included in the model), model content (i.e. rules for determining the simplest model boundary and level of detail to implement the model) and model validation. Originality/value – Paper addresses Robinson (2008a) call for research in defining and developing new approaches for conceptual modelling and Manuj et al., (2009) discussion on improving the rigour of simulation studies in SCM. It is expected that more detailed guidelines will yield benefits to both expert (i.e. avert typical modelling failures) and novice modellers (i.e. guided practice; less reliance on hopeful intuition)

The effect of observing novice and expert performance on acquisition of surgical skills on a robotic platform.

BackgroundObservational learning plays an important role in surgical skills training, following the traditional model of learning from expertise. Recent findings have, however, highlighted the benefit of observing not only expert performance but also error-strewn performance. The aim of this study was to determine which model (novice vs. expert) would lead to the greatest benefits when learning robotically assisted surgical skills.Methods120 medical students with no prior experience of robotically-assisted surgery completed a ring-carrying training task on three occasions; baseline, post-intervention and at one-week follow-up. The observation intervention consisted of a video model performing the ring-carrying task, with participants randomly assigned to view an expert model, a novice model, a mixed expert/novice model or no observation (control group). Participants were assessed for task performance and surgical instrument control.ResultsThere were significant group differences post-intervention, with expert and novice observation groups outperforming the control group, but there were no clear group differences at a retention test one week later. There was no difference in performance between the expert-observing and error-observing groups.ConclusionsSimilar benefits were found when observing the traditional expert model or the error-strewn model, suggesting that viewing poor performance may be as beneficial as viewing expertise in the early acquisition of robotic surgical skills. Further work is required to understand, then inform, the optimal curriculum design when utilising observational learning in surgical training.

FlexiSketch: a lightweight sketching and metamodeling approach for end-users

Engineers commonly use paper and whiteboards to sketch and discuss ideas in early phases of requirements elicitation and software modeling. These physical media foster creativity because they are quick to use and do not restrict in any way the form in which content can be drawn. If the sketched information needs to be reused later on, however, engineers have to spend extra effort for preserving the information in a form that can be processed by a software modeling tool. While saving information in a machine-readable way comes for free with formal software modeling tools, they typically anticipate the use of specific, predefined modeling languages and therefore hamper creativity. To combine the advantages of informal and formal tools, we have developed a flexible tool-supported modeling approach that augments a sketching environment with lightweight metamodeling capabilities. Users can create their own modeling languages by defining sketched constructs on demand and export model sketches as semiformal models. In this article, we first give an overview of FlexiSketch and then focus on an evaluation of our approach with two studies conducted with both novice modelers and experienced practitioners. Our goal was to find out how well modelers manage to use our lightweight metamodeling mechanisms, and how they build notations collaboratively. Results show that experienced modelers adopt our approach quickly, while novices have difficulties to distinguish between the model and metamodel levels and would benefit from additional guidance and user awareness features. The lessons learned from our studies can serve as advice for similar flexible modeling approaches.

An approach to improve the quality of object-oriented models from novice modelers through project practice

The defects in object-oriented models will result in poor quality of applications based on the models, and thus it is necessary to know which defects often occur in practice, to what extent they occur, why they occur, and how they can be prevented. To gain deeper insights into these problems, this paper discusses how to improve the quality of object-oriented models from novice modelers through project practice. This paper summarizes a set of typical quality defect types from a large number of the defects, and confirms them through our project practice. Moreover, the paper analyzes the improvement of the quality of object-oriented models by quantifying the level of occurrence for the defect types in different phases of the project practice, and presents preventive measures by analyzing the causes for the defects to occur in object-oriented models in the aspects of syntax, semantics, and pragmatics.

Assessing the influence of feedback-inclusive rapid prototyping on understanding the semantics of parallel UML statecharts by novice modellers

ContextUML diagrams are the de facto standard for analysing, communicating and designing software systems, as well as automated code generation. However there is a certain degree of difficulty in understanding a system represented by means of UML diagrams. ObjectOur previous research demonstrates a significant improvement in understanding the structural aspects of a system represented as a UML class diagram when using a feedback-inclusive prototype of a model. This paper extends our previous work with an empirical validation study for the effectiveness of the feedback-inclusive rapid prototyping (FIRP) method, on the comprehension of system dynamics represented as multiple interacting UML statecharts. Because models often combine structural and behavioural views that are highly intertwined, we additionally evaluate the effectiveness of the proposed method with respect to comprehension of the between-view consistency. MethodThe FIRP environment was built following the principles of Design Science Research in Information Systems. This study targets the empirical validation of the effectiveness of the proposed technique using an experimental study method. Two experiments were conducted with the participation of 65 final-year master students in the context of different modelling courses from different study programs at KU Leuven using two two-group factorial experimental designs. The effectiveness of the FIRP method was measured by comparing students’ performance between the cycles with and without the use of the method, using the understandability (comprehension test results) as the dependent variable and the use of FIRP as the independent variable. Effects from unknown variables were neutralized by means of randomized group compositions. The effectiveness of FIRP was additionally assessed with respect to personal characteristics (age, gender, previous knowledge, self-efficacy) and user acceptance (perceived ease of use, perceived utility, preference, satisfaction). ResultsThe findings reveal a significant positive impact of the use of the prototyping technique on students’ comprehension of system dynamics represented as multiple interacting statecharts. ConclusionsThe findings provide empirical support for the advantage of the use of FIRP over manual inspection of interacting statecharts. The findings also suggest that the method is suitable for training system's analysis and modelling skills when UML statecharts are involved.