Novice Analysts Research Articles

Thematic Analysis through Artificial Intelligence (AI)

Thematic analysis, a well-enforced qualitative analytic method, is likely to continue evolving with the adoption of AI technologies. This how-to report does not delve into the details of thematic analysis itself, as there are ample existing studies on the topic. Instead, it acknowledges the potential impacts, dynamics, and pitfalls of AI in thematic analysis while offering valuable advice, particularly for novice analysts, on how to incorporate and document AI tools in each phase of a thematic analysis. The author underscores the importance of not allowing AI to overshadow the analyst's critical evaluative and interpretive skills but instead supporting the use of AI as an aid in thematic analysis, enhancing the depth and breadth of analysis, provided certain criteria are adhered to. This approach ensures that AI serves as a complementary tool, augmenting rather than replacing human analytical inquiry.

MetaWin 3: open-source software for meta-analysis.

The rise of research synthesis and systematic reviews over the last 25years has been aided by a series of software packages providing simple and accessible GUI interfaces which are intuitively easy to use by novice analysts and users. Development of many of these packages has been abandoned over time due to a variety of factors, leaving a gap in the software infrastructure available for meta-analysis. To fulfill the continued demand for a GUI-based meta-analytic system, we have now released MetaWin 3 as free, open-source, multi-platform software. MetaWin3 is written in Python and developed from scratch relative to earlier versions. The codebase is available on Github, with pre-compiled executables for both Windows and macOS available from the MetaWin website. MetaWin includes standardized effect size calculations, exploratory and publication bias analyses, and allows for both simple and complex explanatory models of variation within a meta-analytic framework, including meta-regression, using traditional least-squares/moments estimation.

Reliability of ultrasound measurements of muscle thickness and echointensity in older adults with type 2 diabetes

Ultrasound is an emerging tool for muscle-based measurements. However, the analysis of hyperechoic images (high muscle fat infiltration, as in type 2 diabetes [T2D]) can be challenging. ObjectiveTo evaluate the reliability of ultrasound image analysis of muscle thickness and echointensity in older adults with T2D and non-diabetic controls by experienced and novice analysts. We hypothesized that reliability would be high for the experienced analysts, and lower, but acceptable, for a novice analyst. MethodsWe recruited three groups: 1) adults >60 years with T2D; 2) age- and sex-matched normoglycemic controls; and 3) healthy adults 18–35 years. All participants underwent ultrasound imaging of the abdomen and anterior thigh. Ultrasound images were de-identified, randomized, and separately analyzed by two expert analysts (>3 years experience each). A subset of images was also analyzed by a novice analyst with no prior experience. Inter- and intra-rater reliability was assessed using coefficients of variation (CVs), intraclass correlation coefficients (ICCs), and Bland-Altman analysis. ResultsFor the expert analysts, ICCs were >0.90 and CVs were <10 % for all measurements, regardless of participant group. For the novice analyst, all ICCs were >0.90, except for rectus abdominus thickness measurements in the T2D group (ICC: 0.659 [95%CI: -0.134, 0.911]). CVs were <10 % for all measurements, except for rectus abdominus thickness in the T2D group (CV: 12.7 %). Ultrasound image analysis of muscle thickness and echointensity by experienced analysts was highly reliable. ConclusionNovice and expert analysts produced comparable measurements, except for rectus abdominus thickness in images obtained from older adults with T2D.

Developing novice analysts’ videofluoroscopic swallowing study skills in speech-language pathology: A randomised control trial comparing blended and online training approaches

Purpose This research investigates the relative effectiveness of independent online and blended learning approaches for novice analysts’ development of videofluoroscopic swallowing study (VFSS) analytical skills. The secondary aims were to explore the impact of training on decision-making and to describe learners’ perspectives of training outcomes. Method Undergraduate speech-language pathology students (n = 74) who had completed the dysphagia academic curriculum in an undergraduate speech-language pathology program were recruited for a randomised control trial. The ability to identify swallowing impairments in adults was compared pre- and post-training across three conditions: independent online (n = 23), peer-supported (n = 23), and expert-facilitated training (n = 28). The training comprised online VFSS training and practice with a commercially available digital video disc (DVD). Result The three training approaches were equal in improving novice analysts’ identification of impairments on VFSS. Participants’ analysis improved pre- to post-training (p = <.001), with no statistical difference amongst training conditions (p = .280). However, the expert facilitation condition resulted in better decision-making skill for novice analysts, as well as higher levels of confidence and greater engagement in the learning. Conclusion Well-designed independent online methods are appropriate to prepare novice analysts for VFSS analytical training. Expert facilitation and peer-supported environments may have benefits for more advanced skill development and engagement, and should be investigated in future studies.

A new, evidence-based, theory for knowledge reuse in security risk analysis

Security risk analysis (SRA) is a key activity in software engineering but requires heavy manual effort. Community knowledge in the form of security patterns or security catalogs can be used to support the identification of threats and security controls. However, no evidence-based theory exists about the effectiveness of security catalogs when used for security risk analysis. We adopt a grounded theory approach to propose a conceptual, revised and refined theory of SRA knowledge reuse. The theory refinement is backed by evidence gathered from conducting interviews with experts (20) and controlled experiments with both experts (15) and novice analysts (18). We conclude the paper by providing insights into the use of catalogs and managerial implications.

Factors that influence development of speech pathology skills required for videofluoroscopic swallowing studies.

Perceptual, cognitive and previous clinical experience may influence a novice Videofluoroscopic Swallowing Study (VFSS) analyst's trajectory towards competency. Understanding these factors may allow trainees to be better prepared for VFSS training and may allow training to be developed to accommodate differences between trainees. This study explored a range of factors previously suggested in the literature as influencing the development of novice analysts' VFSS skills. We hypothesised that knowledge of swallow anatomy and physiology, visual perceptual skills, self-efficacy and interest, and prior clinical exposure would all influence VFSS novice analysts' skill development. Participants were undergraduate speech pathology students recruited from an Australian university, who had completed the required theoretical units in dysphagia. Data assessing the factors of interest were collected-the participants identified anatomical structures on a still radiographic image, completed a physiology questionnaire, completed subsections of the Developmental Test of Visual Processing-Adults, self-reported the number of dysphagia cases they managed on placement, and self-rated their confidence and interest. Data for 64 participants relating to the factors of interest were compared with their ability to accurately identify swallowing impairments following 15 h of VFSS analytical training, using correlation and regression analysis. Success in VFSS analytical training was best predicted by clinical exposure to dysphagia cases and the ability to identify anatomical landmarks on still radiographic images. Novice analysts vary in the acquisition of beginner-level VFSS analytical skill. Our findings suggest that speech pathologists who are new to VFSS may benefit from clinical exposure to dysphagia cases, sound foundational knowledge of anatomy relevant to swallowing and the ability to see the anatomical landmarks on still radiographic images. Further research is required to equip VFSS trainers and trainees for training, to understand differences between learners during skill development. What is already known on the subject The existing literature suggests that no vice Video fluoroscopic Swallowing Study (VFSS) analysts training may be influenced by their personal characteristics and experience. What this study adds This study found that student clinicians, clinical exposure to dysphagia cases and their ability to identify anatomical landmarks relevant to swallowing on still radiographic images prior to training best predicted their ability to identify swallowing impairments after training. What are the clinical implications of this work? Given the expense of training health professionals, further research is required into the factors that successfully prepare clinicians for VFSS training, including clinical exposure, foundational knowledge of anatomy relevant to swallowing and the ability to identify the anatomical landmarks on still radiographic images.

Identifying common and persistent errors made by novice analysts when modeling business processes using UML activity diagram: utilizing a hierarchical error classification

Identifying common and persistent errors made by novice analysts when modeling business processes using UML activity diagram: utilizing a hierarchical error classification

Exploring Rural Shrink Smart Through Guided Discovery Dashboards

Many small and rural places are shrinking. Interactive dashboards are the most common use cases for data visualization and context for exploratory data tools. In our paper, we will use Iowa data to explore the specific scope of how dashboards are used in small and rural area to empower novice analysts to make data-driven decisions. Our framework will suggest a number of research directions to better support small and rural places from shrinking using an interactive dashboard design, implementation and use for the every day analyst.

Information loss, contextual information, and distinctiveness influence how well novice analysts discriminate fingerprints

AbstractAcross two experimental studies, we investigated the role of information loss, contextual information, and distinctive features of fingerprints on novice's ability to judge whether two fingerprints came from the same source. Distinctive fingerprints resulted in more accurate decisions. Information loss diminished performance on the comparison task, likely due to the loss of distinctive features. When people received contextual information from a fingerprint matching software (Automated Fingerprint Identification System or AFIS) suggesting that the two fingerprints were from the same source, they were less accurate and more biased to render a “same source” decision. These findings suggest that analysts should evaluate the characteristics of a fingerprint prior to engaging in a comparison task, and that fingerprints lacking information, particularly distinctive characteristics, should be more cautiously evaluated.

Minimally trained analysts can perform fast, objective assessment of orthopedic technical skill from fluoroscopic images

Skill assessment in orthopedics has traditionally relied on subjective impressions from a supervising surgeon. The feedback derived from these tools may be limited by bias and practical issues. Objective analysis of intraoperative fluoroscopic images offers an inexpensive and easily repeatable assessment strategy without bias. Despite the advantages, surgeon mentors generally refrain from using the scores of images obtained throughout an operation to evaluate a performer’s technical skill because of other practical constraints. A new system was designed to rapidly facilitate objective analysis of fluoroscopy using minimally trained analysts. Four expert and four novice analysts independently measured one objective metric of orthopedic technical skill using both a custom analysis software and a commercial alternative. Novice analysts using the custom software produced measurements three times faster than experts using the commercial software without a practical difference in annotation error. Expert analysts leveraged the custom software automation features to further improve their measurements. These results suggest that a well-designed fluoroscopy analysis system can facilitate regular, practical, and objective assessment of orthopedic technical skill.

Efficient exploratory clustering analyses in large-scale exploration processes

Clustering is a fundamental primitive in manifold applications. In order to achieve valuable results in exploratory clustering analyses, parameters of the clustering algorithm have to be set appropriately, which is a tremendous pitfall. We observe multiple challenges for large-scale exploration processes. On the one hand, they require specific methods to efficiently explore large parameter search spaces. On the other hand, they often exhibit large runtimes, in particular when large datasets are analyzed using clustering algorithms with super-polynomial runtimes, which repeatedly need to be executed within exploratory clustering analyses. We address these challenges as follows: First, we present LOG-Means and show that it provides estimates for the number of clusters in sublinear time regarding the defined search space, i.e., provably requiring less executions of a clustering algorithm than existing methods. Second, we demonstrate how to exploit fundamental characteristics of exploratory clustering analyses in order to significantly accelerate the (repetitive) execution of clustering algorithms on large datasets. Third, we show how these challenges can be tackled at the same time. To the best of our knowledge, this is the first work which simultaneously addresses the above-mentioned challenges. In our comprehensive evaluation, we unveil that our proposed methods significantly outperform state-of-the-art methods, thus especially supporting novice analysts for exploratory clustering analyses in large-scale exploration processes.

Diagnosing Data Analytic Problems in the Classroom

Teaching data analysis by providing students with real-world problems and datasets allows students to integrate a variety of skills in a situation that mirrors how data analysis actually works. However, whole data analyses may obscure the individual skills of data analytic practice that are generalizable across data analyses. One such skill is the ability to diagnose the cause of unexpected results in a data analysis. While experienced analysts can quickly iterate through a series of potential explanations when confronted with unexpected results, novice analysts often struggle to figure out how to move forward. The goal of this article is to describe an approach to teaching students skills in diagnosing data analytic problems. The exercise described here is targeted to allow students to practice this skill and to assess the depth of their knowledge about the statistical tools they have learned. We take a hypothetical case study approach and focus on the students’ reasoning through their diagnoses and suggestions for follow-up action. We found the implementation of this exercise in a small graduate course to provide valuable information about the students’ diagnostic thought processes, but further work is needed regarding structured approaches to implementation and the design of assessments. Supplementary materials for this article are available online.

How novice analysts understand supply chain process models: an experimental study of using diagrams and texts

PurposeProcess models specific to the supply chain domain are an important tool for the analysis of interorganizational interfaces and requirements of information technology (IT) systems supporting supply chain decision-making. The purpose of this study is to examine the effectiveness of supply chain process models for novice analysts in conveying domain semantics compared to alternative textual representations.Design/methodology/approachA laboratory experiment with graduate students as proxies for novice analysts was conducted. Participants were randomly assigned to either the diagram group, which worked with “thread diagrams” created from the modeling grammar “Supply Chain Operation Reference (SCOR) model”, or the text group, which worked with semantically equivalent textual representations. Domain understanding was measured using cognitively demanding information acquisition for two different domains.FindingsDiagram users were more accurate in identifying product-related information and organizing this information in a graph compared to those using the textual representation. The authors found considerable improvements in domain understanding, and using the diagrams was perceived as easy as using the texts.Originality/valueThe study's findings are unique in providing empirical evidence for supply chain process models being an effective representation for novice analysts. Such evidence is lacking in prior research because of the evaluation methods used, which are limited to scenario, case study and informed argument. This study adds the diagram user's perspective to that literature and provides a rigorous empirical evaluation by contrasting diagrammatic and textual representations.

Might Temporal Logic Improve the Specification of Directed Acyclic Graphs (DAGs)?

Temporality-driven covariate classification had limited impact on: the specification of directed acyclic graphs (DAGs) by 85 novice analysts (medical undergraduates); or the risk of bias in DAG-informed multivariable models designed to generate causal inference from observational data. Only 71 students (83.5%) managed to complete the “Temporality-driven Covariate Classification” task, and fewer still completed the “DAG Specification” task (77.6%) or both tasks in succession (68.2%). Most students who completed the first task misclassified at least one covariate (84.5%), and misclassification rates were even higher among students who specified a DAG (92.4%). Nonetheless, across the 512 and 517 covariates considered by each of these tasks, “confounders” were far less likely to be misclassified (11/252, 4.4% and 8/261, 3.1%) than “mediators” (70/123, 56.9% and 56/115, 48.7%) or “competing exposures” (93/137, 67.9% and 86/138, 62.3%), respectively. Since estimates of total causal effects are biased in multivariable models that: fail to adjust for “confounders”; or adjust for “mediators” (or “consequences of the outcome”) misclassified as “confounders” or “competing exposures,” a substantial proportion of any models informed by the present study’s DAGs would have generated biased estimates of total causal effects (50/66, 76.8%); and this would have only been slightly lower for models informed by temporality-driven covariate classification alone (47/71, 66.2%). Supplementary materials for this article are available online.

A tutorial for joint modeling of longitudinal and time-to-event data in R

In biostatistics and medical research, longitudinal data are often composed of repeated assessments of a variable and dichotomous indicators to mark an event of interest. Consequently, joint modeling of longitudinal and time-to-event data has generated much interest in these disciplines over the previous decade. In behavioural sciences, too, often we are interested in relating individual trajectories and discrete events. Yet, joint modeling is rarely applied in behavioural sciences more generally. This tutorial presents an overview and general framework for joint modeling of longitudinal and time-to-event data, and fully illustrates its application in the context of a behavioral study with the JMbayes R package. In particular, the tutorial discusses practical topics, such as model selection and comparison, choice of joint modeling parameterization and interpretation of model parameters. In the end, this tutorial aims at introducing didactically the theory related to joint modeling and to introduce novice analysts to the use of the JMbayes package.

LOG-Means

Clustering is a fundamental primitive in manifold applications. In order to achieve valuable results, parameters of the clustering algorithm, e.g., the number of clusters, have to be set appropriately, which is a tremendous pitfall. To this end, analysts rely on their domain knowledge in order to define parameter search spaces. While experienced analysts may be able to define a small search space, especially novice analysts often define rather large search spaces due to the lack of in-depth domain knowledge. These search spaces can be explored in different ways by estimation methods for the number of clusters. In the worst case, estimation methods perform an exhaustive search in the given search space, which leads to infeasible runtimes for large datasets and large search spaces. We propose LOG-Means, which is able to overcome these issues of existing methods. We show that LOG-Means provides estimates in sublinear time regarding the defined search space, thus being a strong fit for large datasets and large search spaces. In our comprehensive evaluation on an Apache Spark cluster, we compare LOG-Means to 13 existing estimation methods. The evaluation shows that LOG-Means significantly outperforms these methods in terms of runtime and accuracy. To the best of our knowledge, this is the most systematic comparison on large datasets and search spaces as of today.

SaPeer and ReverseSaPeer: teaching requirements elicitation interviews with role-playing and role reversal

Among the variety of the available requirements elicitation techniques, interviews are the most commonly used. Performing effective interviews is challenging, especially for students and novice analysts, since interviews’ success depends largely on soft skills and experience. Despite their diffusion and their challenging nature, when it comes to requirements engineering education and training (REET), limited resources and few well-founded pedagogical approaches are available to allow students to acquire and improve their skills as interviewers. To overcome this limitation, this paper presents two pedagogical approaches, namely SaPeer and ReverseSaPeer. SaPeer uses role-playing, peer review and self-assessment to enable students to experience first-hand the difficulties related to the interviewing process, reflect on their mistakes, and improve their interview skills by practice and analysis. ReverseSaPeer builds on the first approach and includes a role reversal activity in which participants play the role of a customer interviewed by a competent interviewer. We evaluate the effectiveness of SaPeer through a controlled quasi-experiment, which shows that the proposed approach significantly reduces the amount of mistakes made by the participants and that it is perceived as useful and easy by the participants. ReverseSaPeer and the impact of role reversal are analyzed through a thematic analysis of the participant’s reflections. The analysis shows that not only the students perceive the analysis as beneficial, but also that they have emotional involvement in learning. This work contributes to the body of knowledge of REET with two methods, quantitative and qualitative evaluated, respectively. Furthermore, we share the pedagogical material used, to enable other educators to apply and possibly tailor the approach.

Successful Treatment of Psychosis by Means of Supervised Analysis

The authors present some aspects of their experiences in the treatment of two psychotic patients. The first case suffered from hallucinatory mystical delusions and the second from delusional hypochondria and erythrophobia. Since these cases were followed through regular supervision during their novice analysts’ psychoanalytic training, the usefulness of this kind of treatment for the development of current psychoanalytic practice is emphasized.

Slocum Gliders Provide Accurate Near Real-Time Estimates of Baleen Whale Presence From Human-Reviewed Passive Acoustic Detection Information

Mitigating the effects of human activities on marine mammals often depends on monitoring animal occurrence over long time scales, large spatial scales, and in real time. Passive acoustics, particularly from autonomous vehicles, is a promising approach to meeting this need. We have previously developed the capability to record, detect, classify, and transmit to shore information about the tonal sounds of baleen whales in near real time from long-endurance ocean gliders. We have recently developed a protocol by which a human analyst reviews this information to determine the presence of marine mammals, and the results of this review are automatically posted to a publicly accessible website, sent directly to interested parties via email or text, and made available to stakeholders via a number of public and private digital applications. We evaluated the performance of this system during two 3.75-month Slocum glider deployments in the southwestern Gulf of Maine during the spring seasons of 2015 and 2016. Near real-time detections of humpback, fin, sei, and North Atlantic right whales were compared to detections of these species from simultaneously recorded audio. Data from another 2016 glider deployment in the same area were also used to compare results between three different analysts to determine repeatability of results both among and within analysts. False detection (occurrence) rates on daily time scales were 0% for all species. Daily missed detection rates ranged from 17 to 24%. Agreement between two trained novice analysts and an experienced analyst was greater than 95% for fin, sei, and right whales, while agreement was 83-89% for humpback whales owing to the more subjective process for detecting this species. Our results indicate that the presence of baleen whales can be accurately determined using information about tonal sounds transmitted in near real-time from Slocum gliders. The system is being used operationally to monitor baleen whales in U.S., Canadian, and Chilean waters, and has been particularly useful for monitoring the critically endangered North Atlantic right whale throughout the northwestern Atlantic Ocean.

Can Field Crews Telecommute? Varied Data Quality from Citizen Science Tree Inventories Conducted Using Street-Level Imagery

Street tree inventories are a critical component of urban forest management. However, inventories conducted in the field by trained professionals are expensive and time-consuming. Inventories relying on citizen scientists or virtual surveys conducted remotely using street-level photographs may greatly reduce the costs of street tree inventories, but there are fundamental uncertainties regarding the level of data quality that can be expected from these emerging approaches to data collection. We asked 16 volunteers to inventory street trees in suburban Chicago using Google Street ViewTM imagery, and we assessed data quality by comparing their virtual survey data to field data from the same locations. We also compared virtual survey data quality according to self-rated expertise by measuring agreement within expert, intermediate, and novice analyst groups. Analyst agreement was very good for the number of trees on each street segment, and agreement was markedly lower for tree diameter class and tree identification at the genus and species levels, respectively. Interrater agreement varied by expertise, such that experts agreed with one another more often than novices for all four variables assessed. Compared to the field data, we observed substantial variability in analyst performance for diameter class estimation and tree identification, and some intermediate analysts performed as well as experts. Our findings suggest that virtual surveys may be useful for documenting the locations of street trees within a city more efficiently than field crews and with a high level of accuracy. However, tree diameter and species identification data were less reliable across all expertise groups, and especially novice analysts. Based on this analysis, virtual street tree inventories are best suited to collecting very basic information such as tree locations, or updating existing inventories to determine where trees have been planted or removed. We conclude with evidence-based recommendations for effective implementation of this type of approach.