Code Annotations Research Articles

A survey of experimental stimulus presentation code sharing in major areas of psychology.

Computer code plays a vital role in modern science, from the conception and design of experiments through to final data analyses. Open sharing of code has been widely discussed as being advantageous to the scientific process, allowing experiments to be more easily replicated, helping with error detection, and reducing wasted effort and resources. In the case of psychology, the code used to present stimuli is a fundamental component of many experiments. It is not known, however, the degree to which researchers are sharing this type of code. To estimate this, we conducted a survey of 400 psychology papers published between 2016 and 2021, identifying those working with the open-source tools Psychtoolbox and PsychoPy that openly share stimulus presentation code. For those that did, we established if it would run following download and also appraised the code's usability in terms of style and documentation. It was found that only 8.4% of papers shared stimulus code, compared to 17.9% sharing analysis code and 31.7% sharing data. Of shared code, 70% ran directly or after minor corrections. For code that did not run, the main error was missing dependencies (66.7%). The usability of the code was moderate, with low levels of code annotation and minimal documentation provided. These results suggest that stimulus presentation code sharing lags behind other forms of code and data sharing, potentially due to less emphasis on such code in open-science discussions and in journal policies. The results also highlight a need for improved documentation to maximize code utility.

Mediation Analyses of Intensive Longitudinal Data with Dynamic Structural Equation Modeling

Currently, dynamic structural equation modeling (DSEM) and residual DSEM (RDSEM) are commonly used in testing intensive longitudinal data (ILD). Researchers are interested in ILD mediation models, but their analyses are challenging. The present paper mathematically derived, empirically compared, and step-by-step demonstrated three types (i.e., 1-1-1, 2-1-1, and 2-2-1) of intensive longitudinal mediation (ILM) analyses based on DSEM and RDSEM models. Specifically, each ILM model was demonstrated with a simulated example and illustrated with the corresponding annotated Mplus codes. We compared two types of detrending methods in mediation analyses and showed that RDSEM was superior to DSEM because the latter included the time tj variable as a Level 1 predictor. Lastly, we extended ILM analyses based on DSEM and RDSEM to multilevel autoregressive mediation models, cross-classified DSEM, and intensive longitudinal moderated mediation models.

CoNECo: a Corpus for Named Entity recognition and normalization of protein Complexes.

Despite significant progress in biomedical information extraction, there is a lack of resources for Named Entity Recognition (NER) and Named Entity Normalization (NEN) of protein-containing complexes. Current resources inadequately address the recognition of protein-containing complex names across different organisms, underscoring the crucial need for a dedicated corpus. We introduce the Complex Named Entity Corpus (CoNECo), an annotated corpus for NER and NEN of complexes. CoNECo comprises 1621 documents with 2052 entities, 1976 of which are normalized to Gene Ontology. We divided the corpus into training, development, and test sets and trained both a transformer-based and dictionary-based tagger on them. Evaluation on the test set demonstrated robust performance, with F-scores of 73.7% and 61.2%, respectively. Subsequently, we applied the best taggers for comprehensive tagging of the entire openly accessible biomedical literature. All resources, including the annotated corpus, training data, and code, are available to the community through Zenodo https://zenodo.org/records/11263147 and GitHub https://zenodo.org/records/10693653.

Flux Balance Analysis of Mammalian Cell Systems.

Flux balance analysis (FBA) is a computational methodology to model and analyzethe metabolic behavior of cells. In this chapter, we break down the key steps for formulating an FBA model and other FBA-derived methodologies in the context of mammalian cell biology, including strain design, developingcell line-specific models, and conductingflux sampling. We provide annotated COBRApy code for each step to show how it would work in practice.

Bayesian mediation analysis in trauma research.

Bayesian methods are growing in popularity among social scientists, due to the significant advantages offered to researchers: namely, intuitive probabilistic interpretations of results. Here, we highlight the benefits of using the Bayesian framework in research where collecting large samples is challenging, specifically: the absence of a requirement of large samples for convergence, and the possibility of building on prior research by including informative priors. We demonstrate how to fit a single mediator model and impute missing data in the Bayesian framework using the software JAGS via the R package rjags. To this end, we use open-access data to fit a mediation model and calculate the posterior probability that the mediated effect is above a specified criterion. We replicate the results of the original paper in the Bayesian framework and provide annotated code for mediation analysis in rjags, as well as two additional R packages for Bayesian analysis (brms and rstan) and two additional software packages (SAS and Mplus). We provide guidelines for reporting and interpreting results obtained in the Bayesian framework, and two extensions to the mediation model are discussed: adding covariates to the model and selecting informative priors. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Machine learning code snippets semantic classification

Program code has recently become a valuable active data source for training various data science models, from code classification to controlled code synthesis. Annotating code snippets play an essential role in such tasks. This article presents a novel approach that leverages CodeBERT, a powerful transformer-based model, to classify code snippets extracted from Code4ML automatically. Code4ML is a comprehensive machine learning code corpus compiled from Kaggle, a renowned data science competition platform. The corpus includes code snippets and information about the respective kernels and competitions, but it is limited in the quality of the tagged data, which is ~0.2%. Our method addresses the lack of labeled snippets for supervised model training by exploiting the internal ambiguity in particular labeled snippets where multiple class labels are combined. Using a specially designed algorithm, we effectively separate these ambiguous fragments, thereby expanding the pool of training data. This data augmentation approach greatly increases the amount of labeled data and improves the overall quality of the trained models. The experimental results demonstrate the prowess of the proposed code classifier, achieving an impressive F1 test score of ~89%. This achievement not only enhances the practicality of CodeBERT for classifying code snippets but also highlights the importance of enriching large-scale annotated machine learning code datasets such as Code4ML. With a significant increase in accurately annotated code snippets, Code4ML is becoming an even more valuable resource for learning and improving various data processing models.

Investigating Moderation Effects at the Within-Person Level Using Intensive Longitudinal Data: A Two-Level Dynamic Structural Equation Modelling Approach in Mplus

Recent technological advances have provided new opportunities for the collection of intensive longitudinal data. Using methods such as dynamic structural equation modeling, these data can provide new insights into moment-to-moment dynamics of psychological and behavioral processes. In intensive longitudinal data (t > 20), researchers often have theories that imply that factors that change from moment to moment within individuals act as moderators. For instance, a person’s level of sleep deprivation may affect how much an external stressor affects mood. Here, we describe how researchers can implement, test, and interpret dynamically changing within-person moderation effects using two-level dynamic structural equation modeling as implemented in the structural equation modeling software Mplus. We illustrate the analysis of within-person moderation effects using an empirical example investigating whether changes in spending time online using social media affect the moment-to-moment effect of loneliness on depressive symptoms, and highlight avenues for future methodological development. We provide annotated Mplus code, enabling researchers to better isolate, estimate, and interpret the complexities of within-person interaction effects.

An open-source natural language processing toolkit to support software development: addressing automatic bug detection, code summarisation and code search.

This paper aims to introduce the innovative work carried out in the Horizon 2020 DECODER project - acronym for "DEveloper COmpanion for Documented and annotatEd code Reference" - (Grant Agreement no. 824231) by linking the fields of natural language processing (NLP) and software engineering. The project as a whole addresses the development of a framework, namely the Persistent Knowledge Monitor (PKM), that acts as a central infrastructure to store, access, and trace all the data, information and knowledge related to a given software or ecosystem. This meta-model defines the knowledge base that can be queried and analysed by all the tools integrated and developed in DECODER. Besides, the DECODER project offers a friendly user interface where each of the predefined three roles (i.e., developers, maintainers and reviewers) can access and query the PKM with their personal accounts. The paper focuses on the NLP tools developed and integrated in the PKM, namely the deep learning models developed to perform variable misuse, code summarisation and semantic parsing. These were developed under a common work package - "Activities for the developer" - intended to precisely target developers, who can perform tasks such as detection of bugs, automatic generation of documentation for source code and generation of code snippets from natural languages instructions, among the multiple functionalities that DECODER offers. These tools assist and help the developers in the daily work, by increasing their productivity and avoiding loss of time in tedious tasks such as manual bug detection. Training and validation were conducted for four use cases in Java, C and C++ programming languages in order to evaluate the performance, suitability, usability, etc. of the developed tools.

Calculating risk and prevalence ratios and differences in R: developing intuition with a hands-on tutorial and code

Epidemiologic research questions often focus on evaluating binary outcomes, yet curricula and scientific literature do not always provide clear guidance or examples on selecting and calculating an appropriate measure of association in these scenarios. Reporting inappropriate measures may lead to misleading statistical conclusions. We present a hands-on tutorial that includes annotated code written in an open-source statistical programming language (R) showing readers how to apply, compare, and understand four methods used to estimate a risk or prevalence ratio (or difference), rather than presenting an odds ratio. We will provide guidance on when to use each method, discussing the strengths and limitations of each approach, and compare the results obtained across them. Ultimately, we aim to help trainees, public health researchers, and interdisciplinary professionals develop an intuition for these methods and empower them to implement and interpret these methods in their own research.

Semi-automatic representation of design code based on knowledge graph for automated compliance checking

Automated compliance checking (ACC) intends to verify the compliance of designs in construction industry by design codes. The ability to interpret and represent semantic information of design codes determines the maximum application scope of ACC. However, design codes are clause texts written in natural languages and existing ACC studies usually use relatively low-complexity code clause samples. At present, the lack of an accurate representation model for design codes leads to difficulties in representing the implicit information, nested logic, and complex relations contained in high-complexity clauses in codes. To address this problem, this research establishes a new representation model based on knowledge graph (KG). Four schemas are proposed into the model including order, complex, event and integration schemas. Further, an accompanying methodology for semi-automatic construction of design code KG (DCKG) is proposed. It includes four parts: interpretation, reconstruction, organization, and implementation. Where the implementation part develops a code annotation platform. In the case study and experiment, a scenario of checking a building information model (BIM) of metro station by GB50157–2013 Code for Design of Metro is adopted to validate the newly proposed representation model and the automated compliance process. The results show that the proposed model and method are correct and feasible, and our model outperforms other models in the representation ability of design codes.

Dynamic structural equation models with binary and ordinal outcomes in Mplus.

Intensive longitudinal designs are increasingly popular, as are dynamic structural equation models (DSEM) to accommodate unique features of these designs. Many helpful resources on DSEM exist, though they focus on continuous outcomes while categorical outcomes are omitted, briefly mentioned, or considered as a straightforward extension. This viewpoint regarding categorical outcomes is not unwarranted for technical audiences, but there are non-trivial nuances in model building and interpretation with categorical outcomes that are not necessarily straightforward for empirical researchers. Furthermore, categorical outcomes are common given that binary behavioral indicators or Likert responses are frequently solicited as low-burden variables to discourage participant non-response. This tutorial paper is therefore dedicated to providing an accessible treatment of DSEM in Mplus exclusively for categorical outcomes. We cover the general probit model whereby the raw categorical responses are assumed to come from an underlying normal process. We cover probit DSEM and expound why existing treatments have considered categorical outcomes as astraightforward extension of the continuous case. Data from a motivating ecological momentary assessment study with a binary outcome are used to demonstrate an unconditional model, a model with disaggregated covariates, and a model for data with a time trend. We provide annotated Mplus code for these models and discuss interpretation of the results. We then discuss model specification and interpretation in the case of an ordinal outcome and provide an example to highlight differences between ordinal and binary outcomes. We conclude with a discussion of caveats and extensions.

Embracing asymmetry in nature: How to account for skewness in ecological data

Asymmetric regression is an alternative to conventional linear regression that allows us to model the relationship between predictor variables and the response variable while accommodating skewness. Advantages of asymmetric regression include incorporating realistic ecological patterns observed in data, robustness to model misspecification and less sensitivity to outliers. Bayesian asymmetric regression relies on asymmetric distributions such as the asymmetric Laplace (ALD) or asymmetric normal (AND) in place of the normal distribution used in classic linear regression models. Asymmetric regression concepts can be used for process and parameter components of hierarchical Bayesian models and have a wide range of applications in data analyses. In particular, asymmetric regression allows us to fit more realistic statistical models to skewed data and pairs well with Bayesian inference. We first describe asymmetric regression using the ALD and AND. Second, we show how the ALD and AND can be used for Bayesian quantile and expectile regression for continuous response data. Third, we consider an extension to generalize Bayesian asymmetric regression to survey data consisting of counts of objects. Fourth, we describe a regression model using the ALD, and show that it can be applied to add needed flexibility, resulting in better predictive models compared to Poisson or negative binomial regression. We demonstrate concepts by analyzing a data set consisting of counts of Henslow’s sparrows following prescribed fire and provide annotated computer code to facilitate implementation. Our results suggest Bayesian asymmetric regression is an essential component of a scientist’s statistical toolbox.

A study of programming learning perceptions and effectiveness under a blended learning model with live streaming: comparisons between full-time and working students

ABSTRACT With the rise of big data, artificial intelligence, and other emerging information technologies, an increasing number of students without computer science (CS) backgrounds have begun to learn programming. Programming is considered a complex task for beginners, and instructors find it difficult to quickly address all the problems that students face, especially in the traditional learning environment. This study adopted a blended learning model that integrated conventional face-to-face teaching, asynchronous learning, and live streaming to examine the perceptions and effectiveness of programming learning among students from different degree tracks. The participants were 26 working and 28 full-time students from a university in Taiwan. The results indicated that with code annotations proving an efficient way for working students to review coding knowledge, and flipped courses making videos important for full-time students. Live streaming learning can overcome lab limitations with instant interaction between instructors and learners. The blended model ensures equal programming learning for both groups, while instructors can design learner-centered live streaming environments and innovative learning pedagogy to flip programming.

Testing for Within × Within and Between × Within Moderation using Random Intercept Cross-Lagged Panel Models.

Random-Intercept Cross-Lagged Panel Models allow for the decomposition of measurements into between- and within-person components and have hence become popular for testing developmental hypotheses. Here, we describe how developmental researchers can implement, test and interpret interaction effects in such models using an empirical example from developmental psychopathology research. We illustrate the analysis of Within × Within and Between × Within interactions utilising data from the United Kingdom-based Millennium Cohort Study within a Bayesian Structural Equation Modelling framework. We provide annotated Mplus code, allowing users to isolate, estimate and interpret the complexities of within-person and between person dynamics as they unfold over time.

Code4ML: a large-scale dataset of annotated Machine Learning code.

The use of program code as a data source is increasingly expanding among data scientists. The purpose of the usage varies from the semantic classification of code to the automatic generation of programs. However, the machine learning model application is somewhat limited without annotating the code snippets. To address the lack of annotated datasets, we present the Code4ML corpus. It contains code snippets, task summaries, competitions, and dataset descriptions publicly available from Kaggle-the leading platform for hosting data science competitions. The corpus consists of ~2.5 million snippets of ML code collected from ~100 thousand Jupyter notebooks. A representative fraction of the snippets is annotated by human assessors through a user-friendly interface specially designed for that purpose. Code4ML dataset can help address a number of software engineering or data science challenges through a data-driven approach. For example, it can be helpful for semantic code classification, code auto-completion, and code generation for an ML task specified in natural language.

Primrose: Selecting Container Data Types by Their Properties

Context: Container data types are ubiquitous in computer programming, enabling developers to efficiently store and process collections of data with an easy-to-use programming interface. Many programming languages offer a variety of container implementations in their standard libraries based on data structures offering different capabilities and performance characteristics. Inquiry: Choosing the *best* container for an application is not always straightforward, as performance characteristics can change drastically in different scenarios, and as real-world performance is not always correlated to theoretical complexity. Approach: We present Primrose, a language-agnostic tool for selecting the best performing valid container implementation from a set of container data types that satisfy *properties* given by application developers. Primrose automatically selects the set of valid container implementations for which the *library specifications*, written by the developers of container libraries, satisfies the specified properties. Finally, Primrose ranks the valid library implementations based on their runtime performance. Knowledge: With Primrose, application developers can specify the expected behaviour of a container as a type refinement with *semantic properties*, e.g., if the container should only contain unique values (such as a `set`) or should satisfy the LIFO property of a `stack`. Semantic properties nicely complement *syntactic properties* (i.e., traits, interfaces, or type classes), together allowing developers to specify a container's programming interface *and* behaviour without committing to a concrete implementation. Grounding: We present our prototype implementation of Primrose that preprocesses annotated Rust code, selects valid container implementations and ranks them on their performance. The design of Primrose is, however, language-agnostic, and is easy to integrate into other programming languages that support container data types and traits, interfaces, or type classes. Our implementation encodes properties and library specifications into verification conditions in Rosette, an interface for SMT solvers, which determines the set of valid container implementations. We evaluate Primrose by specifying several container implementations, and measuring the time taken to select valid implementations for various combinations of properties with the solver. We automatically validate that container implementations conform to their library specifications via property-based testing. Importance: This work provides a novel approach to bring abstract modelling and specification of container types directly into the programmer's workflow. Instead of selecting concrete container implementations, application programmers can now work on the level of specification, merely stating the behaviours they require from their container types, and the best implementation can be selected automatically.

CADV: A software visualization approach for code annotations distribution

Code annotations is a widely used feature in Java systems to configure custom metadata on programming elements. Their increasing presence creates the need for approaches to assess and comprehend their usage and distribution. In this context, software visualization has been studied and researched to improve program comprehension in different aspects. This study aimed at designing a software visualization approach that graphically displays how code annotations are distributed and organized in a software system and developing a tool, as a reference implementation of the approach, to generate views and interact with users. We conducted an empirical evaluation through questionnaires and interviews to evaluate our visualization approach considering four aspects: effectiveness for program comprehension, perceived usefulness, perceived ease of use, and suitability for the intended audience. The resulting data was used to perform a qualitative and quantitative analysis. The tool identifies package responsibilities providing visual information about their annotations at different levels. Using the developed tool, the participants achieved a high correctness rate in the program comprehension tasks and performed very well in questions about the overview of the system under analysis. Finally, participants perceived that the tool outperforms existing approaches for code inspection when searching for information related to code annotations. The results show that the visualization approach using the developed tool is effective in program comprehension tasks related to code annotations, which can also be used to identify responsibilities in the application packages. Moreover, it was evaluated as suitable for newcomers to overview the usage of annotations in the system and for architects to perform a deep analysis that can potentially detect misplaced annotations and abnormal growths on their usage.

Classifying literature mentions of biological pathogens as experimentally studied using natural language processing

BackgroundInformation pertaining to mechanisms, management and treatment of disease-causing pathogens including viruses and bacteria is readily available from research publications indexed in MEDLINE. However, identifying the literature that specifically characterises these pathogens and their properties based on experimental research, important for understanding of the molecular basis of diseases caused by these agents, requires sifting through a large number of articles to exclude incidental mentions of the pathogens, or references to pathogens in other non-experimental contexts such as public health.ObjectiveIn this work, we lay the foundations for the development of automatic methods for characterising mentions of pathogens in scientific literature, focusing on the task of identifying research that involves the experimental study of a pathogen in an experimental context. There are no manually annotated pathogen corpora available for this purpose, while such resources are necessary to support the development of machine learning-based models. We therefore aim to fill this gap, producing a large data set automatically from MEDLINE under some simplifying assumptions for the task definition, and using it to explore automatic methods that specifically support the detection of experimentally studied pathogen mentions in research publications.MethodsWe developed a pathogen mention characterisation literature data set —READBiomed-Pathogens— automatically using NCBI resources, which we make available. Resources such as the NCBI Taxonomy, MeSH and GenBank can be used effectively to identify relevant literature about experimentally researched pathogens, more specifically using MeSH to link to MEDLINE citations including titles and abstracts with experimentally researched pathogens. We experiment with several machine learning-based natural language processing (NLP) algorithms leveraging this data set as training data, to model the task of detecting papers that specifically describe experimental study of a pathogen.ResultsWe show that our data set READBiomed-Pathogens can be used to explore natural language processing configurations for experimental pathogen mention characterisation. READBiomed-Pathogens includes citations related to organisms including bacteria, viruses, and a small number of toxins and other disease-causing agents.ConclusionsWe studied the characterisation of experimentally studied pathogens in scientific literature, developing several natural language processing methods supported by an automatically developed data set. As a core contribution of the work, we presented a methodology to automatically construct a data set for pathogen identification using existing biomedical resources. The data set and the annotation code are made publicly available. Performance of the pathogen mention identification and characterisation algorithms were additionally evaluated on a small manually annotated data set shows that the data set that we have generated allows characterising pathogens of interest.Trial registrationN/A.

Impunity: Enforcing Physical Unit Consistency at Definition Time in Python

We introduce impunity, a Python library that enables static analysis of code annotations to ensure the consistency of physical units. It provides a framework for developers to annotate their Python code with physical units and automatically verifies if the units are compatible and adhere to predefined coherence rules. impunity comes as a decorator to apply on functions: it analyses the source code to check for consistency of physical dimensions, and applies minimal code rewriting if conversions are necessary. Overall, this approach takes the best of type-checking based methods and dynamic methods and provides a robust approach with no overhead at runtime.

Intensive longitudinal mediation in Mplus.

Much of the existing longitudinal mediation literature focuses on panel data where relatively few repeated measures are collected over a relatively broad timespan. However, technological advances in data collection (e.g., smartphones, wearables) have led to a proliferation of short duration, densely collected longitudinal data in behavioral research. These intensive longitudinal data differ in structure and focus relative to traditionally collected panel data. As a result, existing methodological resources do not necessarily extend to nuances present in the recent influx of intensive longitudinal data and designs. In this tutorial, we first cover potential limitations of traditional longitudinal mediation models to accommodate unique characteristics of intensive longitudinal data. Then, we discuss how recently developed dynamic structural equation models (DSEMs) may be well-suited for mediation modeling with intensive longitudinal data and can overcome some of the limitations associated with traditional approaches. We describe four increasingly complex intensive longitudinal mediation models: (a) stationary models where the indirect effect is constant over time and people, (b) person-specific models where the indirect effect varies across people, (c) dynamic models where the indirect effect varies across time, and (d) cross-classified models where the indirect effect varies across both time and people. We apply each model to a running example featuring a mobile health intervention designed to improve health behavior of individuals with binge eating disorder. In each example, we provide annotated Mplus code and interpretation of the output to guide empirical researchers through mediation modeling with this increasingly popular type of longitudinal data. (PsycInfo Database Record (c) 2022 APA, all rights reserved).