Research Artifacts Research Articles

Collective knowledge: organizing research projects as a database of reusable components and portable workflows with common interfaces.

This article provides the motivation and overview of the Collective Knowledge Framework (CK or cKnowledge). The CK concept is to decompose research projects into reusable components that encapsulate research artifacts and provide unified application programming interfaces (APIs), command-line interfaces (CLIs), meta descriptions and common automation actions for related artifacts. The CK framework is used to organize and manage research projects as a database of such components. Inspired by the USB 'plug and play' approach for hardware, CK also helps to assemble portable workflows that can automatically plug in compatible components from different users and vendors (models, datasets, frameworks, compilers, tools). Such workflows can build and run algorithms on different platforms and environments in a unified way using the customizable CK program pipeline with software detection plugins and the automatic installation of missing packages. This article presents a number of industrial projects in which the modular CK approach was successfully validated in order to automate benchmarking, auto-tuning and co-design of efficient software and hardware for machine learning and artificial intelligence in terms of speed, accuracy, energy, size and various costs. The CK framework also helped to automate the artifact evaluation process at several computer science conferences as well as to make it easier to reproduce, compare and reuse research techniques from published papers, deploy them in production, and automatically adapt them to continuously changing datasets, models and systems. The long-term goal is to accelerate innovation by connecting researchers and practitioners to share and reuse all their knowledge, best practices, artifacts, workflows and experimental results in a common, portable and reproducible format at https://cKnowledge.io/. This article is part of the theme issue 'Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico'.

Learning from reproducing computational results: introducing three principles and the Reproduction Package.

We carry out efforts to reproduce computational results for seven published articles and identify barriers to computational reproducibility. We then derive three principles to guide the practice and dissemination of reproducible computational research: (i) Provide transparency regarding how computational results are produced; (ii) When writing and releasing research software, aim for ease of (re-)executability; (iii) Make any code upon which the results rely as deterministic as possible. We then exemplify these three principles with 12 specific guidelines for their implementation in practice. We illustrate the three principles of reproducible research with a series of vignettes from our experimental reproducibility work. We define a novel Reproduction Package, a formalism that specifies a structured way to share computational research artifacts that implements the guidelines generated from our reproduction efforts to allow others to build, reproduce and extend computational science. We make our reproduction efforts in this paper publicly available as exemplar Reproduction Packages.This article is part of the theme issue ‘Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico’.

Researching ‘non-sexualities’ via creative notebooks: epistemology, embodiment and empowerment

This article contributes to and extends critical scholarship on the philosophy and practical application of creative methods. I suggest that claims commonly made regarding the ‘potential’ of creative methods can be (re)organised as claims relating to (1) epistemology, (2) embodiment and (3) empowering participants. I evaluate these claims through the contextual lens of a research project on ‘non-sexualities’ wherein I incorporated a creative element (creative notebooks) into the research design. Through analysis of research artefacts and observations of the research process, I reflect upon how the notebooks were particularly good for ‘getting at’ embodiment and had a clear epistemological value in facilitating expressions of complexity, contradictions and ambiguities. However, I also discuss my scepticism with regard to claims made about the empowering potential of creative methods, as the notebooks potentially worked to reproduce certain power dynamics rather than eliminate them. I argue that class in particular needs to be given more attention in critical accounts of creative methods.

Continuously Managing NFRs: Opportunities and Challenges in Practice

Non-functional requirements (NFR), which include performance, availability, and maintainability, are vitally important to overall software quality. However, research has shown NFRs are, in practice, poorly defined and difficult to verify. Continuous software engineering practices, which extend agile practices, emphasize fast paced, automated, and rapid release of software that poses additional challenges to handling NFRs. In this multi-case study we empirically investigated how three organizations, for which NFRs are paramount to their business survival, manage NFRs in their continuous practices. We describe four practices these companies use to manage NFRs, such as offloading NFRs to cloud providers or the use of metrics and continuous monitoring, both of which enable almost real-time feedback on managing the NFRs. However, managing NFRs comes at a cost—as we also identified a number of challenges these organizations face while managing NFRs in their continuous software engineering practices. For example, the organizations in our study were able to realize an NFR by strategically and heavily investing in configuration management and infrastructure as code, in order to offload the responsibility of NFRs; however, this offloading implied potential loss of control. Our discussion and key research implications show the opportunities, trade-offs, and importance of the unique give-and-take relationship between continuous software engineering and NFRs. Research artifacts may be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://doi.org/10.5281/zenodo.3376342</uri> .

(e)Textile New Materialities

This paper explores the intersections within bodily materialism and future textiles by inquiring into embodied practices and materiality in care. By placing the body as a site of research, it centres around concepts of bodily care and the body as an ecosystem, one that is always in flux and considers the fluidity of bodies and bodily fluids, such as urine, discharge, breath and sweat, as fluids with potential to design with. It looks at how bodies are acted upon by outside forces, and explore more-than-human relations as co-creators in co-habiting the space of the body and that around it. To illustrate this, the paper introduces a series of design research artefacts that take a variety of approaches to exploring the materiality of care in the everyday. First, an eTextile toolkit that aims to create bodily awareness through hands-on engagement with textile crafting technology, then a biotextile harvesting toolkit that involves the raw material of the intimate body that explores DIYbio in the context of the home, and lastly a set of wearable living material-based explorations that recognize biomimicry and symbiotic relationships in designing for chronic stress. In embracing notions of bodily materialism, this paper explores the bodily abject, i.e. fluids and the more-than-human as crucial to engendering new modes of knowing in intimate and personal care through textile-based materials. The paper engages critically with textile design research and practice by placing material that embraces care as ambivalent at the forefront and thus challenging traditional approaches to health and care and, importantly, the design of future textiles.

An application of behavioural finance in banking: The Discovery Bank case

Orientation: Behavioural finance research suggests that human biases can cause irrationalities which have a significant impact on decision making. Discovery Bank is an organisation that attempts to apply behavioural finance to improve the financial health of its clients. Research purpose: This study attempts to determine the extent to which the Discovery Bank business model is grounded in behavioural finance theory. Motivation for the study: Discovery Bank is the first bank to leverage behavioural insights to improve personal financial decisions. Research approach/design and method: This study followed an explanatory case study methodology with primary data sources being interviews with key employees, academic research and electronic artefacts. The Discovery Bank business model was evaluated against the behavioural finance theoretical framework to establish the extent to which it conforms to behavioural finance theories. Main findings: The Discovery Bank business model is grounded in behavioural finance theory to a significant extent, with emphasis on modifying the behaviours that inhibit financial well-being. The bank generally uses incentives rather than nudges as behaviour modification tools. Practical/managerial implications: Whilst behavioural finance continues to attract substantial attention in finance research, its practical implications for the banking industry and personal finance are largely unexplored. Furthermore, the study contributes to the literature by examining the behavioural finance theoretical framework in the context of the banking and the broader financial services industry. Contribution/value-add: This research may be of value to practitioners in the financial industry as it explores a unique business model. Researchers on behavioural finance may find value in the practical application of the theoretical framework.

Editors' Note

Editors' Note

Links between research artefacts: use cases for digital libraries

Links between research artefacts: use cases for digital libraries

Links between research artefacts: use cases for digital libraries

Links between research artefacts: use cases for digital libraries

МИНИАТЮРНЫЕ КОЛОНКИ БМАК НА ЕВРОПЕЙСКИХ ИНТЕРНЕТ-АУКЦИОНАХ: ВОЗМОЖНОСТИ ИСПОЛЬЗОВАНИЯ В НАУЧНЫХ ИССЛЕДОВАНИЯХ

The paper is devoted to the consideration of the possibilities of using artifacts from private collections that are sold at European online auctions in scientific research. As an example, the data from the website of the world’s largest auction house Sotheby’s are analyzed. The description of 18 artifacts (miniature columns from Bactria) is given. Such artifacts are cylindrical or biconical shaped stone products with gutters on the bases and sides. The main difficulties when referring to this type of sources in scientific research are such aspects as lack of certification, the problem of authenticity and verification of information, the region of origin, incomplete information. Nevertheless, the analyzed objects fit into the cultural stereotypes of the BMAC and find complete analogies in the well-documented and stratified sits of this cultural area, which is a certain basis for their authenticity. Therefore, despite the noted nuances, it is concluded that it is necessary to take into account such artifacts in scientific research. Keywords: artifacts, BMAC, miniature columns, private collections, online auctions

Trust but Verify: How to Leverage Policies, Workflows, and Infrastructure to Ensure Computational Reproducibility in Publication

This article distills findings from a qualitative study of seven reproducibility initiatives to enumerate nine key decision points for journals seeking to address concerns about the quality and rigor of computational research by expanding the peer review and publication process. We evaluate our guidance in light of the recent National Academies of Science, Engineering, and Medicine (NASEM, 2019) report on Reproducibility and Replicability in Science recommendation for journal reproducibility audits. We present 10 findings that clarify how reproducibility initiatives contend with a variety of social and technical factors, including significant gaps in editorial infrastructure and a lack of uniformity in how research artifacts are packaged for dissemination. We propose and define a novel concept of assessable reproducible research artifacts and point the way to an improved understanding of how changes to author incentives and dissemination requirements impact the quality, rigor, and trustworthiness of published computational research.

Menoci: lightweight extensible web portal enhancing data management for biomedical research projects

BackgroundBiomedical research projects deal with data management requirements from multiple sources like funding agencies’ guidelines, publisher policies, discipline best practices, and their own users’ needs. We describe functional and quality requirements based on many years of experience implementing data management for the CRC 1002 and CRC 1190. A fully equipped data management software should improve documentation of experiments and materials, enable data storage and sharing according to the FAIR Guiding Principles while maximizing usability, information security, as well as software sustainability and reusability.ResultsWe introduce the modular web portal software menoci for data collection, experiment documentation, data publication, sharing, and preservation in biomedical research projects. Menoci modules are based on the Drupal content management system which enables lightweight deployment and setup, and creates the possibility to combine research data management with a customisable project home page or collaboration platform.ConclusionsManagement of research data and digital research artefacts is transforming from individual researcher or groups best practices towards project- or organisation-wide service infrastructures. To enable and support this structural transformation process, a vital ecosystem of open source software tools is needed. Menoci is a contribution to this ecosystem of research data management tools that is specifically designed to support biomedical research projects.

Public Security Sentiment Analysis on Social Web

This article presents (1) the results of a literature review on social web mining and sentiment analysis on public security; (2) the idea of a framework for the analytical process involved in the literature review themes; and (3) a research agenda with a perspective for future studies, considering some elements of the analytical process. The literature review was based on searches of five databases: Scopus, IEEE Xplore, Web of Science, ScienceDirect, and Springer Link. Search strings were applied to retrieve literature material of four kinds, without defining an initial time milestone, to get the historical register of publications associated with the main thematic. After some filtering, primary and secondary findings were separated, enabling the identification of elements for the framework. Finally, the research agenda is presented, containing a set of three research artifacts related to the proposed framework.

'Yes, I comply!'

As science becomes increasingly data-intensive, the requirements for comprehensive Research Data Management (RDM) grow. This often overwhelms scientists, requiring more workload and training. The failure to conduct effective RDM leads to producing research artefacts that cannot be reproduced or reused. Past research placed high value on supporting data science workers, but focused mainly on data production, collection, processing, and sensemaking. In order to understand practices and needs of data science workers in relation to documentation, preservation, sharing, and reuse, we conducted a cross-domain study with 15 scientists and data managers from diverse scientific domains. We identified five core concepts which describe requirements, drivers, and boundaries in the development of commitment for RDM, essential for generating reproducible research artefacts: Practice, Adoption, Barriers, Education, and Impact. Based on those concepts, we introduce a stage-based model of personal RDM commitment evolution. The model can be used to drive the design of future systems that support a transition to open science. We discuss infrastructure, policies, and motivations involved at the stages and transitions in the model. Our work supports designers in understanding the constraints and challenges involved in designing for reproducibility in an age of data-driven science.

Whatsoever things are true: Hypothesis, artefact, and bias in chemical engineering research

AbstractFor experimental research to offer valuable insights and predictions, its results and interpretation need to be properly validated. When the experiments deal with complex systems, such as biological materials, multicomponent mixtures, or multiple phases, the use of the rigorous scientific method is essential. Setting and testing hypotheses and design of experimental programs to include formal positive and negative experiments helps to identify artefacts and to minimize the influence of the biases of the investigators that can invalidate the results of studies. The literature has many examples of well‐executed studies, but there is much less discussion of the pitfalls and traps that can beset experimental research. This paper presents cases in the area of chemical reaction engineering and biochemical engineering. Experimental designs are presented that were successful in validating results by using positive and negative controls. Case studies of experimental artefacts that were published suggest that the biases of the investigators were important in failing to fully verify experimental observations. Both strong experimental designs to identify artefacts and publication of negative results are important in avoiding the persistence of what Stephen Poole calls “zombie ideas”. These zombie ideas may be benign, or they may lead to considerable wasted effort on studies with no chance of success. Graphical representations and schematics are extremely valuable in communicating results and making them memorable, but they can also be seductive in misleading researchers.

Towards a sociomaterial approach to inter-organizational boundaries: How information systems elicit relevant knowledge in government outsourcing

This article furthers a sociomaterial framework to examine inter-organizational boundaries in government IT projects. It engages in a dialogue with the practice theory-based approach to boundary spanning and utilizes analytical tools and epistemologies drawn from the social studies of technology. It aims to contribute to a situated, material understanding of inter-organizational boundaries. We argue that boundaries in outsourcing relationships can be de facto enacted through definitions of what counts as relevant knowledge. Information systems have a key role in eliciting such definitions, thus establishing knowledge asymmetries and regimes of inclusion and exclusion. The article responds to the call to value the role of artefacts in IT research. Furthermore, it eventually shows that understanding knowledge asymmetries triggered at the micro-level of information systems can help to examine macro-scale transformations between the public and the private sectors. To illustrate the framework, two ethnographic case studies of governmental IT projects are discussed. The first case concerns a permit and licence submission service in Italy. The second case analyses a 20-year-long database integration carried on at the Dutch land registry. In the first case, information systems made relevant a form of knowledge developed by contractors; in the second case, the integration process valued knowledge developed in-house. Three sets of implications are drawn for the theory and practice of inter-organizational IT projects. To conclude, the article focuses on inter-organizational boundaries involving the public and the private sectors and foresees a novel interdisciplinary research direction at the confluence of information systems and political studies.

Artifacts annotations in anesthesia blood pressure data by man and machine

Physiologic data from anesthesia monitors are automatically captured. Yet erroneous data are stored in the process as well. While this is not interfering with clinical care, research can be affected. Researchers should find ways to remove artifacts. The aim of the present study was to compare different artifact annotation strategies, and to assess if a machine learning algorithm is able to accept or reject individual data points. Non-cardiac procedures requiring invasive blood pressure monitoring were eligible. Two trained research assistants observed procedures live for artifacts. The same procedures were also retrospectively annotated for artifacts by a different person. We compared the different ways of artifact identifications and modelled artifacts with three different learning algorithms (lasso restrictive logistic regression, neural network and support vector machine). In 88 surgical procedures including 5711 blood pressure data points, the live observed incidence of artifacts was 2.1% and the retrospective incidence was 2.2%. Comparing retrospective with live annotation revealed a sensitivity of 0.32 and specificity of 0.98. The performance of the learning algorithms which we applied ranged from poor (kappa 0.053) to moderate (kappa 0.651). Manual identification of artifacts yielded different incidences in different situations, which were not comparable. Artifact detection in physiologic data collected during anesthesia could be automated, but the performance of the learning algorithms in the present study remained moderate. Future research should focus on optimization and finding ways to apply them with minimal manual work. The present study underlines the importance of an explicit definition for artifacts in database research.

Coming to Terms with Technoscience: The Heideggerian Way

Heidegger’s oeuvre (> 100 volumes) contains a plethora of comments on contemporary science, or rather technoscience because, according to Heidegger, science is inherently technical. What insights can be derived from such comments for philosophers questioning technoscience as it is practiced today? Can Heidegger’s thoughts become a source of inspiration for contemporary scholars who are confronted with automated sequencing machines, magnetic resonance imaging machines and other technoscientific contrivances? This is closely related to the question of method, I will argue. Although Heidegger himself was notoriously ambivalent when it came to method, especially in his later writings, his oeuvre nonetheless contains important hints for how a philosophical questioning of technoscience could be practiced, such as: paying attention to language (to the words that we use) or taking a step backwards (towards the moment of commencement of the type of rationality at work). For Heidegger, method means: being underway, and a philosophical method must be developed along the way. After discussing Heidegger’s views on method, both in his earlier and in his later writings, three dimensions of contemporary technoscience will be addressed, namely: technoscientific objects (research artefacts), technoscientific sites (laboratories as unworldly environments) and technoscience as a global enterprise. In the final section, the question will be addressed whether and how a critical encounter between philosophy and technoscience is possible.

A Meta-Analysis of Neglected Artifacts in Organizational Behavior Mediation Research

Applications of statistical mediation analysis in organizational behavior research are vulnerable to statistical and methodological artifacts whose actual influence on the results of primary studies remains largely undocumented. Unlike traditional meta-analysis, ours neither estimated nor corrected population parameters for statistical artifacts. Instead, we computed the magnitude of spurious variance associated with artifacts across primary studies focused on not only mediation and partial mediation, but also incremental non-mediation, which we defined as the amount of incremental variance explained by the mediator beyond that explained by the independent variable. Analyses of 268 primary studies covering four widely employed organizational behavior mediators – justice, support, leader-member exchange (LMX), and trust – published in high-impact journals revealed that artifacts accounted for up to 16%, 9%, and 17% of the variability in the mediation, partial mediation, and incremental non-mediation effects, respectively. Certain artifact combinations made the likelihood of a specific type of mediation more likely. For instance, partial mediation was larger when (1) the study employed the same method to gauge the independent and dependent variables, (2) the independent variable was reliable, and (3) the mediator was unreliable. We provide caveats intended to help researchers qualify their inferences of mediation drawn from this body of literature.

Serious Financial Games for Youth: An Evolutionary Action Design Science Approach

Supporting lifelong financial education through games for youth requires intricate design consideration of immersion, persuasion, personalization, and evolution. This is important as it ensures the knowledge and skills learnt from the games can be applied in the long term. Furthermore, it serves as a foundation to enable learning, and to learn how to learn. Studying this phenomenon may require calibrated design science methodologies to appropriately address those challenges. Traditionally, a design science approach focuses on creating research artefacts limited to specific timeframes and socio-technological interactions. However, when a design study targets on the orientation of lifelong learning, extant research on design methodologies seem incompatible. This paper proposes methodological contributions that are philosophically oriented in design and are also compatible with lifelong learning.