A modeler's manifesto: Synthesizing modeling best practices with social science frameworks to support critical approaches to data science

M.V Eitzel

doi:10.3897/rio.7.e71553

Abstract

In the face of the "crisis of reproducibility" and the rise of "big data" with its associated issues, modeling needs to be practiced more critically and less automatically. Many modelers are discussing better modeling practices, but to address questions about the transparency, equity, and relevance of modeling, we also need the theoretical grounding of social science and the tools of critical theory. I have therefore synthesized recent work by modelers on better practices for modeling with social science literature (especially feminist science and technology studies) to offer a "modeler’s manifesto": a set of applied practices and framings for critical modeling approaches. Broadly, these practices involve 1) giving greater context to scientific modeling through extended methods sections, appendices, and companion articles, clarifying quantitative and qualitative reasoning and process; 2) greater collaboration in scientific modeling via triangulation with different data sources, gaining feedback from interdisciplinary teams, and viewing uncertainty as openness and invitation for dialogue; and 3) directly engaging with justice and ethics by watching for and mitigating unequal power dynamics in projects, facing the impacts and implications of the work throughout the process rather than only afterwards, and seeking opportunities to collaborate directly with people impacted by the modeling.

Highlights

Data science has the potential to work towards a more sustainable, more equitable world
Statistical analysis is in the hands of private companies who have no motivation to be transparent (Davies 2017)
I reflected on how these social science framings could inform my own modeling practices

Summary

Introduction

Data science has the potential to work towards a more sustainable, more equitable world. Seemingly innocuous details can have profound impacts on the results." Porter even goes so far as to say that "experimental regularities should perhaps be interpreted in terms of human skill rather than of stable underlying entities." So while modeling the observation process can be crucial to re-purposing high-variety data, this can be difficult if there is insufficient information available This is problematic in situations where many ecological datasets are "going dark" -- meaning that they are unpublished work which may be forgotten and lost, metadata and all (Heidorn 2008). During 2015-2018, through one-on-one meetings, participation in group events, auditing courses, and joining reading groups, I compiled a bibliography of pieces that addressed my questions regarding "better" modeling that could mitigate or avoid some of the issues I'd encountered In reading this material, I reflected on how these social science framings (especially those from feminist STS) could inform my own modeling practices. How can we create and use models, especially algorithmic and big data models, in a more just way (to ensure equitable benefit for those who are impacted by the models)?

How are modelers already discussing and addressing these issues?

Findings

Discussion

Conclusions