Interactivity/Reciprocity (Online Discussions/Discussion Quality)

Abstract

Interactivity (or reciprocity) is a key dimension to assess the deliberative quality of online discussions. In quantitative content analyses, this dimension measures if participants engage in dialog with each other and refer to each other. Field of application/Theoretical foundation Most studies on online discussions draw on deliberative norms to measure the quality of their discourse (e.g., Esau et al., 2017; Friess et al., 2021; Rowe, 2015; Ziegele et al., 2020; Zimmermann, 2017). Deliberation is an important concept for the study of (political) online discussions (Ziegele et al., 2020). It focuses on a free and equal exchange of arguments to bridge social differences and legitimize political decisions (Dryzek et al., 2019; Fishkin, 1991, Habermas, 2015). Interactivity is a key dimension of deliberative quality, since deliberation is always a reciprocal and dialogical process (Goodin, 2000; Zimmermann, 2017). Participants engage in a dialogic exchange with each other, reflecting on other views and perspectives, and referring to each other (Friess et al., 2021; Ziegele et al., 2020). This reciprocal process includes both responding and listening (Barber, 1984; Graham, 2009). Interactivity is considered essential for desirable effects of deliberation such as learning, tolerance building and opinion change (Estlund & Landmore, 2018; Friess et al., 2021).   References/Combination with other methods Besides quantitative content analyses, the (deliberative) quality of online discussions is examined with qualitative content analyses and discourse analyses (e.g., Graham & Witschge, 2003; Price & Capella, 2002). Furthermore, participants’ perceptions of the quality of online discussions are investigated with qualitative interviews (e.g., Engelke, 2019; Ziegele, 2016) or a combination of qualitative interviews and content analy­sis (Díaz Noci et al., 2012). Cross-references Interactivity is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on inclusivity, rationality, explicit civility, and storytelling regarding theoretical background, references/combinations with other methods, and some example studies.   Information on Esau et al. (2017) Authors: Katharina Esau, Dennis Friess, & Christiane Eilders Research question: “How does platform design affect the level of deliberative quality?” (p. 323) Object of analysis: “We conducted a quantitative content analysis of user comments left in a news forum, on news websites, and on Facebook news pages concerning the same journalistic content on two topics […]  A sample of news articles […] with related user comments, was drawn from the online platforms of four German news media […] The first step of the sampling process consisted of 18 news articles from which 3,341 comments were collected […] In the second step for each article, up to 100 sequential comments were randomly selected for content analysis, leading to a total sample of 1,801 comments (979 on Facebook, 591 on news websites, and 231 in the news forum)” (p. 331). Time frame of analysis: December 2015 Info about variables Level of analysis: individual comment Variables and reliability: see Table 1 Table 1: Variables and Reliability (Esau et al., 2017, pp. 332-333): Dimension Measure Definition RCA Cohen’s Kappa Reciprocity General engagement This measure captures whether a comment addresses another comment. .92 -   Argumentative engagement This measure captures whether a comment addresses a specific argument made in another comment. .77 .542   Critical engagement This measure captures whether a comment is critical of another comment. .89 -         n = 40; 12 coders Values: Dichotomous measures (yes, no)    Information on Heinbach & Wilms (2022) Authors: Dominique Heinbach & Lena K. Wilms (Codebook by Dominique Heinbach, Marc Ziegele, & Lena K. Wilms) Research question: Which attributes differentiate moderated from unmoderated comments? Object of analysis: The quantitative content analysis was based on a stratified random sample of moderated and not moderated comments (N = 1.682) from the German online participation platform “#meinfernsehen202” [#myTV2021], a citizen participation platform to discuss the future of public broadcasting in Germany. Time frame of analysis: November 24, 2020 to March 3, 2021 Info about variables Level of analysis: User comment Variables and reliability: see Table 2 Table 2: Variables and reliability (Heinbach & Wilms, 2022) Dimension Measure Definition Krippendorff’s α Reciprocity Reference to other users or to the community Does the comment refer to at least one other user, a group of users, or all users in the community? .78   Reference to the content of other comments Does the comment refer to content, arguments or positions in other comments? .78   Critical reference Does the comment refer to other comments in a critical manner? .86       n = 159, 3 coders   Values: All variables were coded on a four-point scale (1 = clearly not present; 2 = rather not present; 3 = rather present; 4 = clearly present). Detailed explanations and examples for each value are provided in the Codebook (in German). Codebook: in the appendix of this entry (in German)   Information on Stromer-Galley (2007) Author: Jennifer Stromer-Galley Research question: The aim of the paper was developing a coding scheme for academics and practitioners of deliberation to systematically measure what happens during group deliberations (p. 1; p. 7). Object of analysis: The author conducted a secondary analysis of online group discussions (23 groups with 5-12 participants) in an experiment called “The Virtual Agora Project” at Carnegie Mellon Unversitiy in Pittsburgh, Pennsylvania. Participants attended the discussions from dormitory rooms that were equipped with a computer, headphones, and microphone. The group discussions were recorded and transcribed for analysis (pp. 7-8). Although strictly speaking the study does not analyze media content, the coding scheme has provided the basis for numerous other studies on the deliberative quality of online discussions (e.g., Rowe, 2015; Stroud et al., 2015; Ziegele et al., 2020). Time frame of analysis: Three weeks in July 2004 (p. 7). Info about variables Level of analysis: Level of the turn: Speaking contribution of a participant. Participants had to get “in line” to speak. When a speaker had finished their turn, the software activated the next speaker (max. 3 minutes per turn) (p. 8). Level of the thought: Coders segmented each turn into thought units before coding the categories. “A thought is defined as an utterance (from a single sentence to multiple sentences) that expresses an idea on a topic. A change in topic signaled a change in thought. A second indicator of a change in thought was a change in the type of talk. The distinct types of talk that this coding captured were the following: talk about the problem of public schools, talk about the process of the talk, talk about the process of the deliberation, and social talk” (p. 9). Variables an values: see Table 3 Reliability: “Two coders spent nearly two months developing and training with the coding scheme. The intercoder agreement measures […] were established from coding 3 of the 23 groups, which were randomly selected. […] Cohen’s Kappas of the coding elements described above are as follows: thought statements on the problem of public schools, .95; […] turn type (new topic, continuing self, responding to others) .97; meta-talk, 1.0 […]” (p. 13-14). Codebook: in the appendix (pp. 22-33) Table 3: Variables and values of the dimension “engagement” (Stromer-Galley, 2007, p.12; pp. 24-26). Category Level Description Value Definition Turn-type Turn Identify whether and to whom this turn is referring. Starting a new topic A new topic (not prompted by the moderator).       Respond on topic A turn that is in response to a prior speaker or is on a topic that has been discussed. Includes responding to multiple speakers.       Respond to moderator A turn that is a response to a prompt or question from the moderator.       Continue self A turn that seems not to respond to anything a prior speaker said but to continue the current speaker’s ideas from one of his or her prior turns. Problem Thought Talk about the problem is talk that focuses on the issue under consideration. Question A genuine question directed to another speaker that is trying to seek information or an opinion from others. Metatalk   Thought Metatalk is talk about the talk. It attempts to step back and assess what has transpired or is transpiring in the interaction. Consensus Consensus metatalk is talk about the speaker’s sense of consensus of the group (“I think we all agree that . . . .”), including an explanation for the collective’s opinions or the collective’s behavior (We’re asking you these questions because . .).       Conflict Highlighting some disagreement or conflict in the group (“I sense some disagreement around . . . .”).       Clarify own Clarify the speaker’s own opinion or fact statement (“what I’m trying to say is”). It’s an attempt to clarify what the speaker means. This will arise ONLY after they’ve provided an opinion, NOT a question, and are now trying to clarify their original opinion on the problem, likely because they believe someone has misunderstood them.       Clarify other Clarify someone else’s argument/opinion or fact statement (“Sally, so, what you’re saying is . . . “). It is an attempt to clarify what someone else means. Pay attention to the use of another participants’ name. That can be a sign of metatalk of another’s position.   Example studies Esau, K., Fleuß, D. & Nienhaus, S.‑M. (2021). Different Arenas, Different Deliberative Quality? Using a Systemic Framework to Evaluate Online Deliberation on Immigration Policy in Germany. Policy & Internet, 13(1), 86–112. https://doi.org/10.1002/poi3.232 Esau, K., Friess, D. & Eilders, C. (2017). Design Matters! An Empirical Analysis of Online Deliberation on Different News Platforms. Policy & Internet, 9(3), 321–342. https://doi.org/10.1002/poi3.154 Esau, K., Frieß, D. & Eilders, C. (2019). Online-Partizipation jenseits klassischer Deliberation: Eine Analyse zum Verhältnis unterschiedlicher Deliberationskonzepte in Nutzerkommentaren auf Facebook-Nachrichtenseiten und Beteiligungsplattformen. In I. Engelmann, M. Legrand & H. Marzinkowski (Hrsg.), Digital Communication Research: Bd. 6. Politische Partizipation im Medienwandel (S. 221–245). Friess, D., Ziegele, M. & Heinbach, D. (2021). Collective Civic Moderation for Deliberation? Exploring the Links between Citizens’ Organized Engagement in Comment Sections and the Deliberative Quality of Online Discussions. Political Communication, 38(5), 624–646. https://doi.org/10.1080/10584609.2020.1830322 Heinbach, D. & Wilms, L. K. (2022): Der Einsatz von Moderation bei #meinfernsehen2021 [The deployment of moderation at #meinfernsehen2021]. In: F. Gerlach, C. Eilders & K. Schmitz (Eds.): #meinfernsehen2021. Partizipationsverfahren zur Zukunft des öffentlich-rechtlichen Fernsehens. Baden-Baden: Nomos. Rowe, I. (2015). Deliberation 2.0: Comparing the Deliberative Quality of Online News User Comments Across Platforms. Journal of Broadcasting & Electronic Media, 59(4), 539–555. https://doi.org/10.1080/08838151.2015.1093482 Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12. Ziegele, M., Quiring, O., Esau, K. & Friess, D. (2020). Linking News Value Theory With Online Deliberation: How News Factors and Illustration Factors in News Articles Affect the Deliberative Quality of User Discussions in SNS’ Comment Sections. Communication Research, 47(6), 860-890. https://doi.org/10.1177/0093650218797884 Zimmermann, T. (2017). Digitale Diskussionen: Über politische Partizipation mittels Online-Leserkommentaren. Edition Politik: Bd. 44. transcript Verlag. http://www.content-select.com/index.php?id=bib_view&ean=9783839438886 Further references Barber, B. R. (1984). Strong democracy: Participatory politics for a new age. University of California Press. Díaz Noci, J., Domingo, D., Masip, P., Micó, J. L. & Ruiz, C. (2012). Comments in news, democracy booster or journalistic night­mare: Assessing the quality and dynamics of citizen debates in Catalan online new­spapers. #ISOJ, 2(1), 46–64. https://isoj.org/ wp-content/uploads/2016/10/ISOJ_Jour­nal_V2_N1_2012_Spring.pdf#page=46 Dryzek, J. S., Bächtiger, A., Chambers, S., Cohen, J., Druckman, J. N., Felicetti, A., Fishkin, J. S., Farrell, D. M., Fung, A., Gutmann, A., Landemore, H., Mansbridge, J., Marien, S., Neblo, M. A., Niemeyer, S., Setälä, M., Slothuus, R., Suiter, J., Thompson, D. & Warren, M. E. (2019). The crisis of democracy and the science of deliberation. Science (New York, N.Y.), 363(6432), 1144–1146. https://doi.org/10.1126/science.aaw2694 Engelke, K. M. (2019). Enriching the Conversation: Audience Perspectives on the Deliberative Nature and Potential of User Comments for News Media. Digital Journalism, 8(4), 1–20. https://doi.org/10.1080/21670811.2019.1680567 Estlund, D. & Landemore, H. (2018). The epistemic value of democratic deliberation. In A. Bächtiger, J. S. Dryzek, J. J. Mansbridge & M. E. Warren (Hrsg.), Oxford handbooks online. The Oxford handbook of deliberative democracy: An introduction (S. 113–131). Oxford University Press. Fishkin, J. S. (1991). Democracy and deliberation: New directions for democratic reform. Yale University Press. http://www.jstor.org/stable/10.2307/j.ctt1dt006v https://doi.org/10.2307/j.ctt1dt006v Goodin, R. E. (2000). Democratic Deliberation Within. Philosophy & Public Affairs, 29(1), 81–109. https://doi.org/10.1111/j.1088-4963.2000.00081.x Graham, T. (2009). What's Wife Swap got to do with it? Talking politics in the net-based public sphere Amsterdam: University of Amsterdam DOI: 10.13140/RG.2.1.3413.0088 Graham, T. & Witschge, T. (2003). In Search of Online Deliberation: Towards a New Method for Examining the Quality of Online Discussions. Communications, 28(2). https://doi.org/10.1515/comm.2003.012 Habermas, J. (2015). Between facts and norms: Contributions to a discourse theory of law and democracy (Reprinted.). Polity Press. Price, V. & Cappella, J. N. (2002). Online deliberation and its influence: The Electronic Dialogue Project in Campaign 2000. IT&Society, 1(1), 303–329. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.9.5945&rep=rep1&type=pdf Stroud, N. J., Scacco, J. M., Muddiman, A., & Curry, A. L. (2015). Changing Deliberative Norms on News Organizations' Facebook Sites. Journal of Computer-Mediated Communication, 20(2), 188–203. https://doi.org/10.1111/jcc4.12104 Ziegele, M. (2016). Nutzerkommentare als Anschlusskommunikation: Theorie und qualitative Analyse des Diskussionswerts von Online-Nachrichten [The Discussion Value of Online News. An Analysis of User Comments on News Platforms]. Springer VS.