Storytelling (Online Discussions/Discussion Quality)

Abstract

Storytelling is a variable to assess the quality of online discussions beyond traditional deliberation criteria, such as rationality, interactivity, and inclusiveness. It belongs to the so-called type II deliberation (Bächtiger et al., 2010) and includes personal experiences and narratives from participants’ own lives and environments.

Field of application/Theoretical foundation

In addition to traditional criteria such as rationality, more inclusive forms are increasingly playing a role in deliberation research, for example, the so-called "type II deliberation" (Bächtiger et al., 2010) or "democratic deliberation" (Mansbridge, 2007). Following these approaches, the use of expressive forms of communication such as participants’ personal experiences and stories, emotions, and humor are also legitimate in deliberative discourse (Esau et al., 2019; Esau et al., 2021). Of these inclusive quality criteria, the most important is probably storytelling, where participants contribute personal stories, narratives, and experiences to the discussion (Bächtiger et al., 2010; Polletta & Lee, 2006). By now, storytelling is widely accepted as an indicator of quality in deliberation research (Bächtiger et al., 2010; Zimmermann, 2017). It is considered the most significant alternative to rational argumentation, which has been so crucial in traditional deliberation literature, since positions can also be justified with personal stories and experiences (Bächtiger et al., 2010; Polletta & Lee, 2006; Steiner, 2012).

References/Combination with other methods

Besides quantitative content analyses, storytelling is examined  by means of qualitative content analyses and discourse analyses (Graham & Witschge, 2003; Steiner et al., 2017) or a combination of qualitative and quantitative analyses (Polletta & Lee, 2006).

Cross-references

Storytelling is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on interactivity, inclusivity, rationality, and explicit civility regarding theoretical background, references/combinations with other methods, and some example studies.

Information on Esau et al. (2021)

Authors: Katharina Esau, Dannica Fleuß, & Sarah-Michelle Nienhaus

Research question: “RQ1: How does the deliberative quality of online participatory practices vary between different deliberative arenas? RQ2: Does deliberative quality, according to a classic concept of deliberation, vary between different arenas? RQ 1.2: Does the proportion of alternative forms of communication (expressions of emotion and storytelling) vary between arenas?” (p. 88)

Object of analysis: The authors conducted comparative content analyses of three different deliberative arenas: A government-run online consultation platform (all user comments, N = 603), the comment sections of mass media platforms (stratified random sample, N = 794), and a civil society Facebook community page (all user comments, N = 767). This resulted in a final sample of 2,164 user comments across all three platforms (pp. 101-102).

Time frame of analysis: 2015

Info about variables

Level of analysis: User comment (p. 101)

Variables: see Table 1

Values: Dichotomous measure (yes,no)

Reliability: “The comments were analyzed by a team of five trained coders. […] All variables had a Krippendorff’s α > 0.70“ (p. 102).

Table 1: Variables (Esau et al., 2021, p. 102).

Dimension	Measure	Definition
Storytelling	Personal storytelling	This measure captured whether a comment reported a personal experience (or an experience of known others), expressed in a narrative form.

Information on Heinbach & Wilms (2022)

Authors: Dominique Heinbach & Lena K. Wilms (Codebook by Dominique Heinbach, Marc Ziegele, & Lena K. Wilms)

Research question: Which attributes differentiate moderated from unmoderated comments?

Object of analysis: The quantitative content analysis was based on a stratified random sample of moderated and not moderated comments (N = 1.682) from the German online participation platform “#meinfernsehen202” [#myTV2021], a citizen participation platform to discuss the future of public broadcasting in Germany.

Time frame of analysis: November 24, 2020 to March 3, 2021

Info about variables

Level of analysis: User comment

Variables and reliability: see Table 2

Values: All variables were coded on a four-point scale (1 = clearly not present; 2 = rather not present; 3 = rather present; 4 = clearly present). Detailed explanations and examples for each value are provided in the Codebook (in German).

Codebook: in the appendix of this entry (in German)

Table 2: Variables and reliability (Heinbach & Wilms, 2022)

Dimension	Measure	Definition	Krippendorff’s α
Storytelling	Storytelling	Does the comment contain personal stories and experiences?	.85
	Narrative reasoning	Does the comment contain personal stories or experiences to support an argument or position?	.75
			n = 159, 3 coders

Information on Zimmermann (2017)

Author: Tobias Zimmermann

Research question: Which role do online reader comments play for a deliberative-democratic understanding of a digital public sphere? (p. 11)

Object of analysis: To compare discursive participation online and offline, the author conducted a full-sample content analysis of online reader comments (N = 1.176) and letters to the editor (N = 381) from German local newspapers on three similar conflicts in local politics concerning the renaming of streets and squares. Because the coding scheme was based on the discourse quality index (DQI), only contributions that contained a demand were included in the analysis, that is, “a proposal on what decision should or should not be made” Steenbergen et al., 2003, p. 27). Only then, a speech act is considered relevant from a discourse ethics perspective.

Time frame of analysis: June 2012 to May 2013

Info about variables

Variables: Following Bächtiger et al. (2011, p. 17), the variable storytelling measures “whether participants use personal narratives or experiences”. However, the author interprets storytelling in a narrow sense, that is, if users clarify the narration as their personal story of experience. Consequently, only own experiences are considered (p. 173). Following Steiner (2012), the study operationalizes storytelling on an ordinal scale in order to measure its deliberative quality (p. 174).

Level of analysis: Individual contribution

Variables and values: see Table 3

Reliability: Intracoder reliability for storytelling was tested on a subset of 100 comments and exceeded a Krippendorff’s Alpha (ordinal) above .73 (p. 201).

Codebook: pp. 159-185 (in German)

Table 3: Variables and values (Zimmermann, 2017, p. 174)

Variable	Value	Definition
Storytelling	No storytelling	The comment does not contain any personal story of experience of the author.
	Unfocused storytelling	The comment does contain at least one personal story of experience of the author.
	Justifying storytelling	The comment does contain at least one personal story of experience of the author, which serves as sole justification for the demand.
	Complementary storytelling	The comment does contain at least one personal story of experience of the author that supports the justification.

Example studies

Esau, K., Fleuß, D. & Nienhaus, S.‑M. (2021). Different Arenas, Different Deliberative Quality? Using a Systemic Framework to Evaluate Online Deliberation on Immigration Policy in Germany. Policy & Internet, 13(1), 86–112. https://doi.org/10.1002/poi3.232

Heinbach, D. & Wilms, L. K. (2022): Der Einsatz von Moderation bei #meinfernsehen2021 [The deployment of moderation at #meinfernsehen2021]. In: F. Gerlach, C. Eilders & K. Schmitz (Eds.): #meinfernsehen2021. Partizipationsverfahren zur Zukunft des öffentlich-rechtlichen Fernsehens. Baden-Baden: Nomos.

Rowe, I. (2015). Deliberation 2.0: Comparing the Deliberative Quality of Online News User Comments Across Platforms. Journal of Broadcasting & Electronic Media, 59(4), 539–555. https://doi.org/10.1080/08838151.2015.1093482

Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.

Zimmermann, T. (2017). Digitale Diskussionen: Über politische Partizipation mittels Online-Leserkommentaren [Digital discussions: On political participation trough online reader comments]. Edition Politik: Bd. 44. transcript Verlag. http://www.content-select.com/index.php?id=bib_view&ean=9783839438886

Further references

Bächtiger, A., Gerber, M., & Shikano, S. (2011, August). Deliberative Abilities of Ordinary Citizens. In 6th ECPR General Conference, Reykjavik.

Bächtiger, A., Shikano, S., Pedrini, S. & Ryser, M. (2010). Measuring Deliberation 2.0: Standards, Discourse Types, and Sequentialization. University of Konstanz and University of Bern. https://ash.harvard.edu/files/ash/files/baechtiger_0.pdf

Graham, T. & Witschge, T. (2003). In Search of Online Deliberation: Towards a New Method for Examining the Quality of Online Discussions. Communications, 28(2). https://doi.org/10.1515/comm.2003.012

Mansbrige, J. (2007). “Deliberative Democracy” or “Democratic Deliberation”? In S. W. Rosenberg (Hrsg.), Deliberation, Participation and Democracy (S. 251–271). Palgrave Macmillan, London. https://doi.org/10.1057/9780230591080_12

Polletta, F. & Lee, J. (2006). Is Telling Stories Good for Democracy? Rhetoric in Public Deliberation after 9/11. American Sociological Review, 71(5), 699–721. https://doi.org/10.1177/000312240607100501

Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1(1), 21–48. https://doi.org/10.1057/palgrave.cep.6110002

Steiner, J. (2012). The Foundations of Deliberative Democracy. Cambridge University Press.

Steiner, J., Jaramillo, M. C., Maia, R. C. M. & Mameli, S. (2017). Deliberation Across Deeply Divided Societies. Cambridge University Press. https://doi.org/10.1017/9781316941591