DOCA - Database of Variables for Content Analysis

Inclusivity (Online Discussions/Discussion Quality)

2022-11-29T20:45:46+01:00

Inclusivity is a key dimension to assess the deliberative quality of online discussions. In quantitative content analyses, this dimension measures the openness and accessibility of and the equality and diversity within a discussion.

Field of application/Theoretical foundation

Most studies on online discussions draw on deliberative norms to measure the quality of their discourse (e.g., Esau et al., 2017; Friess et al., 2021; Rowe, 2015; Ziegele et al., 2020; Zimmermann, 2017). Deliberation is an important concept for the study of (political) online discussions (Ziegele et al., 2020). It focuses on a free and equal exchange of arguments to bridge social differences and legitimize political decisions (Dryzek et al., 2019; Fishkin, 1991, Habermas, 2015). Inclusivity or open participation is one of the central criteria of Habermas’ discourse ethics. Deliberative discussions should be open to everyone and all participants should be able to express their attitudes, desires, and needs (Habermas, 2015; Steenbergen et al., 2003). Inclusivity occurs on two levels: On the one hand, it is a matter of open and free access for all citizens, which precedes the actual discussion process (input, Friess & Eilders, 2015). This precondition is often referred to as universalism or openness (Engelke, 2019; Kersting, 2008). In the discussion process itself (troughput, Friess & Eilders, 2015), all voices should have an equal opportunity to be heard and responded to, regardless of factors such as gender, race, or social background. Inclusivity usually implies opinion diversity, since one-sided discussions carry the risk of marginalizing other positions (Habermas, 2006; Manin, 1987; Zimmermann, 2017).

References/Combination with other methods

Besides quantitative content analyses, the (deliberative) quality of online discussions is examined with qualitative content analyses and discourse analyses (e.g., Graham & Witschge, 2003; Price & Capella, 2002). Furthermore, participants’ perceptions of the quality of online discussions are investigated with qualitative interviews (e.g., Engelke, 2019; Ziegele, 2016) or a combination of qualitative interviews and content analysis (Díaz Noci et al., 2012).

Cross-references

Inclusivity is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on rationality, interactivity, explicit civility, and storytelling regarding the theoretical background, references/combinations with other methods, and some example studies.

Information on Stromer-Galley (2007)

Author: Jennifer Stromer-Galley

Research question: The aim of the paper was developing a coding scheme for academics and practitioners of deliberation to systematically measure what happens during group deliberations (p. 1; p. 7).

Object of analysis: The author conducted a secondary analysis of online group discussions (23 groups with 5-12 participants) in an experiment called “The Virtual Agora Project” at Carnegie Mellon Unversitiy in Pittsburgh, Pennsylvania. Participants attended the discussions from dormitory rooms that were equipped with a computer, headphones, and microphone. The group discussions were recorded and transcribed for analysis (pp. 7-8). Although strictly speaking the study does not analyze media content, the coding scheme has provided the basis for numerous other studies on the deliberative quality of online discussions (e.g., Rowe, 2015; Stroud et al., 2015; Ziegele et al., 2020).

Time frame of analysis: Three weeks in July 2004 (p. 7).

Info about variables

Level of analysis: Equality was measured on the level of the group discussion as well as on the level of the thought. Coders segmented each speaking contribution into thought units as first stage of the coding process. “A thought is defined as an utterance (from a single sentence to multiple sentences) that expresses an idea on a topic. A change in topic signaled a change in thought. A second indicator of a change in thought was a change in the type of talk. The distinct types of talk that this coding captured were the following: talk about the problem of public schools, talk about the process of the talk, talk about the process of the deliberation, and social talk” (p. 9).

Variables and values: For measuring the variable equality, the number of speakers within a group was counted. Furthermore, the thoughts were counted for the number of words per thought. Additionally, the total number of thoughts spoken in a given group was counted (p. 15).

Reliability: “Two coders spent nearly two months developing and training with the coding scheme. The intercoder agreement measures […] were established from coding 3 of the 23 groups, which were randomly selected. […] The coders of the unitizing process achieved a statistically significant correlation of .86 (p < .001)” (p. 14).

Codebook: in the appendix (pp. 22-33)

Information on Zimmermann (2017)

Author: Tobias Zimmermann

Research question: Which role do online reader comments play for a deliberative-democratic understanding of a digital public sphere? (p. 11)

Object of analysis: To compare discursive participation online and offline, the author conducted a full-sample content analysis of online reader comments (N = 1.176) and letters to the editor (N = 381) from German local newspapers on three similar conflicts in local politics concerning the renaming of streets and squares. Because the coding scheme was based on the discourse quality index (DQI), only contributions that contained a demand were included in the analysis, that is, “a proposal on what decision should or should not be made” Steenbergen et al., 2003, p. 27). Only then, a speech act is considered relevant from a discourse ethics perspective.

Time frame of analysis: June 2012 to May 2013

Info about variables

Level of analysis: see Table 1

Variables: Following Stromer-Galley (2007) and Bächtiger et al. (2010), the author operationalizes participation (egalitarian openness) based on frequency and volume of the comments. Furthermore, the study assigns the comments to a pro or contra side in regard to their content. This allows conclusions regarding the equality of different positions (pp. 161-163). Additionally, based on the DQI (Steenbergen et al., 2003), he included the variable common good reference, because reasoning oriented to common interests represents the most inclusive form of reasoning (pp. 190-191).

Reliability: Intracoder reliability was tested on a subset of 100 comments. The variable “common good reference” reached a Krippendorff’s Alpha of .71 (p. 201).

Codebook: pp. 159-185 (in German)

Table 1: Variables, values and level of analysis (Zimmermann, 2017, p. 163; p. 191)

Indicator	Category	Definition	Level of analysis
Egalitarian openness	Egalitarian openness (a)	Length of a comment (or letter to the editor)	Individual contribution
	Egalitarian openness (b)	Number of contributions per participant	Discussion
	Egalitarian openness (c)	Number of contributions per thematic position	Discussion
Common good reference	No common good reference	No reference to the common good is explicitly made	Individual contribution
	Explicit common good reference	The contribution includes at least one explicit reference to the common good (utilitarian or disadvantaged-oriented)	Individual contribution

Example studies

Ruiz, C., Domingo, D., Micó, J. L., Díaz-Noci, J., Meso, C. & Masip, P. (2011). Public Sphere 2.0? The Democratic Qualities of Citizen Debates in Online Newspapers. The International Journal of Press/Politics, 16, 463–487.

Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.

Ziegele, M., Quiring, O., Esau, K. & Friess, D. (2020). Linking News Value Theory With Online Deliberation: How News Factors and Illustration Factors in News Articles Affect the Deliberative Quality of User Discussions in SNS’ Comment Sections. Communication Research, 47(6), 860-890. https://doi.org/10.1177/0093650218797884

Zimmermann, T. (2017). Digitale Diskussionen: Über politische Partizipation mittels Online-Leserkommentaren [Digital discussions: On political participation trough online reader comments]. Edition Politik: Bd. 44. transcript Verlag. http://www.content-select.com/index.php?id=bib_view&ean=9783839438886

Further references

Bächtiger, A., Shikano, S., Pedrini, S. & Ryser, M. (2010). Measuring Deliberation 2.0: Standards, Discourse Types, and Sequentialization. University of Konstanz and University of Bern. https://ash.harvard.edu/files/ash/files/baechtiger_0.pdf

Díaz Noci, J., Domingo, D., Masip, P., Micó, J. L. & Ruiz, C. (2012). Comments in news, democracy booster or journalistic nightmare: Assessing the quality and dynamics of citizen debates in Catalan online newspapers. #ISOJ, 2(1), 46–64. https://isoj.org/ wp-content/uploads/2016/10/ISOJ_Journal_V2_N1_2012_Spring.pdf#page=46

Dryzek, J. S., Bächtiger, A., Chambers, S., Cohen, J., Druckman, J. N., Felicetti, A., Fishkin, J. S., Farrell, D. M., Fung, A., Gutmann, A., Landemore, H., Mansbridge, J., Marien, S., Neblo, M. A., Niemeyer, S., Setälä, M., Slothuus, R., Suiter, J., Thompson, D. & Warren, M. E. (2019). The crisis of democracy and the science of deliberation. Science (New York, N.Y.), 363(6432), 1144–1146. https://doi.org/10.1126/science.aaw2694

Engelke, K. M. (2019). Enriching the Conversation: Audience Perspectives on the Deliberative Nature and Potential of User Comments for News Media. Digital Journalism, 8(4), 1–20. https://doi.org/10.1080/21670811.2019.1680567

Esau, K., Friess, D. & Eilders, C. (2017). Design Matters! An Empirical Analysis of Online Deliberation on Different News Platforms. Policy & Internet, 9(3), 321–342. https://doi.org/10.1002/poi3.154

Fishkin, J. S. (1991). Democracy and deliberation: New directions for democratic reform. Yale University Press. http://www.jstor.org/stable/10.2307/j.ctt1dt006v https://doi.org/10.2307/j.ctt1dt006v

Friess, D. & Eilders, C. (2015). A systematic review of online deliberation research. Policy & Internet, 7(3), 319–339. https://doi.org/10.1002/poi3.95

Friess, D., Ziegele, M. & Heinbach, D. (2021). Collective Civic Moderation for Deliberation? Exploring the Links between Citizens’ Organized Engagement in Comment Sections and the Deliberative Quality of Online Discussions. Political Communication, 38(5), 624–646. https://doi.org/10.1080/10584609.2020.1830322

Graham, T. & Witschge, T. (2003). In Search of Online Deliberation: Towards a New Method for Examining the Quality of Online Discussions. Communications, 28(2). https://doi.org/10.1515/comm.2003.012

Habermas, J. (2006). Political communication in media society: Does democracy still enjoy an epistemic dimension? The impact of normative theory on empirical research. Communication Theory, 16(4), 411–426.

Habermas, J. (2015). Between facts and norms: Contributions to a discourse theory of law and democracy (Reprinted.). Polity Press.

Kersting, N. (2008). Innovative Partizipation: Legitimation, Machtkontrolle und Transformation. Eine Einführung [Innovative participation. Legitimation, control of power, and transformation. An introduction]. In N. Kersting (Hrsg.), Politische Beteiligung: Einführung in dialogorientierte Instrumente politischer und gesellschaftlicher Partizipation (S. 11–39). VS Verlag für Sozialwissenschaften.

Manin, B. (1987). On Legitimacy and Political Deliberation. Political Theory, 15(3), 338–368. https://doi.org/10.1177/0090591787015003005

Price, V. & Cappella, J. N. (2002). Online deliberation and its influence: The Electronic Dialogue Project in Campaign 2000. IT&Society, 1(1), 303–329. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.9.5945&rep=rep1&type=pdf

Rowe, I. (2015). Deliberation 2.0: Comparing the Deliberative Quality of Online News User Comments Across Platforms. Journal of Broadcasting & Electronic Media, 59(4), 539–555. https://doi.org/10.1080/08838151.2015.1093482

Steenbergen, M. R., Bächtiger, A., Spörndli, M. & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1(1), 21–48. https://doi.org/10.1057/palgrave.cep.6110002

Stroud, N. J., Scacco, J. M., Muddiman, A., & Curry, A. L. (2015). Changing Deliberative Norms on News Organizations' Facebook Sites. Journal of Computer-Mediated Communication, 20(2), 188–203. https://doi.org/10.1111/jcc4.12104

Ziegele, M. (2016). Nutzerkommentare als Anschlusskommunikation: Theorie und qualitative Analyse des Diskussionswerts von Online-Nachrichten [The Discussion Value of Online News. An Analysis of User Comments on News Platforms]. Springer VS.

Storytelling (Online Discussions/Discussion Quality)

2022-11-29T20:36:41+01:00

Storytelling is a variable to assess the quality of online discussions beyond traditional deliberation criteria, such as rationality, interactivity, and inclusiveness. It belongs to the so-called type II deliberation (Bächtiger et al., 2010) and includes personal experiences and narratives from participants’ own lives and environments.

Field of application/Theoretical foundation

In addition to traditional criteria such as rationality, more inclusive forms are increasingly playing a role in deliberation research, for example, the so-called "type II deliberation" (Bächtiger et al., 2010) or "democratic deliberation" (Mansbridge, 2007). Following these approaches, the use of expressive forms of communication such as participants’ personal experiences and stories, emotions, and humor are also legitimate in deliberative discourse (Esau et al., 2019; Esau et al., 2021). Of these inclusive quality criteria, the most important is probably storytelling, where participants contribute personal stories, narratives, and experiences to the discussion (Bächtiger et al., 2010; Polletta & Lee, 2006). By now, storytelling is widely accepted as an indicator of quality in deliberation research (Bächtiger et al., 2010; Zimmermann, 2017). It is considered the most significant alternative to rational argumentation, which has been so crucial in traditional deliberation literature, since positions can also be justified with personal stories and experiences (Bächtiger et al., 2010; Polletta & Lee, 2006; Steiner, 2012).

References/Combination with other methods

Besides quantitative content analyses, storytelling is examined by means of qualitative content analyses and discourse analyses (Graham & Witschge, 2003; Steiner et al., 2017) or a combination of qualitative and quantitative analyses (Polletta & Lee, 2006).

Cross-references

Storytelling is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on interactivity, inclusivity, rationality, and explicit civility regarding theoretical background, references/combinations with other methods, and some example studies.

Information on Esau et al. (2021)

Authors: Katharina Esau, Dannica Fleuß, & Sarah-Michelle Nienhaus

Research question: “RQ1: How does the deliberative quality of online participatory practices vary between different deliberative arenas? RQ2: Does deliberative quality, according to a classic concept of deliberation, vary between different arenas? RQ 1.2: Does the proportion of alternative forms of communication (expressions of emotion and storytelling) vary between arenas?” (p. 88)

Object of analysis: The authors conducted comparative content analyses of three different deliberative arenas: A government-run online consultation platform (all user comments, N = 603), the comment sections of mass media platforms (stratified random sample, N = 794), and a civil society Facebook community page (all user comments, N = 767). This resulted in a final sample of 2,164 user comments across all three platforms (pp. 101-102).

Time frame of analysis: 2015

Info about variables

Level of analysis: User comment (p. 101)

Variables: see Table 1

Values: Dichotomous measure (yes,no)

Reliability: “The comments were analyzed by a team of five trained coders. […] All variables had a Krippendorff’s α > 0.70“ (p. 102).

Table 1: Variables (Esau et al., 2021, p. 102).

Dimension	Measure	Definition
Storytelling	Personal storytelling	This measure captured whether a comment reported a personal experience (or an experience of known others), expressed in a narrative form.

Information on Heinbach & Wilms (2022)

Authors: Dominique Heinbach & Lena K. Wilms (Codebook by Dominique Heinbach, Marc Ziegele, & Lena K. Wilms)

Research question: Which attributes differentiate moderated from unmoderated comments?

Object of analysis: The quantitative content analysis was based on a stratified random sample of moderated and not moderated comments (N = 1.682) from the German online participation platform “#meinfernsehen202” [#myTV2021], a citizen participation platform to discuss the future of public broadcasting in Germany.

Time frame of analysis: November 24, 2020 to March 3, 2021

Info about variables

Level of analysis: User comment

Variables and reliability: see Table 2

Values: All variables were coded on a four-point scale (1 = clearly not present; 2 = rather not present; 3 = rather present; 4 = clearly present). Detailed explanations and examples for each value are provided in the Codebook (in German).

Codebook: in the appendix of this entry (in German)

Table 2: Variables and reliability (Heinbach & Wilms, 2022)

Dimension	Measure	Definition	Krippendorff’s α
Storytelling	Storytelling	Does the comment contain personal stories and experiences?	.85
	Narrative reasoning	Does the comment contain personal stories or experiences to support an argument or position?	.75
			n = 159, 3 coders

Information on Zimmermann (2017)

Author: Tobias Zimmermann

Research question: Which role do online reader comments play for a deliberative-democratic understanding of a digital public sphere? (p. 11)

Time frame of analysis: June 2012 to May 2013

Info about variables

Variables: Following Bächtiger et al. (2011, p. 17), the variable storytelling measures “whether participants use personal narratives or experiences”. However, the author interprets storytelling in a narrow sense, that is, if users clarify the narration as their personal story of experience. Consequently, only own experiences are considered (p. 173). Following Steiner (2012), the study operationalizes storytelling on an ordinal scale in order to measure its deliberative quality (p. 174).

Level of analysis: Individual contribution

Variables and values: see Table 3

Reliability: Intracoder reliability for storytelling was tested on a subset of 100 comments and exceeded a Krippendorff’s Alpha (ordinal) above .73 (p. 201).

Codebook: pp. 159-185 (in German)

Table 3: Variables and values (Zimmermann, 2017, p. 174)

Variable	Value	Definition
Storytelling	No storytelling	The comment does not contain any personal story of experience of the author.
	Unfocused storytelling	The comment does contain at least one personal story of experience of the author.
	Justifying storytelling	The comment does contain at least one personal story of experience of the author, which serves as sole justification for the demand.
	Complementary storytelling	The comment does contain at least one personal story of experience of the author that supports the justification.

Example studies

Esau, K., Fleuß, D. & Nienhaus, S.‑M. (2021). Different Arenas, Different Deliberative Quality? Using a Systemic Framework to Evaluate Online Deliberation on Immigration Policy in Germany. Policy & Internet, 13(1), 86–112. https://doi.org/10.1002/poi3.232

Heinbach, D. & Wilms, L. K. (2022): Der Einsatz von Moderation bei #meinfernsehen2021 [The deployment of moderation at #meinfernsehen2021]. In: F. Gerlach, C. Eilders & K. Schmitz (Eds.): #meinfernsehen2021. Partizipationsverfahren zur Zukunft des öffentlich-rechtlichen Fernsehens. Baden-Baden: Nomos.

Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.

Further references

Bächtiger, A., Gerber, M., & Shikano, S. (2011, August). Deliberative Abilities of Ordinary Citizens. In 6th ECPR General Conference, Reykjavik.

Mansbrige, J. (2007). “Deliberative Democracy” or “Democratic Deliberation”? In S. W. Rosenberg (Hrsg.), Deliberation, Participation and Democracy (S. 251–271). Palgrave Macmillan, London. https://doi.org/10.1057/9780230591080_12

Polletta, F. & Lee, J. (2006). Is Telling Stories Good for Democracy? Rhetoric in Public Deliberation after 9/11. American Sociological Review, 71(5), 699–721. https://doi.org/10.1177/000312240607100501

Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1(1), 21–48. https://doi.org/10.1057/palgrave.cep.6110002

Steiner, J. (2012). The Foundations of Deliberative Democracy. Cambridge University Press.

Steiner, J., Jaramillo, M. C., Maia, R. C. M. & Mameli, S. (2017). Deliberation Across Deeply Divided Societies. Cambridge University Press. https://doi.org/10.1017/9781316941591

Explicit civility (Online Discussions/Discussion Quality)

2022-11-29T20:28:23+01:00

Explicit civility or respect is a key dimension to assess the (deliberative) quality of online discussions. In contrast to operationalizations that address civility through the mere absence of incivility or offensive language, this construct measures explicit indicators for respect, such as acknowledgements, endorsement, and explicit valuation.

Field of application/Theoretical foundation

References/Combination with other methods

Besides quantitative content analyses, the (deliberative) quality of online discussions is examined with qualitative content analyses and discourse analyses (Graham & Witschge, 2003; Price & Capella, 2002). Furthermore, perceptions of civility and incivility are investigated with qualitative interviews and focus groups (Bormann, 2022; Engelke, 2019; Ziegele, 2016), surveys (Kenski et al., 2020; Stryker et al., 2016) and experimental designs (Muddiman, 2017; Ziegele et al., 2019).

Cross-references

Explicit civility is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on interactivity, inclusivity, rationality, and storytelling regarding theoretical background, references/combinations with other methods, and some example studies.

Information on Heinbach & Wilms (2022)

Authors: Dominique Heinbach & Lena K. Wilms (Codebook by Dominique Heinbach, Marc Ziegele, & Lena K. Wilms)

Research question: Which attributes differentiate moderated from unmoderated comments?

Object of analysis: The quantitative content analysis was based on a stratified random sample of moderated and not moderated comments (N = 1.682) from the German online participation platform “#meinfernsehen2021” [#myTV2021], a citizen participation platform to discuss the future of public broadcasting in Germany.

Time frame of analysis: November 24, 2020 to March 3, 2021

Info about variables

Level of analysis: User comment

Variables and reliability: see Table 1

Codebook: in the appendix of this entry (in German)

Table 1: Variables and reliability (Heinbach & Wilms, 2022)

Dimension

Measure

Definition

Krippendorff’s α

(n = 159, 3 coders)

Explicit

respect

Polite salutation

Does the comment contain formulations of polite salutations, greetings or farewell phrases, e.g., “Dear …”, “Have a good weekend”?

.80

Expression of respect

Does the comment contain appreciation, approval, praise, or acknowledgements, e.g, “Kudos!”, “You are doing a great job, “Thank you”, “I respect your opinion“?

.71

Information on Stromer-Galley (2007)

Author: Jennifer Stromer-Galley

Time frame of analysis: Three weeks in July 2004 (p. 7).

Info about variables

Level of analysis: Social talk was measured on the level of the thought. Coders segmented each speaking contribution into thought units before coding the categories. “A thought is defined as an utterance (from a single sentence to multiple sentences) that expresses an idea on a topic. A change in topic signaled a change in thought. A second indicator of a change in thought was a change in the type of talk. The distinct types of talk that this coding captured were the following: talk about the problem of public schools, talk about the process of the talk, talk about the process of the deliberation, and social talk” (p. 9).

Variables and values: see Table 2

Reliability: Reliability scores for social talk are not provided.

Codebook: in the appendix (pp. 22-33)

Table 2: Variables and values (Stromer-Galley, 2007, p. 9; pp. 26-27).

Variable	Description	Value	Definition
Social talk	Social talk is talk that brings the strangers together by building social bonds, including salutations, praise, and apologies.	Salutations	Statements of welcome, greeting, hello, and good bye, see you later, and the like.
		Apologies	Statements of apology: I’m sorry, and ‘I hope’ statements, such as “I hope I haven’t been too obnoxious.” Includes statements of reflection of how the participant performed in the group (likely comes at the end of the group’s discussion): “I hope my few ideas did get across.”
		Praise	Includes thank you, you’re welcome, as well as praise for other individuals or the group (“you’ve been a good group.” I’ve really enjoyed myself,” “this has been fun”) Praise in the service of an argument about the problem is coded as a problem thought (“I want to commend Sally for volunteering at her school. We need more people to be volunteers”).
		ChitChat	Thought statements that are not on topic relative to the deliberation. These could be jokes or puns (but not as they relate to the problem of schools), social chit chat about the weather, and the like.

Information on Zimmermann (2017)

Author: Tobias Zimmermann

Research question: Which role do online reader comments play for a deliberative-democratic understanding of a digital public sphere? (p. 11)

Time frame of analysis: June 2012 to May 2013

Info about variables

Variables: For the variable respect towards groups, the author extends the respect variable of the Discourse Quality Index (DQI, Steenbergen et al., 2003) by adding anti-democratic behavior following Papacharissi (2004). Besides the ordinal variable “respect”, the author measured the dichotomous variable “democratic respect”.

Level of analysis: Individual contribution

Variables and values: see Table 3

Reliability: Intracoder reliability was tested on a subset of 100 comments. The ordinal respect variable exceeded a Krippendorff’s Alpha above .73 (p. 201). The dichotomous variable “democratic respect” that measured if a comment contained anti-democratic behavior or not reached a Krippendorff’s Alpha of .89 (p. 200-201).

Codebook: pp. 159-185 (in German)

Table 3: Variables and values (Zimmermann, 2017, p. 171; p. 189)

Variable	Value	Definition
Respect	Anti-democratic behavior	Only or predominantly statements are made that either contain at least one threat against the democratic system as a whole, use negative stereotypes, or question other individuals’ individual liberty rights.
	No respect	This category applies when exclusively or predominantly negative statements about others, but not anti-democratic statements are made.
	Implicit respect	There are no explicitly positive or negative statements to others, that is, a comment is neither explicitly respectful nor disrespectful.
	Balanced respect	Both positive and negative statements in relation to others are expressed in the same degree.
	Explicit respect	This category is applied when there is at least one explicitly positive statement regarding others and no negative statements, or if the positive statement clearly dominates the negative.
Democratic respect	Anti-democratic behavior	Only or predominantly statements are made that either contain at least one threat against the democratic system as a whole, use negative stereotypes, or question other individuals’ individual liberty rights.
	Democratic respect	Anti-democratic behavior does not occur.

Example studies

Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.

Further references

Bormann, M. (2022). Perceptions and Evaluations of Incivility in Public Online Discussions—Insights From Focus Groups With Different Online Actors. Frontiers in Political Science, 4, Article 812145. https://doi.org/10.3389/fpos.2022.812145

Habermas, J. (2007). Moral consciousness and communicative action (Repr). Polity.

Kenski, K., Coe, K. & Rains, S. A. (2020). Perceptions of Uncivil Discourse Online: An Examination of Types and Predictors: An examination of types and predictors. Communication Research, 47(6), 795–814. https://doi.org/10.1177/0093650217699933

Muddiman, A. (2017). Personal and Public Levels of Political Incivility. International Journal of Communication, 11, 3182–3202.

Papacharissi, Z. (2004). Democracy online: civility, politeness, and the democratic potential of online political discussion groups. new media & society, 6(2), 259–283. https://doi.org/10.1177/1461444804041444

Steiner, J. (2012). The Foundations of Deliberative Democracy. Cambridge University Press. https://doi.org/10.1017/CBO9781139057486

Stryker, R., Conway, B. A. & Danielson, J. T. (2016). What is political incivility? Communication Monographs, 83(4), 535–556. https://doi.org/10.1080/03637751.2016.1201207

Ziegele, M. (2016). Nutzerkommentare als Anschlusskommunikation: Theorie und qualitative Analyse des Diskussionswerts von Online-Nachrichten [The Discussion Value of Online News. An Analysis of User Comments on News Platforms]. Springer VS.

Ziegele, M., Naab, T. K. & Jost, P. (2020). Lonely together? Identifying the determinants of collective corrective action against uncivil comments. New Media & Society, 22(5), 731-751. https://doi.org/10.1177/1461444819870130

Interactivity/Reciprocity (Online Discussions/Discussion Quality)

2022-11-29T20:08:23+01:00

Interactivity (or reciprocity) is a key dimension to assess the deliberative quality of online discussions. In quantitative content analyses, this dimension measures if participants engage in dialog with each other and refer to each other.

Field of application/Theoretical foundation

Most studies on online discussions draw on deliberative norms to measure the quality of their discourse (e.g., Esau et al., 2017; Friess et al., 2021; Rowe, 2015; Ziegele et al., 2020; Zimmermann, 2017). Deliberation is an important concept for the study of (political) online discussions (Ziegele et al., 2020). It focuses on a free and equal exchange of arguments to bridge social differences and legitimize political decisions (Dryzek et al., 2019; Fishkin, 1991, Habermas, 2015). Interactivity is a key dimension of deliberative quality, since deliberation is always a reciprocal and dialogical process (Goodin, 2000; Zimmermann, 2017). Participants engage in a dialogic exchange with each other, reflecting on other views and perspectives, and referring to each other (Friess et al., 2021; Ziegele et al., 2020). This reciprocal process includes both responding and listening (Barber, 1984; Graham, 2009). Interactivity is considered essential for desirable effects of deliberation such as learning, tolerance building and opinion change (Estlund & Landmore, 2018; Friess et al., 2021).

References/Combination with other methods

Cross-references

Interactivity is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on inclusivity, rationality, explicit civility, and storytelling regarding theoretical background, references/combinations with other methods, and some example studies.

Information on Esau et al. (2017)

Authors: Katharina Esau, Dennis Friess, & Christiane Eilders

Research question: “How does platform design affect the level of deliberative quality?” (p. 323)

Object of analysis: “We conducted a quantitative content analysis of user comments left in a news forum, on news websites, and on Facebook news pages concerning the same journalistic content on two topics […] A sample of news articles […] with related user comments, was drawn from the online platforms of four German news media […] The first step of the sampling process consisted of 18 news articles from which 3,341 comments were collected […] In the second step for each article, up to 100 sequential comments were randomly selected for content analysis, leading to a total sample of 1,801 comments (979 on Facebook, 591 on news websites, and 231 in the news forum)” (p. 331).

Time frame of analysis: December 2015

Info about variables

Level of analysis: individual comment

Variables and reliability: see Table 1

Table 1: Variables and Reliability (Esau et al., 2017, pp. 332-333):

Dimension	Measure	Definition	RCA	Cohen’s Kappa
Reciprocity	General engagement	This measure captures whether a comment addresses another comment.	.92	-
	Argumentative engagement	This measure captures whether a comment addresses a specific argument made in another comment.	.77	.542
	Critical engagement	This measure captures whether a comment is critical of another comment.	.89	-
				n = 40; 12 coders

Values: Dichotomous measures (yes, no)

Information on Heinbach & Wilms (2022)

Authors: Dominique Heinbach & Lena K. Wilms (Codebook by Dominique Heinbach, Marc Ziegele, & Lena K. Wilms)

Research question: Which attributes differentiate moderated from unmoderated comments?

Time frame of analysis: November 24, 2020 to March 3, 2021

Info about variables

Level of analysis: User comment

Variables and reliability: see Table 2

Table 2: Variables and reliability (Heinbach & Wilms, 2022)

Dimension	Measure	Definition	Krippendorff’s α
Reciprocity	Reference to other users or to the community	Does the comment refer to at least one other user, a group of users, or all users in the community?	.78
	Reference to the content of other comments	Does the comment refer to content, arguments or positions in other comments?	.78
	Critical reference	Does the comment refer to other comments in a critical manner?	.86
			n = 159, 3 coders

Codebook: in the appendix of this entry (in German)

Information on Stromer-Galley (2007)

Author: Jennifer Stromer-Galley

Time frame of analysis: Three weeks in July 2004 (p. 7).

Info about variables

Level of analysis: Level of the turn: Speaking contribution of a participant. Participants had to get “in line” to speak. When a speaker had finished their turn, the software activated the next speaker (max. 3 minutes per turn) (p. 8). Level of the thought: Coders segmented each turn into thought units before coding the categories. “A thought is defined as an utterance (from a single sentence to multiple sentences) that expresses an idea on a topic. A change in topic signaled a change in thought. A second indicator of a change in thought was a change in the type of talk. The distinct types of talk that this coding captured were the following: talk about the problem of public schools, talk about the process of the talk, talk about the process of the deliberation, and social talk” (p. 9).

Variables an values: see Table 3

Reliability: “Two coders spent nearly two months developing and training with the coding scheme. The intercoder agreement measures […] were established from coding 3 of the 23 groups, which were randomly selected. […] Cohen’s Kappas of the coding elements described above are as follows: thought statements on the problem of public schools, .95; […] turn type (new topic, continuing self, responding to others) .97; meta-talk, 1.0 […]” (p. 13-14).

Codebook: in the appendix (pp. 22-33)

Table 3: Variables and values of the dimension “engagement” (Stromer-Galley, 2007, p.12; pp. 24-26).

Category	Level	Description	Value	Definition
Turn-type	Turn	Identify whether and to whom this turn is referring.	Starting a new topic	A new topic (not prompted by the moderator).
			Respond on topic	A turn that is in response to a prior speaker or is on a topic that has been discussed. Includes responding to multiple speakers.
			Respond to moderator	A turn that is a response to a prompt or question from the moderator.
			Continue self	A turn that seems not to respond to anything a prior speaker said but to continue the current speaker’s ideas from one of his or her prior turns.
Problem	Thought	Talk about the problem is talk that focuses on the issue under consideration.	Question	A genuine question directed to another speaker that is trying to seek information or an opinion from others.
Metatalk	Thought	Metatalk is talk about the talk. It attempts to step back and assess what has transpired or is transpiring in the interaction.	Consensus	Consensus metatalk is talk about the speaker’s sense of consensus of the group (“I think we all agree that . . . .”), including an explanation for the collective’s opinions or the collective’s behavior (We’re asking you these questions because . .).
			Conflict	Highlighting some disagreement or conflict in the group (“I sense some disagreement around . . . .”).
			Clarify own	Clarify the speaker’s own opinion or fact statement (“what I’m trying to say is”). It’s an attempt to clarify what the speaker means. This will arise ONLY after they’ve provided an opinion, NOT a question, and are now trying to clarify their original opinion on the problem, likely because they believe someone has misunderstood them.
			Clarify other	Clarify someone else’s argument/opinion or fact statement (“Sally, so, what you’re saying is . . . “). It is an attempt to clarify what someone else means. Pay attention to the use of another participants’ name. That can be a sign of metatalk of another’s position.

Example studies

Esau, K., Frieß, D. & Eilders, C. (2019). Online-Partizipation jenseits klassischer Deliberation: Eine Analyse zum Verhältnis unterschiedlicher Deliberationskonzepte in Nutzerkommentaren auf Facebook-Nachrichtenseiten und Beteiligungsplattformen. In I. Engelmann, M. Legrand & H. Marzinkowski (Hrsg.), Digital Communication Research: Bd. 6. Politische Partizipation im Medienwandel (S. 221–245).

Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.

Zimmermann, T. (2017). Digitale Diskussionen: Über politische Partizipation mittels Online-Leserkommentaren. Edition Politik: Bd. 44. transcript Verlag. http://www.content-select.com/index.php?id=bib_view&ean=9783839438886

Further references

Barber, B. R. (1984). Strong democracy: Participatory politics for a new age. University of California Press.

Estlund, D. & Landemore, H. (2018). The epistemic value of democratic deliberation. In A. Bächtiger, J. S. Dryzek, J. J. Mansbridge & M. E. Warren (Hrsg.), Oxford handbooks online. The Oxford handbook of deliberative democracy: An introduction (S. 113–131). Oxford University Press.

Fishkin, J. S. (1991). Democracy and deliberation: New directions for democratic reform. Yale University Press. http://www.jstor.org/stable/10.2307/j.ctt1dt006v https://doi.org/10.2307/j.ctt1dt006v

Goodin, R. E. (2000). Democratic Deliberation Within. Philosophy & Public Affairs, 29(1), 81–109. https://doi.org/10.1111/j.1088-4963.2000.00081.x

Graham, T. (2009). What's Wife Swap got to do with it? Talking politics in the net-based public sphere Amsterdam: University of Amsterdam DOI: 10.13140/RG.2.1.3413.0088

Habermas, J. (2015). Between facts and norms: Contributions to a discourse theory of law and democracy (Reprinted.). Polity Press.

Ziegele, M. (2016). Nutzerkommentare als Anschlusskommunikation: Theorie und qualitative Analyse des Diskussionswerts von Online-Nachrichten [The Discussion Value of Online News. An Analysis of User Comments on News Platforms]. Springer VS.

Rationality (Online Discussions/Discussion Quality)

2022-11-29T18:03:59+01:00

Rationality is considered the most important dimension to assess the deliberative quality of online discussions. In quantitative content analyses, it is usually measured with a set of variables, including (among others) reasoning, justification, fact claims, evidence, additional knowledge, and topic relevance.

Field of application/Theoretical foundation

Most studies on online discussions draw on deliberative theories to measure the quality of their discourse (e.g., Esau et al., 2017; Friess et al., 2021; Rowe, 2015; Ziegele et al., 2020; Zimmermann, 2017). Deliberation is an important concept for the study of (political) online discussions (Ziegele et al., 2020). It focuses on a free and equal exchange of arguments to bridge social differences and legitimize political decisions (Dryzek et al., 2019; Fishkin, 1991, Habermas, 2015). Rationality is considered the most important dimension of deliberative quality, which is inherent in most conceptualizations (Frieß & Eilders, 2015). Rationality is primarily about reasoning, justifications, and facts (Engelke, 2019). Discussion participants should provide justifications and evidence to support their positions (Friess et al., 2021). These reasons and arguments must be both criticizable and verifiable or falsifiable (Esterling, 2011; Habermas, 1995). Counterarguments and different perspectives should also be included (Engelke, 2019; Ziegele et al., 2020). This allows the elaboration of the best arguments in the deliberation process and an informed opinion formation based on these arguments (“the unforced force of the better argument”, Habermas, 2015). A rational discourse and a constructive discussion atmosphere are also considered necessary for reaching a rationally motivated consensus, a central aim of formal deliberation (Cohen, 1989; Friess & Eilders, 2015; Stromer-Galley, 2007).

References/Combination with other methods

Cross-references

Rationality is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on interactivity, inclusivity, explicit civility, and storytelling regarding theoretical background, references/combinations with other methods, and some example studies.

Information on Heinbach & Wilms (2022)

Authors: Dominique Heinbach & Lena K. Wilms (Codebook by Dominique Heinbach, Marc Ziegele, & Lena K. Wilms)

Research question: Which attributes differentiate moderated from unmoderated comments?

Object of analysis: The quantitative content analysis was based on a stratified random sample of moderated and unmoderated comments (N = 1.682) from the German online participation platform “#meinfernsehen202” [#myTV2021], a citizen participation platform to discuss the future of public broadcasting in Germany.

Time frame of analysis: November 24, 2020 to March 3, 2021

Info about variables

Level of analysis: User comment

Variables and reliability: see Table 1

Table 1: Variables and reliability (Heinbach & Wilms, 2022)

Dimension	Measure	Definition	Krippendorff’s α (ordinal)
Rationality	Topic relevance	Does the comment refer to the topic of the post?	.70
	Fact claims	Does the comment contain at least one objectively falsifiable statement with a claim to truth?	.78
	Reasoning	Does the comment contain at least one justification to support a statement (e.g., an assertion, opinion, or claim)?	.73
	Solution proposal	Does the comment contain at least one suggestion on how to resolve problems or issues?	.75
	Additional knowledge	Does the comment contain additional information that is of a knowledge nature of and adds content-related value?	.72
	Genuine questions	Does the comment contain at least one question with a genuine need for information, e.g. questions of knowledge, understanding, justification or opinion?	.75
			n = 159, 3 coders

Codebook: in the appendix of this entry (in German)

Information on Zimmermann (2017)

Author: Tobias Zimmermann

Research question: Which role do online reader comments play for a deliberative-democratic understanding of a digital public sphere? (p. 11)

Time frame of analysis: June 2012 to May 2013

Info about variables

Variables: Based on the DQI (Steenbergen et al., 2003) the author operationalizes the level of justification as an indicator for rationality. This variable distinguishes four levels of justification (p. 164). Besides the ordinal variable “Level of justification”, the author also uses a dichotomous measurement to distinguish between substantiated and unsubstantiated claims.

Level of analysis: Individual contribution

Values: see Table 2

Table 2: Variables and Values (pp. 163-166; p. 188)

Variable	Value	Definition
Level of Justification	No justification	The author makes a demand without justifying it argumentatively. The demand stands for itself.
	Indirect justification	The author introduces an argument but its connection to the demand is incomplete, or its justification is not falsifiable.
	Qualified justification	An argument substantiates a demand. A (falsifiable) link is made as to why one should expect that X contributes to or detracts from Y.
	Detailed justification	At least two complete justifications are given, either two complete justifications for the same demand or complete justifications for two different demands (broad justification). Or one justification explains the represented position in depth from several points of view (deep justification).
Justification	No justification	A user makes a demand that X should (not) be done or happen without giving a justification.
Justification	Justification	A user substantiates a demand why X should (not) be done or happen.

Reliability: Intracoder reliability was tested on a subset of 100 comments. The ordinal variable “level of justification” exceeded a Krippendorff’s Alpha above .73. The dichotomous variable “justification” reached a Krippendorff’s Alpha of .75 (p. 200-201).

Codebook: pp. 159-185 (in German)

Information on Ziegele et al. (2020)

Authors: Marc Ziegele, Oliver Quiring, Katharina Esau, & Dennis Friess

Research questions: RQ1: “Which news factors predict the civility and rationality of reactive user comments?” (p. 869) RQ3: “Which illustration factors predict civil and rational reactive user comments?” (p. 871)

Object of analysis: The quantitative content analysis was based on a sample of top-level comments (i.e., comments responding to the article) from the Facebook pages of nine established German news media outlets (N = 11.218). Three artificial weeks were constructed for the sampling of news articles and user comments. On each access day, three or four news articles and the corresponding user comments were randomly selected from each news page. Then, for each article, the oldest five top-level comments, the most recent five top-level comments, five random top-level comments from the middle of the discussion, and the five most popular comments were selected (20 comments per article) (pp. 872-873).

Time frame of analysis: May 2015 to August 2015

Info about variables

Level of analysis: User comment

Variables and reliability: see Table 3

Table 3: Variables and reliability (p. 874)

Dimension	Measure	Definition	Krippendorff’s α
Rationality	Topic relevance	Is the comment on-topic?	.67
	Balance	Does the comment include a balanced view on the commented issue?	.74
	Additional knowledge	Does the comment contain additional knowledge?	.79
	Elaboration	Does the comment appear elaborate to the coders?	.81
	Arguments	Does the comment provide reasons for its claims?	.74
	Analytical	Does the comment analyze the background of the issue at hand?	.70
	Factual claims	Does the comment provide facts and factual claims?	.72
	Questions	Does the comment include genuine questions?	.80
			n = 100, 9 coders

Values: “Each factor was coded on 3-point scales (0 = absent, 1 = sporadically present, 2 = highly present)” (p. 874).

Example studies

Monnoyer-Smith, L. & Wojcik, S. (2012). Technology and the quality of public deliberation: a comparison between on and offline participation. International Journal of Electronic Governance, 5(1), Artikel 47443, 24. https://doi.org/10.1504/IJEG.2012.047443

Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.

Stroud, N. J., Scacco, J. M., Muddiman, A. & Curry, A. L. (2015). Changing Deliberative Norms on News Organizations' Facebook Sites. Journal of Computer-Mediated Communication, 20(2), 188–203. https://doi.org/10.1111/jcc4.12104

Further references

Cohen, J. (1989). Deliberation and democratic legitimacy. In A. P. Hamlin & P. Pettit (Hrsg.), The good polity: Normative analysis of the state (S. 67–92). Blackwell.

Díaz Noci, J., Domingo, D., Masip, P., Micó, J. L. & Ruiz, C. (2012). Comments in news, democracy booster or journalistic nightmare: Assessing the quality and dynamics of citizen debates in Catalan online newspapers. #ISOJ, 2(1), 46–64. https://isoj.org/wp-content/uploads/2016/10/ISOJ_Journal_V2_N1_2012_Spring.pdf#page=46

Esterling, K. M. (2011). “Deliberative Disagreement” in U.S. Health Policy Committee Hearings. Legislative Studies Quarterly, 36(2), 169–198. https://doi.org/10.1111/j.1939-9162.2011.00010.x

Fishkin, J. S. (1991). Democracy and deliberation: New directions for democratic reform. Yale University Press. http://www.jstor.org/stable/10.2307/j.ctt1dt006v https://doi.org/10.2307/j.ctt1dt006v

Friess, D. & Eilders, C. (2015). A systematic review of online deliberation research. Policy & Internet, 7(3), 319–339. https://doi.org/10.1002/poi3.95

Habermas, J. (2015). Between facts and norms: Contributions to a discourse theory of law and democracy (Reprinted.). Polity Press.

Ziegele, M. (2016). Nutzerkommentare als Anschlusskommunikation: Theorie und qualitative Analyse des Diskussionswerts von Online-Nachrichten [The Discussion Value of Online News. An Analysis of User Comments on News Platforms]. Springer VS.

Consent Communication (Portrayals of Sexuality in Pornography)

2022-10-22T21:19:47+02:00

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices or relational dynamics and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This entry focuses on the representation of consent communication as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of consent communication as one dimension of the portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Manual quantitative content analyses of pornographic material can be combined with qualitative (e.g., Keft-Kennedy, 2008) as well as computational (e.g., Seehuus et al., 2019) content analyses. Furthermore, content analyses can be complemented with qualitative interviews and quantitative surveys to investigate perceptions and evaluations of the portrayals of sexuality in pornography among pornography’s creators and performers (e.g., West, 2019) and audiences (e.g., Cowan & Dunn, 1994; Hardy et al., 2022; Paasoonen, 2021; Shor, 2022). Additionally, experimental studies are helpful to measure directly how different dimensions of pornographic portrayals of sexuality are perceived and evaluated by recipients, and if and how these portrayals can affect audiences’ sexuality-related thoughts, feelings, and behaviors (e.g., Kohut & Fisher, 2013; Miller et al., 2019).

Example studies for manual quantitative content analyses:

A common research hypothesis states that pornography depicts sex mostly without consent communication, especially explicit verbal communication. To test this hypothesis and code pornographic material accordingly, it is necessary to clarify the concept of “consent communication” and use valid and reliable measures for different types of consent communication.

Here it is important to conceptually differentiate between consent communication between characters in the fictional world of the porn scene and consent communication between performers on set (or consent of performers to have their image be recorded and disseminated as pornography). This distinction becomes murky with regard to amateur pornography, which ostensibly is meant to depict “authentic” sex (although this sex may still be performative), and also professional pornography in which a performer is playing “themselves” as opposed to a character. Some production studios (especially those specializing in BDSM content) embed interviews with performers into their videos, in which performers indicate that they consented to the activities presented.

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Consent Communication:* Whenever in the sequence of sexual activities depicted in pornography a new activity is started, the question arises if all participants have consented to the new behavior. Sexual consent between characters can be communicated verbally and nonverbally (Willis et al., 2020). Further, verbal or nonverbal communication may be explicit or implicit (Willis et al., 2020). Apart from issues of performer health protection, explicit consent communication on camera is also regarded as relevant in terms of modelling behaviors for audiences.
N=50 segments (length 20 min. each) from a random sample of 50 bestselling pornographic films (1 segment per film) depicting a total of 1,109 sexual behaviors	Explicit verbal sexual consent	“Straightforward statements, questions, or responses expressing agreement to engage in sexual behavior stated using words for actual sexual behavior or a very close synonym”. Binary coding (1: present; 2: not present).	Cohen’s Kappa: .72	Willis et al. (2020)
	Implicit verbal sexual consent	“Verbally initiating sexual behavior or communicating agreement to engage in sexual behavior without explicitly using the word sex or other close synonyms. The content of the words may not be sexual in nature, but the connotation or tone of voice used by the characters implies sex or is sexual in nature”. Binary coding (1: present; 2: not present).	Cohen’s Kappa: .63
	Explicit nonverbal sexual consent	“Behaviors or actions that are sexually explicit including bodily touching in a sexual way”. Binary coding (1: present; 2: not present).	Cohen’s Kappa: .76
	Implicit nonverbal sexual consent	“Behaviors or actions that imply interest in engagement in sexual behavior”. Binary coding (1: present; 2: not present).	Cohen’s Kappa: .65
	No response	“Characters do not say anything, do not resist, or let the sexual activity happen without much action. The person is a passive participant in sexual behavior, but not uncomfortable, distressed, or showing signs of disinterest”. Binary coding (1: present; 2: not present).	Cohen’s Kappa: .72
	No sexual consent shown	“Scene begins or video cuts away and comes back with characters engaging in sexual behavior without any preceding actions to assess consent”. Binary coding (1: present; 2: not present).	Cohen’s Kappa: .95

For more nuanced analyses of consent communication, the sex/gender of the persons involved in consent communication can be coded (see DOCA entry “Performer Demographics (Portrayals of Sexuality in Pornography)”) and the respective sex acts that are to be consented to (see DOCA entry “Sex Acts (Portrayals of Sexuality in Pornography)”). Furthermore, relationship status between characters might play an important role for consent communication (Willis et al., 2020): Characters in established relationships might be more likely to communicate sexual consent nonverbally than those in casual encounters (see DOCA entry "Relational Context of Sex (Portrayals of Sexuality in Pornography").

References

Carrotte, E. R., Davis, A. C., & Lim, M. S. (2020). Sexual behaviors and violence in pornography: Systematic review and narrative synthesis of video content analyses. Journal of Medical Internet Research, 22(5), Article e16702. https://doi.org/10.2196/16702

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Hardy, J., Kukkonen, T., & Milhausen, R. (2022). Examining sexually explicit material use in adults over the age of 65 years. The Canadian Journal of Human Sexuality, 31(1), 117–129. https://doi.org/10.3138/cjhs.2021-0047

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

Kohut, T., & Fisher, W. A. (2013). The impact of brief exposure to sexually explicit video clips on partnered female clitoral self-stimulation, orgasm and sexual satisfaction. The Canadian Journal of Human Sexuality, 22(1), 40–50. https://doi.org/10.3138/cjhs.935

McKee, A., Byron, P., Litsou, K., & Ingham, R. (2020). An interdisciplinary definition of pornography: Results from a global Delphi panel. Archives of Sexual Behavior, 49(3), 1085–1091. https://doi.org/10.1007/s10508-019-01554-4

Miller, D. J., & McBain, K. A. (2022). The content of contemporary, mainstream pornography: A literature review of content analytic studies. American Journal of Sexuality Education, 17(2), 219–256. https://doi.org/10.1080/15546128.2021.2019648

Miller, D. J., McBain, K. A., & Raggatt, P. T. F. (2019). An experimental investigation into pornography’s effect on men’s perceptions of the likelihood of women engaging in porn-like sex. Psychology of Popular Media Culture, 8(4), 365–375. https://doi.org/10.1037/ppm0000202

Miller, D. J., Raggatt, P. T. F., & McBain, K. (2020). A literature review of studies into the prevalence and frequency of men’s pornography use. American Journal of Sexuality Education, 15(4), 502–529. https://doi.org/10.1080/15546128.2020.1831676

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Seehuus, M., Stanton, A. M., & Handy, A. B. (2019). On the content of "real-world" sexual fantasy: Results from an analysis of 250,000+ anonymous text-based erotic fantasies. Archives of Sexual Behavior, 48(3), 725–737. https://doi.org/10.1007/s10508-018-1334-0

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.

Willis, M., Canan, S. N., Jozkowski, K. N., & Bridges, A. J. (2020). Sexual consent communication in best-selling pornography films: A content analysis. Journal of Sex Research, 57(1), 52–63. https://doi.org/10.1080/00224499.2019.1655522

Relational Context of Sex (Portrayals of Sexuality in Pornography)

2022-10-22T21:11:48+02:00

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices or relational dynamics and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This entry focuses on the representation of relational context of sex as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of relational context of sex as one dimension of the portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Example studies for manual quantitative content analyses:

Common research hypotheses state that sex in pornography is mostly depicted as casual and/or extrarelational, even though real life sex predominantly occurs in committed relationships. To test such hypotheses and code pornographic material accordingly, it is necessary to clarify the concept of “relational context of sex” and use valid and reliable measures for different types of relational contexts. In addition, it is necessary to code the sex/gender of the persons involved.

It is important to note that the relational context of sex may be determined based on the interactions and dialogue between performers or based on video titles and descriptions. For example, a video might depict sex with little or no dialogue indicative of the nature of the relationship between performers, but include a title or description that contextualizes this relationship (e.g., “Cheating wife has sex with stranger” or “woman surprises her fiancé”).

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Relational Context of Sex*: Two people engaging in sex (a dyad) can have different types of romantic or non-romantic relationships with each other and with further people outside this dyad. If a person is having sex with a person they just met, this is defined as casual sex; and if a person is in a monogamous relationship and engages in sex with another person outside this relationship, this is considered extrarelational sex or infidelity (Rasmussen et al., 2019).
N=190 scenes (average length 14 min.) taken from the highest rated section of PornHub (86 scenes) and Xvideos (104 scenes)	Casual sex and further relationship contexts with sex partners	Relationship between dyad members for each dyad engaging in sex during the scene. Polytomous coding (0: no relational information; 1: just met / casual sex; 2: acquaintances/friends; 3: dating; 4: married; 5: not enough information).	Krippendorff’s Alpha average of .74 for all variables in codebook	Rasmussen et al. (2019)
	Extrarelational sex	Sexual scene with at least one of the sexual participants being in a romantic relationship with someone not present in the sexual encounter. Binary coding (1: yes; 2: no).
	- Extrarelational participant dating (type of extrarelational sex)	Sexual scene with at least one participant dating someone not present in the sexual encounter. Binary coding (1: yes; 2: no).
	- Extrarelational participant married (type of extrarelational sex)	Sexual scene with at least one participant being married to someone not present in the sexual encounter. Binary coding (1: yes; 2: no).

If sex is determined to be extrarelational it is possible to further code whether this extrarelational sex is happening with the knowledge, encouragement, or participation of the individual’s partner (e.g., as part of a cuckold fantasy). Rasmussen et al. (2019) refer to this as consensual non-monogamy.

References

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Rasmussen, K. R., Millar, D., & Trenchuk, J. (2019). Relationships and infidelity in pornography: An analysis of pornography streaming websites. Sexuality & Culture, 23(2), 571–584. https://doi.org/10.1007/s12119-018-9574-7

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.

Safer Sex Practices (Portrayals of Sexuality in Pornography)

2022-10-22T21:05:02+02:00

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices or relational dynamics and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This entry focuses on the representation of safer sex practices as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of safer sex practices as one dimension of the portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Example studies for manual quantitative content analyses:

A common research hypothesis states that pornography displays sexuality – and even high-risk sex acts, such as penetrative sex – predominately without condoms. To test such hypotheses and code pornographic material accordingly, it is necessary to clarify the concept of “safer sex” and use valid and reliable measures for different types of safer sex practices.

It is important to note that safer sex practices could be assessed at the sexual encounter level (e.g., Is a condom used during the sexual encounter?) or at the level of specific sexual acts (e.g., Is a condom used during anal sex?). It is also possible that a scene depicts a condom being used, but only for the purpose of it being removed (e.g., to highlight the unrestricted nature of the sex being shown or as part of an impregnation roleplay). Given that such scenarios would depict condom use while simultaneously presenting sex with a condom as being less pleasurable, researchers may need to consider these contextual factors when developing their coding schemes. In gay male pornography “barebacking” (deliberately forgoing condom use during anal sex) is its own sub-genre (Tollini, 2019), thus there is a possibility for indirect coding based on meta-information about whether material sits within this sub-genre.

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Safer Sex Practices:* Condom use during penetrative sex with a penis provides a high degree of protection against pregnancies and several types of STIs (Sexually Transmitted Infections) including HIV (Human Immunodeficiency Virus). While condom use is recommended for private sexual encounters (and demanded by some legislatures for porn performers), condoms can increase pain, risk of injury and risk of STI/HIV transmission among performers (particularly those engaging in receptive penetrative practices) because professional performances are typically much longer and more demanding than the average private sexual encounters. For this reason, some porn performers have challenged the idea that legal obligations for condom use on set effectively protect them (Shachner, 2015). Alternative protection measures are often preferred by professional performers (e.g., long acting contraception methods, regular STI/HIV testing, HIV-Pre-Exposure Prophylaxis). The availability of HIV-Pre-Exposure Prophylaxis has resulted in a trend toward condomless anal sex (“barebacking”) in gay male pornography (Tollini, 2019). Apart from issues of performer health protection, safer sex practices in pornography are also regarded as relevant in terms of modelling behaviors for audiences.
	Condom use
N=50 scenes from 50 best-selling heterosexual adult films (1 scene per film) and N=50 scenes from 50 bestselling male homosexual films (1 scene per film)	- Condom use during penile-oral contact (type of condom use)	Penile-oral contact is shown and a condom is used at least some of the time during this act. Binary coding (1: yes; 2: no).	Cohen’s Kappa across all four variables: .78	Grudzen et al. (2009)
	- Condom use during penile-vaginal contact (type of condom use)	Penile-vaginal contact is shown and a condom is used at least some of the time during this act. Binary coding (1: yes; 2: no).
	- Condom use during penile-anal contact (type of condom use)	Penile-anal contact is shown and a condom is used at least some of the time during this act. Binary coding (1: yes; 2: no).
	- Condom use during anal-to-oral penile insertion	Anal-to-oral penile insertion (penile-oral insertion immediately following penile-anal insertion) is shown and a condom is used at least some of the time during this act. Binary coding (1: yes; 2: no).

Apart from condom use, further safer sex practices can be measured (e.g., visible use of lube to prevent pain or injuries during penetrative sex; observable communication between characters about sexual health status, STI testing, or use of contraception).

References

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Grudzen, C. R., Elliott, M. N., Kerndt, P. R., Schuster, M. A., Brook, R. H., & Gelberg, L. (2009). Condom use and high-risk sexual acts in adult films: A comparison of heterosexual and homosexual films. American Journal of Public Health, 99(S1), S152-156. https://doi.org/10.2105/AJPH.2007.127035

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Shachner, J. (2015). Unwrapped: How the Los Angeles County Safer Sex in the Adult Film Industry Act's condom mandate hurts performers & violates the First Amendment. Health Matrix: The Journal of Law Medicine, 24(1), 345–375.

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

Tollini, C. (2019). How two holdouts went bareback: CockyBoys and Men. com's initial transition to producing videos without condoms. Porn Studies, 6(3), 282-300. https://doi.org/10.1080/23268743.2019.1602958

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.

Performer Bodily Appearance (Portrayals of Sexuality in Pornography)

2022-10-22T20:57:37+02:00

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices or relational dynamics and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This entry focuses on the representation of performer bodily appearance as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of performer bodily appearance as one dimension of the portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Example studies for manual quantitative content analyses:

Common research hypotheses state that performers in pornography are mainly selected and presented to conform to gendered norms of sexual attractiveness but also potentially unhealthy beauty standards or current beauty trends. To test such hypotheses and code pornographic material accordingly, it is necessary to clarify the concept of “performer bodily appearance” and use valid and reliable measures for different aspects of appearance. In addition, it is necessary to code the sex/gender of the persons depicted.

Two different approaches to coding are available: Direct coding based on the performer’s appearance (e.g., breast size) versus indirect coding based on meta-information about the material, such as the sub-genre pornography category the material belongs to (e.g., the “big tits”, “BBW” [big beautiful women], “tattoed women” categories on PornHub) or statistics provided as part of performer profiles published on online platforms (e.g., height, weight, bra or penis size).

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Performer Bodily Appearance:* Among the many aspects of performer appearance, those conventionally related to sexual attractiveness are measured most often in the context of pornography research. Researchers may also measure variables related to general beauty trends in society (e.g., shaving of pubic and body hair) or assess aspects of performer appearance which could be consider to promote unhealthy/unrealistic beauty standards (e.g., performers being unhealthily underweight or extremely muscular). Apart from issues of performer health protection, unhealthy standards of beauty and sexual attractiveness are also regarded as relevant in terms of modelling behaviors for audiences.
N=50 best-selling pornographic videos and DVDs in Australia in 2003 with 838 sexual scenes	Performer body type	Performer body type. Polytomous coding (1: unhealthy underweight; 2: slim / undertoned; 3: average (untoned); 4: average (toned); 5: bulked up / very muscular; 6: overweight).	Not available	McKee et al. (2008)
	Performer breast size	Performer breast size. Polytomous coding (1: smaller than average breasts; 2: average-sized breasts; 3: larger than average breasts).
	Performer breast surgery	Performer breast surgery is obvious. Polytomous coding (1: yes; 2: no; 3: unsure).
	Performer penis size	Performer penis size. Polytomous coding (1: smaller than average penis; 2: average-sized penis; 3: larger than average penis).
N > 6,900 performer profiles from 10 gay male adult websites	Performer penis size	Performer penis size (as listed in performer profile). Polytomous coding (1: 5–6.5 inches; 2: 7–8 inches, 3: 8.5–10 inches, 4: 10.5–13 inches)	Not available	Brennan (2018)
N=50 MILF [“Mother I’d like to fuck” sub-genre category] and 50 “Teen” pornographic videos randomly selected from 10 different adult websites (10 videos per website)	Performer pubic hair	Performer pubic hair. Polytomous coding (1: none; 2. groomed; 3: natural).	Percentage agreement across all variables in codebook: 90.3%	Vannier et al. (2014)

References

Brennan, J. (2018). Size matters: Penis size and sexual position in gay porn profiles. Journal of Homosexuality, 65(7), 912-933. https://doi.org/10.1080/00918369.2017.1364568

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

McKee, A., Albury, K., & Lumby, C. (2008). The porn report. Melbourne University Press.

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

Vannier, S. A., Currie, A. B., & O'Sullivan, L. F. (2014). Schoolgirls and soccer moms: A content analysis of free “teen” and “MILF” online pornography. Journal of Sex Research, 51(3), 253-264. https://doi.org/10.1080/00224499.2013.829795

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.

Performer Demographics (Portrayals of Sexuality in Pornography)

2022-10-22T15:10:51+02:00

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices or relational dynamics and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This entry focuses on the representation of performer demographics (such as sex/gender, age, and race/ethnicity) as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of performer demographics as one dimension of portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Example studies for manual quantitative content analyses:

Common research hypotheses in relation to performer demographics state that pornography portrays sexuality in a sexist manner entailing violence towards and degradation of women, usually perpetrated by men. In addition, it is hypothesized that pornographic portrayals of sexuality are asymmetric in terms of showing men in superior and dominating, and women in subordinate and submissive, positions. This sex/gender asymmetry can be reflected in demographic variables such as social status (difference) or age (difference). Furthermore, mainstream pornography is critized for its racist portrayal of sexuality. This means that non-White performers are underrepresented and if they are represented are often depicted according to racial/ethnic stereotypes. To test such hypotheses and code pornographic material accordingly, it is necessary to clarify demographic concepts such as sex/gender, age, and race/ethnicity and use valid and reliable measures.

It is important to note that in the context of pornographic content research, researchers conceptualize demographic characteristics differently and that two different approaches to coding are available: Direct coding based on the person’s appearance (e.g., apparent sex/gender, age or skin color) versus indirect coding based on meta-information about the material, such as the sub-genre category the material belongs to (e.g., pornography category “Asian” displaying Asian-looking performers or “Teen” displaying adult performers who look very young). If applying an intersectional theoretical framework (see DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)”) the researcher would need to code each performer in terms of multiple demographic variables.

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Sex/gender:* Most analyses of the way sexuality is portrayed in pornography hypothesize (or, at least, acknowledge the possibility) that men and women are depicted differently (e.g., that men are more likely to be depicted as the perpetrators of violent behaviors and that women are more likely to be depicted as recipients of violent behaviors). Accordingly, coding the sex/gender of performers is often essential to addressing research questions in this area. The term gender is often preferred when referring to people as groups, as gender reflects a social categorization, whereas sex reflects a biological categorization (American Psychological Association, 2020). While many content analyses of pornography address sex/gender differences they do not present any standardized measures for the demographic variable of sex/gender. The measure presented below is one of the rare exceptions, but it remains vague in its coding instructions and the meanings of the value “other”.
N=50 segments (length 20 min. each) from a random sample of 50 bestselling pornographic films (1 segment per film) depicting a total of 1,109 sexual behaviors	Sex/gender (based on performer appearance)	“Coder’s perception of character’s sex based on primary and secondary sex characteristics.” Polytomous coding (1: male; 2: female; 3: other).	Cohen’s Kappa: 1.0	Willis et al. (2020)
*Age:* Performer age may be a variable of interest in its own right (e.g., if investigating whether pornography has a bias toward depicting performers in their early 20s). Alternatively, performer age may be recorded to investigate differential depictions by age group (e.g., investigating whether younger female performers are more likely to be depicted as submissive than older female performers). It should be noted that a performer’s age may be different to their character’s age (as is often the case in the legal pornography category “Teen”, where young characters are played by adult performers; Willis et al, 2020). It should be noted that reliable coding of age (of the performer or of the performed character) is difficult as tools such as make-up, costume, lighting, filters can greatly bias impressions. This problem is reflected in the available measures that cannot ensure sufficient reliability.
N=50 best-selling pornographic videos and DVDs in Australia in 2003 with 838 sexual scenes	Age of performer (based on performer appearance)	Performer age. Polytomous coding (1: 18-30 years; 2: 31-40 years; 3: 41-50 years; 4: 51+ years).	Not available	McKee et al. (2008)
N=50 segments (length 20 min. each) from a random sample of 50 bestselling pornographic films (1 segment per film) depicting a total of 1,109 sexual behaviors	Age of character (based on character appearance)	“Coder’s perception of character’s age—not the actor’s—based on physical appearance.” Note: some characters were clearly intended to be under 18 years of age, but their actors were likely older. Polytomous coding (1: <18; 2: 18-20; 3: 21-30; 4: 31-40; 5: 41-50; 6: >50 years).	Cohen’s Kappa: .47	Willis et al. (2020)
*Race/Ethnicity:* Critical analyses of racism in pornography address the mere visibility of different races/ethnicities as well as racial/ethnic stereotypes, such as Black men being depicted as sexually aggressive and well-endowed or Asian women being depicted as petite, submissive and docile (Miller & McBain, 2022).
N=269 popular pornographic videos from different PornHub.com sub-genre categories	Race/ethnicity (based on pornographic sub-genre category)	Videos selected/coded according to race/ethnicity-related sub-genre categories on PornHub. Polytomous coding (1: “Asian/Japanese” PornHub categeory; 2: “Interracial” PornHub category; 3: “Ebony” PornHub category; 4: “Latina” PornHub category).	Not applicable	Shor & Seida (2019)
N=45 pornographic videos from 15 different adult websites (3 videos per website)	Race/ethnicity (based on performer appearance)	Performers coded according to physical appearance. Binary coding (1: White; 2: non-White/other race).	Not available	Gorman et al. (2010)
N=50 segments (length 20 min. each) from a random sample of 50 bestselling pornographic films (1 segment per film) depicting a total of 1,109 sexual behaviors	Race/ethnicity (based on performer appearance)	“Coder’s perception of character’s race based on physical appearance.” Polytomous coding (1: White; 2: Black; 3: Asian; 4: Latina/o; 5: Native American; 6: other).	Cohen’s Kappa: .94	Willis et al. (2020)

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.).

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Gorman, S., Monk-Turner, E., & Fish, J. N. (2010). Free adult internet web sites: How prevalent are degrading acts? Gender Issues, 27(3-4), 131–145. https://doi.org/10.1007/s12147-010-9095-7

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

McKee, A. (2015). Methodological issues in defining aggression for content analyses of sexually explicit material. Archives of Sexual Behavior, 44(1), 81–87. https://doi.org/10.1007/s10508-013-0253-3

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

Shor, E., & Seida, K. (2019). "Harder and harder"? Is mainstream pornography becoming increasingly violent and do viewers prefer violent content? Journal of Sex Research, 56(1), 16–28. https://doi.org/10.1080/00224499.2018.1451476

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.Willis, M., Canan, S. N., Jozkowski, K. N., & Bridges, A. J. (2020). Sexual consent communication in best-selling pornography films: A content analysis. Journal of Sex Research, 57(1), 52–63. https://doi.org/10.1080/00224499.2019.1655522