DOCA - Database of Variables for Content Analysis

Frames (Automated Content Analysis)

Valerie Hase — Fri, 26 Mar 2021 00:00:00 +0100

Frames describe the way issues are presented, i.e., what aspects are made salient when communicating about these issues.

Field of application/theoretical foundation:

The concept of frames is directly based on the theory of “Framing”. However, many studies using automated content analysis are lacking a clear theoretical definition of what constitutes a frame. As an exception, Walter and Ophir (2019) use automated content analysis to explore issue and strategy frames as defined by Cappella and Jamieson (1997). Vu and Lynn (2020) refer to Entman’s (1991) understanding of frames.

The datasets referred to in the table are described in the following paragraph:

Van der Meer et al. (2010) use a dataset consisting of Dutch newspaper articles (1991-2015, N = 9,443) and LDA topic modeling in combination with k-means clustering to identify frames. Walter and Ophir (2019) use three different datasets and a combination of topic modeling, network analysis and community detection algorithms to analyze frames. Their datasets consist of political newspaper articles and wire service coverage (N = 8,337), newspaper articles on foreign nations (2010-2015, N = 18,216) and health-related newspaper coverage (2009-2016, N = 5,005). Lastly, Vu and Lynn (2020) analyze newspaper coverage of the Rohingya crisis (2017-2018, N = 747) concerning frames.

References/combination with other methods of data collection:

While most approaches only rely on automated data collection and analyses, some also combine automated and manual coding. For example, a recent study by Vu and Lynn (2020) proposes to combine semantic networks and manual coding to identify frames.

Table 1. Measurement of “Frames” using automated content analysis.

Author(s)	Sample	Procedure	Formal validity check with manual coding as benchmark*	Code
Vu & Lynn (2020)	Newspaper articles	Semantic networks; manual coding	Reported	Not available
van der Meer et al. (2019)	Newspaper articles	LDA topic modeling; k-means clustering	Not reported	Not available
Walter & Ophir (2019)	(a) U.S. newspapers and wire service articles (b) Newspaper articles (c) Newspaper articles	LDA topic modeling, network analysis; community detection algorithms	Not reported	https://github.com/DrorWalt/ANTMN

*Please note that many of the sources listed here are tutorials on how to conducted automated analyses – and therefore not focused on the validation of results. Readers should simply read this column as an indication in terms of which sources they can refer to if they are interested in the validation of results.

References

Cappella, J. N., & Jamieson, K. H. (1997). Spiral of cynicism: The press and the public good. New York: Oxford University Press.

Entman, R. M. 1991. Framing U.S. coverage of international news: contrasts in narratives of the KAL and Iran Air incidents. Journal of Communication, 41(4), 6-7.

van der Meer, T. G. L. A., Kroon, A. C., Verhoeven, P., & Jonkman, J. (2019). Mediatization and the disproportionate attention to negative news: The case of airplane crashes. Journalism Studies, 20(6), 783–803.

Walter, D., & Ophir, Y. (2019). News frame analysis: an inductive mixed-method computational approach. Communication Methods and Measures, 13(4), 248–266.

Vu, H. T., & Lynn, N. (2020). When the news takes sides: Automated framing analysis of news coverage of the rohingya crisis by the elite press from three countries. Journalism Studies. Online first publication. doi:10.1080/1461670X.2020.1745665

Actors (Automated Content Analysis)

Valerie Hase — Fri, 26 Mar 2021 00:00:00 +0100

Actors in coverage might be individuals, groups, or organizations, which are discussed, described, or quoted in the news.

The datasets referred to in the table are described in the following paragraph:

Benoit and Matuso (2020) uses fictional sentences (N = 5) to demonstrate how named entities and noun phrases can be identified automatically. Lind and Meltzer (2020) demonstrate the use of organic dictionaries to identify actors in German newspaper articles (2013-2017, N = 348,785). Puschmann (2019) uses four data sets to demonstrate how sentiment/tone may be analyzed by the computer. Using tweets (2016, N = 18,826), German newspaper articles (2011-2016, N = 377), Swiss newspaper articles (2007-2012, N = 21,280), and debate transcripts (1970-2017, N = 7,897), he extracts nouns and named entities from text. Lastly, Wiedemann and Niekler (2017) extract proper nouns from State of the Union speeches (1790-2017, N = 233).

Field of application/theoretical foundation:

Related to theories of “Agenda Setting” and “Framing”, analyses might want to know how much weight is given to a specific actor, how these actors are evaluated and what perspectives and frames they might bring into the discussion how prominently.

References/combination with other methods of data collection:

Oftentimes, studies use both manual and automated content analysis to identify actors in text. This might be a useful tool to extend the lists of actors that can be found as well as to validate automated analyses. For example, Lind and Meltzer (2020) combine manual coding and dictionaries to identify the salience of women in the news.

Table 1. Measurement of “Actors” using automated content analysis.

Author(s)	Sample	Procedure	Formal validity check with manual coding as benchmark*	Code
Benoit & Matuso (2020)	Fictional sentences	Part-of-Speech tagging; syntactic parsing	Not reported	https://cran.r-project.org/web/packages/spacyr/vignettes/using_spacyr.html
Lind & Meltzer (2020)	Newspapers	Dictionary approach	Reported	https://osf.io/yqbcj/?view_only=369e2004172b43bb91a39b536970e50b
Puschmann (2019)	(a) Tweets (b) German newspaper articles (c) Swiss newspaper articles (d) United Nations General Debate Transcripts	Part-of-Speech tagging; syntactic parsing	Not reported	http://inhaltsanalyse-mit-r.de/ner.html
Wiedemann & Niekler (2017)	State of the Union speeches	Part-of-Speech tagging	Not reported	https://tm4ss.github.io/docs/Tutorial_8_NER_POS.html

References

Benoit, K., & Matuso. (2020). A Guide to Using spacyr. Retrieved from https://cran.r-project.org/web/packages/spacyr/vignettes/using_spacyr.html

Lind, F., & Meltzer, C. E. (2020). Now you see me, now you don’t: Applying automated content analysis to track migrant women’s salience in German news. Feminist Media Studies, 1–18.

Puschmann, C. (2019). Automatisierte Inhaltsanalyse mit R. Retrieved from http://inhaltsanalyse-mit-r.de/index.html

Wiedemann, G., Niekler, A. (2017). Hands-on: a five day text mining course for humanists and social scientists in R. Proceedings of the 1st Workshop Teaching NLP for Digital Humanities (Teach4DH@GSCL 2017), Berlin. Retrieved from https://tm4ss.github.io/docs/index.html

Sentiment/tone (Automated Content Analysis)

Valerie Hase — Fri, 26 Mar 2021 00:00:00 +0100

Sentiment/tone describes the way issues or specific actors are described in coverage. Many analyses differentiate between negative, neutral/balanced or positive sentiment/tone as broader categories, but analyses might also measure expressions of incivility, fear, or happiness, for example, as more granular types of sentiment/tone. Analyses can detect sentiment/tone in full texts (e.g., general sentiment in financial news) or concerning specific issues (e.g., specific sentiment towards the stock market in financial news or a specific actor).

The datasets referred to in the table are described in the following paragraph:

Puschmann (2019) uses four data sets to demonstrate how sentiment/tone may be analyzed by the computer. Using Sherlock Holmes stories (18th century, N = 12), tweets (2016, N = 18,826), Swiss newspaper articles (2007-2012, N = 21,280), and debate transcripts (2013-2017, N = 205,584), he illustrates how dictionaries may be applied for such a task. Rauh (2019) uses three data sets to validate his organic German language dictionary for sentiment/tone. His data consists of sentences from German parliament speeches (1991-2013, N = 1,500), German-language quasi-sentences from German, Austrian and Swiss party manifestos (1998-2013, N = 14,008) and newspaper, journal and news wire articles (2011-2012, N = 4,038). Silge and Robinson (2020) use six Jane Austen novels to demonstrate how dictionaries may be used for sentiment analysis. Van Atteveldt and Welbers (2020) use state of the Union speeches (1789-2017, N = 58) for the same purpose. The same authors (van Atteveldt & Welbers, 2019) show based on a dataset of N = 2,000 movie reviews how supervised machine learning might also do the trick. In their Quanteda tutorials, Watanabe and Müller (2019) demonstrate the use of dictionaries and supervised machine learning for sentiment analysis on UK newspaper articles (2012-2016, N = 6,000) as well as the same set of movie reviews (n = 2,000). Lastly, Wiedemann and Niekler (2017) use state of the Union speeches (1790-2017, N = 233) to demonstrate how sentiment/tone can be coded automatically via a dictionary approach.

Field of application/theoretical foundation:

Related to theories of “Framing” and “Bias” in coverage, many analyses are concerned with the way the news evaluates and interprets specific issues and actors.

References/combination with other methods of data collection:

Manual coding is needed for many automated analyses, including the ones concerned with sentiment. Studies for example use manual content analysis to develop dictionaries, to create training sets on which algorithms used for automated classification are trained, or to validate the results of automated analyses (Song et al., 2020).

Table 1. Measurement of “Sentiment/Tone” using automated content analysis.

Author(s)	Sample	Procedure	Formal validity check with manual coding as benchmark*	Code
Puschmann (2019)	(a) Sherlock Holmes stories (b) Tweets (c) Swiss newspaper articles (d) German Parliament transcripts	Dictionary approach	Not reported	http://inhaltsanalyse-mit-r.de/sentiment.html
Rauh (2018)	(a) Bundestag speeches (b) Quasi-sentences from German, Austrian and Swiss party manifestos (c) Newspapers, journals, agency reports	Dictionary approach	Reported	https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/BKBXWD
Silge & Robinson (2020)	Books by Jane Austen	Dictionary approach	Not reported	https://www.tidytextmining.com/sentiment.html
van Atteveldt & Welbers (2020)	State of the Union speeches	Dictionary approach	Reported	https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/sentiment_analysis.md
van Atteveldt & Welbers (2019)	Movie reviews	Supervised Machine Learning Approach	Reported	https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/r_text_ml.md
Watanabe & Müller (2019)	Newspaper articles	Dictionary approach	Not reported	https://tutorials.quanteda.io/advanced-operations/targeted-dictionary-analysis/
Watanabe & Müller (2019)	Movie reviews	Supervised Machine Learning Approach	Reported	https://tutorials.quanteda.io/machine-learning/nb/
Wiedemann & Niekler (2017)	State of the Union speeches	Dictionary approach	Not reported	https://tm4ss.github.io/docs/Tutorial_3_Frequency.html

References

Puschmann, C. (2019). Automatisierte Inhaltsanalyse mit R. Retrieved from http://inhaltsanalyse-mit-r.de/index.html

Rauh, C. (2018). Validating a sentiment dictionary for German political language—A workbench note. Journal of Information Technology & Politics, 15(4), 319–343. doi:10.1080/19331681.2018.1485608

Silge, J., & Robinson, D. (2020). Text mining with R. A tidy approach. Retrieved from https://www.tidytextmining.com/

Song, H., Tolochko, P., Eberl, J.-M., Eisele, O., Greussing, E., Heidenreich, T., Lind, F., Galyga, S., & Boomgaarden, H.G. (2020) In validations we trust? The impact of imperfect human annotations as a gold standard on the quality of validation of automated content analysis. Political Communication, 37(4), 550-572.

van Atteveldt, W., & Welbers, K. (2019). Supervised Text Classification. Retrieved from https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/r_text_ml.md

van Atteveldt, W., & Welbers, K. (2020). Supervised Sentiment Analysis in R. Retrieved from https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/sentiment_analysis.md

Watanabe, K., & Müller, S. (2019). Quanteda tutorials. Retrieved from https://tutorials.quanteda.io/

Topics (Automated Content Analysis)

Valerie Hase — Fri, 26 Mar 2021 00:00:00 +0100

Topics describe the main issue discussed in an article, for example: Does an article deal with politics, economics or sports?

Field of application/theoretical foundation:

In the context of “Agenda Setting”, studies analyze which issues are on the public agenda. In the context of “News Values”, studies may analyze why some topics are more prominently covered than others.

References/combination with other methods of data collection:

Many studies combine manual inspection of topics with their automated detection. Quinn et al. (2010) demonstrate for their analyses of legislative speeches how manual inspection may increase the validity of results. Similarly, Hase et al. (2020) use automated content analysis to find and map similar topics for which manual coding is then conducted. Such combinations may contribute to a better and more detailed understanding of topics than automated analyses by themselves.

The datasets referred to in the table are described in the following paragraph:

Puschmann (2019a) uses New York Times articles (1996-2006, N = 30,862) as well as articles from Die Zeit (2011-2016, N = 377) to identify topics using supervised machine learning. In another tutorial, Puschmann (2019b) uses Sherlock Holmes stories (18^th century, N = 12), articles from Die Zeit (2011-2016, N = 377) and debate transcripts (1970-2017, N = 7,897) to apply LDA and structural topic modeling. In her tutorials, Silge (2018a, 2018b) also uses Sherlock Holmes stories (18^th century, N = 12) and a news corpus also containing comments (2006-ongoing, N = 100,000). Silge and Robinson (2020) apply LDA topic modeling on news stories by the Associated Press (1992, N = 2,246) as well as books by Dickens, Wells, Verne and Austen (18^th century, N = 4). Roberts et al. (2019) use blogposts (2008, N = 13,248) for structural topic modeling. Watanabe and Müller (2019) apply LDA topic modeling on newspaper articles from The Guardian (2016, N = 6,000). Van Atteveldt and Welbers (2019, 2020) use State of the Union speeches (1981-2017, N = 10 and 1789-2017, N = 58) for their analyses. Lastly, Wiedemann and Niekler (2017) use the same data containing State of the Union speeches (1790-2017, N = 223).

Table 1. Measurement of “Topics” using automated content analysis.

Author(s)	Sample	Procedure	Formal validity check with manual coding as benchmark*	Code
Puschmann (2019a)	(a) Newspaper articles (b) Newspaper articles	Supervised machine learning	Reported	http://inhaltsanalyse-mit-r.de/maschinelles_lernen.html
Puschmann (2019b)	(a) Sherlock Holmes stories (b) Newspaper articles (c) United Nations General Debate Transcripts	LDA topic modeling; structural topic modeling	Not reported	http://inhaltsanalyse-mit-r.de/themenmodelle.html
Silge (2018a) & Silge (2018b)	(a) Sherlock Holmes stories (b) News stories and comments	t Structural topic modeling	Not reported	https://juliasilge.com/blog/sherlock-holmes-stm/ & https://juliasilge.com/blog/evaluating-stm/
Silge & Robinson (2020)	(a) News articles (b) Books	LDA topic modeling	Not reported	https://www.tidytextmining.com/topicmodeling.html
Roberts et al. (2019)	Blogposts	Structural topic modeling	Not reported	https://www.jstatsoft.org/article/view/v091i02
Watanabe & Müller (2019)	Newspaper articles	LDA topic modeling	Not reported	https://tutorials.quanteda.io/machine-learning/topicmodel/
van Atteveldt & Welbers (2019)	State of the Union speeches	Structural topic modeling	Not reported	https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/r_text_stm.md
van Atteveldt & Welbers (2020)	State of the Union speeches	LDA topic modeling	Not reported	https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/r_text_lda.md
Wiedemann & Niekler (2017)	State of the Union speeches	LDA topic modeling	Not reported	https://tm4ss.github.io/docs/Tutorial_6_Topic_Models.html
Wiedemann & Niekler (2017)	State of the Union speeches	Supervised machine learning	Reported	https://tm4ss.github.io/docs/Tutorial_7_Klassifikation.html

References

Hase, V., Engelke, K., Kieslich, K. (2020). The things we fear. Combining automated and manual content analysis to uncover themes, topics and threats in fear-related news. Journalism Studies, 21(10), 1384-1402.

Puschmann, C. (2019). Automatisierte Inhaltsanalyse mit R. Retrieved from http://inhaltsanalyse-mit-r.de/index.html

Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010). How to analyze political attention with minimal assumptions and costs. American Journal of Political Science, 54(1), 209–228.

Roberts, M. E., Stewart, B. M., & Tingley, D. (2019). stm: An R Package for Structural Topic Model. Journal of Statistical Software, 91(2), 1–40.

Silge, J. (2018a). The game is afoot! Topic modeling of Sherlock Holmes stories. Retrieved from https://juliasilge.com/blog/sherlock-holmes-stm/

Silge, J. (2018b). Training, evaluating, and interpreting topic models. Retrieved from https://juliasilge.com/blog/evaluating-stm/

Silge, J., & Robinson, D. (2020). Text Mining with R. A tidy approach. Retrieved from https://www.tidytextmining.com/

van Atteveldt, W., & Welbers, K. (2019). Structural Topic Modeling. Retrieved from https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/r_text_stm.md

van Atteveldt, W., & Welbers, K. (2020). Fitting LDA models in R. Retrieved from https://github.com/ccs-amsterdam/r-course-material/blob/master/tutorials/r_text_lda.md

Watanabe, K., & Müller, S. (2019). Quanteda tutorials. Retrieved from https://tutorials.quanteda.io/

Political issues (Self-Presentation of Political Actors)

Sina Blassnig — Fri, 26 Mar 2021 00:00:00 +0100

Political issues, in general, focus on the content of political actors’ communication and most often describe either the main issue or several issues that are in the focus of a political actor’s statement or any other relevant text (e.g., press release, news article, tweet, etc.). The basic premise of analyzing political issues in the self-presentation of political actors is that one major goal of political actors’ communication is to place specific issues on the political agenda (Strömbäck & Esser, 2017). Political issues are most often coded based on a list of pre-defined issues that refer to different policies and sometimes also to polity or politics. The scope and detail of the individual issues depend on the purpose and the focus of the analysis.

Field of application/theoretical foundation:

Apart from being a common descriptive and control variable, the coding of issues in political actors’ communication can serve as the basis for more complex variables or concepts such as agenda building or issue ownership.

Agenda building, at large, refers to the process of how media content is shaped by societal forces (Lang & Lang, 1981). With regard to analyses of politicians’ self-presentation, most work focuses on the processes of communication by which political actors aim to obtain media coverage for their issues (Norris et al., 1999; Seethaler & Melischek, 2019). Analyses on agenda building usually compare issue agendas between at least two different forms of communication, e.g., between channels where political actors have high control (such as press releases, party manifestos, social media messages) and journalistic outlets where political actors have less control (e.g., Harder et al., 2017; Kiousis et al., 2006; Seethaler & Melischek, 2019).

Content analyses on agenda building usually start by, first, identifying relevant issue fields and categories (inductively or deductively). Second, the dominant political issues in political actors’ communication and/or other forms of communication (e.g., news articles) are coded according to predefined lists. Third, the occurrence of specific issues or issue agendas are compared between the different forms of communication, often over time (see, e.g., Seethaler & Melischek, 2019).

Issue ownership, in broad terms, means that some parties are considered by the public in general as being more adept to deal with, or more attentive to, certain issues (Lachat, 2014; Petrocik, 1996; Walgrave et al., 2015). Traditionally, issue ownership has been analyzed from a demand-side perspective, based on surveys, as the connection between issues and parties in voters’ minds. Definitions of issue ownership usually comprise at least two dimensions: competence issue ownership (parties’ perceived capacity to competently handle or solve a certain issue) and associative issue ownership (the spontaneous link between some parties and some issues) (Walgrave et al., 2015). Content analyses build on these definitions to investigate to what extent political actors focus on issues that they (respectively their parties) own and what factors may explain the (non-)reliance on owned issues (e.g., Dalmus et al., 2017; Peeters et al., 2019). Other content analyses use issue ownership as an independent variable, for example, to explain user reactions to parties’ social media messages (e.g., Staender et al., 2019).

Content analyses on issue ownership usually start by, first, identifying relevant issue fields and categories (inductively or deductively). Second, the dominant political issues in political actors’ communication are coded according to predefined lists. Third, political actors are assigned issue ownership for specific issues based on theoretical considerations, existing literature, and/or survey data. Fourth, an index for owned issues is calculated at the statement or text level based on the coded issues and the predefined ownership for specific issues.

References/combination with other methods of data collection:

Political issues can be analyzed using both manual and automated content analysis (e.g. topic modeling or dictionary approach). Analyses use both inductive or deductive approaches and/or a combination of both to identify issue categories and extend or amend previous lists of political issues.

Example studies:

Dalmus et al. (2019), Peeters et al. (2019); Seethaler & Melischek (2019)

Table 1: Summary of a selection of studies on agenda building and/or issue ownership

Author(s)

Sample

Unit of Analysis

Values

Reliability

Seethaler & Melischek (2019)

Content type: parties’ news releases and tweets, media reports

Country: Austria

Political actors: all parliamentary parties (ÖVP, SPÖ, FPÖ, Grüne, NEOS, Liste Pilz)

Outlets: all party news releases, parties’ and top candidates’ twitter accounts, five legacy media outlets

Sampling period: 6 weeks before the national election day in 2017 (4 September 2017–14 October 2017)

Sample size: 1,009 news releases, 9,088 tweets, 2,422 news stories

Unit of analysis: individual news releases, tweets, and news stories

Level of analysis: issue agendas

Dominant issue: 13 issue areas based on the Comparative Agendas Project: civil rights, government operations, law and crime, international affairs and defence, European integration, macroeconomics, domestic commerce, transportation and technology, environment and agriculture, education, labour, social welfare and housing, health

Cohen’s Kappa between .91 and .95

Harder, Sevenans, & Van Aelst (2017)

Content type: newspaper, television, radio, news website, and Twitter items featuring a political topic, a domestic political actor, or an election-specific term

Country: Belgium

(Political) actors: tweets by 678 professional journalists, 44 accounts affiliated with legacy media organizations, 467 politicians, 19 civil society organizations, 109 “influentials”

Outlets: 5 print newspapers, 3 news websites, 2 daily television newscasts, 6 daily radio newscasts, current affairs tv programs, and election-specific tv shows

Sampling period: Belgian 2014 election campaign (1 May to 24 May 2014)

Sample size: n = 9,935

Unit of analysis: news items and tweets

Level of analysis: news items (n = 5,260) / news stories (n = 414)

Issues (up to three issues per item): list of 28 broad issues based on the Comparative Agendas Project

Categorization of news stories: inductive coding of individual time- and place-specific events based on news items from traditional news outlets. Non-news items and tweets were then assigned to the already-identified news stories

Krippendorff’s alpha = .70

Krippendorff’s alpha = .76 (for assigning news story to tweet)

Dalmus, Hänggli, Bernhard (2019)

Content type: party manifestos, party press releases, and newspaper coverage

Countries: CH, DE, FR, UK

Political actors: parties

Outlets: 1 quality newspaper and 1 tabloid per country, all party press releases and manifestos

Sampling period: election campaigns between 2010 and 2013 (8 weeks prior to the respective election days)

Sample size: 4,191

Unit of analysis: Actor statements on issues concerning national politics and containing either an explicitly mentioned position or interpretation/ elaboration on the issue

Level of analysis: text level

Main issue: Economy, Welfare, Budget, Freedom and Rights, Europe/ Globalization, Education, Immigration, Army, Security, Ecology, Institutional Reforms, Infrastructure, Elections and Events (each of these top-issue categories is made up of several more detailed sub-issues leading to a total of 127 issue options)

Issue emphasis: percentage of statements devoted to a certain issue

Issue ownership: issue fully belongs to one party (1), issue belongs to center-left / center-right parties (0.5), issue is unowned (0) (based on Seeberg, 2016; Tresch et al., 2017, for more details see the paper)

Cohen’s Kappa ?.3 for sub-issues; Cohen’s Kappa ?.5 for top-issues

Peeters, Van Aelst, & Praet (2019)

Content type: politicians’ tweets, online media coverage, and parliamentary documents

Country: Belgium (Flemish part)

Political actors: 144 MPs from the 6 parties represented in the Flemish and federal parliament

Outlets: 13 Flemish news outlets

Sampling period: 1 January to 1 September, 2018

Sample size: n = 51,691 tweets, n = 8,857 articles, n = 12,638 parliamentary documents

Unit of analysis: text level

Level of analysis: issue agendas

Index for issue concentration: Herfindahl index (to assess how diverse/ concentrated the individual issue agendas are across platforms)

Issues: automated coding of 20 issue topics using the Dutch dictionary based on the Comparative Agendas Project

Issue ownership: operationalization based on survey data; relative party ownership scores for each politician were assigned based on the percentage of respondents that linked a certain party with the topic

(A manual check on 200 randomly selected documents shows that a little over 70% of the automated non-codings were in fact non-classifiable documents. For the other 30%, the dictionary was not able to properly classify the documents.)

References

Dalmus, C., Hänggli, R., & Bernhard, L. (2017). The charm of salient issues? Parties’ strategic behavior in press releases. In P. van Aelst & S. Walgrave (Eds.), How Political Actors Use the Media: A Functional Analysis of the Media’s Role in Politics (pp. 187–205). Springer International Publishing. https://doi.org/10.1007/978-3-319-60249-3_10

Harder, R. A., Sevenans, J., & van Aelst, P. (2017). Intermedia Agenda Setting in the Social Media Age: How Traditional Players Dominate the News Agenda in Election Times. The International Journal of Press/Politics, 22(3), 275–293. https://doi.org/10.1177/1940161217704969

Kiousis, S., Mitrook, M., Wu, X., & Seltzer, T. (2006). First- and Second-Level Agenda-Building and Agenda-Setting Effects: Exploring the Linkages Among Candidate News Releases, Media Coverage, and Public Opinion During the 2002 Florida Gubernatorial Election. Journal of Public Relations Research, 18(3), 265–285. https://doi.org/10.1207/s1532754xjprr1803_4

Lachat, R. (2014). Issue ownership and the vote: the effects of associative and competence ownership on issue voting. Swiss Political Science Review, 20(4), 727–740. https://doi.org/10.1111/spsr.12121

Lang, G.E., & Lang, K. (1981). Watergate: An exploration of the agenda-building process. In: Wilhoit, G.C., & De Bock, H. (Eds.). Mass Communication Review Yearbook. SAGE, pp. 447–468.

Norris, P., Curtice, J., Sanders, D., et al. (1999). On Message: Communicating the Campaign. SAGE.

Peeters, J., van Aelst, P., & Praet, S. (2019). Party ownership or individual specialization? A comparison of politicians’ individual issue attention across three different agendas. Party Politics, 55(4), 135406881988163. https://doi.org/10.1177/1354068819881639

Petrocik, J.R. (1996). Issue ownership in presidential elections, with a 1980 case study. American Journal of Political Science, 40(3), 825–850.

Seeberg, H. B. (2017). How stable is political parties’ issue ownership? A cross-time, cross-national analysis. Political Studies, 65(2), 475–492. https://doi.org/10.1177/0032321716650224

Seethaler, J., & Melischek, G. (2019). Twitter as a tool for agenda building in election campaigns? The case of Austria. Journalism, 20(8), 1087–1107. https://doi.org/10.1177/1464884919845460

Staender, A., Ernst, N., & Steppat, D. (2019). Was steigert die Facebook-Resonanz? Eine Analyse der Likes, Shares und Comments im Schweizer Wahlkampf 2015. SCM Studies in Communication and Media, 8(2), 236–271. https://doi.org/10.5771/2192-4007-2019-2-236

Strömbäck, J., & Esser, F. (2017). Political Public Relations and Mediatization: The Strategies of News Management. In P. van Aelst & S. Walgrave (Eds.), How Political Actors Use the Media: A Functional Analysis of the Media’s Role in Politics (pp. 63–83). Springer International Publishing. https://doi.org/10.1007/978-3-319-60249-3_4

Tresch, A., Lefevere, J., Walgrave, S. (2018). How parties’ issue emphasis strategies vary across communication channels: The 2009 regional election campaign in Belgium. Acta Politica, 53(1), 25–47. https://doi.org/10.1057/s41269-016-0036-7

Walgrave, S., Tresch, A., & Lefevere, J. (2015). The Conceptualisation and Measurement of Issue Ownership. West European Politics, 38(4), 778–796. https://doi.org/10.1080/01402382.2015.1039381

Populist communication: content and style elements (Self-Presentation of Political Actors)

Sina Blassnig — Fri, 26 Mar 2021 00:00:00 +0100

Populist communication, in this entry, refers to the occurrence of a) specific messages that are seen as the expression of populist ideology and b) characteristic style elements that are often associated with these messages expressing populist ideology in political actors’ (or other actors such as journalists’ or citizens’) communication (Ernst et al., 2019; De Vreese et al., 2018).

Field of application/theoretical foundation:

Populism has been defined in various terms; e.g., as Ideology (Canovan, 1999; Mudde, 2004), set of ideas (Hawkins et al., 2018, Taggart, 2000), discourse (Laclau, 2005; Mouffe, 2018), political style (Moffit, 2016), communication style (Jagers & Walgrave, 2007), or political strategy (Weyland, 2017). Thus, there have been numerous operationalizations of populism or populist communication in content analyses that cannot all be accounted for here. This entry specifically follows a communication-centered perspective (Stanyer et al., 2016; De Vreese et al., 2018). Jagers & Walgrave (2007), in a pioneer study on populist communication, define populism as a political communication style “essentially displaying proximity of the people, while at the same time taking an anti-establishment stance and stressing the (ideal) homogeneity of the people by excluding specific population segments.” In a more recent study, Ernst et al. (2019) differentiate between populist communication content and populist communication style. Populist communication content refers to the communicative representation of the populist ideology (what is being said) that can be expressed in the form of populist key messages. Depending on the parsimony of the definition, populist ideology comprises three or four dimensions: people-centrism, anti-elitism, restoring sovereignty, and exclusion (e.g., De Vreese et al., 2018; Mudde, 2004; Jagers & Walgrave, 2007; Wirth et al., 2016). In distinction to the content, Ernst et al. (2019) define populist communication style as the use of populism-related style elements (how something is said) (see also De Vreese et al., 2018; Bracciale & Martella, 2018).

Communication-centered content analyses of populist communication are often carried out in three steps. First, specific characteristics of populist communication (e.g., populist key messages or stylistic elements) are identified. Second, the occurrence of these individual elements is then coded either on the statement level (e.g. Ernst et al., 2019; Wirth et al., 2016), excerpts level (Jagers & Walgrave, 2007), or on the text/article level (e.g. Blassnig et al, 2019). Third, the level of populism is determined using different indices for populist communication as a whole (e.g. maximum indices; Blassnig et al., 2019; Ernst et al., 2019) or for the individual dimensions separately (e.g., Jagers & Walgrave, 2007). Populism indices can be calculated at the statement level, text level, or actor level.

References/combination with other methods of data collection:

Whereas this entry focuses on quantitative and deductive approaches, populist communication has also been investigated using qualitative or inductive approaches (e.g., Wodak, 2015), especially in studies following a more actor-centered approach (Stanyer et al., 2016). Most studies on populist communication have used manual content analysis. Yet, some analyses have also applied automated approaches to investigate the occurrence of populist communication in texts (e.g., Hawkins & Castanho Silva, 2018).

Example studies:

Blassnig et al., (2019); Bracialle & Martella (2017); Ernst et al., (2019); Jagers & Walgrave, (2007)

Table 2: Summary of a selection of studies on populist communication

Author(s)

Sample

Unit of Analysis

Values

Reliability

Jagers & Walgrave, 2007

Content type: political party broadcasts (PPB)

Country: Belgium (Flemish part)

Political actors: six Belgian-Flemish parties

Outlets: 20 PPBs per party

Sampling period: 1999 - 2001

Sample size: 1,200 PPB excerpts

Unit of analysis: excerpts including ‘thin’ populism (references to the people)

Level of analysis: excerpt level and actor level

People-index: multiplication of the proportion and intensity of references to the people for each party

Anti-state-index: number of anti-state excerpts * average intension anti-state excerpts (1-5) per party

Anti-politics-index: number of anti-politics excerpts * average intension anti-politics excerpts (1-5) per party

Anti-media-index: number of anti-media excerpts * average intension anti-media excerpts (1-5) per party

Anti-establishment-index: anti-state + anti-politics + anti-media per party

Exclusivity-index: J-scores; (positive – negative evaluations) / (positive + neutral + negative evaluations of specific population categories)

References to the people: terms referring to the population (as a whole or population categories), that cover the people “in political terms”, meaning the “political entity”

Anti-state: failure of the state with regard to (1) single failure, (2) systematic failure, (3) public service should be abolished, (4) all public services are criticized at once, (5) the system

Anti-politics: criticism directed towards (1) policy measure or present situation, (2) policy, (3) politician, (4) party, (5) group of parties, (6) all parties. (7) the system

Anti-media: media targets of criticism; (1) newspaper/ magazine/ tv channel, (2) group of media, (3) all (the) media

Evaluation of specific population categories: positive, neutral, negative

(for further restrictions for the individual variables and more detailed instructions see the methodological appendix by Jagers & Walgrave, 2007)

Reliability is not reported

Ernst, Blassnig, Engesser, Büchel, & Esser (2019)

(See also Ernst et al., 2018; Ernst, Esser et al., 2019; Wirth et al., 2016)

Content type: statements by politicians expressing either a political position, an elaboration on a political issue, or an evaluation/ attribution of a target actor

Countries: CH, DE, IT, FR, UK, US

Political actors: 98 politicians from 31 parties

Outlets: political talk shows (2 per country), politicians’ Facebook and Twitter accounts

Sampling period: April through May 2015

Sample size: n = 2’067 (n = 969 talk show statements, n = 734 Facebook posts, and n = 364 Tweets

Unit of analysis: a single statement by a politician on a target actor or an issue

Level of analysis: statement level and actor level

Populism index: Maximum index based on the nine populist key messages and seven stylistic elements (0/1)

Populist key messages:

Anti-elitism: discrediting the elite, blaming the elite, detaching the elite from the people

People-centrism: stressing the people’s virtues, praising the people’s achievements, stating a monolithic people, demonstrating closeness to the people

Restoring sovereignty: demanding popular sovereignty, denying elite sovereignty

Populist style elements:

Negativity: negativism, crisis rhetoric

Emotionality: emotional tone, absolutism, patriotism)

Sociability: colloquialism, intimization

(all items were coded as dummy variables based on more detailed sub-categories)

Brennan & Prediger’s kappa average = 0.91 (³0.65)

Blassnig, Ernst, Büchel, Engesser, & Esser (2019)

Content type: election news coverage about immigration and adjacent reader comments

Countries: CH, FR, UK

Actors/Speakers: politicians, journalists, and citizens

Outlets: 6 online news outlets per country

Sampling period: six weeks before the respective election days. CH: September to October 2015; FR: April to May 2017; UK: April to May 2015

Sample size: n = 493 news articles and n = 2904 reader comments

Unit of analysis: news article / reader comment

Level of analysis: article level

Populism index: Maximum index based on the twelve populist key messages (0/1)

Populist key messages:

Anti-elitism: discrediting the elite, blaming the elite, detaching the elite from the people

People-centrism: praising the people’s virtues, praising the people’s achievements, describing the people as homogenous, demonstrating closeness to the people

Restoring sovereignty: demanding popular sovereignty, denying elite sovereignty

Exclusion: discrediting specific social groups, blaming specific social groups, excluding specific social groups from the people

(all items were coded as dummy variables)

Brennan & Prediger’s kappa average = 0.75

Bracciale & Martella (2017)

Content type: politicians’ tweets

Country: Italy

Political actors: 5 party leaders

Outlets: leaders’ Twiter timelines

Sampling period: 1 January 2015 to 1 July 2016

Sample size: n = 7,772

Unit of analysis: tweets

Level of analysis: tweets, actors

Indices:

Populist ideology: three additive synthetic dichotomous indices adding together the indicators for each of the three dimensions of populism (sovereignty of the people, attacking the elite, ostracizing others)

The variables for political communication style were summarized using multiple correspondence analysis (MCA) into two dimensions: communicative mode (positive vs. negative) and communicative focus (personalization vs. political/ campaign)

Political communication style:

Stagecraft: emotionalisation; informality, instrumental actualization, intimisation, negative affect, simplification, storytelling, taboo breaker, vulgarism

Register (communicative tone): referential/ neutral, aggressive/ provocative, humorous/ ironic, conversational/ participatory

Topic: political issues, policy issues, campaign issues, personal issues, current affairs

Function: campaign updating, self-promotion, setting the agenda, position-taking, call to action, opposition/ violence, endorsement, irony, request for interaction, pointless babble

Populist ideology:

Emphasizing sovereignty of the people: refers to the people, refers to ‘ad hoc’ people, direct representation

Attacking the elite: generic anti-establishment, political anti-establishment, economic anti-establishment, EU anti-establishment, institutional anti-establishment, anti-elitism media, anti-elitism intellectuals

Ostracizing others: dangerous others, authoritarianism

(all individual indicators were coded as dummy variables)

Krippendorff's Alpha > .83

References

Blassnig, S., Ernst, N., Büchel, F., Engesser, S., & Esser, F. (2019). Populism in online election coverage. Journalism Studies, 20(8), 1110–1129. https://doi.org/10.1080/1461670X.2018.1487802

Bracciale, R., & Martella, A. (2017). Define the populist political communication style: the case of Italian political leaders on Twitter. Information, Communication & Society, 20(9), 1310–1329. https://doi.org/10.1080/1369118X.2017.1328522

Canovan, M. (1999). Trust the people! Populism and the two faces of democracy. Political Studies, 47(1), 2–16. https://doi.org/10.1111/1467-9248.00184

Cranmer, M. (2011). Populist communication and publicity: An empirical study of contextual differences in Switzerland. Swiss Political Science Review, 17(3), 286–307. https://doi.org/10.1111/j.1662-6370.2011.02019.x

De Vreese, C. H., Esser, F., Aalberg, T., Reinemann, C., & Stanyer, J. (2018). Populism as an expression of political communication content and style: A new perspective. The International Journal of Press/Politics, 23(4), 423-438. https://doi.org/10.1177/1940161218790035

Engesser, S., Fawzi, N., & Larsson, A. O. (2017). Populist online communication: Introduction to the special issue. Information, Communication & Society, 20(9), 1279–1292. https://doi.org/10.1080/1369118X.2017.1328525

Ernst, N., Blassnig, S., Engesser, S., Büchel, F., & Esser, F. (2019). Populists prefer social media over talk shows: An analysis of populist messages and stylistic elements across six countries. Social Media + Society, 5(1), 1-14. https://doi.org/10.1177/2056305118823358

Hawkins, K. A., Carlin, R. E., Littvay, L., & Rovira Kaltwasser, C. (Eds.). (2018). Extremism and democracy. The ideational approach to populism: Concept, theory, and analysis. Routledge.

Haswkins, K. A., & Castanho Silva, B. (2018). Textual analysis: big data approaches. In K. A. Hawkins, R. E. Carlin, L. Littvay, & C. Rovira Kaltwasser (Eds.). Extremism and democracy. The ideational approach to populism: Concept, theory, and analysis (pp. 27-48). Routledge.

Jagers, J., & Walgrave, S. (2007). Populism as political communication style: An empirical study of political parties' discourse in Belgium. European Journal of Political Research, 46(3), 319–345. https://doi.org/10.1111/j.1475-6765.2006.00690.x

Laclau, E. (2005). On populist reason. London: Verso.

Moffitt, B. (2016). The global rise of populism: Performance, political style, and representation. Stanford University Press.

Mudde, C. (2004). The populist Zeitgeist. Government and Opposition, 39(4), 542–563. https://doi.org/10.1111/j.1477-7053.2004.00135.x

Stanyer, J., Salgado, S., & Strömbäck, J. (2017). Populist actors as communicators or political actors as populist communicators: Cross-national findings and perspectives. In T. Aalberg, F. Esser, C. Reinemann, J. Strömbäck, & C. H. de Vreese (Eds.), Populist political communication in Europe (pp. 353–364). Routledge.

Taggart, P. (2000). Populism. Concepts in the social sciences. Open University Press.

Wirth, W., Esser, F., Engesser, S., Wirz, D. S., Schulz, A., Ernst, N., . . . Schemer, C. (2016). The appeal of populist ideas, strategies and styles: A theoretical model and research design for analyzing populist political communication. Zurich: NCCR Democracy, Working Paper No. 88, pp. 1–60. https://doi.org/10.5167/uzh-127461

Wodak, R. (2015). The Politics of Fear: What Right-Wing Populist Discourses Mean. SAGE Publications.

Negative campaigning (Election Campaigning Communication)

Desiree Steppat, Laia Castro Herrero — Sun, 18 Apr 2021 00:00:00 +0200

One of the most crucial decisions political candidates make ahead of an election is whether they want to focus on their image or that of their their political opponents in their advertisement (Lau and Rovner , 2009). During electoral campaigns, candidates need to decide whether they use political advertisement to display a positive image of themselves or whether they try to make the opponent look bad. The first strategy is referred to as Acclaim or Positive Ads. The second approach, according to Surlin and Gordon is called Negative Campaigning and is applied by a political candidate when (s)he “attacks the other candidate personally, the issues for which the other candidate stands, or the party of the other candidate” (1977, p. 93). However, measuring negative campaigning poses a challenge to academic research since content analyses often fail to address the grey areas of this concept. To begin with, many political ads compare positive characteristics of a candidate against opponents’ more negative ones. (Lau & Rovner, 2009). Ads that contain both strategies, shedding positive light on the candidate while also highlighting negative aspects about the opponent’s character or policies are called Comparison or Comparative Ads. These comparisons are difficult to code with straightforward approaches. For example, analyzing campaigns along a positive/negative dichotomy by discounting attacks to the opponent from positive self-presentations may equate strongly positively and negatively charged political advertising to neutral campaigns. Also, negativity in political campaigning is studied in different contexts and has been extended as a number of studies on negative campaigning look in particular at Attacks and Rebuttals/Defense from opponents after an attack (Benoit, 2000; Benoit & Airne, 2009; Erigha & Charles, 2012; Lee & Benoit, 2004; Torres, Hyman, & Hamilton, 2012). This distinction raises other important methodological and theoretical implications. Sweeping measures of negativity based on common scholarly definitions do not consider voters’ tolerance towards the use of certain forms of negativity by candidates (for example, rebutting an attack from an opponent) that may be perceived as legitimate. Not accounting for such nuances is what makes many negativity measures unable to accurately gauge the effects of negative campaigning among the electorate (Sigelman & Kugler, 2003).

Field of application/theoretical foundation:

Negative campaigning and its related constructs (such as attacks or rebuttals) have been often associated with current trends in political communication of modernization and professionalization of election campaigns (Voltmer, 2004). Negative campaigning is indeed a development that can be observed across many different political contexts (Kaid & Holtz-Bacha, 2006). Campaign strategies using negative messages about a political opponent have been studied relying on theories from social and cognitive psychology (Kahn & Kenney, 1999; Lau, 1985) and mostly in regard to their potential consequences for a healthy democracy (Lau & Rovner, 2009). Their operationalization follows a simple schema by coding whether a certain construct is present in a given advertising piece or not. Alternatively, it is coded which kind of category best reflects on the content of a given political advertisement.

References/combination with other methods of data collection:

Negative campaigning and related constructs have been studied through content analysis both of paid advertisement (Benoit, 2000) and news coverage by the mass media (Lau & Pomper, 2004); The features and effects of negative campaigning have also been analyzed through voter surveys (Brader, 2005, 2006) and interviews with campaign managers (Kahn & Kenney, 1999). Its effects were furthermore more precisely measured through numerous experimental studies (Ansolabehere, Iyengar, Simon, & Valentino, 1994; overview see: Lau et al., 2007).

Example studies:

Table 1: Overview exemplary studies measuring of negative campaigning and related constructs

Authors	Sample	Unit of analysis	Constructs	Values	Reliability
Benoit (2000), Benoit & Airne (2009), Lee & Benoit (2004)		Television ads, direct mail, newspaper ads, and candidate web pages	Acclaim	Acclaims portray the sponsored candidate in a favorable light, both his/her character and/or policy (Benoit, 2000, 281, 295) 0 = not present 1 = present	Cohen’s kappa average = .96
Erigha & Charles (2012)		Television and web advertisements	Non-negative/ advocacy	A non-negative/advocacy ad favors a party’s candidate, focusing solely on that individual. 1 = non-negative / advocacy 2 = comparison 3= attack ads (exclusive options)	Cohen’s kappa average = .96
Torres et al. (2012)		Presidential candidate–sponsored TV ads	Non-comparative ad	If the ad simply mentions positive attributes of a particular candidate without mentioning an opponent, the ad is coded as a non-comparison (positive) ad (p. 196) 1 = comparative ad 2 = negative ad 3= non-comparative ad (exclusive options)	Cohen’s kappa average = .98
Steffan & Venema (2019)		Campaign posters	Textual negative campaigning Visual negative campaigning	Based on Lau and Pomper’s (2002), textual/visual negative campaiging indicates whether the image / text on the campaign posters referred to other political parties or candidates. (p. 273) 0 = not present 1 = present	Visual negative campaigning: Krippendorff’s α = .82 Textual negative campaigning: Krippendorff’s α = .84
Torres et al. (2012)		Presidential candidate–sponsored TV ads	Negative ad	If the ad criticizes the opposing party and/or candidate but offers no alternative (in essence, the ad presents negative information about an opponent but no information about the candidate on whose behalf it is run), then the ad is coded as a negative ad. 1 = comparative ad 2 = negative ad 3= non-comparative ad (exclusive options)	Cohen’s kappa average = .98
Ceccobelli (2018)		Facebook posts	Negative rhetorical strategy	The posts taken into consideration are those in which leaders employ a purely negative campaigning strategy. Cases in which a hypothetic leader A attacks one or more political opponents by comparing his/her own figure or policy proposal with the one(s) of her/his competitor(s) are not coded, since they denote a comparative rhetorical strategy (p. 129) 0 = not present 1 = present	Krippendorff’s α average = .85
Benoit (2000), Benoit & Airne (2009), Lee & Benoit (2004)		Television spots, direct mail pieces, newspaper ads, and candidate web pages	Attack	Portrays the opposing candidate in an unfavorable light, both his/her character and/or policy (Benoit, 2000, 281, 295) 0 = not present 1 = present	Cohen’s kappa average = .96
Erigha & Charles (2012)		Television and web advertisements	Attack ads	Attack ads criticize the opposing candidate without referencing the sponsoring party’s candidate (p. 443) 1 = non-negative / advocacy 2 = comparison 3= attack ads (exclusive options)	Cohen's kappa average = .96
Benoit (2000), Benoit & Airne (2009), Lee & Benoit (2004)		Television spots, direct mail pieces, newspaper ads, and candidate web pages	Defense	Defense responds to (refutes) an attack on the candidate, both on his/her character and/or policy (Benoit, 2000, 281, 295) 0 = not present 1 = present	Cohen’s kappa average = .96
Erigha & Charles (2012)		Television and web advertisements	Comparison	A comparison ad weighs two credentials, characteristics, or policystances (p. 443) 1 = non-negative / advocacy 2 = comparison 3= attack ads (exclusive options)	Cohen's kappa average = .956
Torres et al. (2012)		Presidential candidate–sponsored TV ads	Comparative ad	If the ad criticizes the opposing party and/or candidate and recommends alternative courses of action by comparing two candidates on specific points so as to present one in a more positive and the other in a more negative light, then the ad is coded as a comparative ad (p. 195) 1 = comparative ad 2 = negative ad 3= non-comparative ad (exclusive options)	Cohen’s kappa average = .98

References

Ansolabehere, S., Iyengar, S., Simon, A., & Valentino, N. (1994). Does Attack Advertising Demobilize the Electorate? American Political Science Review, 88(4), 829–838. https://doi.org/10.2307/2082710

Benoit, W. L. (2000). A Functional Analysis of Political Advertising across Media, 1998. Communication Studies, 51(3), 274–295. https://doi.org/10.1080/10510970009388524

Benoit, W. L., & Airne, D. (2009). Non-Presidential Political Advertising in Campaign 2004. Human Communication, 12(1), 91–117.

Brader, T. (2005). Striking a Responsive Chord: How Political Ads Motivate and Persuade Voters by Appealing to Emotions. American Journal of Political Science, 49(2), 388. https://doi.org/10.2307/3647684

Brader, T. (2006). Campaigning for hearts and minds: How emotional appeals in political ads work. Studies in communication, media, and public opinion. Chicago, Ill.: Univ. of Chicago Press. Retrieved from http://www.loc.gov/catdir/enhancements/fy0622/2005009159-b.html

Buell, E. H., & Sigelman, L. (2008). Attack politics: Negativity in presidential campaigns since 1960. Studies in government and public policy. Lawrence, Kan.: Univ. Press of Kansas.

Ceccobelli, D. (2018). Not Every Day is Election Day: a Comparative Analysis of Eighteen Election Campaigns on Facebook. Journal of Information Technology & Politics, 15(2), 122–141. https://doi.org/10.1080/19331681.2018.1449701

Erigha, M., & Charles, C. Z. (2012). Other, Uppity Obama: A Content Analysis of Race Appeals in the 2008 U.S. Presidential Election. Du Bois Review: Social Science Research on Race, 9(2), 439–456. https://doi.org/10.1017/S1742058X12000264

Geer, J. G. (2010). In defense of negativity: Attack ads in presidential campaigns. Studies in communication, media, and public opinion. Chicago: University of Chicago Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=319130

Kahn, K. F., & Kenney, P. J. (1999). Do Negative Campaigns Mobilize or Suppress Turnout? Clarifying the Relationship between Negativity and Participation. American Political Science Review, 93(4), 877–889. https://doi.org/10.2307/2586118

Kaid, L. L., & Holtz-Bacha, C. (Eds.) (2006). The SAGE handbook of political advertising. Thousand Oaks, Calif.: SAGE Publications.

Kanouse, D. E., & Hansen, L. R. (1987). Negativity in evaluations. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior. Hillsdale, N.J.: Erlbaum.

Lau, R. R. (1985). Two explanations for negativity effects in political behavior. American Journal of Political Science. (29), 119–138.

Lau, R. R., & Pomper, G. M. (2004). Negative campaigning: An analysis of U.S. Senate elections. Campaigning American style. Lanham, Md.: Rowman & Littlefield.

Lau, R. R., & Rovner, I. B. (2009). Negative Campaigning. Annual Review of Political Science, 12(1), 285–306. https://doi.org/10.1146/annurev.polisci.10.071905.101448

Lau, R. R., Sigelman, L., & Rovner, I. B. (2007). The Effects of Negative Political Campaigns: A Meta-Analytic Reassessment. The Journal of Politics, 69(4), 1176–1209. https://doi.org/10.1111/j.1468-2508.2007.00618.x

Lee, C., & Benoit, W. L. (2004). A Functional Analysis of Presidential Television Spots: A Comparison of Korean and American Ads. Communication Quarterly, 52(1), 68–79. https://doi.org/10.1080/01463370409370179

Sigelman, L., & Kugler, M. (2003). Why Is Research on the Effects of Negative Campaigning So Inconclusive? Understanding Citizens’ Perceptions of Negativity. The Journal of Politics, 65(1), 142–160. https://doi.org/10.1111/1468-2508.t01-1-00007

Steffan, D., & Venema, N. (2019). Personalised, De-Ideologised and Negative? A Longitudinal Analysis of Campaign Posters for German Bundestag Elections, 1949–2017. European Journal of Communication, 34(3), 267–285. https://doi.org/10.1177/0267323119830052

Surlin, S. H., & Gordon, T. F. (1977). How Values Affect Attitudes Toward Direct Reference Political Advertising. Journalism Quarterly, 54(1), 89–98. https://doi.org/10.1177/107769907705400113

Torres, I. M., Hyman, M. R., & Hamilton, J. (2012). Candidate-Sponsored TV Ads for the 2004 U.S. Presidential Election: A Content Analysis. Journal of Political Marketing, 11(3), 189–207. https://doi.org/10.1080/15377857.2012.703907

Voltmer, K. (2004). Mass media and political communication in new democracies: Routledge.

Interaction (Election Campaigning Communication)

Desiree Steppat, Laia Castro Herrero — Sun, 18 Apr 2021 00:00:00 +0200

Interaction is described as a way to persuade citizens through direct contact allowing for a dialogical encounter between political actors and citizens (Magin, Podschuweit, Haßler, & Russmann, 2017). Although the new online environment can facilitate direct communication between politicians and citizens, empirical findings indicate that, to date, a unidirectional communication style between voters and politicians predominates (Jackson & Lilleker, 2010; Lilleker & Koc-Michalska, 2013; Stromer-Galley, 2000). To a large extent, politicians still employ the broadcasting style for campaign communication (Graham, Broersma, Hazelhoff, & van 't Haar, 2013) and retain communication strategies from the mass media era (Margolis & Resnick, 2000), as few voters visit their websites on a regular basis (Gibson & McAllister, 2011) or follow politicians' profiles on social media (Vaccari & Nielsen, 2013). However, research in campaign communication also shows that the Web 2.0 provide new opportunities for politicians to address an expanded, new electorate and engage them. As an example, studies show that posts that are frequently liked, commented, or shared can reach a much wider circle of users known as secondary audience or second-degree followers (Jacobs & Spierings, 2016; Vaccari & Valeriani, 2015). Interaction through social media channels furthermore enables face-to-face-like communication with individual voters, with whom politicians can also exchange ideas and negotiate campaign strategies (Magin et al., 2017).

Field of application/theoretical foundation:

In recent years, interaction has been recognized as a central aspect of dialogical communication in the field of public relations (Sweetser & Lariscy, 2008; Taylor & Kent, 2004). The theory states that symmetrical and dialogical two-way communication between an organization and its audience can sustainably support relationship building and their maintenance (Zhang & Seltzer, 2010). By applying this approach to the field of online political communication, it is possible to understand the interactions between politicians and citizens as a form of strategic communication and how they attract and persuade voters.

References/combination with other methods of data collection

Interaction in the last twenty years has been mostly studied in the context of the online environment either by looking at structural features of candidates’ online tools that enable interactions with users (e.g., Druckman, Kifer, & Parkin, 2007, 2009; Schweitzer, 2008); or by studying actual interactions between candidates and citizens on social media (e.g., Graham et al., 2013; Klinger, 2013). Both quantitative manual and automated content analyses thereof have been employed to in research on social media interactions). Quantitative content analysis have been also been combined and compared with qualitative interviews with campaign managers (e.g., Magin et al., 2017).

Example studies

Table 1: Overview exemplary studies measuring interaction, discussion, participation, and related constructs

Study	Medium	Constructs	Operationalization	Coding
Druckman et al. (2007); Druckman et al. (2009)	Candidate websites	Interactivity	Web sites were scrutinized in light of their ability to create someform of interaction by e.g.enabling users to personalize information, arrange information, add information, and/or communicate with other voters and/or the candidate	Additive index
Schweitzer (2008)	Candidate websites	Interactivity provision strategies	Possibility to comment on news; Agenda can be updated by visitors; A channel on video sharing websites; Possibility to comment (a video sharing website); Life webcam; Online photo gallery; Possibility to comment (online photo gallery); Easy contact; Online polls; Profile on SNS; Online forum or chat (among visitors); Online forum or chat (with politicians); Possibility to share content of the website; Possibility to share content on social media; Information about political program (interactive format)	Additive index
Magin et al. (2017)	Facebook posts	Interaction	Index including (1) number of parties’ comments, (2) the number of users’ comments per 1,000,000 eligible voters, and (3) the share of posts in which the parties encourage the voters to discuss politics on the parties’ Facebook page (reciprocity).	Combined index
Graham et al. (2013), Graham, Jackson, & Broersma (2016)	Twitter posts	Interaction	Tweets including: Debating/position taking; Acknowledging; Organizing/mobilizing; Advice giving/helping; and/or Consulting Furthermore @Tweets were scrutinized with whom politicians interacted: Public; Politician/candidate; journalist/media; Party activist; Lobbyist; Expert; Celebrity; Industry; and/or Authority	(0) Not present (1) Present
Lukamto & Carson (2016)	Twitter comments, @mentions, and retweets (RTs)	Discussion	Measures quantity of one-way and two-way messages between members of parliament (MPs) and citizens and who they interact with: ‘citizen to politician’; ‘politician to citizen’; or ‘politician to politician’	Count variable
Bene (2017)	Facebook posts	Engagement	Engagement content is coded if the post contains either requests for likes, comments, and/or sharing or whether it poses a question. All of these individual elements were also coded on their own and analyzed in specified models with all dependent variables	(0) Not present (1) Present
Klinger (2013)	Facebook & Twitter posts	Participation	Posts including calls for discussion, appeals to collect signatures and mobilize other people to participate and to vote as well as general community-building	(0) Not present (1) Present
Keller & Kleinen-von Königslöw (2018)	Facebook & Twitter posts	Pseudo discursive style		(0) Not present (1) Present
Graham et al. (2013), Graham et al. (2016)	Twitter posts	Interaction	Tweets including: Debating/position taking; Acknowledging; Organizing/mobilizing; Advice giving/helping; and/or Consulting @Tweets were also scrutinized with whom politicians interacted: Public; Politician/candidate; journalist/media; Party activist; Lobbyist; Expert; Celebrity; Industry; and/or Authority	(0) Not present (1) Present

References

Bene, M. (2017). Go Viral on the Facebook! Interactions between Candidates and Followers on Facebook during the Hungarian General Election Campaign of 2014. Information, Communication & Society, 20(4), 513–529. https://doi.org/10.1080/1369118X.2016.1198411

Druckman, J. N., Kifer, M. J., & Parkin, M. (2007). The Technological Development of Congressional Candidate Web Sites. Social Science Computer Review, 25(4), 425–442. https://doi.org/10.1177/0894439307305623

Druckman, J. N., Kifer, M. J., & Parkin, M. (2009). Campaign Communications in U.S. Congressional Elections. American Political Science Review, 103(3), 343–366. https://doi.org/10.1017/S0003055409990037

Gibson, R. K., & McAllister, I. (2011). Do online election campaigns win votes? The 2007 Australian “YouTube” election. Political Communication, 28(2), 227–244.

Graham, T., Broersma, M., Hazelhoff, K., & van 't Haar, G. (2013). Between Broadcasting Political Messages and Interacting with Voters. Information, Communication & Society, 16(5), 692–716. https://doi.org/10.1080/1369118X.2013.785581

Graham, T., Jackson, D., & Broersma, M. (2016). New Platform, Old Habits? Candidates’ Use of Twitter during the 2010 British and Dutch General Election Campaigns. New Media & Society, 18(5), 765–783. https://doi.org/10.1177/1461444814546728

Jackson, N., & Lilleker, D. G. (2010). Tentative Steps towards Interaction. Internet Research, 20(5), 527–544. https://doi.org/10.1108/10662241011084103

Jacobs, K., & Spierings, N. (2016). Social Media, Parties, and Political Inequalities. New York: Palgrave Macmillan US. https://doi.org/10.1057/9781137533906

Keller, T. R., & Kleinen-von Königslöw, K. (2018). Pseudo-Discursive, Mobilizing, Emotional, and Entertaining: Identifying Four Successful Communication Styles of Political Actors on Social Media during the 2015 Swiss National Elections. Journal of Information Technology & Politics, 15(4), 358–377. https://doi.org/10.1080/19331681.2018.1510355

Klinger, U. (2013). Mastering the Art of Social Media. Information, Communication & Society, 16(5), 717–736. https://doi.org/10.1080/1369118X.2013.782329

Lilleker, D. G., & Koc-Michalska, K. (2013). Online Political Communication Strategies: MEPs, E-Representation, and Self-Representation. Journal of Information Technology & Politics, 10(2), 190–207. https://doi.org/10.1080/19331681.2012.758071

Lukamto, W., & Carson, A. (2016). Politweets: Social Media as a Platform for Political Engagement between Victorian Politicians and Citizens. Communication Research and Practice, 2(2), 191–212. https://doi.org/10.1080/22041451.2016.1186485

Magin, M., Podschuweit, N., Haßler, J., & Russmann, U. (2017). Campaigning in the Fourth Age of Political Communication. A Multi-Method Study on the Use of Facebook by German and Austrian Parties in the 2013 National Election Campaigns. Information, Communication & Society, 20(11), 1698–1719. https://doi.org/10.1080/1369118X.2016.1254269

Margolis, M., & Resnick, D. (2000). Politics as Usual: The Cyberspace "Revolution". Contemporary American politics. Thousand Oaks: Sage.

Schweitzer, E. J. (2008). Innovation or Normalization in E-Campaigning? European Journal of Communication, 23(4), 449–470. https://doi.org/10.1177/0267323108096994

Stromer-Galley, J. (2000). On-line Interaction and Why Candidates Avoid It. Journal of Communication, 50(4), 111–132.

Sweetser, K. D., & Lariscy, R. W. (2008). Candidates make good friends: An analysis of candidates' uses of Facebook. International Journal of Strategic Communication, 2(3), 175–198.

Taylor, M., & Kent, M. L. (2004). Congressional web sites and their potential for public dialogue. Atlantic Journal of Communication, 12(2), 59–76.

Vaccari, C., & Nielsen, R. K. (2013). What drives politicians' online popularity? An analysis of the 2010 US midterm elections. Journal of Information Technology & Politics, 10(2), 208–222.

Vaccari, C., & Valeriani, A. (2015). Follow the leader! Direct and indirect flows of political communication during the 2013 Italian general election campaign. New Media & Society, 17(7), 1025–1042. https://doi.org/10.1177/1461444813511038

Zhang, W., & Seltzer, T. (2010). Another piece of the puzzle: Advancing social capital theory by examining the effect of political party relationship quality on political and civic participation. International Journal of Strategic Communication, 4(3), 155–170.

Issue salience (Public Diplomacy) (see also theme or media coverage salience)

Sarah Marschlich — Sun, 18 Apr 2021 00:00:00 +0200

The variable “issue salience” refers to the visibility or prominence of a given topic or theme occurring in the news coverage and is used to explore first-level agenda-setting (McCombs & Shaw, 1972). In addition to actor salience and valence, issue salience is analyzed to describe and explore the news coverage on different events and public debates. Mostly, issue salience is measured as the number of mentioning a particular issue, topic, or theme.

Field of application/theoretical foundation:

Issue salience is analyzed using content analysis across different subfields of communication and media research, including the field of public diplomacy. In public diplomacy research, scholars measure issue salience in the context of governmental communication on their official channels online and offline or the representation of countries in social or mass media. Researchers embed the concept of issue salience primarily in agenda-setting theory (McCombs & Shaw, 1972), analyzing it as an independent variable from which to derive implications of news media coverage on audiences’ perceptions on a certain object or examining the relationship between issue salience in the media and the public agenda.

References/combination with other methods of data collection:

When it comes to analyses on issue salience and its link to public perceptions, a mixed-method study design incorporating content analysis in combination with surveys is used to validate issue salience.

Example study:

Zhou et al., 2013

Information about Zhou et al., 2013

Authors: Zhang et al.

Research question/research interest: Comparison between news coverage on Great Britain (in terms of themes) in U.S.-American and Chinese news media during the Olympic Games 2012

RQ: What were the most salient themes in British, U.S., and Chinese media when they covered the opening ceremony of the London Olympics?

Object of analysis: Newspaper (30 media outlets across three countries (the United States, the United Kingdom, and China, not explicated)

Time frame of analysis: 24 July 2012 to 12 August 2012

Information about variable

Variable name/definition: Media coverage salience: Number of mentions given to a particular theme

Level of analysis: Story

Values:

(1) Countryside (e.g., emphasis of British natural beauty and scenic sites)

(2) Creativity (e.g., focus on British creative sector, such as arts, film, and literature)

(3) Entrepreneurship (e.g., portrayals on entrepreneurs and investors, or global investment)

(4) Green (e.g., emphasis on Great Britain’s sustainability and environmental protection efforts)

(5) Heritage (e.g., focus on British royalty, museums, and historical landmarks)

(6) Innovation (e.g., discussion of science and technology in Great Britain)

(7) Knowledge (e.g., portrayals of research and development at British universities)

(8) Music (e.g., mentions of British and music artists)

(9) Shopping (e.g., emphasis on British shopping venues such as London as shopping city)

(10) Sport (e.g., emphasis on sporting events or athletes, such as David Beckham)

(11) Technology (e.g., focus on digital media, e-commerce, and IT services in Great Britain)

Scales: Nominal

Reliability: Krippendorf’s alpha = .90

References

McCombs, M. E., & Shaw, D. L. (1972). The Agenda-Setting Function of Mass Media. Public Opinion Quarterly, 36(2), 176–187.

Zhou, S., Shen, B., Zhang, C., & Zhong, X. (2013). Creating a Competitive Identity: Public Diplomacy in the London Olympics and Media Portrayal. Mass Communication & Society, 16(6), 869–887.

Attribute salience (Public Diplomacy) (see also issue attributes)

Sarah Marschlich — Sun, 18 Apr 2021 00:00:00 +0200

The variable “attribute salience” is described as the characteristics of a given issue that is portrayed in media coverage or other communication channels. It is generally measured in addition to issue salience and issue valence in order to analyze media portrayals of events, actors, or public discourses. Attribute salience is often measured in order to explore how particular issues are presented (instead of which in general), thereby contributing to second-level agenda-setting effects (McCombs et al., 1997).

Field of application/theoretical foundation:

Attribute salience is analyzed across different subfields of communication and media research, including the field of public diplomacy. In public diplomacy research, scholars measure attribute salience in the context of political communication or the representation of countries in the news media as well as on social media. Researchers embed the concept of attribute salience or issue attributes mainly in agenda-setting theory (McCombs & Shaw, 1972), analyzing it as an independent variable to derive with implications of news media coverage on audiences’ evaluations of certain issues.

References/combination with other methods of data collection:

When it comes to analyses on attribute salience in the context of issues and its link to public perceptions, a mixed-method study design incorporating content analysis in combination with surveys is used to validate attribute salience.

Example study:

Zhang et al. (2018)

Information on Zhang et al., 2018

Authors: Zhang et al.

Research question/research interest: Effects of agenda-building of Chinese state-sponsored media on news media coverage in Taiwan and Singapore during Hong Kong Protest

Object of analysis: Newspaper (several English newspapers and newswires published in China, Singapore, and Taiwan; not explicated)

Time frame of analysis: 1 May 2014 to 30 April 2015

Information about Variable

Level of analysis: Article

Values:

(1) Substantive issue attributes (frame):

(a) Conflict

(b) Cooperation

(d) Proposed solution to the problem

(e) Responsibility attribution

(f) Human interest

(g) Consequences and outcomes

(h) Morality and motivation to take actions

(2) Affective issue attributes (tone):

(a) Negative

(b) Neutral or mixed

(d) N/A.

Scales: Nominal

Reliability: Cohen‘s kappa = 0.76

References

McCombs, M. E., & Shaw, D. L. (1972). The Agenda-Setting Function of Mass Media. Public Opinion Quarterly, 36(2), 176–187.

McCombs, M. E., Llamas, J. P., Lopez-Escobar, E., & Rey, F. (1997). Candidate Images in Spanish Elections: Second-Level Agenda-Setting Effects. Journalism & Mass Communication Quarterly, 74(4), 703–717.

Zhang, T., Khalitova, L., Myslik, B., Mohr, T. L., Kim, J. Y., & Kiousis, S. (2018). Comparing Chinese state-sponsored media’s agenda-building influence on Taiwan and Singapore media during the 2014 Hong Kong Protest. Chinese Journal of Communication, 11(1), 66–87.

Public diplomacy message strategy (Public Diplomacy) (see also public diplomacy approach)

Sarah Marschlich — Sun, 18 Apr 2021 00:00:00 +0200

The variable “public diplomacy message strategy” (or “public diplomacy approach”) refers to public diplomacy efforts in a given country in order to investigate how and with which goal public diplomacy is strategically communicated in the given context. The variable reflects the communication style of a specific actor (a politician, government, or country).

Field of application/theoretical foundation:

Analyses of public diplomacy message strategies or approaches mostly build on the taxonomy of public diplomacy (Cull, 2008) or the proposed categories of public diplomacy by Fitzpatrick (2010).

References/combination with other methods of data collection:

Public diplomacy message strategies can, in addition to content analysis, be analyzed by conducting interviews or surveys with public diplomacy actors, which allow validating the results from content analyses.

Example study:

Dodd & Collins (2017)

Information on Dodd & Collins (2017)

Authors: Dodd & Collins

Research question/research interest: Comparison between public diplomacy approaches between Central Eastern European (not explicated) and Western countries (Canada, the United Kingdom, and the United States)

Object of analysis: Twitter content posted by 41 embassy accounts (not explicated)

Time frame of analysis: March 2015

Information about variable

Variable name/definition: Public diplomacy practices: Communication strategy

Level of analysis: Tweet

Values:

Building on Cull’s (2008) taxonomy of public diplomacy:

(1) Listening (attempts to collect and collate information about foreign publics and their opinions)

(2) Advocacy (activities that promote the country’s policies or general interests among foreign publics)

(3) Cultural (efforts to promote cultural resources and achievements of a country)

(4) International (activities that involve sending national actors abroad or receiving international actors to strategically manage the international environment)

(5) News (use of radio, television and digital media to inform and involve foreign audiences)

(6) Other

Scales: Nominal

Reliability: Krippendorf’s alpha = .50

References

Cull, N. J. (2008). Public Diplomacy: Taxonomies and Histories. The ANNALS of the American Academy of Political and Social Science, 616(1), 31–54.

Dodd, M. D., & Collins, S. J. (2017). Public relations message strategies and public diplomacy 2.0: An empirical analysis using Central-Eastern European and Western Embassy Twitter accounts. Public Relations Review, 43(2), 417–425.

Fitzpatrick, K. (2010). The future of U.S. public diplomacy: An uncertain fate. Martinus Nijhoff/Brill.

Publishers/sources (Disinformation)

Anna Staender, Edda Humprecht — Fri, 26 Mar 2021 00:00:00 +0100

Recent research has mainly used two approaches to identify publishers or sources of disinformation: First, alternative media are identified as potential publishers of disinformation. Second, potential publishers of disinformation are identified via fact-checking websites. Samples created using those approaches can partly overlap. However, the two approaches differ in terms of validity and comprehensiveness of the identified population. Sampling of alternative media outlets is theory-driven and allows for cross-national comparison. However, researchers face the challenge to identify misinforming content published by alternative media outlets. In contrast, fact-checked content facilitates the identification of a given disinformation population; however, fact-checker often have a publication bias focusing on a small range of (elite) actors or sources (e.g. individual blogs, hyper partisan news outlets, or politicians). In both approaches it is important to describe, compare and, if possible, assign the outlets to already existing categories in order to enable a temporal and spatial comparison.

Approaches to identify sources/publishers:

Besides the operationalization of specific variables analyzed in the field of disinformation, the sampling procedure presents a crucial element to operationalize disinformation itself. Following the approach of detecting disinformation through its potential sources or publishers (Li, 2020), research analyzes alternative media (Bachl, 2018; Boberg, Quandt, Schatto-Eckrodt, & Frischlich, 2020; Heft et al., 2020) or identifies a various range of actors or domains via fact-checking sites (Allcott & Gentzkow, 2017; Grinberg et al., (2019); Guess, Nyhan & Reifler, 2018). Those two approaches are explained in the following.

Alternative media as sources/publishers

The following procedure summarizes the approaches used in current research for the identification of relevant alternative media outlets (following Bachl, 2018; Boberg et al., 2020; Heft et al., 2020).

Snowball sampling to define the universe of alternative media outlets may consists of the following steps:
1. Sample of outlets identified in previous research
2. Consultation of search engines and news articles

Departing from a potential prototype, websites provide information about digital metrics (Alexa.com or Similarweb.com). For example, Similarweb.com shows three relevant lists per outlet: “Top Referring Sites” (which websites are sending traffic to this site), “Also visited websites” (overlap with users of other websites), and “Competitors & Similar Sites” (similarity defined by the company)

Definition of alternative media outlets
1. Journalistic outlets (for example, excluding blogs and forums) with current, non-fictional and regular content
2. Self-description of the outlets in a so-called “about us” section or in a mission statement, which underlines the relational perspective of being an alternative to the mainstream media. This description may for example include keywords such as alternative, independent, unbiased, critical or is in line with statements like “presenting the real/true views/facts” or “covering what the mainstream media hides/leaves out”.

Use of predefined dimensions and categories of alternative media (Frischlich, Klapproth, & Brinkschulte, 2020; Holt, Ustad Figenschou, & Frischlich, 2019)

Sources/publishers via fact-checking sites

Following previous research in the U.S., Guess et al. (2018) identified “Fake news domains” (focusing on pro-Trump and pro-Clinton content) which published two or more articles that were coded as “fake news” by fact-checkers (derived from Allcott & Gentzkow, 2017). Grinberg et al. (2019) identified three classes of “fake news sources” differentiated by severity and frequency of false content (see Table 1). These three sources are part of a total of six website labels. The researchers additionally coded the sites into reasonable journalism, low quality journalism, satire and sites that were not applicable. The coders reached a percentual agreement of 60% for the labeling of the six categories, and 80% for the distinction of fake and non-fake categories.

Table 1. Three classes of “fake news sources” by Grinberg et al. (2019)

Label	Specification	Identification	Definition
Black domains	Based on previous studies: These domains published at least two articles which were declared as “fake news” by fact-checking sites.	Based on preexisting lists constructed by fact-checkers, journalists and academics (Allcott & Gentzkow, 2017; Guess et al., 2018)	Almost exclusively fabricated stories
Red domains	Major or frequent falsehoods that are in line with the site's political agenda. Prejudiced: Site presents falsehoods that focus upon one group with regards to race / religion / ethnicity / sexual orientation. Major or frequent falsehoods with little regard for the truth, but not necessarily to advance a certain political agenda.	By the fact-checker snopes.com as sources of questionable claims; then manually differentiated between red and orange domains	Falsehoods that clearly reflected a flawed editorial process
Orange domains	Moderate or occasional falsehoods to advance political agenda. Sensationalism: exaggerations to the extent that the article becomes misleading and inaccurate. Occasionally prejudiced articles: Site at times presents individual articles that contain falsehoods regarding race / religion / ethnicity / sexual orientation Openly states that the site may not be inaccurate, fake news, or cannot be trusted to provide factual news. Moderate or frequent falsehoods with little regard for the truth, but not necessarily to advance a certain political agenda. Conspiratorial: explanations of events that involves unwarranted suspicion of government cover ups or supernatural agents.	By the fact-checker snopes.com as sources of questionable claims; then manually differentiated between red and orange domains	Negligent and deceptive information but are less systemically flawed

Supplementary materials: https://science.sciencemag.org/content/sci/suppl/2019/01/23/363.6425.374.DC1/aau2706_Grinberg_SM.pdf (S5 and S6)

Coding scheme and source labels: https://zenodo.org/record/2651401#.XxGtJJgzaUl (LazerLab-twitter-fake-news-replication-2c941b8\domains\domain_coding\data)

References

Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2), 211–236.

Bachl, M. (2018). (Alternative) media sources in AfD-centered Facebook discussions. Studies in Communication | Media, 7(2), 256–270.

Bakir, V., & McStay, A. (2018). Fake news and the economy of emotions. Digital Journalism, 6(2), 154–175.

Boberg, S., Quandt, T., Schatto-Eckrodt, T., & Frischlich, L. (2020, April 6). Pandemic populism: Facebook pages of alternative news media and the corona crisis -- A computational content analysis. Retrieved from http://arxiv.org/pdf/2004.02566v3

Farkas, J., Schou, J., & Neumayer, C. (2018). Cloaked Facebook pages: Exploring fake Islamist propaganda in social media. New Media & Society, 20(5), 1850–1867.

Frischlich, L., Klapproth, J., & Brinkschulte, F. (2020). Between mainstream and alternative – Co-orientation in right-wing populist alternative news media. In C. Grimme, M. Preuss, F. W. Takes, & A. Waldherr (Eds.), Lecture Notes in Computer Science. Disinformation in open online media (Vol. 12021, pp. 150–167). Cham: Springer International Publishing.

Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019). Fake news on Twitter during the 2016 U.S. Presidential election. Science (New York, N.Y.), 363(6425), 374–378.

Guess, A., Nagler, J., & Tucker, J. (2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science Advances, 5(1). https://doi.org/10.1126/sciadv.aau4586

Guess, A., Nyhan, B., & Reifler, J. (2018). Selective exposure to misinformation: Evidence from the consumption of fake news during the 2016 US presidential campaign. European Research Council, 9(3), 1–14.

Heft, A., Mayerhöffer, E., Reinhardt, S., & Knüpfer, C. (2020). Beyond Breitbart: Comparing right?wing digital news infrastructures in six Western democracies. Policy & Internet, 12(1), 20–45.

Holt, K., Ustad Figenschou, T., & Frischlich, L. (2019). Key dimensions of alternative news media. Digital Journalism, 7(7), 860–869.

Nelson, J. L., & Taneja, H. (2018). The small, disloyal fake news audience: The role of audience availability in fake news consumption. New Media & Society, 20(10), 3720–3737.

Topics (Disinformation)

Anna Staender, Edda Humprecht — Fri, 26 Mar 2021 00:00:00 +0100

The topic variable is used in research on disinformation to analyze thematic differences in the content of false news, rumors, conspiracies, etc. Those topics are frequently based on national news agendas, i.e. producers of disinformation address current national or world events (e.g. elections, immigration, etc.) (Humprecht, 2019).

Field of application/theoretical foundation:

Topics are a central yet under-researched aspect of research on online disinformation (Freelon & Wells, 2020). The research interest is to find out which topics are taken up and spread by disinformation producers. The focus of this research is both on specific key topics for which sub-themes are identified (e.g. elections, climate change, Covid-19) and, more generally, on the question of which misleading content is disseminated (mostly on social media). Methodologically, the identification of topics is often a first step followed by further analysis of the content (Ferrara, 2017). Thus, the analysis of topics is linked to the detection of disinformation, which represents a methodological challenge. Topics can be identified inductively or deductively. Inductive analyses often use a data corpus, for example social media data, and try to identify topics using techniques such as topic modeling (e.g. Boberg et al., 2020). Deductive analyses frequently use topic lists to classify contents. Topics lists are initially created based on the literature on the respective topic or with the help of databases, e.g. by fact-checkers.

References/combination with other methods of data collection:

Studies on topics of disinformation use both manual and automated content analysis or combinations of both to investigate the occurrence of different topics in texts (Boberg et al., 2020; Bradshaw, Howard, Kollanyi, & Neudert, 2020). Inductive and deductive approaches have been combined with qualitative text analyses to identify topic categories which are subsequently coded (Humprecht, 2019; Marchal, Kollanyi, Neudert, & Howard, 2019).

Example studies:

Ferrara (2017); Humprecht (2019), Marchal et al. (2019)

Table 1. Summary of selected studies

Author(s)	Sample	Values	Reliability
Ferrara (2017)	Content type: Tweets Sampling period: April 27, 2017 to May 7, 201 Sample size: 16.65 million tweets Sampling: List of 23 key words and top 20 hashtags	Keywords: France2017, Marine2017, AuNomDuPeuple, FrenchElection, FrenchElections, Macron, LePen, MarineLePen, FrenchPresidentialElection, JeChoisisMarine, JeVoteMarine, JeVoteMacron JeVote, Presidentielle2017, ElectionFracaise, JamaisMacron, Macron2017, EnMarche, MacronPresident Hashtags: #Macron, #Presidentielle2017, #fn, #JeVote, #LePen, #France, #2017LeDebat, #MacronLeaks, #Marine2017, #debat2017, #2017LeDébat, #MacronGate, #MarineLePen, #Whirlpool, #EnMarche, #JeVoteMacron, #MacronPresident, #JamaisMacron, #FrenchElection	-
Humprecht (2019)	Content type: fact checks Outlet/ country: 2 fact checkers per country (AT, DE, UK, US) Sampling period: June 1, 2016 to September 30, 2017 Sample size: N=651 Unit of analysis: story/ fact-check No. of topics coded: main topic per fact-check Level of analysis: fact checks and fact-checker	conspiracy theory, education, election campaign, environment, government/public administration (at the time when the story was published), health, immigration/integration, justice/crime, labor/employment, macroeconomics/economic regulation, media/journalism, science/ technology, war/terror, others	Krippendorff’s alpha = 0.71
Marchal et al. (2019)	Content type: tweets related to the European elections 2019 Sampling: hashtags in English, Catalan, French, German, Italian, Polish, Spanish, Swedish Sampling criteria: (1) contained at least one of the relevant hashtags; (2) contained the hashtag in the URL shared, or the title of its webpage; (3) were a retweet of a message that contained a relevant hashtag or mention in the original message; (4) were a quoted tweet referring to a tweet with a relevant hashtag or mention Sampling period: 5 April and 20 April, 2019 Sample size: 584,062 tweets from 187,743 unique users	Religion Islam (Muslim, Islam, Hijab, Halal, Muslima, Minaret) Religion Christianity (Christianity, Church, Priest) Immigration (Asylum Seeker, Refugee, Migrants, Child Migrant, Dual Citizenship, Social Integration) Terrorism (ISIS, Djihad, Terrorism, Terrorist Attack) Political Figures/Parties (Vladimir Putin, Enrico Mezzetti, Emmanuel Macron, ANPI, Arnold van Doorn, Islamic Party for Unity, Nordic Resistance Movement) Celebrities (Lara Trump, Alba Parietti) Crime (Vandalism, Rape, Sexual Assault, Fraud, Murder, Honour Killing) Notre-Dame Fire (Notre-Dame Fire, Reconstruction) Political Ideology (Anti-Fascism, Fascism, Nationalism) Social Issues (Abortion, Bullying, Birth Rate)	-

References

Boberg, S., Quandt, T., Schatto-Eckrodt, T., & Frischlich, L. (2020). Pandemic Populism: Facebook Pages of Alternative News Media and the Corona Crisis -- A Computational Content Analysis, 2019. Retrieved from http://arxiv.org/abs/2004.02566

Bradshaw, S., Howard, P. N., Kollanyi, B., & Neudert, L. M. (2020). Sourcing and Automation of Political News and Information over Social Media in the United States, 2016-2018. Political Communication, 37(2), 173–193. https://doi.org/10.1080/10584609.2019.1663322

Ferrara, E. (2017). Disinformation and social bot operations in the run up to the 2017 French presidential election. First Monday, 22(8). https://doi.org/10.5210/FM.V22I8.8005

Freelon, D., & Wells, C. (2020). Disinformation as Political Communication. Political Communication, 37(2), 145–156. https://doi.org/10.1080/10584609.2020.1723755

Humprecht, E. (2019). Where ‘fake news’ flourishes: a comparison across four Western democracies. Information Communication and Society, 22(13), 1973–1988. https://doi.org/10.1080/1369118X.2018.1474241

Marchal, N., Kollanyi, B., Neudert, L., & Howard, P. N. (2019). Junk News During the EU Parliamentary Elections?: Lessons from a Seven-Language Study of Twitter and Facebook. Oxford, UK. Retrieved from https://comprop.oii.ox.ac.uk/wp-content/uploads/sites/93/2019/05/EU-Data-Memo.pdf

Types (Disinformation)

Anna Staender, Edda Humprecht — Fri, 26 Mar 2021 00:00:00 +0100

Disinformation can appear in various forms. Firstly, different formats can be manipulated, such as texts, images, and videos. Secondly, the amount and degree of falseness can vary, from completely fabricated content to decontextualized information to satire that intentionally misleads recipients. Therefore, the forms and format of disinformation might vary and differ not only between the supposedly clear categories of “true” and “false”.

Field of application/theoretical foundation:

Studies on types of disinformation are conducted in various fields, e.g. political communication, journalism studies, and media effects studies. Among other things, the studies identify the most common types of mis- or disinformation during certain events (Brennen, Simon, Howard, & Nielsen, 2020), analyze and categorize the behavior of different types of Twitter accounts (Linvill & Warren, 2020), and investigate the existence of serveral types of “junk news” in different national media landscapes (Bradshaw, Howard, Kollanyi, & Neudert, 2020; Neudert, Howard, & Kollanyi, 2019).

References/combination with other methods of data collection:

Only relatively few studies use combinations of methods. Some studies identify different types of disinformation via qualitative and quantitative content analyses (Bradshaw et al., 2020; Brennen et al., 2020; Linvill & Warren, 2020; Neudert et al., 2019). Others use surveys to analyze respondents’ concerns as well as exposure towards different types of mis- and disinformation (Fletcher, 2018).

Example studies:

Brennen et al. (2020); Bradshaw et al. (2020); Linvill and Warren (2020)

Information on example studies:

Types of disinformation are defined by the presentation and contextualization of content and sometimes additionally by details (e.g. professionalism) about the communicator. Studies either deductively identify different types of disinformation (Brennen et al., 2020) by applying the theoretical framework by Wardle (2019), or additionally inductively identify and build different categories based on content analyses (Bradshaw et al., 2020; Linvill & Warren, 2020).

Table 1. Types of mis-/disinformation by Brennen et al. (2020)

Category	Specification
Satire or parody	-
False connection	Headlines, visuals or captions don’t support the content
Misleading content	Misleading use of information to frame an issue or individual, when facts/information are misrepresented or skewed
False context	Genuine content is shared with false contextual information, e.g. real images which have been taken out of context
Imposter content	Genuine sources, e.g. news outlets or government agencies, are impersonated
Fabricated content	Content is made up and 100% false; designed to deceive and do harm
Manipulated content	Genuine information or imagery is manipulated to deceive, e.g. deepfakes or other kinds of manipulation of audio and/or visuals

Note. The categories are adapted from the theoretical framework by Wardle (2019). The coding instruction was: “To the best of your ability, what type of misinformation is it? (Select one that fits best.)” (Brennen et al., 2020, p. 12). The coders reached an intercoder reliability of a Cohen’s kappa of 0.82.

Table 2. Criteria for the “junk news” label by Bradshaw et al. (2020)

Criteria	Reference	Specification
Professionalism	refers to the information about authors and the organization	“Sources do not employ the standards and best practices of professional journalism, including information about real authors, editors, and owners” (pp. 174-175). “Distinct from other forms of user-generated content and citizen journalism, junk news domains satisfy the professionalism criterion because they purposefully refrain from providing clear information about real authors, editors, publishers, and owners, and they do not publish corrections of debunked information” (p. 176). Procedure: - Systematically checked the about pages of domains: Contact information, information about ownership and editors, and other information relating to professional standards - Reviewed whether the sources appeared in third-party fact-checking reports - Checked whether sources published corrections of fact-checked reporting. Examples: zerohedge.com, conservative- fighters.org, deepstatenation.news
Counterfeit	refers to the layout and design of the domain itself	“(…) [S]ources mimic established news reporting by using certain fonts, having branding, and employing content strategies. (…) Junk news is stylistically disguised as professional news by the inclusion of references to news agencies and credible sources as well as headlines written in a news tone with date, time, and location stamps. In the most extreme cases, outlets will copy logos and counterfeit entire domains” (p. 176). Procedure: - Systematically reviewed organizational information about the owner and headquarters by checking sources like Wikipedia, the WHOIS database, and third-party fact-checkers (like Politico or MediaBiasFactCheck) - Consulted country-specific expert knowledge of the media landscape in the US to identify counterfeiting websites. Examples: politicoinfo.com, NBC.com.co
Style	refers to the content of the domain as a whole	“ (…) [S]tyle is concerned with the literary devices and language used throughout news reporting. (…) Designed to systematically manipulate users for political purposes, junk news sources deploy propaganda techniques to persuade users at an emotional, rather than cognitive, level and employ techniques that include using emotionally driven language with emotive expressions and symbolism, ad hominem attacks, misleading headlines, exaggeration, excessive capitalization, unsafe generalizations, logical fallacies, moving images and lots of pictures or mobilizing memes, and innuendo (Bernays, 1928; Jowette & O’Donnell, 2012; Taylor, 2003). (…) Stylistically, problematic sources will employ propaganda and clickbait techniques to varying degrees. As a result, determining style can be highly complex and context dependent” (p. 177). Procedure: - Examined at least five stories on the front page of each news source in depth during the US presidential campaign in 2016 and the SOTU address in 2018 - Checked the headlines of the stories and the content of the articles for literary and visual propaganda devices - Considered as stylistically problematic if three of the five stories systematically exhibited elements of propaganda Examples: 100percentfedup.com, barenakedislam.com, theconservativetribune.com, dangerandplay.com
Credibility	refers to the content of the domain as a whole	“(…) [S]ources rely on false information or conspiracy theories and do not post corrections” (p. 175). “[They] typically report on unsubstantiated claims and rely on conspiratorial and dubious sources. (…) Junk news sources that satisfy the credibility criterion frequently fail to vet their sources, do not consult multiple sources, and do not fact-check” (p. 178). Procedure: - Examined at least five front page stories and reviewed the sources that were cited - Reviewed pages to see if they included known conspiracy theories on issues such as climate change, vaccination, and “Pizzagate” - Checked third-party fact-checkers for evidence of debunked stories and conspiracy theories Examples: infowars.com, endingthefed.com, thegatewaypundit.com, newspunch.com
Bias	refers to the content of the domain as a whole	“(…) [H]yper-partisan media websites and blogs (…) are highly biased, ideologically skewed, and publish opinion pieces as news. Basing their stories on the same events, these sources manage to convey strikingly different impressions of what actually transpired. It is such systematic differences in the mapping from facts to news reports that we call bias. (…) Bias exists on both sides of the political spectrum. Like determining style, determining bias can be highly complex and context dependent” (pp. 177-178). Procedure: - Checked third-party sources that systematically evaluate media bias - If the domain was not evaluated by a third party, the authors examined the ideological leaning of the sources used to support stories appearing on the domain - Evaluation of the labeling of politicians (are there differences between the left and the right?) - Identified bias created through the omission of unfavorable facts, or through writing that is falsely presented as being objective Examples on the right: breitbart.com, dailycaller.com, infowars.com, truthfeed.com Examples on the left: occupydemocrats.com, addictinginfo.com, bipartisanreport.com

Note. The coders reached an intercoder reliability of a Krippendorff’s kappa of 0.89. The label of “junk news” is defined by fulfilling at least three of the five criteria. It refers to sources that deliberately publish misleading, deceptive, or incorrect information packaged as real news.

Table 3. Identified types of IRA-associated Twitter accounts by Linvill and Warren (2020)

Category	Specification
Right troll	“Twitter-handles broadcast nativist and right-leaning populist messages. These handles’ themes were distinct from mainstream Republicanism. (…) They rarely broadcast traditionally important Republican themes, such as taxes, abortion, and regulation, but often sent divisive messages about mainstream and moderate Republicans. (…) The overwhelming majority of handles, however, had limited identifying information, with profile pictures typically of attractive, young women” (p. 5). Hashtags frequently used by these accounts: #MAGA (i.e., “Make America Great Again,”), #tcot (i.e. “Top Conservative on Twitter), #AmericaFirst, and #IslamKills
Left troll	“These handles sent socially liberal messages, with an overwhelming focus on cultural identity. (…) They discussed gender and sexual identity (e.g., #LGBTQ) and religious identity (e.g., #MuslimBan), but primarily focused on racial identity. Just as the Right Troll handles attacked mainstream Republican politicians, Left Troll handles attacked mainstream Democratic politicians, particularly Hillary Clinton. (…) It is worth noting that this account type also included a substantial portion of messages which had no clear political motivation” (p. 6). Hashtags frequently used by these accounts: #BlackLivesMatter, #PoliceBrutality, and #BlackSkinIsNotACrime
Newsfeed	“These handles overwhelmingly presented themselves as U.S. local news aggregators and had descriptive names (…). These accounts linked to legitimate regional news sources and tweeted about issues of local interest (…). A small number of these handles, (…) tweeted about global issues, often with a pro-Russia perspective” (p. 6). Hashtags frequently used by these accounts: #news, #sports, and #local
Hashtag gamer	“These handles are dedicated almost entirely to playing hashtag games, a popular word game played on Twitter. Users add a hashtag to a tweet (e.g., #ThingsILearnedFromCartoons) and then answer the implied question. These handles also posted tweets that seemed organizational regarding these games (…). Like some tweets from Left Trolls, it is possible such tweets were employed as a form of camouflage, as a means of accruing followers, or both. Other tweets, however, often using the same hashtag as mundane tweets, were socially divisive (…)” (p. 7). Hashtags frequently used by these accounts: #ToDoListBeforeChristmas, #ThingsYouCantIgnore, #MustBeBanned, and #2016In4Words
Fearmonger	“These accounts spread disinformation regarding fabricated crisis events, both in the U.S. and abroad. Such events included non-existent outbreaks of Ebola in Atlanta and Salmonella in New York, an explosion at the Columbian Chemicals plan in Louisiana, a phosphorus leak in Idaho, as well as nuclear plant accidents and war crimes perpetrated in Ukraine. (…) These accounts typically tweeted a great deal of innocent, often frivolous content (i.e. song lyrics or lines of poetry) which were potentially automated. With this content these accounts often added popular hashtags such as #love (…) and #rap (…). These accounts changed behavior sporadically to tweet disinformation, and that output was produced using a different Twitter client than the one used to produce the frivolous content. (…) The Fearmonger category was the only category where we observed some inconsistency in account activity. A small number of handles tweeted briefly in a manner consistent with the Right Troll category but switched to tweeting as a Fearmonger or vice-versa” (p. 7). Hashtags frequently used by these accounts: #Fukushima2015 and #ColumbianChemicals

Note. The categories were identified qualitatively analyzing the content produced and were then refined and explored more detailed via a quantitative analysis. The coders reached a Krippendorff’s alpha intercoder-reliability of 0.92.

References

Bradshaw, S., Howard, P. N., Kollanyi, B., & Neudert, L.?M. (2020). Sourcing and automation of political news and information over social media in the United States, 2016-2018. Political Communication, 37(2), 173–193.

Brennen, J. S., Simon, F. M., Howard, P. N. [P. N.], & Nielsen, R. K. (2020). Types, sources, and claims of covid-19 misinformation. Reuters Institute. Retrieved from http://www.primaonline.it/wp-content/uploads/2020/04/COVID-19_reuters.pdf

Fletcher, R. (2018). Misinformation and disinformation unpacked. Reuters Institute. Retrieved from http://www.digitalnewsreport.org/survey/2018/misinformation-and-disinformation-unpacked/

Linvill, D. L., & Warren, P. L. (2020). Troll factories: Manufacturing specialized disinformation on Twitter. Political Communication, 1–21.

Neudert, L.?M., Howard, P., & Kollanyi, B. (2019). Sourcing and automation of political news and information during three European elections. Social Media + Society, 5(3). https://doi.org/10.1177/2056305119863147

Wardle, C. (2019). First Draft's essential guide to understanding information disorder. UK: First Draft News. Retrieved from https://firstdraftnews.org/wp-content/uploads/2019/10/Information_Disorder_Digital_AW.pdf?x76701

Public engagement of scientists (Science Communication)

Nina Wicke — Fri, 26 Mar 2021 00:00:00 +0100

Public engagement of scientists is defined as “all kinds of publicly accessible communication carried out by people presenting themselves as scientists. This includes scholarly communication directed at peers as well as science communication directed at lay publics” (Jünger & Fähnrich, 2019, p. 7).

Field of application/theoretical foundation:

The variable “public engagement of scientists” can be differentiated according to the following three main dimensions (Jünger & Fähnrich, 2019):

Directions of engagement: Describes the extent to which communication scientists on Twitter connect with people from different sectors of society (e.g. science, politics, media, economy). This allows conclusions to the potential influence of scientists reaching specific audiences beyond the scientific community (Jünger & Fähnrich, 2019).
Topics of engagement: Previous research reveals that social scientists not only act as experts in their research field, but often present themselves as public intellectuals by also referring to political and social issues (Albæk, Christiansen, & Togeby, 2003; Fähnrich & Lüthje, 2017). For this reason, communication scientists are expected to communicate not only on scientific but also on political or economic issues.
Modes of engagement: In addition to disseminating information, social networking sites also allow for more interactive ways of maintaining relationships. Thus, following Ellison and Boyd (2013), it can be assumed that communication on social networking sites can be both content-centered and user-centered. This dimension can be linked to the speech act theory (Klemm, 2000; Searle, 1990), according to which every use of language has a performative function.

References/combination with other methods of data collection:

In some cases, a mixed method approach, employing two data collection methods, is applied: a content analysis is complemented by a survey to gain information about the science communicators such as demographic information (Hara, Abbazio, & Perkins, 2019). Furthermore, their social networks are investigated by means of network analysis (Walter, Lörcher, & Brüggemann, 2019).

Example studies:

Hara et al. (2019); Jahng & Lee (2018); Kouper (2010); Mahrt & Puschmann (2014); Walter et al. (2019)

Information on Jünger & Fähnrich, 2019

Authors: Jakob Jünger & Birte Fähnrich, 2019

Research questions: How can the public engagement of scientists in the context of online communication be conceptualized? Which types of engagement occur in the Twitter activity of communication scholars?

Object of analysis: Tweets and followers belonging to the Twitter profiles of communication scientists who are following the International Communication Association (ICA) on Twitter (only German- and English-speaking users)

Timeframe of analysis: Data collection in September 2017

Info about variables

Variable name/definition: Subject area of the content of the tweets

Level of analysis: Tweet

Values:

- Science-related topics (research, teaching)

- Non-scientific topics (politics, economy, media, sports, environment, society, leisure time, and others)

Scale of measurement: Nominal

Reliability: Gwet’s AC1: 0,71 – 1,00; Holsti: 0,82 – 1,00

Variable name/definition: Language patterns of communication scientists (Speech acts)

Level of analysis: Tweet

Values:

- Actor-centered patterns (discussing, activating, socializing),

- Content-centered patterns (reporting, commenting),

- Other language patterns

Scale of measurement: Nominal

Reliability: Gwet’s AC1: 0,54 – 0,95; Holsti: 0,75 – 1,00

Variable name/definition: References of the communication scientists on Twitter

Level of analysis: Tweet

Values:

- Self-reference,

- Reference to specific actor,

- Reference to other unspecific actor,

- No reference to actors

Scale of measurement: Nominal

Reliability: Gwet’s AC1: 0,83 – 0,87; Holsti: 0,88 – 0,93

Variable name/definition: Type of actor (followers of the investigated scientists)

Level of analysis: Self description in profile

Values: Person, Organization

Scale of measurement: Nominal

Reliability: Gwet’s AC1: 0,89; Holsti: 0,91; Kappa: 0,84; Krippendorffs’ Alpha: 0,84

Variable name/definition: Social sphere of action of the followers

Level of analysis: Self description in profile

Values:

- Science (communication science, other sciences, science in general)

- Politics (party, state/administration, activists & lobbyists)

- Media (media & journalism, news & comments)

- Economy (communication industry, other economic sectors)

- Arts & Entertainment

- Health

- Other (Other areas of activity, personal interests)

Scale of measurement: Nominal

Reliability: Gwet’s AC1: 0,81 – 0,87; Holsti: 0,82 – 0,88; Kappa: 0,83 – 0,85; Krippendorffs’ Alpha: 0,83 – 0,85

Codebook: in the appendix (in German)

Information on Walter, Lörcher & Brüggemann, 2019

Authors: Stefanie Walter, Ines Lörcher & Michael Brüggemann

Research question: How do scientists interact with politicians and civil society on Twitter?

Object of analysis: Climate-related English-language Tweets posted by scientists from the United States (to classify the Twitter users, an automated content analysis, a dictionary approach, was applied; Krippendorffs’ Alpha: 0,74)

Timeframe of analysis: Data collection took place from October 1, 2017 to March 31, 2018

Variable name/definition: Mode and content of communication

Level of analysis: Tweet

Values: Negative emotion, Certainty

Scale of measurement: Linguistic Inquiry and Word Count (LIWC) program for computerized text analysis

Reliability: –

Codebook: in the appendix (R-Script)

Information on Hara et al., 2019

Authors: Noriko Hara, Jessica Abbazio & Kathryn Perkins

Research questions: What kind of demographic characteristics do the scientists participating in “Science” subreddit AMAs have? [survey] What was the experience like to host an AMA in the “Science” subreddit? [survey] What type of discussions did “Science” subreddit AMA participants engage in? Do questions receive answers? What are posters’ intentions? What kind of content features appear? Who is posting comments? What kind of responses do posts receive?

Object of analysis: Six Ask Me Anything (AMA) sessions on Reddit’s “Science” subreddit (r/science)

Timeframe of analysis: –

Info about variable

Variable name/definition: Poster’s intentions (PI); Answer status (AS); Comment status (CS); Poster’s identity (PID); Content features (CF)

Level of analysis: Post

Values:

- PI: Seeking information, Seeking discussion, Non-questions/comments, Further discussion/interaction among users, Answering a question

- AS: Answered, Not answered

- CS: Commented on, Not commented on

- PID: Host, Participant – flair, Participant – no flair

- CF: Providing factual information, Providing opinions, Providing resources, Providing personal experience, Providing guidance on forum governance, Making an inquiry – initial question, Making an inquiry – embedded question, Requesting resources, Off-topic comment

Scale of measurement: Nominal

Reliability: Intercoder reliability ranged between 0.66 and 1.0 calculated by Cohen’s Kappa

Codebook: in the appendix (in English)

References

Albæk, E., Christiansen, P. M., & Togeby, L. (2003). Experts in the mass media: Researchers as sources in Danish daily newspapers, 1961–2001. Journalism & Mass Communication Quarterly, 80(4), 937–948.

Ellison, N. B., & Boyd, D. M. (2013). Sociality through social network sites. In W. H. Dutton, N. B. Ellison, & D. M. Boyd (Eds.), The Oxford Handbook of Internet Studies (pp. 151–172). Oxford: Oxford University Press.

Fähnrich, B., & Lüthje, C. (2017). Roles of Social Scientists in Crisis Media Reporting: The Case of the German Populist Radical Right Movement PEGIDA. Science Communication, 39(4), 415–442.

Hara, N., Abbazio, J., & Perkins, K. (2019). An emerging form of public engagement with science: Ask Me Anything (AMA) sessions on Reddit r/science. PloS One, 14(5), e0216789.

Jahng, M. R., & Lee, N. (2018). When scientists tweet for social changes: Dialogic communication and collective mobilization strategies by flint water study scientists on Twitter. Science Communication, 40(1), 89–108. https://doi.org/10.1177/1075547017751948

Jünger, J., & Fähnrich, B. (2019). Does really no one care?: Analyzing the public engagement of communication scientists on Twitter. New Media & Society, 7(2), 146144481986341.

Klemm, M. (2000). Zuschauerkommunikation: Formen und Funktionen der alltäglichen kommunikativen Fernsehaneignung [Audience Communication: Forms and Functions of Everyday Communicative Appropriation of Television]. Frankfurt am Main: Lang.

Kouper, I. (2010). Science blogs and public engagement with science: Practices, challenges, and opportunities. Journal of Science Communication, 09(01).

Mahrt, M., & Puschmann, C. (2014). Science blogging: An exploratory study of motives, styles, and audience reactions. Journal of Science Communication, 13(03).

Searle, J. R. (1990). Sprechakte: Ein sprachphilosophischer Essay [Speech Acts: An Essay on the Philosophy of Language]. Frankfurt am Main: Suhrkamp.

Walter, S., Lörcher, I., & Brüggemann, M. (2019). Scientific networks on Twitter: Analyzing scientists’ interactions in the climate change debate. Public Understanding of Science, 28(6), 696–712.

Characteristics of university websites (Science Communication)

Nina Wicke — Fri, 26 Mar 2021 00:00:00 +0100

A topic-independent systematic approach according to Deuze (2003) enables to describe websites of universities based on three main characteristics: Hypertextuality, multimediality and interactivity (Metag & Schäfer, 2017). Other important dimensions to characterize a website are multilingualism (e. g. Chapleo et al., 2011) as well as the content of those websites (e. g. Bozyigit & Akkan, 2014), but also their dialogical dimension (e. g. McAllister-Spooner & Kent, 2009) and the prevalence of ethnic and gender diversity (Bal & Sharik, 2019).

Field of application/theoretical foundation:

According to Deuze (2003), whose systematic approach was applied in the content analysis of Metag and Schäfer (2017), three basic characteristics of online communication can be distinguished:

Hypertextuality can be understood as the linking of individual pages on the internet, whereby a differentiation is made between internal (links lead to one's own website) and external hypertextuality (links lead to other websites).
Multimediality means offering information in various formats (e.g. text, or audio and video formats).
The term interactivity encompasses different ways for users to disseminate or access information (Kopper, Kolthoff, & Czepek, 2000), but also different options for producers and users to interact with each other (e.g. giving feedback).

References/combination with other methods of data collection:

In some studies, such as those by Chapleo et al. (2011) or Metag and Schäfer (2017), cluster analyses were carried out following the content analyses. The study by Bal and Sharik (2019) also incorporated enrollment data from the universities in order to compare it with the results of the content analysis and to examine whether the website portrays the actual diversity of students at the university. In other cases, combinations of quantitative and qualitative techniques were applied (Lederbogen & Trebbe, 2003) or a critical discourse analysis (Zhang & O’Halloran, 2013) was conducted.

Example studies:

Bal & Sharik (2019); Bozyigit & Akkan (2014); Chapleo et al. (2011); Else & Crookes (2015); Gordon & Berhow (2009); Lederbogen & Trebbe (2003); McAllister-Spooner & Kent (2009); Metag & Schäfer (2017); Shadinger (2013); Zhang & O'Halloran (2013)

Information on Metag & Schäfer 2017

Authors: Julia Metag, Mike S. Schäfer

Research questions: What is the content of the online communication of German, Austrian and Swiss universities? Is it possible to distinguish different types of universities in terms of their communication methods? Which structural features are constituting the identified types?

Object of analysis: Websites, Facebook and Twitter accounts of all universities which are entitled to award doctorates in Germany (N = 146), Austria (N = 33) and German-speaking Switzerland (N = 11)

Unit of analysis I: Website, Facebook and Twitter presence as a whole

Unit of analysis II: The three largest posts on the website, the first five Facebook posts and the first five tweets of the research week

Timeframe of analysis: 19.-25.05.2014 (Switzerland) & 02.-08.06.2014 (Germany and Austria)

Information about variables

Variable name/definition: Formal categories: Structure of the website

Level of analysis: Website

Values:

- Language: German, English, other

- Ressort structure: Information about studying; information about the university

- Clearly visible navigation point for public/press/media

- Number of employees in the press office

- Multimediality: Number of pictures in general & referring to science; number of video files; number of graphics in general & referring to science; number of audio files

- Hypertextuality and Interactivity: Integration of the Twitter feed or Twitter icon on the home page; integration of the Facebook icon on the home page; integration of other social media icons on the homepage; embedded app for campus life on the homepage

Scale of measurement: Nominal, except for number of employees and Multimediality (metric)

Reliability: intercoder reliability according to Holsti: 0,90 (in total; no variable below 0,71)

Codebook: in the appendix (in German)

Variable name/definition: Content variables for articles on the website

Level of analysis: Articel on website

Values:

- Type of article: Presentation of scientific results, scientific interview with a scientist, scientific news article, event information with scientific relevance, article/interview without scientific reference, event information without scientific reference

- Author

- Multimediality: number of pictures; number of graphics; number of audio files; number of video files

- Topic of the article

- Addressee of the article

- Number of speakers in the article

- Different speakers in the article

- Address (direct/indirect/unclear & form of address)

- Language (scientific/formal/everyday language)

- Hypertextuality and Interactivity: Possibility to link on LinkedIn/blogs; links to other websites/further information; possibility to rate/like/share the article; possibility to give feedback/to comment; number of comments; reactions of university towards e.g. comments; number of comments from university; response time to comments from other users

Scale of measurement: Nominal, except for Multimediality, number of speakers in the article, number of comments (from university), response time (metric)

Reliability: intercoder reliability according to Holsti: 0,90 (in total; no variable below 0,71)

Codebook: in the appendix (in German)

References

Bal, T. L., & Sharik, T. L. (2019). Web content analysis of university forestry and related natural resources landing webpages in the United States in relation to student and faculty diversity. Journal of Forestry, 117(4), 379–397.

Bozyigit, S., & Akkan, E. (2014). Linking universities to the target market via web sites: A content analysis of Turkish private universities’ web sites. Procedia - Social and Behavioral Sciences, 148, 486–493.

Chapleo, C., Carrillo Durán, M. V., Castillo Díaz, A. (2011). Do UK universities communicate their brands effectively through their websites? Journal of Marketing for Higher Education 21(1), 25-46.

Deuze, M. (2003). The Web and its journalisms: Considering the consequences of different types of newsmedia online. New Media & Society, 5(2), 203–230.

Kopper, G. G., Kolthoff, A., & Czepek, A. (2000). Research review: Online Journalism - a report on current and continuing research and major questions in the international discussion. Journalism Studies, 1(3), 499–512.

Lederbogen, U., & Trebbe, J. (2003). Promoting science on the web. Science Communication, 24(3), 333–352.

McAllister-Spooner, S. M., & Kent, M. L. (2009). Dialogic public relations and resource dependency: New Jersey community colleges as models for web site effectiveness. Atlantic Journal of Communication, 17(4), 220–239.

Metag, J., & Schäfer, M. S. (2017). Hochschulen zwischen Social Media-Spezialisten und Online-Verweigerern.: Eine Analyse der Online-Kommunikation promotionsberechtigter Hochschulen in Deutschland, Österreich und der Schweiz [Universities between social media specialists and holdouts. An analysis of universities’ online communication in Germany, Austria, and Switzerland]. Studies in Communication | Media, 6(2), 160–195.

Zhang, Y., & O'Halloran, K. L. (2013). ‘Toward a global knowledge enterprise’: University websites as portals to the ongoing marketization of higher education. Critical Discourse Studies, 10(4), 468–485.

Dialogical strategies of science communicators (Science Communication)

Nina Wicke — Fri, 26 Mar 2021 00:00:00 +0100

According to Taylor et al. (2001), dialogue can be defined "as a tool for effective and mutually rewarding interpersonal communication" (p. 267) and refers to “any negotiated exchange of ideas and opinions” (Kent & Taylor, 1998, p. 325), whereby all parties in a relationship strive to engage in an honest, open and ethically communicative give and take (Bortree & Seltzer, 2009). From a public relations perspective, dialogue is a necessary tool to build an effective relationship with the public (Kent & Taylor, 1998). Dialogical strategies therefore serve to build and maintain dynamic and lasting relationships with publics (Kent & Taylor, 1998).

Field of application/theoretical foundation:

Many studies investigating the dialogical strategies of organizations on the Internet refer in terms of their theoretical foundation to the preliminary work of Kent and Taylor (1998) and Taylor et al. (2001) (e. g. Bortree & Seltzer, 2009; Yang & Taylor, 2010). Kent and Taylor (1998) developed five dimensions, so-called “principles of dialogue” (Taylor et al., 2001, p. 269), which can be utilized by organizations in their strategically designed websites in order to engage in dialogical relationships Among these principles are:

Ease of interface: This point refers to the usability of a site which is a prerequisite for online dialogue. If visitors do not have an easy time navigating a site and finding information, they will not have a positive experience at the website and may not use it again.
Usefulness of information: This principle is relevant because it clarifies the reason for visiting a website. Visitors turn to a site when it provides them with information that is useful, trustworthy, and of lasting value.
Conservation of visitors: The goal of websites should be to keep visitors on their own site and not to encourage them to visit other sites, e. g. through advertisements.
Generation of return visits: A relationship can only be established by users who are encouraged to return and visit the website on a regular basis.
Dialogic loop: Websites have to offer two-way communication (interactivity) in order to be fully dialogic.

Other studies adapted the theoretical framework and modified it for application to different research objects such as social media platforms (Bortree & Seltzer, 2009; Yang & Taylor, 2010). For instance, the study by Bortree & Seltzer (2009) extended it by adding organization engagement as a new strategy, since organizations can also encourage dialogue via posts on their own sites (e. g. walls and discussion boards). Besides the dialogic strategies, their outcomes are often identified and coded as well.

Example studies:

Bortree & Seltzer (2009); Cho et al. (2014); Reber & Kim (2006); Taylor et al. (2001); Waters et al. (2009); Waters & Jamal (2011); Yang & Taylor (2010)

Information on Bortree & Seltzer, 2009

Authors: Denise Sevick Bortree & Trent Seltzer, 2009

Research objective: “This study sought to determine to what degree advocacy organizations are utilizing dialogic strategies on their social networking profiles as well as the degree to which these strategies are related to actual dialogic outcomes” (p. 317).

Objective of analysis: 50 Facebook profiles of environmental advocacy groups

Timeframe of analysis: –

Information about variables

Variable name/definition: Dialogic strategies

Level of analysis: Facebook page

Values:

- Links to organization homepage

- Number of advertisements on a site

- Use of applications

- Ease of donations

- Join now option

- Offers of regular information through email

- Profile sharing

- Content sharing

- Organization comments in dialogic spaces (i. e. wall and discussion boards)

Scale of measurement: Nominal (Present/absent)

Reliability: Intercoder reliability according to Scott’s Pi: .61 to .87

Codebook: –

Variable name/definition: Outcomes of dialogic communication

Level of analysis: Facebook page

Values:

- User posts (number of user posts on wall and discussion board)

- Network activity (number of user posts in one week)

- User responses to others (number of user posts in response to inquiries by the organization or others)

- Organization response to users (number of organization posts in response to user inquiries)

- Network extensiveness (total number of friends or fans)

- Network growth (one week increase in number of friends or fans)

Scale of measurement: Metric

Reliability: Intercoder reliability according to Holsti: 90 % - 100 %

Codebook: –

Information onYang & Taylor, 2010

Authors: Aimei Yang & Maureen Taylor, 2010

Research questions: How do Chinese ENGOs’ websites incorporate features that facilitate interaction? How do Chinese ENGOs’ websites provide information to key stakeholders (members, volunteers, general public, and the media)? How do Chinese ENGOs’ websites incorporate relationship-building features?

Object of analysis: 68 Chinese ENGO’s websites

Timeframe of analysis: –

Information about variable

Level of analysis: Website features

Values:

- Ease of interaction (site map, major link to rest of site, search engine box)

- Usefulness of information to members/volunteers (details of how to become affiliated, how to contribute money, links (email, telephone) to organization leaders, chat room/BBS/Blog, links to affiliate websites)

- Usefulness of information to general public (tips of how to practice environmentally friendly activities in everyday life, tips of how to lead a healthy life, games and other entertainment function)

- Usefulness of information to the media (press release room/search engine, FAQ section aimed at media, news published or aired about the organization, editorial stories written by organization staff, organization-in-action photos/stories, downloadable graphics/video/other material, organization fact sheets, organization logos for use in publication, clearly stated position on policy issues, organization perspective on current issues/trends, annual reports/financial)

- Relationship-building (opportunity for user-response, opportunity to vote on issues, survey to voice opinion on issues, offers regular information through email, positing calendar of events)

- Mission statement (statement of mission/organization value/goals from amoral authority perspective, statement of mission/organization value/goals from a grassroots perspective)

Scale of measurement: Nominal (1 = absence; 2 = linked through at least three levels of hyperlinks from the front page; 3 = linked through at least two levels of hyperlinks from the front page; 4 = placed on the front page)

Reliability: Intercoder reliability according to Scott’s Pi: .77 to 1.00

(M = .84)

Codebook: –

References

Bortree, D. S., & Seltzer, T. (2009). Dialogic strategies and outcomes: An analysis of environmental advocacy groups’ Facebook profiles. Public Relations Review, 35(3), 317–319. https://doi.org/10.1016/j.pubrev.2009.05.002

Cho, M., Schweickart, T., & Haase, A. (2014). Public engagement with nonprofit organizations on Facebook. Public Relations Review, 40(3), 565–567. https://doi.org/10.1016/j.pubrev.2014.01.008

Kent, M. L., Taylor, M. (1998). Building dialogic relationships through the World Wide Web. Public Relations Review, 24(3), 321-334.

Reber, B. H., & Kim, J. K. (2006). How activist groups use websites in media relations: Evaluating online press rooms. Journal of Public Relations Research, 18(4), 313–333. https://doi.org/10.1207/s1532754xjprr1804_2

Taylor, M., Kent, M. L., & White, W. J. (2001). How activist organizations are using the Internet to build relationships. Public Relations Review, 27(3), 263–284. https://doi.org/10.1016/S0363-8111(01)00086-8

Waters, R. D., Burnett, E., Lamm, A., & Lucas, J. (2009). Engaging stakeholders through social networking: How nonprofit organizations are using Facebook. Public Relations Review, 35(2), 102–106. https://doi.org/10.1016/j.pubrev.2009.01.006

Waters, R. D., & Jamal, J. Y. (2011). Tweet, tweet, tweet: A content analysis of nonprofit organizations’ Twitter updates. Public Relations Review, 37(3), 321–324. https://doi.org/10.1016/j.pubrev.2011.03.002

Yang, A., & Taylor, M. (2010). Relationship-building by Chinese ENGOs’ websites: Education, not activation. Public Relations Review, 36(4), 342–351. https://doi.org/10.1016/j.pubrev.2010.07.001

Arguments (User Comments/Interactions)

Teresa K. Naab, Constanze Küchler — Fri, 26 Mar 2021 00:00:00 +0100

The occurrence of ‘arguments’ indicates the quality of statements made in online discussions. User comments below news content on websites or in social media can be examined whether they give reasons for or against positions or assertions. The occurrence of arguments in user comments or entire online discussions can be investigated in various ways depending on the exact research focus. Thus, the occurrence of arguments, their number, quality, or content (what reasons are raised?) can be examined. To determine the occurrence and number of arguments, arguments must first be identified. To ascertain the quality of arguments an evaluation standard is necessary (what constitutes high- or low-quality arguments?).

Field of application/theoretical foundation:

As a complement to various further criteria (e.g. coherence, clarity), the occurrence of arguments in user comments indicates the comments’ contribution to the deliberative quality of online discussions. Normative approaches to discourse ethics (e.g. Habermas, 1992; Steenberger et al., 2003) assume that contributions to a discussion that provide arguments for a position are more valuable to the success of a discussion than contributions that do not provide arguments.

In addition, studies with various other theoretical backgrounds analyze arguments in user comments. For instance, the occurrence of arguments in user comments is interpreted as an indicator for factuality. Factuality, in turn, is regarded as a discussion factor comparable to news factors in journalism research (Ziegele, Breiner & Quiring, 2014).

Example studies:

 Medium
 Measure
 Unit of analysis
 Studies
 

 Online; online discussions below news posts
 Occurence of arguments
 Individual user comment
 Naab & Küchler (work in progress)
Ziegele & Quiring (2015)
 

Info about variables

Variable name/definition: Vorhandensein von Argument(en) in einem Nutzerkommentar

Operationalization/coding instructions: Argumente sind Aussagen, die dazu dienen sollen, Behauptungen zu begründen oder zu widerlegen. Es wird kodiert, ob ein Nutzerkommentar Argumente verwendet, um eine oder mehrere geäußerte(n) Meinung(en) zu begründen.

Von Interesse ist hier, ob die Autor*innen von Nutzerkommentaren ihre eigenen Aussagen mit Argumenten unterstützen oder ob diese unbegründet bleiben. Die Leser*innen müssen sich bei zusammenhängenden Aussagen im Kommentar fragen, ob die Frage nach dem „Warum“ beantwortet wird. Anders gesagt: Gut erkennbar ist ein Argument, wenn man es problemlos mit einem Kausalsatz (weil...) an eine Behauptung anfügen kann oder wenn man eine begründende „Wenn-Dann-Beziehung“ zwischen den Sätzen herstellen kann. Eine Aneinanderreihung von Aussagen zählt nicht als Argumentation.

Es geht hier nicht um die Qualität der Argumente. Es wird nur kodiert, ob Begründungen für Aussagen/Behauptungen/Positionen angeführt werden.

Level of analysis: einzelner Nutzerkommentar

Values: 0/ keine Argumente, 1/ mind. ein Argument, 99/ nicht eindeutig zuzuordnen

References

Naab, T.K. & Küchler, C. (work in progress). Unveröffentlichtes Codebuch aus dem DFG-Projekt „Gegenseitige Sanktionierung unter NutzerInnen von Kommentarbereichen auf Nachrichtenwebseiten und auf Facebook“. Augsburg.

Habermas, J. (1992). Faktizität und Geltung: Beiträge zur Diskurstheorie des Rechts und des demokratischen Rechtsstaates. Suhrkamp.

Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring political deliberation: A discourse quality index. Comparative European Politics, 1(1), 21–48. doi:10.1057/palgrave.cep.6110002

Ziegele, M., Breiner, T., & Quiring, O. (2014). What creates interactivity in online news discussions? An exploratory analysis of discussion factors in user comments on news items. Journal of Communication, 64(6), 1111–1138. doi:10.1111/jcom.12123

Ziegele, M. & Quiring, O. (2015). Codebuch: Der Diskussionswert von Online-Nachrichten. Unveröffentlichtes Codebuch aus dem DFG-Projekt „Vom Nachrichtenwert zum Diskussionswert“. Mainz.

Clarity (User Comments/Interactions)

Teresa K. Naab, Constanze Küchler — Fri, 26 Mar 2021 00:00:00 +0100

The ‘clarity’ of user comments is an indicator for the quality of statements made in online discussions. User comments below news content on websites or in social media can be examined to determine whether they are clearly written, that is comprehensible to the reader in terms of form, style, and content. Clarity of user comments is essential for the contribution of a comment to a discussion and the exchange between commenters.

Field of application/theoretical foundation:

As a complement to various further criteria (e.g. coherence, occurrence of arguments), the variable ‘clarity’ of a user comment indicates the comment’s contribution to the deliberative quality of online discussions. Normative approaches to discourse ethics (e.g. Habermas, 1992) assume that contributions to discussions should be phrased understandably to be more valuable for the success of a discussion.

Example studies:

 Medium
 Measure
 Unit of analysis
 Studies
 

 Online; online discussions below news posts
 Clarity
 Individual user comment
 Naab & Küchler (work in progress)
Ziegele & Quiring (2015)
 

Info about variables

Variable name/definition: Verständlichkeit eines Nutzerkommentars

Operationalization/coding instructions: Es wird kodiert, wie leicht sich Leser*innen der Sinn eines Nutzerkommentars erschließt, wie gut er nachvollziehbar ist. Hierbei zählt der Gesamteindruck. Indikatoren für eine hohe Verständlichkeit sind:

Ein verständlicher Sprachstil
- überwiegend Standardsprache
- ohne akademische Begriffe bzw. Fremdwörter
- Vermeidung von Reimen oder literarischen Schreibformen
- Vermeidung von übertriebenem Cyberslang oder Umgangssprache
- Keine Verwendung von Fremdsprachen
Eine klare, wenig verschachtelte Satzstruktur
Keine auffälligen Rechtschreib- und Grammatikfehler
Eine eindeutige rhetorische Gestaltung durch den Verzicht auf Ironie, Metaphern und abstrakte Bilder
Eine hohe Prägnanz der Aussagen im Sinne der Verbindung von „Bedeutungsreichtum mit einem hohen Maß an Klarheit, Angemessenheit, Anschaulichkeit und Einfachheit“
Verzicht auf das Voraussetzen von speziellem Hintergrundwissen, das beim Durchschnittsleser nicht vorausgesetzt werden kann.

Der Kodierer kann sich die Frage stellen: Wie leicht erschließt sich mir der Sinn des Kommentars, wie nachvollziehbar ist er? „Baseline“ ist der Code „0“.

Level of analysis: einzelner Nutzerkommentar

Values: 0/ normal verständlich, 1/ schwer verständlich, 2/ überhaupt nicht verständlich, 99/ nicht eindeutig zuzuordnen

Intercoder reliability: The variable showed good performance in tests for intercoder agreement (percentage agreement = 86%; Krippendorf’s alpha = .72) in the study by Ziegele, Breiner, & Quiring (2014).

References

Habermas, J. (1992). Faktizität und Geltung: Beiträge zur Diskurstheorie des Rechts und des demokratischen Rechtsstaates. Suhrkamp.

Ziegele, M. & Quiring, O. (2015). Codebuch: Der Diskussionswert von Online-Nachrichten. Unveröffentlichtes Codebuch aus dem DFG-Projekt „Vom Nachrichtenwert zum Diskussionswert“. Mainz.

Number of reply comments (User Comments/Interactions)

Teresa K. Naab, Constanze Küchler — Fri, 26 Mar 2021 00:00:00 +0100

The variable ‘number of reply comments’ is an indicator of interactivity in online discussions. The number of reply comments is a simple measure of how much response a user comment has received. It is applicable if platforms that host comment sections offer the technical possibility to users to respond directly to existing user comments. The reply comments (also called ‘sub-level comments’ or ‘child comments’) then usually appear indented below the existing user comment they refer to (also called ‘top-level comment’ or ‘parent comment’).

The measure does not provide information about the quality of the interaction between the commenters. It neither covers interactivity that occurs “outside” of comment threads below top-level comments, i.e. commenters responding in new top-level comments instead of sub-level comments.

Field of application/theoretical foundation:

Normative approaches to discourse ethics (e.g. Habermas, 1992) evaluate interactivity as a prerequisite for high-quality discourses.

Example studies:

Medium	Measure	Unit of analysis	Studies
Online; online discussions below news posts	Number of reply comments (sub-level comments) to a top-level comment	Individual user comments	Naab & Küchler (work in progress)

Info about variables

Variable name/definition: Anzahl der Antwortkommentare auf einen Nutzerkommentar

Operationalization/coding instructions: Manuell: Zählen Sie die Kind-Kommentare (Antwortkommentare, Second-Level-Kommentare), die zu einem Eltern-Kommentar (Top-Level-Kommentar) verfasst wurden und tragen die Anzahl ein.

Automatisiert: Sofern ein Datensatz alle Nutzerkommentare eines Kommentarthreads, Informationen über das Eltern- bzw. Kind-Level jedes Kommentars sowie eine Zuordnung aller Kind-Kommentare zu einem Eltern-Kommentar enthält, ist es möglich, die Anzahl der Kind-Kommentare für jeden Eltern-Kommentar per Auswertungssoftware zu aggregieren.

Level of analysis: Kommentarthread (Eltern-Kommentar + alle zugehörigen Kind-Kommentare/Antwortkommentare)

References

Habermas, J. (1992). Faktizität und Geltung: Beiträge zur Diskurstheorie des Rechts und des demokratischen Rechtsstaates. Suhrkamp.

Incivility (Hate Speech/Incivility)

Katharina Esau — Fri, 26 Mar 2021 00:00:00 +0100

The variable incivility is an indicator used to describe violations of communication norms. These norms can be social norms established within a society, a culture or parts of a society (e.g. a social class, milieu or group) or democratic norms established within a democratic society. In this sense incivility is associated with behaviors that threaten a collective face or a democratic society, deny people their personal freedoms, and stereotype individuals or social groups. Furthermore, some scholars include impoliteness into the concept of incivility and argue that the two concepts have no clear boundaries (e.g. Seely, 2017). They therefore describe incivility as aggressive, offensive or derogatory communication expressed directly or indirectly to other individuals or parties. In many studies a message is classified as uncivil if the message contains at least one instance of incivility (e.g. one violent threat). The direction of an uncivil statement is coded as ‘interpersonal’/‘personal’ or ‘other-oriented’/‘impersonal’ or sometimes also as ‘neutral’, meaning it is not directed at any group or individual.

Field of application/theoretical foundation:

One unifying element to communication that is labelled as incivility is that it has to be a violation of an existing norm. Which norms are seen as violated depends on the theoretical tradition. Incivility research is related to theories on social norms of communication and conversation: conversational-maxims (Grice, 1975), face-saving concepts (Brown & Levinson, 1987; Goffman, 1989) or conversational-contract theories (Fraser, 1990). Further, incivility research has ties to theories that view public communication as part of democratic opinion formation and decision-making processes, e.g. theories on deliberative democracy and deliberation (Dryzek, 2000; Gutmann & Thompson, 1996; Habermas, 1994).

References/combination with other methods of data collection:

Incivility is examined through content analysis and sometimes combined with comparative designs (e.g., Rowe, 2015) or experimental designs (Muddiman, 2017; Oz, Zheng, & Chen, 2017). In addition, content analyses can be accompanied by interviews or surveys, for example to validate the results of the content analysis (Erjavec & Kovačič, 2012).

Example studies:

Research question/research interest: Previous studies have been interested in the extent, levels and direction of incivility in online communication (e.g. in one specific online discussion, in discussions on a specific topic, in discussions on a specific platform or on different platforms comparatively).

Object of analysis: Previous studies have investigated incivility in user comments on political newsgroups, news websites, social media platforms (e.g. Twitter, Facebook), political blogs, science blogs or online consultation platforms.

Timeframe of analysis: Many studies investigate incivility in user comments focusing on periods between 2 months and 1 year. It is common to use constructed weeks.

Level of analysis: Most manual content analyses measure incivility on the level of a message, for example on the level of user comments. On a higher level of analysis, the level of incivility for a whole discussion thread or online platform can be measured or estimated. On a lower level of analysis incivility can be measured on the level of utterances, sentences or words which are the preferred levels of analysis in automated content analyses.

Table 1. Previous manual content analysis studies and measures of incivility

Example study	Construct	Dimensions/Variables	Explanation/ example	Reliability
Papacharissi (2004)	incivility (separate from impoliteness)	threat to democracy	e.g. propose to overthrow a democratic government by force	I_r = .89
		stereotype	e.g. association of a person with a group by using labels, whether those are mild – “liberal”, or more offensive – “faggot”)?	I_r = .91
		threat to other individuals’ rights	e.g. personal freedom, freedom to speak	I_r = .86
		incivility		I_r = .89
Coe, Kenski, and Rains (2014)	incivility (impoliteness is included)	name-calling	mean-spirited or disparaging words directed at a person or group of people	K-α = .67
		aspersion	mean-spirited or disparaging words directed at an idea, plan, policy, or behavior	K-α = .61
		reference to lying	stating or implying that an idea, plan, or policy was disingenuous	K-α = .73
		vulgarity	using profanity or language that would not be considered proper (e.g., “pissed”, “screw”) in professional discourse	K-α = .91
		pejorative for speech	disparaging remark about the way in which a person communicates	K-α = .74
		incivility / impoliteness		K-α = .73
Rowe (2015)	incivility (separate from impoliteness)	threat to democracy	proposes to overthrow the government (e.g. proposes a revolution) or advocates an armed struggle in opposition to the government (e.g. threatens the use of violence against the government)	κ = .66
		threat to individual rights	advocates restricting the rights or freedoms of certain members of society or certain individuals	κ = .86
		stereotype	asserts a widely held but fixed and oversimplified image or idea of a particular type of person	κ = .80
		incivility		κ = .77
Seely (2017)	incivility (impoliteness is included)	insulting language	name calling and other derogatory remarks often seen in pejorative speech and aspersions	K-α = .84
		vulgarity	e.g. “lazy fkers”, “aholes”	K-α = 1
		stereotyping of political party/ideology	e.g. “typical lying lefties”	K-α = .88
		stereotyping using “isms”/discriminatory language	e.g. “if we don’t get rid of idiotic Muslim theologies, we will have growing problems”	K-α = 1
		other stereotyping language	e.g. “GENERALS LIKE TO HAVE A MALE SOLDIER ON THEIR LAP AT ALL TIMES.”	K-α = .78
		sarcasm	e.g. “betrayed again by the Repub leadership . . . what a shock”	K-α = .79
		accusations of lying	e.g. “typical lying lefties”	K-α = .80
		shouting	excessive capitalization and/or exclamation points	K-α = .83
		incivility / impoliteness		K-α = .81

Note: Previous studies used different inter-coder reliability statistics; I_r = reliability index by Perreault and Leigh (1989); K-α = Krippendorff’s-α; κ = Cohen’s Kappa

Codebook used in the study Rowe (2015) is available under: https://www.tandfonline.com/doi/full/10.1080/1369118X.2014.940365

References

Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage. Cambridge: Cambridge University Press.

Coe, K., Kenski, K., & Rains, S. A. (2014). Online and Uncivil? Patterns and Determinants of Incivility in Newspaper Website Comments. Journal of Communication, 64(4), 658–679. https://doi.org/10.1111/jcom.12104

Dryzek, J. S. (2000). Deliberative democracy and beyond: Liberals, Critics, Contestations. Oxford political theory. Oxford, New York: Oxford University Press.

Erjavec, K., & Kovačič, M. P. (2012). “You Don't Understand, This is a New War! ” Analysis of Hate Speech in News Web Sites' Comments. Mass Communication and Society, 15(6), 899–920. https://doi.org/10.1080/15205436.2011.619679

Fraser, B. (1990). Perspectives on politeness. Journal of Pragmatics, 14(2), 219–236. https://doi.org/10.1016/0378-2166(90)90081-n

Goffman, E. (1989). Interaction ritual: Essays on face-to-face behavior. New York: Pantheon Books.

Grice, P. H. (1975). Logic and conversation. In P. Cole (Ed.), Syntax and Semantics: Speech acts (pp. 41–58). New York: Academic Press.

Gutmann, A., & Thompson, D. F. (1996). Democracy and disagreement. Cambridge, Massachusetts: Belknap Press of Harvard University Press.

Habermas, J. (1994). Three Normative Models of Democracy. Constellations, 1(1), 1–10.

Muddiman, A. (2017). : Personal and public levels of political incivility. International Journal of Communication, 11, 3182–3202.

Oz, M., Zheng, P., & Chen, G. M. (2017). Twitter versus Facebook: Comparing incivility, impoliteness, and deliberative attributes. New Media & Society, 20(9), 3400–3419. https://doi.org/10.1177/1461444817749516

Papacharissi, Z. (2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6(2), 259–283. https://doi.org/10.1177/1461444804041444

Rowe, I. (2015). Civility 2.0: A comparative analysis of incivility in online political discussion. Information, Communication & Society, 18(2), 121–138. https://doi.org/10.1080/1369118X.2014.940365

Seely, N. (2017). Virtual Vitriol: A Comparative Analysis of Incivility Within Political News Discussion Forums. Electronic News, 12(1), 42–61. https://doi.org/10.1177/1931243117739060

Impoliteness (Hate Speech/Incivility)

Katharina Esau — Fri, 26 Mar 2021 00:00:00 +0100

The variable impoliteness is an indicator used to describe violations of communication norms. These norms can be social norms established within a society, a culture or parts of a society (e.g. a social class, milieu or group). In this sense impoliteness is associated with, among other things, aggressive, offensive or derogatory communication expressed directly or indirectly to other individuals or parties. More specifically name calling, vulgar expressions or aspersions are classified as examples of impolite statements (e.g. Papacharissi, 2004; Seely, 2017). While some scholars distinguish between impoliteness and incivility and argue that impoliteness is more spontaneous, unintentional and more frequently regretted than incivility (e.g. Papacharissi, 2004; Rowe, 2015), other scholars include impoliteness into the concept of incivility and argue that the two concepts have no clear boundaries (Coe, Kenski, & Rains, 2014; e.g. Seely, 2017). In many studies a message is classified as impolite if the message contains at least one instance of impoliteness (e.g. a swear word). The direction of an impolite statement is coded as ‘interpersonal’/‘personal’ or ‘other-oriented’/‘impersonal’ or sometimes also as ‘neutral’, meaning it is not directed at any group or individual.

Field of application/theoretical foundation:

Impoliteness is a broader concept of violations of norms in communication that, in digital communication research, is often referred to in studies on incivility. Politeness can be related to theories on social norms of communication and conversation, for example conversational-maxims (Grice, 1975), face-saving concepts (Brown & Levinson, 1987; Goffman, 1989) or conversational-contract theories (Fraser, 1990).

References/combination with other methods of data collection:

Impoliteness is examined through content analysis and is sometimes combined with comparative designs (e.g., Rowe, 2015) or experimental designs (Muddiman, 2017; Oz, Zheng, & Chen, 2017). In addition, content analyses can be accompanied by interviews or surveys, for example to validate the results of the content analysis (Erjavec & Kovačič, 2012).

Example studies:

Research question/research interest: Previous studies have been interested in the extent, levels and direction of impoliteness in online communication (e.g. in one specific online discussion, in discussions on a specific topic, in discussions on a specific platform or on different platforms comparatively).

Object of analysis: Previous studies have investigated impoliteness in user comments on political newsgroups, news websites, social media platforms (e.g. Twitter, Facebook), political blogs, science blogs or online consultation platforms.

Timeframe of analysis: Content analysis studies investigate impoliteness in user comments focusing on periods between 2 months and 1 year (Coe et al., 2014; Rowe, 2015; Seely, 2017). It is common to use constructed weeks.

Level of analysis: Most manual content analysis studies measure impoliteness on the level of a message, for example on the level of user comments. On a higher level of analysis, the level of impoliteness for a whole discussion thread or online platform could be measured or estimated. On a lower level of analysis impoliteness can be measured on the level of utterances, sentences or words which are the preferred levels of analysis in automated content analyses.

Table 1. Previous manual content analysis studies and measures of impoliteness

Example study	Construct	Dimensions/Variables	Explanation/ example	Reliability
Papacharissi (2004)	impoliteness (separate from incivility)	name-calling	e.g. “weirdo”, “traitor”, “crackpot”	I_r = .91
		aspersion	e.g. “reckless”, “irrational”, “un-American”	I_r = .91
		synonyms for liar	e.g. “hoax”, “farce”	N/A
		hyperboles	e.g. “outrageous”, “heinous”	N/A
		non-cooperation	-	N/A
		pejorative speak	-	N/A
		vulgarity	e.g. ”shit”, “damn”, “hell”	I_r = .89
		sarcasm	-	N/A
		all-capital letters	used online to reflect shouting	N/A
		impoliteness		I_r = .90
Coe et al. (2014)	impoliteness (included in incivility)	name-calling	mean-spirited or disparaging words directed at a person or group of people	K-α = .67
		aspersion	mean-spirited or disparaging words directed at an idea, plan, policy, or behavior	K-α = .61
		reference to lying	stating or implying that an idea, plan, or policy was disingenuous	K-α = .73
		vulgarity	using profanity or language that would not be considered proper (e.g., “pissed”, “screw”) in professional discourse	K-α = .91
		pejorative for speech	disparaging remark about the way in which a person communicates	K-α = .74
		impoliteness/incivility		K-α = .73
Rowe (2015)	impoliteness (separate from incivility)	name-calling	e.g., “gun-nut”, “idiot”, “fool”	κ = .82
		aspersion	comments containing an attack on the reputation or integrity of someone or something	κ = .72
		lying	comments implying disingenuousness	N/A
		vulgarity	e.g., “crap”, “shit”, any swear-words/cursing, sexual innuendo	κ = 1
		pejorative	comments containing language which disparage the manner in which someone communicates (e.g., blather, crying, moaning)	κ = 1
		hyperbole	a massive overstatement (e.g., makes pulling teeth with pliers look easy)	κ = .75
		non-cooperation	a situation in a discussion in terms of a stalemate	κ = .66
		sarcasm	-	κ = .71
		other impoliteness	any other type of impoliteness	κ = .72
		impoliteness		κ = .78
Seely (2017)	impoliteness (included in incivility)	insulting language	name calling and other derogatory remarks often seen in pejorative speech and aspersions	K-α = .84
		vulgarity	e.g. “lazy fkers”, “aholes”	K-α = 1
		stereotyping of political party/ideology	e.g. “typical lying lefties”	K-α = .88
		stereotyping using “isms”/discriminatory language	e.g. “if we don’t get rid of idiotic Muslim theologies, we will have growing problems”	K-α = 1
		other stereotyping language	e.g. “GENERALS LIKE TO HAVE A MALE SOLDIER ON THEIR LAP AT ALL TIMES.”	K-α = .78
		sarcasm	e.g. “betrayed again by the Repub leadership . . . what a shock”	K-α = .79
		accusations of lying	e.g. “typical lying lefties”	K-α = .80
		shouting	excessive capitalization and/or exclamation points	K-α = .83
		impoliteness/incivility		K-α = .81

Note: Previous studies used different inter-coder reliability statistics: I_r = reliability index by Perreault and Leigh (1989); K-α = Krippendorff’s-α; κ = Cohen’s Kappa

Codebook used in the study Rowe (2015) is available under: https://www.tandfonline.com/doi/full/10.1080/1369118X.2014.940365

References

Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage. Cambridge: Cambridge University Press.

Fraser, B. (1990). Perspectives on politeness. Journal of Pragmatics, 14(2), 219–236. https://doi.org/10.1016/0378-2166(90)90081-n

Goffman, E. (1989). Interaction ritual: Essays on face-to-face behavior. New York: Pantheon Books.

Grice, P. H. (1975). Logic and conversation. In P. Cole (Ed.), Syntax and Semantics: Speech acts (pp. 41–58). New York: Academic Press.

Muddiman, A. (2017). : Personal and public levels of political incivility. International Journal of Communication, 11, 3182–3202.

Seely, N. (2017). Virtual Vitriol: A Comparative Analysis of Incivility Within Political News Discussion Forums. Electronic News, 12(1), 42–61. https://doi.org/10.1177/1931243117739060

Hate speech (Hate Speech/Incivility)

Katharina Esau — Fri, 26 Mar 2021 00:00:00 +0100

The variable hate speech is an indicator used to describe communication that expresses and/or promotes hatred towards others (Erjavec & Kovačič, 2012; Rosenfeld, 2012; Ziegele, Koehler, & Weber, 2018). A second element is that hate speech is directed against others on the basis of their ethnic or national origin, religion, gender, disability, sexual orientation or political conviction (Erjavec & Kovačič, 2012; Rosenfeld, 2012; Waseem & Hovy, 2016) and typically uses terms to denigrate, degrade and threaten others (Döring & Mohseni, 2020; Gagliardone, Gal, Alves, & Martínez, 2015). Hate speech and incivility are often used synonymously as hateful speech is considered part of incivility (Ziegele et al., 2018).

Field of application/theoretical foundation:

Hate speech (see also incivility) has become an issue of growing concern both in public and academic discourses on user-generated online communication.

References/combination with other methods of data collection:

Hate speech is examined through content analysis and can be combined with comparative or experimental designs (Muddiman, 2017; Oz, Zheng, & Chen, 2017; Rowe, 2015). In addition, content analyses can be accompanied by interviews or surveys, for example to validate the results of the content analysis (Erjavec & Kovačič, 2012).

Example studies:

Research question/research interest: Previous studies have been interested in the extent of hate speech in online communication (e.g. in one specific online discussion, in discussions on a specific topic or discussions on a specific platform or different platforms in comparatively) (Döring & Mohseni, 2020; Poole, Giraud, & Quincey, 2020; Waseem & Hovy, 2016).

Object of analysis: Previous studies have investigated hate speech in user comments for example on news websites, social media platforms (e.g. Twitter) and social live streaming services (e.g. YouTube, YouNow).

Level of analysis: Most manual content analysis studies measure hate speech on the level of a message, for example on the level of user comments. On a higher level of analysis, the level of hate speech for a whole discussion thread or online platform could be measured or estimated. On a lower level of analysis hate speech can be measured on the level of utterances, sentences or words which are the preferred levels of analysis in automated content analyses.

Table 1. Previous manual and automated content analysis studies and measures of hate speech

Example study *(type of content analysis)*	Construct	Dimensions/variables	Explanation/ example	Reliability
Waseem & Hovy (2016) (automated content analysis)	hate speech	sexist or racial slur	-	-
		attack of a minority	-	-
		silencing of a minority		-
		criticizing of a minority without argument or straw man argument	-	-
		promotion of hate speech or violent crime	-	-
		misrepresentation of truth or seeking to distort views on a minority	-	-
		problematic hash tags. e.g. “#BanIslam”, “#whoriental”, “#whitegenocide”	-	-
		negative stereotypes of a minority	-	-
		defending xenophobia or sexism	-	-
		user name that is offensive, as per the previous criteria	-	-
		hate speech	-	κ = .84
Döring & Mohseni (2020) (manual content analysis)	hate speech	explicitly or aggressively sexual hate	e. g. “are you single, and can I lick you?”	κ = .74; PA = .99
Döring & Mohseni (2020) (manual content analysis)	hate speech	racist or sexist hate	e.g. “this is why ignorant whores like you belong in the fucking kitchen”, “oh my god that accent sounds like crappy American”	κ = .66; PA = .99
		hate speech		κ = .70

Note: Previous studies used different inter-coder reliability statistics; κ = Cohen’s Kappa; PA = percentage agreement.

More coded variables with definitions used in the study Döring & Mohseni (2020) are available under: https://osf.io/da8tw/

References

Döring, N., & Mohseni, M. R. (2020). Gendered hate speech in YouTube and YouNow comments: Results of two content analyses. SCM Studies in Communication and Media, 9(1), 62–88. https://doi.org/10.5771/2192-4007-2020-1-62

Gagliardone, I., Gal, D., Alves, T., & Martínez, G. (2015). Countering online hate speech. UNESCO Series on Internet Freedom. Retrieved from http://unesdoc.unesco.org/images/0023/002332/233231e.pdf

Muddiman, A. (2017). : Personal and public levels of political incivility. International Journal of Communication, 11, 3182–3202.

Poole, E., Giraud, E. H., & Quincey, E. de (2020). Tactical interventions in online hate speech: The case of #stopIslam. New Media & Society, 146144482090331. https://doi.org/10.1177/1461444820903319

Rosenfeld, M. (2012). Hate Speech in Constitutional Jurisprudence. In M. Herz & P. Molnar (Eds.), The Content and Context of Hate Speech (pp. 242–289). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139042871.018

Waseem, Z., & Hovy, D. (2016). Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In J. Andreas, E. Choi, & A. Lazaridou (Chairs), Proceedings of the NAACL Student Research Workshop.

Ziegele, M., Koehler, C., & Weber, M. (2018). Socially Destructive? Effects of Negative and Hateful User Comments on Readers’ Donation Behavior toward Refugees and Homeless Persons. Journal of Broadcasting & Electronic Media, 62(4), 636–653. https://doi.org/10.1080/08838151.2018.1532430

Online Incel Speech (Hate Speech/Incivility)

M. Rohangis Mohseni, Jessica Grau Chopite — Sat, 18 Jun 2022 00:00:00 +0200

Involuntarily celibate men (Incels) form online communities in which they “often bemoan their lack of a loving relationship with a woman while simultaneously dehumanizing women and calling for misogynistic violence” (Glace et al., 2021, p. 288). Several studies investigate this dehumanization and misogyny including (gendered) hate speech in online comments from Incels (e.g., Glace et al., 2021). However, not all online comments from Incels contain misogyny or gendered hate speech. To get a better understanding of the phenomenon of Incels, it would be better to not only focus on these problematic comments. Thus, we propose a new construct called “Online Incel speech”, which is defined as the sum of all online comments from Incels that are related to Inceldom, that is, being or becoming an Incel.

In an approach to provide an extensive system of categorization, Grau Chopite (2022) synthesized codebooks from several studies on Incels (see example studies table note) and put it to an empirical test. She found that most Incel comments found online can be categorized into three subdimensions. The first two subdimensions cover framing by Incels, namely how Incels frame the subjective causes of becoming an Incel and how they frame the subjective emotional consequences of being an Incel. Both subdimensions can also be interpreted as part of a subjective theory (sensu Groeben et al., 1988) of Inceldom. In contrast to this, the third subdimension does not consist of framing, but of observable verbal behaviors, which are often linked to gendered hate speech.

When trying to categorize online comments from Incels, former studies often applied the construct “Hybrid Masculinities” (e.g., Glace et al, 2021). This construct from Bridge and Pascoe (2014) suggests that “some men develop masculinities which appear to subvert, but actually reaffirm, White hegemonic masculinities” (Glace et al., 2021, p. 289). Glace et al. (2021) structure the construct into three subdimensions, namely (1) discursive distancing (claiming distance from hegemonic masculine roles without actually relinquishing masculine power), (2) strategic borrowing (appropriating the cultures of nondominant groups of men), and (3) fortifying boundaries (continually using hegemonic standards to constrain masculinity and demeaning men who fail to meet them). However, the construct only covers a part of Inceldom, which Glace et al. (2021) indirectly acknowledge by adding two inductive categories, that is, hostile sexism (shaming and degrading women) and suicidality (reporting suicidal thoughts, feelings, and intentions).

Field of application/theoretical foundation:

The construct “Online Incel speech” was coined by Grau Chopite (2022), and there are currently no other studies making use of it. However, there are studies (e.g., Vu & Lynn, 2020; also see the entry “Frames (Automated Content Analysis”) based on the framing theory by Entman (1991) where the subdimension “subjective causes” would correspond to Entman’s “causal interpretation frame”, while the “subjective emotional consequences” would correspond to Entman’s “problem definition frame”. The “subjective causes” also correspond to the “discursive distancing” and the “emotional consequences” to “suicidality” in the construct of Hybrid Masculinities.

The third subdimension “verbal behavior” corresponds to gendered online hate speech (e.g., Döring & Mohseni, 2019), but also to “hostile sexism” and “fortifying boundaries” in the construct of Hybrid Masculinities.

References/combination with other methods:

The study by Grau Chopite (2022) employs a quantitative manual content analysis using a deductive approach. Studies based on the construct of Hybrid Masculinities also employ manual online content analyses or manual thematic analyses, but those are often qualitative in nature (e.g., Glace et al., 2021).

Framing is also often assessed with manual content analyses (e.g., Nitsch & Lichtenstein, 2019), but newer studies try to assess it computationally (e.g., Vu & Lynn, 2020). Hate speech is often assessed with manual content analyses (e.g., Döring & Mohseni, 2019) and surveys (e.g., Oksanen et al., 2014), but some newer studies try to assess it computationally (e.g., Al-Hassan & Al-Dossari, 2019).

As Online Incel Speech is related to framing and gendered hate speech, it seems plausible that manual content analyses of Online Incel Speech could be combined with computational analyses, too, to enable the investigation of large samples. However, computational analyses of subtle forms of verbal behavior can be challenging because the number of wrong categorizations increases (e.g., for sexism detection see Samory et al., 2021; for hate speech detection see Ruiter et al., 2022).

Example studies:

Example study	Construct	Dimensions	Explanation	Reliability
Online Incel speech
Grau Chopite (2022)	Subjective Causes of Inceldom	Race/Ethnicity	having certain racial features and/or belonging to a certain ethnic	κ = .55; AC1 = .80
		Mental Health	suffering from any mental health issue	κ = .58; AC1 = .90
		Employment	difficulties with getting and/or maintaining employment; experiencing dissatisfaction in the workplace	κ = .85; AC1 = .98
		Family	having family issues (e.g., an abusive family member)	κ = .66; AC1 = .98
	Subjective Emotional Consequences of Inceldom	Hopelessness	expressing hopelessness	κ = .37; AC1 = .89
		Sadness	expressing sadness	κ = .26; AC1 = .91
		Suicidality	expressing suicidality	κ = .24; AC1 = .95
		Anger	expressing anger	κ = .44; AC1 = .87
		Hatred	expressing hatred	κ = .40; AC1 = .83
	Verbal Behavior of Incels	Using Gendered Hate Speech Against Women	hostile sexism against women and misogynistic speech	κ = .80; AC1 = .87
		Adopting Social Justice Language	claiming unfairness/ injustice of being discriminated by society or groups (e.g., other men, other races)	κ = .48; AC1 = .82
		Claiming Lack of Masculine Traits	lacking masculine traits (e.g., muscles, a big penis)	κ = .62; AC1 = .86
		Shaming Other Men	shaming of other men directly by calling them terms related to being “effeminate” or “unmanly”	κ = .71; AC1 = .91
		Claiming Lack of Female Interest	being unable to attract women or being rejected by women	κ = .61; AC1 = .87
Hybrid Masculinities
Glace et al. (2021)	Discursive Distancing	Lack of Female Interest	claiming a lack of ability to attract female romantic companionship and sexual interest	n/a
	Discursive Distancing	Lack of Masculine Traits	claiming a lack of traditionally attractive masculine physical traits	n/a
	Strategic Borrowing	Race and Racism	appropriating the culture of racial and ethnic minority men	n/a
	Strategic Borrowing	Social Justice Language	using the language of the marginalized to diminish one’s own position of power	n/a
	Fortifying Boundaries	Soyboys	deriding non-Incel men as weak and desperate	n/a
	Fortifying Boundaries	Cucks	deriding non-Incel men as being cheated or exploited by women	n/a
	Hostile Sexism	Women are Ugly	deriding women for being unattractive	n/a
		Slut-Shaming	deriding women for having sex	n/a
		False Rape Claims	claiming that women make false rape claims (e.g., when approached by an Incel)	n/a
		Women’s Only Value is Sex	claiming that women’s only value is their sexuality	n/a
		Women are Subhuman	dehumanizing women	n/a
	Suicidality	Due to Incel Experience	attributing suicidal thoughts, feelings, and intentions to Incel status	n/a
	Suicidality	The “Clown World”	claiming that the world is meaningless and nonsensical	n/a

Note: The codebook from Grau Chopite (2022) is based on the codebook and findings of Glace et al. (2021) and other studies (Baele et al., 2019; Bou-Franch & Garcés-Conejos Blitvich, 2021; Bridges & Pascoe, 2014; Cottee, 2020; Döring & Mohseni, 2019; D’Souza et al., 2018; Marwick & Caplan, 2018; Mattheis & Waltman, 2021; Maxwell et al., 2020; Rogers et al., 2015; Rouda & Siegel, 2020; Scaptura & Boyle, 2019; Williams & Arntfield, 2020; Williams et al., 2021). Gwet’s AC1 was calculated in addition to Cohen’s Kappa because some categories were rarely coded, which biases Cohen’s Kappa. The codebook is available at http://doi.org/10.23668/psycharchives.5626

References

Al-Hassan, A., & Al-Dossari, Hmood (2019). Detection of hate speech in social networks: A survey on multilingual corpus. In D. Nagamalai & D. C. Wyld (Eds.), Computer Science & Information Technology. Proceedings of the 6th International Conference on Computer Science and Information Technology (pp. 83–100). AIRCC Publishing. doi:10.5121/csit.2019.90208

Baele, S. J., Brace, L., & Coan, T. G. (2019). From “Incel” to “Saint”: Analyzing the violent worldview behind the 2018 Toronto attack. Terrorism and Political Violence, 1–25. doi:10.1080/09546553.2019.1638256

Bou-Franch, P., & Garcés-Conejos Blitvich, P. (2021). Gender ideology and social identity processes in online language aggression against women. In R. M. DeKeyser (Ed.), Benjamins Current Topics: Vol. 116. Aptitude-Treatment Interaction in Second Language Learning (Vol. 86, pp. 59–81). John Benjamins Publishing Company. doi:10.1075/bct.86.03bou

Bridges, T., & Pascoe, C. J. (2014). Hybrid masculinities: New directions in the sociology of men and masculinities. Sociology Compass, 8(3), 246–258. doi:10.1111/soc4.12134

Cottee, S. (2021). Incel (e)motives: Resentment, shame and revenge. Studies in Conflict & Terrorism, 44(2), 93–114. doi:10.1080/1057610X.2020.1822589

Döring, N., & Mohseni, M. R. (2018). Male dominance and sexism on YouTube: Results of three content analyses. Feminist Media Studies, 19(4), 512–524. doi:10.1080/14680777.2018.1467945

D'Souza, T., Griffin, L., Shackelton, N., & Walt, D. (2018). Harming women with words: The failure of Australian law to prohibit gendered hate speech. University of New South Wales Law Journal, 41(3), 939–976.

Entman, R. M. 1991. Framing U.S. coverage of international news: contrasts in narratives of the KAL and Iran Air incidents. Journal of Communication, 41(4), 6-7.

Glace, A. M., Dover, T. L., & Zatkin, J. G. (2021). Taking the black pill: An empirical analysis of the “Incel”. Psychology of Men & Masculinities, 22(2), 288–297. doi:10.1037/men0000328

Grau Chopite, J. (2022). Framing of Inceldom on incels.is: A content analysis [Master’s thesis, TU Ilmenau]. Psycharchives. doi:10.23668/psycharchives.5626

Groeben, N., Wahl, D., Schlee, J., & Scheele, B. (Eds.). (1988). Das Forschungsprogramm Subjektive Theorien: eine Einführung in die Psychologie des reflexiven Subjekts. Francke. Retrieved from https://nbn-resolving.org/urn:nbn:de:0168-ssoar-27658

Marwick, A. E., & Caplan, R. (2018). Drinking male tears: language, the manosphere, and networked harassment. Feminist Media Studies, 18(4), 543–559. doi:10.1080/14680777.2018.1450568

Mattheis, A. A., & Waltman, M. S. (2021). Gendered hate online. In K. Ross & I. Bachmann (Eds.), The Wiley Blackwell-ICA international encyclopedias of communication. The international encyclopedia of gender, media, and communication (pp. 1–5). John Wiley & Sons Inc. doi:10.1002/9781119429128.iegmc019

Maxwell, D., Robinson, S. R., Williams, J. R., & Keaton, C. (2020). “A short story of a lonely guy”: A qualitative thematic analysis of involuntary celibacy using Reddit. Sexuality & Culture, 24(6), 1852–1874. doi:10.1007/s12119-020-09724-6

Nitsch, C. & Lichtenstein, D. (2019). Satirizing international crises. The depiction of the Ukraine, Greek debt and migration crises in political satire. Studies in Communication Science (SComS), 19(1), 85-103. doi:10.24434/j.scoms.2019.01.007

Oksanen, A., Hawdon, J., Holkeri, E., Näsi, M., & Räsänen, P. (2014). Exposure to online hate among young social media users. In N. Warehime (Ed.), Soul of Society: A focus on the lives of children & youth (p. 253-273). doi:10.1108/S1537-466120140000018021

Rogers, D. L., Cervantes, E., & Espinosa, J. C. (2015). Development and validation of the belief in female sexual deceptiveness scale. Journal of Interpersonal Violence, 30(5), 744–761. doi:10.1177/0886260514536282

Rouda, B., & Siegel, A. (2020). I’d kill for a girl like that”: The black pill and the Incel uprising. International Multidisciplinary Program in the Humanities, Tel Aviv University. Retrieved from https://www.academia.edu/43663741/_Id_kill_for_a_girl_like_that_The_Black_Pill_and_the_Incel_Uprising

Ruiter, D., Reiners, L., Geet D’Sa, A., Kleinbauer, Th., Fohr, D., Illina, I., Klakow. D., Schemer, Ch., & Monnier, A. (2022). Placing m-phasis on the plurality of hate. A feature-based corpus of hate online. Preprint. Retrieved from https://doi.org/10.48550/arXiv.2204.13400

Samory, M., Sen, I., Kohne, J., Flöck, F., & Wagner, C. (2021). “Call me sexist, but...”: Revisiting sexism detection using psychological scales and adversarial samples. Proceedings of the International AAAI Conference on Web and Social Media, 15(1), 573-584. Retrieved from https://ojs.aaai.org/index.php/ICWSM/article/view/18085

Scaptura, M. N., & Boyle, K. M. (2019). Masculinity threat, “Incel” traits, and violent fantasies among heterosexual men in the United States. Feminist Criminology, 15(3), 278–298. doi:10.1177/1557085119896415

Vu, H. T., & Lynn, N. (2020). When the news takes sides: Automated framing analysis of news coverage of the Rohingya crisis by the elite press from three countries. Journalism Studies. Online first publication. doi:10.1080/1461670X.2020.1745665

Williams, D. J., & Arntfield, M. (2020). Extreme sex-negativity: An examination of helplessness, hopelessness, and misattribution of blame among “Incel” multiple homicide offenders. Journal of Positive Sexuality, 6(1), 33–42. doi:10.51681/1.613

Williams, D. J., Arntfield, M., Schaal, K., & Vincent, J. (2021). Wanting sex and willing to kill: Examining demographic and cognitive characteristics of violent "involuntary celibates". Behavioral Sciences & the Law, 39(4), 386–401. doi:10.1002/bsl.2512

Theoretical typology of deceptive content (Conspiracy Theories)

Jing Zeng — Fri, 26 Mar 2021 00:00:00 +0100

The conceptual fuzziness of terms like misinformation, disinformation, rumour, gossip, conspiracy theories has been discussed by various scholars (e.g. DiFonzo & Bordia, 2007; Rojecki & Meraz, 2016). In both academic research and media reports, it is common to see these terms being used interchangeably. To develop better understanding of how and why different forms of misinformation operate, it is important to clarify the conceptual boundaries between these terms in a meaningful way.

Field of Application/Theoretical foundation:

In their social psychology research, DiFonzo and Bordia (2007) propose an effective way to differentiate rumour from other terms, which emphasises the content, function, and context of each concept. This three-dimensional framework can be applied to systematically differentiate concepts related to misinformation.

In the field of media and communication studies, as research on digital misinformation continues to grow, it is crucial for researchers to understand the contexts of each concept and choose the appropriate term in accordance with their research agenda. It is worth noting that there are also overlapping dimensions between these concepts. For instance, rumour can contain misinformation, and conspiracy theories can be used for propaganda.

Example study:

Zeng (2018)

Information on Zeng, 2018

Author: Jing Zeng

Interest of the study: In her study on online rumours during times of crises, Zeng (2018) applies DiFonzo and Bordia’s (2007) framework to differentiate seven seemingly similar concepts: misinformation, disinformation, rumour, gossip, urban legend, propaganda, and conspiracy theories.

Table 1. Summary of main features of seven concepts related to misinformation

	Content	Context	Function
Misinformation	Inaccurate information	Any circumstances of information circulation.	Does not have to have any specific function
Gossip	Talk about individuals or private behaviour	Shared between individuals or in casual social events.	Maintaining network, interpersonal relations
Urban legend	Meaning-making, value-endorsing stories related to contemporary life	Casual setting for storytelling.	Promote cultural and moral values
Disinformation	Deliberately deceptive information	Disseminated by institutions.	Undermine public support
Propaganda	Messages instrumentally disseminated to promote a political cause	Following a top-down pattern, disseminated by official sources.	Promote political and ideological values
Rumour	Unofficial information unverified by authorities	Circulated in circumstances of ambiguity, danger or threat.	Make sense of an uncertain circumstance
Conspiracy theory	Proposed explanations of an event or a practice that refer to the machinations of influential people, institutions, or a secret society	Emerged in responses to uncertainty and perceived threats posed by a coalition of elites/secret actors, and constructed as ‘alternative’ explanations challenging narratives provided by governments, mainstream media or scientific institutions.	Serves as a threat management response, and often also as an anti-establishment/anti-science, political and ideological stance

Misinformation is the most generic one among these terms, in the sense that it does not emphasise the social and political dimension of information. The concept of misinformation is mostly discussed in tandem with information, particularly by Information Science scholars. As a form of information, the defining feature of misinformation is its inaccuracy. Such inaccuracy is not necessarily caused by false information, but can also be caused by irrelevant and incomplete information.

Gossip is also a form of unverified information; however, the content is more private, and is mostly circulated in an interpersonal context (DiFonzo & Bordia, 2007; Rojecki & Meraz, 2016). In terms of function, instead of sense making, gossip is propagated for social purposes. As summarised by Foster (2004), gossip functions to achieve the formation, adaptation, and maintenance of social networks. This is to say, individuals share gossip in the context of managing their relationship with members within their social group. For instance, early literature on gossip associated the practice of gossiping with female social networking. As Rysman (1977) pointed out, one key reason behind the patriarchal criticism on female gossiping is gossip’s ability ‘to develop social ties outside the institution of male dominance’ (p. 176). This personal and interpersonal focus on the concept of gossip is its most distinctive feature.

Disinformation and propaganda are two very closely related concepts, because they are both disseminated for political purposes (Lewandowsky, Stritzke, & Freund, 2013). In terms of content, disinformation is deliberately deceptive information that is used to undermine public support of a regime, whereas propaganda is information used to mobilise public support for a political cause or a regime (Rojecki & Meraz, 2016). The word disinformation originated from ‘dezinformacija’, a Russian term coined in the former Soviet Union (Karlova & Fisher, 2013). Given the particular cultural and political context in the region at that time, this term was originally closely associated with intelligence operations and political tactics. However, the definition of disinformation has, over time, expanded to include the propagation of misinformation that is non-politically motivated.

Urban legends are contemporary tales that are shared to promote moral and cultural values (DiFonzo & Bordia, 2007). Construed in this way, urban legends are similar to propaganda in the sense that they encode and transmit values, but they are used mythologically rather than strategically. Furthermore, where propaganda emphasises political and ideological values, urban legends focus on cultural and moral values. One key criterion for an urban legend is that it must be grounded in the day-to-day affairs of contemporary life, hence the ‘urban’ in urban legend (Bennett & Smith, 2013). A classic example of an urban legend is the claim that a tooth left in a glass of Coca Cola can dissolve overnight. This story promotes health messages to avoid excessive consumption of soda drinks. Another case is the tale of a drugged traveller awakening in an ice-filled bathtub, only to discover one of his kidneys has been removed by organ thieves. This tale echoes a classic form of legend that teaches the moral lesson to avoid dangerous situations.

Rumour, in terms of content, is unofficial information – that is, information whose authenticity is not verified by an appeal to authority. By this definition, the defining characteristic of rumour content is not its falsity, but its ‘unofficial’ status and therefore its relationship to social institutions (Fine, 2007). Second, rumour arises in contexts that are ambiguous, threatening or potentially threatening (DiFonzo & Bordia, 2007, p. 20). In such uncertain contexts, rumour functions to make sense of the unknown situation. This may explain why rumour goes hand in hand with crisis events, during which there is often a paucity of information and a state of anxiety among the populace.

Conspiracy theories’ content is proposed explanations of an event or a practice that refer to the machinations of powerful people, institutions, or a secret society (e.g. Coady, 2003; Goertzel, 1994; Keeley, 1999). One distinctive feature of conspiracy theories is its reference to a coordinated group of deliberate actors. For instance, in anti-vaccination conspiracy theories, ‘big pharma’ companies are accused of conspiring with politicians; likewise, flat earthers implicate NASA in the plot to keep the truth about the “true nature” of the Earth from the public. Similar to rumour mongering, the context in which conspiracy theories emerge is often one of uncertainty and perceived risks, and conspiracy theorising represents a form of ‘alternative’ collective sensemaking to challenge established narratives provided by the mainstream media and institutions. Therefore, conspiracy theorising is not merely sensemaking, but also has anti-establishment and anti-science undercurrents. Therefore, in terms of function, conspiracy theories serve as (1) a threat management response to ‘secret coalitions’ that are perceived to pose direct threats to the collective well-being, health, and safety of the society (van Prooijen et al., 2018); and (2) a political and ideological stance (Hofstadter, 2012).

References

Bennett, G., & Smith, P. (2013). Contemporary legend: A reader. London: Routledge.

Coady, D. (2003). Conspiracy theories and official stories. International Journal of Applied Philosophy, 17(2), 197–209.

DiFonzo, N., & Bordia, P. (2007). Rumor, gossip and urban legends. Diogenes, 54(1), 19-35.

Fine, G. A. (2007). Rumor, trust and civil society: Collective memory and cultures of judgment. Diogenes, 54(1), 5-18.

Foster, E. K. (2004). Research on gossip: Taxonomy, methods, and future directions. Review of General Psychology, 8(2), 78-99.

Goertzel, T. (1994). Belief in conspiracy theories. Political Psychology, 15, 731–742.

Hofstadter, R. (2012). The paranoid style in American politics. New York: Vintage.

Karlova, N. A., & Fisher, K. E. (2013). A social diffusion model of misinformation and disinformation for understanding human information behaviour. Information Research, 18(1), paper 573.

Keeley, B. L. (1999). Of conspiracy theories. The Journal of Philosophy, 96(3), 109–126.

Lewandowsky, S., Stritzke, W. G. K., Freund, A. M, Oberauer, K., and Krueger, J. I. (2013). Misinformation, disinformation, and violent conflict: From Iraq and the ‘War on Terror’ to future threats to peace. American Psychologist, 68(7), 487-501.

Losee, R. M. (1997). A discipline independent definition of information. Journal of the American Society for Information Science, 48(3), 254-269

Rojecki, A., & Meraz, S. (2016). Rumors and factitious informational blends: The role of the web in speculative politics. New Media & Society, 18(1), 25-43.

Rysman, A. (1977). How the ‘gossip’ became a woman. Journal of Communication, 27(1), 176-180.

van Prooijen, J. W., & Van Vugt, M. (2018). Conspiracy theories: Evolved functions and psychological mechanisms. Perspectives on psychological science, 13(6), 770-788.

Zeng, J. (2018). Contesting rumours on social media during acute events: The 2014 Sydney siege and 2015 Tianjin blasts (Doctoral dissertation, Queensland University of Technology). Available at https://eprints.qut.edu.au/115786/.

Alcohol Portrayals on Social Media (Social Media)

Nicola Döring, Roberto Walter — Fri, 27 May 2022 00:00:00 +0200

The depiction of alcohol is the focus of a growing number of content analyses in the field of social media research. Typically, the occurrence and nature of alcohol representations are coded to measure the prevalence, normalization, or even glorification of alcohol and its consumption on different social media platforms (Moreno et al., 2016; Westgate & Holliday, 2016) and smartphone apps (Ghassemlou et al., 2020). But social media platforms and smartphone apps also play a role in the prevention of alcohol abuse when they disseminate messages about alcohol risks and foster harm reduction, abstinence, and sobriety (Davey, 2021; Döring & Holz, 2021; Tamersoy et al., 2015; Westgate & Holliday, 2016).

Field of application/theoretical foundation:

Social Cognitive Theory (SCT; Bandura 1986, 2009) as the dominant media effects theory in communication science, is applicable and widely applied to social media representations of alcohol: According to SCT, positive media portayals of alcohol and attractive role models consuming alcohol can influence the audience’s relation to alcohol. That’s why positive alcohol portayals in the media are considered a public health threat as they can foster increased and risky alcohol consumption among media users in general and young people in particular. The negative health impact predicted by SCT depends on different aspects of alcohol portrayals on social media that have been traditionally coded in manual content analyses (Beullens & Schepers, 2013; Mayrhofer & Naderer, 2019; Moreno et al., 2010) and most recently by studies relying on computational methods for content analysis (e.g. Ricard & Hassanpour, 2021). Core aspects of alcohol representations on social media are: a) the type of communicator / creator of alcohol-related social media content, b) the overall valence of the alcohol portrayal, c) the people consuming alcohol, d) the alcohol consumption behaviors, e) the social contexts of alcohol consumption, f) the types and brands of consumed alcohol, g) the consequences of alcohol consumption, and h) alcohol-related consumer protection messages in alcohol marketing (Moreno et al., 2016; Westgate & Holliday, 2016).

For example, a normalizing portrayal shows alcohol consumption as a regular and normal behavior of diverse people in different contexts, while a glorifying portrayal shows alcohol consumption as a behavior that is strongly related to positive effects such as having fun, enjoying social community, feeling sexy, happy, and carefree (Griffiths & Casswell, 2011). While criticism of glorifying alcohol portrayals in entertainment media (e.g., music videos; Cranwell et al., 2015), television (e.g., Barker et al., 2021), and advertising (e.g., Curtis et al., 2018; Stautz et al., 2016) has a long tradition, the concern about alcohol representations on social media is relatively new and entails the phenomenon of alcohol brands and social media influencers marketing alcohol (Critchlow & Moodie, 2022; Turnwald et al., 2022) as well as ordinary social media users providing alcohol-related self-presentations (e.g., showing themselves partying and drinking; Boyle et al., 2016). Such alcohol-related self-presentations might elicit even stronger identification and imitation effects among social media audiences compared to regular advertising (Griffiths & Casswell, 2011).

Because of its psychological and health impact, alcohol-related social media content – and alcohol marketing in particular – is also an issue of legal regulation. The World Health Organization states that “Europe is the heaviest-drinking region in the world” and strongly advocates for bans or at least stricter regulations of alcohol marketing both offline and online (WHO, 2020, p. 1). At the same time, the WHO points to the problem of clearly differentiating between alcohol marketing and other types of alcohol representations on social media.

Apart from normalizing and glorifying alcohol portayals, there are also anti-alcohol posts and comments on social media. They usually point to the health risks of alcohol consumption and the dangers of alcohol addiction and, hence, try to foster harm reduction, abstincence and sobriety. While such negative alcohol portayals populate different social media platforms, an in-depth investigation of the spread, scope and content of anti-alcohol messages on social media is largely missing (Davey, 2021; Döring & Holz, 2021; Tamersoy et al., 2015).

References/combination with other methods of data collection:

Manual and computational content analyses of alcohol representations on social media platforms can be complemented by qualitative interview and quantitative survey data addressing alcohol-related beliefs and behaviors collected from social media users who a) create and publish alcohol-related social media content and/or b) are exposed to or actively search for and follow alcohol-related social media content (e.g., Ricard & Hassanpour, 2021; Strowger & Braitman, 2022). Furthermore, experimental studies are helpful to directly measure how different alcohol-related social media posts and comments are perceived and evaluated by recipients and if and how they can affect their alcohol-related thoughts, feelings, and behaviors (Noel, 2021). Such social media experiments can build on respective mass media experiments (e.g., Mayrhofer & Naderer, 2019). Insights from content analyses help to select or create appropriate stimuli for such experiments. Last but not least, to evaluate the effectiveness of alcohol marketing regulations, social media content analyses conducted within a longitudinal or trend study design (including measurements before and after new regulations came into effect) should be preferred over cross-sectional studies (e.g., Chapoton et al., 2020).

Example Studies for Manual Content Analyses:

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
a) Creators of alcohol-related social media content
Extensive explorations on Facebook, Instagram and TikTok	Creators of alcohol-related social media content on Facebook, Instagram and TikTok	Polytomous variable “Type of content creator” (1: alcohol industry; 2: media organization/media professional; 3: health organization/health professional; 4: social media influencer; 5: ordinary social media user; 6: other)	Not available	Döring & Tröger (2018) Döring & Holz (2021)
b) Valence of alcohol-related social media content
N = 3 015 Facebook comments N = 100 TikTok videos	Valence of alcohol-related social media content (posts or comments)	Binary variable “Valence of alcohol-related social media content” (1: positive/pro-alcohol sentiment; 2: negative/anti-alcohol sentiment)	Cohen’s Kappa average of .72 for all alcohol-related variables in codebook*	Döring & Holz (2021) *Russell et al. (2021)
c) People consuming alcohol
N = 160 Facebook profiles (profile pictures, personal photos, and text)	Portrayal of people consuming alcohol on Facebook profiles	Binary variable “Number of persons on picture” (1: alone; 2: with others)	Cohen’s Kappa > .90	Beullens & Schepers (2013)
d) Alcohol consumption behaviors
N = 160 Facebook profiles (profile pictures, personal photos, and text)	Type of depicted alcohol use/consumption	Polytomous variable “Type of depicted alcohol use/consumption” (1: explicit use such as depiction of person drinking alcohol; 2: implicit use such as depiction of alcohol bottle on table; 3: alcohol logo only)	Cohen’s Kappa = .89	Beullens & Schepers (2013)
N = 100 TikTok videos	Multiple alcoholic drinks consumed per person	Binary variable “Multiple alcoholic drinks consumed per person” as opposed to having only one drink or no drink per person (1: present; 2: not present)	Cohen’s Kappa average of .72 for all alcohol-related variables in codebook	Russell et al. (2021)
N = 100 TikTok videos	Alcohol intoxication	Binary variable “Alcohol intoxication” (1: present; 2: not present)	Cohen’s Kappa average of .72 for all alcohol-related variables in codebook	Russell et al. (2021)
N = 4 800 alcohol-related Tweets	Alcohol mentioned in combination with other substance use	Binary variable “Alcohol mentioned in combination with tobacco, marijuana, or other drugs” (1: yes; 2: no)	Cohen’s Kappa median of .73 for all pro-drinking variables in codebook	Cavazos-Rehg et al. (2015)
e) Social contexts of alcohol consumption
N = 192 Facebook and Instagram profiles (profile pictures, personal photos, and text)	Portrayal of social evaluative contexts of alcohol consumption on Facebook and Instagram profiles	Polytomous variable “Social evaluative context” (1: negative context such as someone looking disapprovingly at a drunk person; 2: neutral context such as no explicit judgment or emotion is shown; 3: positive context such as people laughing and toasting with alcoholic drinks)	Cohen’s Kappa ranging from .68 to .91 for all variables in codebook	Hendriks et al. (2018), based on previous work by Beullens & Schepers (2013)
N = 51 episodes with a total of N = 1 895 scenes of the American adolescent drama series “The OC”	Portrayal of situational contexts of alcohol consumption in scenes of a TV series	Polytomous variable “Setting of alcohol consumption” (1: at home; 2: at adult / youth party; 3: in a bar; 4: at work; 5: at other public place) Polytomous variable “Reason of alcohol consumption” (1: celebrating/partying; 2: habit; 3: stress relief; 4: social facilitation)	Cohen’s Kappa for setting of alcohol consumption .90 Cohen’s Kappa for reason of alcohol consumption .71	Van den Bulck et al. (2008)
f) Types and brands of consumed alcohol
N = 17 800 posts of Instagram influencers and related comments	Portrayal of different alcohol types and alcohol brands in Instagram posts	Polytomous variable “Alcohol type” (1: wine; 2: beer; 3: cocktails; 4: spirits; 5: non-alcoholic drinks/0% alcohol) Binary variable “Alcohol brand visibility” (1: present if full brand name, recognizable logo, or brand name in header or tag are visible; 2: non-present) String variable “Alcohol brand name” (open text coding)	Krippendorff’s Alpha ranging from .69 to 1.00 for all variables in codebook	Hendriks et al. (2019)
g) Consequences of alcohol consumption
N = 400 randomly selected public MySpace profiles	Portayal of consequences of alcohol consumption on MySpace profiles	Five individually coded binary variables for different consequences associated with alcohol use (1: present; 2: not present): a) “Positive emotional consequence highlighting positive mood, feeling or emotion associated with alcohol use” b) “Negative emotional consequence highlighting negative mood, feeling or emotion associated with alcohol use” c) “Positive social consequences highlighting perceived social gain associated with alcohol use” d) “Negative social consequences highlighting perceived poor social outcomes associated with alcohol use” e) “Negative physical consequences describing adverse physical consequences or outcomes associated with alcohol use”	Cohen’s Kappa ranging from 0.76 to 0.82 for alcohol references and alcohol use	Moreno et al. (2010)
h) Alcohol-related consumer protection messages in alcohol marketing
N = 554 Tweets collected from 13 Twitter accounts of alcohol companies in Ireland	Alcohol-related consumer protection messages in alcohol marketing (covers both mandatory and voluntary messages depending on national legislation)	Four individually coded binary variables for different alcohol-related consumer protection messages in alcohol marketing (1: present; 2: not present): a) “Warning about the risks/danger of alcohol consumption” b) “Warning about the risks/danger of alcohol consumption when pregnant” c) “Warning about the link between alcohol consumption and fatal cancers” d) “Link/reference to website with public health information about alcohol”	Not available	Critchlow & Moodie (2022)

The presented measures were developed for specific social media platforms, but are so generic that they can be used across different social media platforms and even across mass media channels such as TV, cinema, and advertisement. The presented measures cover different aspects of media portrayals of alcohol and can be used individually or in combination. Depending on the research aim, more detailed measures can be developed and added: for example, regarding the media portrayal of people consuming alcohol, additional measures can code people’s age, gender, ethnicity and further characteristics relevant to the respective research question. In the course of a growing body of content analyses addressing alcohol-related prevention messages on social media, respective measures can be added as well.

References

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.

Bandura, A. (2009). Social cognitive theory of mass communication. In J. Bryant & M. B. Oliver (Eds.), Communication series. Media effects: Advances in theory and research (3rd ed., pp. 94–124). Routledge.

Barker, A. B., Britton, J., Thomson, E., & Murray, R. L. (2021). Tobacco and alcohol content in soap operas broadcast on UK television: A content analysis and population exposure. Journal of Public Health (Oxford, England), 43(3), 595–603. https://doi.org/10.1093/pubmed/fdaa091

Boyle, S. C., LaBrie, J. W., Froidevaux, N. M., & Witkovic, Y. D. (2016). Different digital paths to the keg? How exposure to peers' alcohol-related social media content influences drinking among male and female first-year college students. Addictive Behaviors, 57, 21–29. https://doi.org/10.1016/j.addbeh.2016.01.011

Beullens, K., & Schepers, A. (2013). Display of alcohol use on Facebook: A content analysis. Cyberpsychology, Behavior and Social Networking, 16(7), 497–503. https://doi.org/10.1089/cyber.2013.0044

Cavazos-Rehg, P. A., Krauss, M. J., Sowles, S. J., & Bierut, L. J. (2015). "Hey everyone, I'm drunk." An evaluation of drinking-related Twitter chatter. Journal of Studies on Alcohol and Drugs, 76(4), 635–643. https://doi.org/10.15288/jsad.2015.76.635

Chapoton, B., Werlen, A.‑L., & Regnier Denois, V. (2020). Alcohol in TV series popular with teens: A content analysis of TV series in France 22 years after a restrictive law. European Journal of Public Health, 30(2), 363–368. https://doi.org/10.1093/eurpub/ckz163

Cranwell, J., Murray, R., Lewis, S., Leonardi-Bee, J., Dockrell, M., & Britton, J. (2015). Adolescents' exposure to tobacco and alcohol content in YouTube music videos. Addiction (Abingdon, England), 110(4), 703–711. https://doi.org/10.1111/add.12835

Critchlow, N., & Moodie, C. (2022). Consumer protection messages in alcohol marketing on Twitter in Ireland: A content analysis. Drugs: Education, Prevention and Policy, 1–8. https://doi.org/10.1080/09687637.2022.2028730

Curtis, B. L., Lookatch, S. J., Ramo, D. E., McKay, J. R., Feinn, R. S., & Kranzler, H. R. (2018). Meta-analysis of the association of alcohol-related social media use with alcohol consumption and alcohol-related problems in adolescents and young adults. Alcoholism, Clinical and Experimental Research, 42(6), 978–986. https://doi.org/10.1111/acer.13642

Davey, C. (2021). Online sobriety communities for women's problematic alcohol use: A mini review of existing qualitative and quantitative research. Frontiers in Global Women's Health, 2, 773921. https://doi.org/10.3389/fgwh.2021.773921

Döring, N., & Tröger, C. (2018). Zwischenbericht: Durchführung und Ergebnisse der summativen Evaluation des Facebook-Kanals „Alkohol? Kenn dein Limit.“ [Intermediate report: Implementation and results of the summative evaluation of the Facebook channel "Alcohol? Know your limit."].

Döring, N., & Holz, C. (2021). Alkohol in sozialen Medien: Wo ist der Platz für Prävention? [Alcohol in social media: Where is the space for prevention?]. Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz, 64(6), 697–706. https://doi.org/10.1007/s00103-021-03335-8

Ghassemlou, S., Marini, C., Chemi, C., Ranjit, Y. S., & Tofighi, B. (2020). Harmful smartphone applications promoting alcohol and illicit substance use: A review and content analysis in the United States. Translational Behavioral Medicine, 10(5), 1233–1242. https://doi.org/10.1093/tbm/ibz135

Griffiths, R., & Casswell, S. (2010). Intoxigenic digital spaces? Youth, social networking sites and alcohol marketing. Drug and Alcohol Review, 29(5), 525–530. https://doi.org/10.1111/j.1465-3362.2010.00178.x

Hendriks, H., van den Putte, B., Gebhardt, W. A., & Moreno, M. A. (2018). Social drinking on social media: Content analysis of the social aspects of alcohol-related posts on Facebook and Instagram. Journal of Medical Internet Research, 20(6), e226. https://doi.org/10.2196/jmir.9355

Hendriks, H., Wilmsen, D., van Dalen, W., & Gebhardt, W. A. (2019). Picture me drinking: Alcohol-related posts by Instagram influencers popular among adolescents and young adults. Frontiers in Psychology, 10, 2991. https://doi.org/10.3389/fpsyg.2019.02991

Mayrhofer, M., & Naderer, B. (2019). Mass media as alcohol educator for everyone? Effects of portrayed alcohol consequences and the influence of viewers’ characteristics. Media Psychology, 22(2), 217–243. https://doi.org/10.1080/15213269.2017.1378112

Moreno, M. A., Briner, L. R., Williams, A., Brockman, L., Walker, L., & Christakis, D. A. (2010). A content analysis of displayed alcohol references on a social networking web site. The Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, 47(2), 168–175. https://doi.org/10.1016/j.jadohealth.2010.01.001

Moreno, M. A., D’Angelo, J., & Whitehill, J. (2016). Social media and alcohol: Summary of research, intervention ideas and future study directions. Media and Communication, 4(3), 50–59. https://doi.org/10.17645/mac.v4i3.529

Noel, J. K. (2021). Using social media comments to reduce alcohol purchase intentions: An online experiment. Drug and Alcohol Review, 40(6), 1047–1055. https://doi.org/10.1111/dar.13262

Ricard, B. J., & Hassanpour, S. (2021). Deep learning for identification of alcohol-related content on social media (Reddit and Twitter): Exploratory analysis of alcohol-related outcomes. Journal of Medical Internet Research, 23(9), e27314. https://doi.org/10.2196/27314

Russell, A. M., Davis, R. E., Ortega, J. M., Colditz, J. B., Primack, B., & Barry, A. E. (2021). #Alcohol: Portrayals of alcohol in top videos on TikTok. Journal of Studies on Alcohol and Drugs, 82(5), 615–622. https://doi.org/10.15288/jsad.2021.82.615

Stautz, K., Brown, K. G., King, S. E., Shemilt, I., & Marteau, T. M. (2016). Immediate effects of alcohol marketing communications and media portrayals on consumption and cognition: A systematic review and meta-analysis of experimental studies. BMC Public Health, 16, 465. https://doi.org/10.1186/s12889-016-3116-8

Strowger, M., & Braitman, A. L. (2022). Using social network methodology to examine the effects of exposure to alcohol-related social media content on alcohol use: A critical review. Experimental and Clinical Psychopharmacology. Advance online publication. https://doi.org/10.1037/pha0000561

Tamersoy, A., Choudhury, M. de, & Chau, D. H. (2015). Characterizing smoking and drinking abstinence from social media. HT '15: The Proceedings of the 26th ACM Conference on Hypertext and Social Media, 2015, 139–148. https://doi.org/10.1145/2700171.2791247

Turnwald, B. P., Anderson, K. G., Markus, H. R., & Crum, A. J. (2022). Nutritional analysis of foods and beverages posted in social media accounts of highly followed celebrities. JAMA Network Open, 5(1), e2143087. https://doi.org/10.1001/jamanetworkopen.2021.43087

Van den Bulck, H., Simons, N., & van Gorp, B. (2008). Let's drink and be merry: The framing of alcohol in the prime-time American youth series The OC. Journal of Studies on Alcohol and Drugs, 69(6), 933–940. https://doi.org/10.15288/jsad.2008.69.933

Westgate, E. C., & Holliday, J. (2016). Identity, influence, and intervention: The roles of social media in alcohol use. Current Opinion in Psychology, 9, 27–32. https://doi.org/10.1016/j.copsyc.2015.10.014

World Health Organization Regional Office for Europe (WHO). (2020). Alcohol marketing in the WHO European Region: update report on the evidence and recommended policy actions. https://apps.who.int/iris/bitstream/handle/10665/336178/WHO-EURO-2020-1266-41016-55678-eng.pdf?sequence=1&isAllowed=y

Visual Gender Stereotypes (Advertisement, Social Media)

Nicola Döring — Wed, 15 Jun 2022 00:00:00 +0200

The depiction of gender is the focus of a growing number of content analyses in the fields of both mass media (e.g., Goffman, 1979; Grau & Zotos, 2016; Mitchell & McKinnon, 2019; Sink & Mastro, 2017; Ward & Grower, 2020) and social media (e.g., Baker & Walsh, 2018; Döring, 2019; Döring & Mohseni, 2019; Döring et al., 2016). Typically, the depiction of gender follows traditional gender roles and, hence, does not include at lot of individuality and diversity but sticks to established gender stereotypes (Collins, 2011). Gender steoreotypes are defined as beliefs about how men versus women are (descriptive beliefs) or should be (prescriptive beliefs). Relevant dimensions of gender stereotyping are occupations (e.g., the man as the hero, breadwinner, or executive; the woman as the mother, housewife, or subordinate), sexual and romantic behaviors (e.g., the man seeking sex; the woman seeking love), personality traits (e.g., the man being active, aggressive, rational, and instrumental; the woman being passive, affectionate, emotional, and social), or body types (e.g., the man being tall, muscular and older; the woman being petite, slim, and younger). Gender stereotypes in the media cover different dimensions of traditional masculinity and feminity and are represented textually and/or (audio-)visually. Typically, the occurrence and nature of gender stereotyping in different media is measured and changes over time are of particular interest (e.g., Bhatia & Bhatia, 2020; Maker & Childs, 2003).

Field of application/theoretical foundation:

According to the Social Cognitive Theory (SCT; Bandura 1986, 2009), gender-stereotyped protagonists in the media can influence how media audiences perceive gender roles and to which degree they imitate them as role models. Cultivation theory (Gerbner & Gross, 1976; Kim & Lowry, 2005) predicts, that exposure to distorted media images of reality will shape the audiences’ worldviews. Repeated or constant exposure to gender stereotpyes in the media, according to cultivation theory, will influence the audiences’ perceptions of the roles of women and men in society. Against the background of human rights and gender equality, exaggerated gender stereotypes and the related subordination of women in the media are criticized (e.g. Döring et al., 2016; Goffman, 1979; Grau & Zotos, 2016). Often times, gender-related media content analyses support feminist claims about gender-based inequalities (Collins, 2011; Rudy et al., 2010).

When criticizing gender steoreotypes in the media, it is important to realize, though, that media do not one-directionally influence public perception and opinion (mold theory) but also bi-directionally reflect existing social gender relations and societal attitudes (mirror theory). Last but not least, based on an understanding of stereotypes as cognitive shortcuts and simplifications (Windels, 2016) it needs to be acknowledged that using stereotypes in media representations makes it easier to disseminate clear messages, inform or entertain the audience. Hence, the use of gender-related or other group-related stereotypes is not only an issue of societal relations and equality but also an issue of information processing and message creation.

References/combination with other methods of data collection:

Manual (e.g., Döring et al., 2016) and computational (e.g., Bhatia & Bhatia, 2020) content analyses of gender representations in mass media and social media can be combined. Furthermore, content analyses can be complemented with qualitative interviews and quantitative surveys to investigate both media creators’ and media audiences’ perceptions and evaluations of gender stereotypes in the media. Additionally, experimental studies are helpful to measure directly how different gender stereotypes in the media are perceived and evaluated by recipients and if and how they can affect their gender-related thoughts, feelings, and behaviors (Bast et al., 2021).

Example Studies for Manual Content Analyses:

Acknowledging the multidimensionality and complexity of gender stereotypes in the media, this DOCA entry focuses on the analysis of gender displays in the tradition of Erving Goffman (1979, 1988). Goffman’s approach originally addressed press adversitements and was qualitative in nature. It has been adopted for quantitative content analyses and extended regarding relevant dimensions with a focus on press advertisments (Kang, 1997), magazine titles (Mortensen et al., 2020) as well as social media images such as selfies on Instagram (Döring et al., 2016; Baker & Walsh, 2018). Extending Goffman’s gender display framework to social media contexts and user-generated content does make sense from a theoretical point of view (Butkowski, 2020). Usually, dichotomous or polytomous variables are used to code stereotypical gender displays in the Goffman tradition, however, some content researchers also have developed and used rating scales for coding (Butkowski et al., 2020). So far, published codebooks with example pictures are scarce.

Table 1. Example studies for manual content analyses.

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
a) *Six categories of gender display according to Goffman (1979, 1988)*
	Relative size (between 2 or more persons)	One person (usually the man) is depicted as larger in height and greater in girth through positioning or perspective of the image compared to the other person(s) (usually the woman). Can only be coded with 2 or more persons in the picture. Binary coding (1: yes; 2: no).	Not available
N=500 selfies on Instagram	Feminine touch	One person (usually the woman) is pictured using their fingers and hands to trace the outlines of an object or to cradle it or to caress its surface or to touch their own body (e.g., their hair). The so-called feminine touch is not goal-oriented or functional. Binary coding (1: yes; 2: no). Example image for femine touch:	Cohen’s Kappa = .79	Döring et al. (2016)
	Function ranking (between 2 or more persons)	One person (usually the man) is pictured in the executive or dominant role, the other person in the subordinate or assisting role (usually the woman). Can only be coded with 2 or more persons in the picture. Binary coding (1: yes; 2: no)	Not available
	Family (nuclear family of four persons)	The typical nuclear family is depicted with mother, father, daughter, and son. Typically, closer bonds between mother and daughter on the one side, and father and son on the other side are depicted. Can only be coded with a whole family in the picture. Multidimensional qualitative variable that has not been adopted for quantitative coding yet.	Not available
N=500 selfies on Instagram	Ritualization of subordination	One person (usually the woman) is depicted in a posture of subordination that deviates from a stable, upright position and includes lying/sitting postures and imbalance. Posture of subordination includes lying or sitting versus standing: Polytomous coding (1: lying, 2: sitting, 3: standing) Example image for lying posture: Imbalance in body posture includes canting positions and knee bending. Binary coding (1: yes; 2: no). Example image for imbalance posture:	Lying, sitting, standing posture Cohen’s Kappa = 1.00 Imbalance posture: Cohen’s Kappa = .90	Döring et al. (2016)
N=500 selfies on Instagram	Licensed withdrawal	One person (usually the woman) is depicted in a situation of licensed withdrawal meaning that she does not fully turn to the camera. This includes withdrawing gaze and loss of control. Withdrawing gaze means that one person (usually the woman) is depicted gazing away from the camera. Binary coding (1: yes; 2: no). Example image withdrawing gaze: Loss of control means that one person (usually the woman) is depicted expressing strong emotions implying that she is not fully focusing on the current scene . Binary coding (1: yes; 2: no). Example image loss of control:	Withdrawing gaze: Cohen’s Kappa = 1.00 Loss of control: Cohen’s Kappa = 1.00	Döring et al. (2016)
b) *Two additional gender display categories according to Kang (1997)*
N=500 selfies on Instagram	Body Display	Body display of persons vary with the type of clothing. One person (usually the man) is depicted in full clothing. Binary coding (1: yes; 2: no). One person (usually the woman) is depicted in sparse clothing or nudity. Binary coding (1: yes; 2: no). Example image sparse closing	Full clothing Cohen’s Kappa = .73 Sparse clothing: Cohen’s Kappa = .73	Döring et al. (2016)
	Independence and self-assertiveness	One person (usually the man) is depicted in a position of independence and self-assertivenesss. Binary coding (1: yes; 2: no).	Not available
c) *Three categories of social media related gender stereotypes (Döring et al., 2016)*
N=500 selfies on Instagram	Kissing pout	One person (usually the woman) is depicted showing a kissing pout (“duck face”). Binary coding (1: yes; 2: no). Example image for kissing pout:	Cohen’s Kappa = 1.00	Döring et al. (2016)
N=500 selfies on Instagram	Muscle presentation	One person (usually the man) is depicted presenting their muscles (e.g., biceps, sixpack). Binary coding (1: yes; 2: no). Example image for muscle presentation:	Cohen’s Kappa = 1.00	Döring et al. (2016)
N=500 selfies on Instagram	Faceless portrayal	One person (usually the woman) is depicted without the face in the picture. Binary coding (1: yes; 2: no). Example image for faceless portayal:	Cohen’s Kappa = 1.00	Döring et al. (2016)

Note. In order to ensure anonymity, no original Instagram posts are displayed. All example pictures shown are re-enactments to visually illustrate the categories and all protagonists gave their informed consent for publication of the pictures. The pictures are also used in the original study Döring et al. (2016).

The categories of gender display in the tradition of Erving Goffman (1979, 1988) can be complemented with further categories that go into more detail of physical appearance in terms of body type, attire or sexualization. Furthermore, additional dimensions of gender stereotyping such as occupations or activities can be added.

References

Baker, S. A., & Walsh, M. J. (2018). ‘Good morning fitfam’: Top posts, hashtags and gender display on Instagram. New Media & Society, 20(12), 4553–4570. https://doi.org/10.1177/1461444818777514

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.

Bast, J., Oschatz, C., & Renner, A.‑M. (2021). Successfully overcoming the “bouble bind”? A mixed-method analysis of the self-presentation of female right-wing populists on Instagram and the impact on voter attitudes. Political Communication, 1–25. https://doi.org/10.1080/10584609.2021.2007190

Bhatia, N., & Bhatia, S. (2021). Changes in gender stereotypes over time: A computational analysis. Psychology of Women Quarterly, 45(1), 106–125. https://doi.org/10.1177/0361684320977178

Butkowski, C. P. (2020). Beyond “commercial realism”: Extending Goffman’s gender display framework to networked media contexts. Communication, Culture and Critique, 14(1), 89-108.

Butkowski, C. P., Dixon, T. L., Weeks, K. R., & Smith, M.A. (2020). Quantifying the feminine self(ie): Gender display and social media feedback in young women’s Instagram selfies. New Media & Society, 22(5), 817-837. https://doi.org/10.1177/1461444819871669

Collins, R. L. (2011). Content analysis of gender roles in media: Where are we now and where should we go? Sex Roles, 64(3-4), 290–298. https://doi.org/10.1007/s11199-010-9929-5

Döring, N. (2019). Videoproduktion auf YouTube: Die Bedeutung von Geschlechterbildern [Video production on YouTube: The relevance of gender images]. In J. Dorer, B. Geiger, B. Hipfl, & V. Ratković (Eds.), Handbuch Medien und Geschlecht: Perspektiven und Befunde der feministischen Kommunikations- und Medienforschung (pp. 1–11). Springer Fachmedien. https://doi.org/10.1007/978-3-658-20712-0_53-1

Döring, N., & Mohseni, M. R. (2019). Fail videos and related video comments on YouTube: A case of sexualization of women and gendered hate speech? Communication Research Reports, 36(3), 254–264. https://doi.org/10.1080/08824096.2019.1634533

Döring, N., Reif, A., & Poeschl, S. (2016). How gender-stereotypical are selfies? A content analysis and comparison with magazine adverts. Computers in Human Behavior, 55, 955–962. https://doi.org/10.1016/j.chb.2015.10.001

Gerbner, G., & Gross, L. (1976). Living with television: The violence profile. The Journal of Communication, 26(2), 173–199. https://doi.org/10.1111/j.1460-2466.1976.tb01397.x

Goffman, E. (1979). Gender advertisements. Harper & Row.

Goffman, E. (1988). Gender advertisements (revised edition). Harpercollins College Div.

Grau, S. L., & Zotos, Y. C. (2016). Gender stereotypes in advertising: A review of current research. International Journal of Advertising, 35(5), 761–770. https://doi.org/10.1080/02650487.2016.1203556

Kang, M.‑E. (1997). The portrayal of women’s images in magazine advertisements: Goffman’s gender analysis revisited. Sex Roles, 37(11-12), 979–996. https://doi.org/10.1007/BF02936350

Kim, K., & Lowry, D. T. (2005). Television commercials as a lagging social indicator: Gender role stereotypes in Korean television advertising. Sex Roles, 53(11-12), 901–910. https://doi.org/10.1007/s11199-005-8307-1

Maker, J. K., & Childs, N. M. (2003). A longitudinal content analysis of gender roles in children's television advertisements: A 27 year review. Journal of Current Issues & Research in Advertising, 25(1), 71–81. https://doi.org/10.1080/10641734.2003.10505142

Mitchell, M., & McKinnon, M. (2019). 'Human' or 'objective' faces of science? Gender stereotypes and the representation of scientists in the media. Public Understanding of Science (Bristol, England), 28(2), 177–190. https://doi.org/10.1177/0963662518801257

Mortensen, T. M., Ejaz, K., & Pardun, C. J. (2020). Quantifying gender stereotypes? Visually assessing stereotypes of women in People Magazine. Journal of Magazine Media, 21(1), 30–50. https://doi.org/10.1353/jmm.2020.0002

Rudy, R. M., Popova, L., & Linz, D. G. (2010). The context of current content analysis of gender roles: An introduction to a special issue. Sex Roles, 62(11-12), 705–720. https://doi.org/10.1007/s11199-010-9807-1

Sink, A., & Mastro, D. (2017). Depictions of gender on primetime television: A quantitative content analysis. Mass Communication and Society, 20(1), 3–22. https://doi.org/10.1080/15205436.2016.1212243

Ward, L. M., & Grower, P. (2020). Media and the development of gender role stereotypes. Annual Review of Developmental Psychology, 2(1), 177–199. https://doi.org/10.1146/annurev-devpsych-051120-010630

Windels, K. (2016). Stereotypical or just typical: How do US practitioners view the role and function of gender stereotypes in advertisements? International Journal of Advertising, 35(5), 864–887. https://doi.org/10.1080/02650487.2016.1160855

Conceptual Overview (Portrayals of Sexuality in Pornography)

Nicola Döring, Dan J. Miller — Mon, 24 Oct 2022 00:00:00 +0200

Pornography is neither a documentary media genre that documents what real sex in everyday life looks like, nor is it a pedagogical or moral media genre aimed at showing what ideal sex (in terms of health or morality) should look like. Instead, pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022).

It should be noted that the conceptual differentiation between erotica and pornography is complex and that “pornography” remains an ideologically charged, and often negatively connotated, concept. Hence, the research literature sometimes uses the broader and more neutral term “sexually explicit material” (SEM) in place of “pornographic material” (McKee et al., 2020). Furthermore, it must be emphasized that in the context of content analyses of SEM the focus is typically on legal pornography. Legal visual pornography is produced with adults who have given their informed consent for their image to be recorded, and then disseminated and marketed as SEM. Illegal pornography is usually beyond the scope of media content research, as the acquisition and use of illegal material would be unethical and illegal for researchers (e.g., the analysis of so-called “child pornography”, or what might be more accurately labeled “images of child sexual abuse”). Criminological and forensic research projects are exceptions to this rule.

Field of application/theoretical foundation:

The theories applied in pornographic media content research primarily come from four academic disciplines: communication science, psychology, sex research, and gender studies. These different theories are fairly similar in their core assumption that pornography users’ sexual cognitions and behaviors are molded by the ways in which sexuality is portrayed in pornographic material. Some of the theories also explain the typical content of pornography and point to the fact that audiences might not only be influenced by pornography but can also shape porn production through their preferences. All theories demand content analyses of pornographic material to back up their predictions.

General Media Effects Theories

Cultivation Theory and Social Cognitive Theory are the most commonly used media effects theories, irrespective of specific media content. They are often applied to pornographic material.

Cultivation Theory (CT) was developed by communication researcher George Gerbner in the 1960s (Gerbner, 1998). CT claims that heavy media users’ perceptions of the prevalence of different societal phenomena (e.g., crimes) are shaped by the prevalence with which these phenomena occur in the media they consume (e.g., cop shows on TV). Applied to pornography, CT predicts that heavy users of pornography will severely overestimate the prevalence of sexual practices that are rare in reality, but widespread in pornography. Young people who lack real life sexual experience are regarded as particularly vulnerable for sexual cultivation effects in terms of biased perceptions of the popularity and normalcy of different performances of sexuality (e.g., name calling and slapping during sex).
Another classic media effects theory that is widely adopted in pornography research is psychologist Albert Bandura’s Social Learning Theory (Bandura, 1971), later re-labeled as Social Cognitive Theory (SCT; Bandura, 2001). SCT claims that people imitate the behaviors of media role models. Applied to pornographic material, SCT predicts that media audiences will develop more favorable attitudes towards, and engage more frequently in, sexual behaviors portrayed positively in sexually explicit material. Such sexual imitation effects may influence not only attitudes toward, and engagement in, sex acts represented in pornography (e.g., anal sex), but also gender role behaviors (e.g., men acting dominantly, women acting submissively during sex), safer sex measures (e.g., lack of condom use), bodily appearance (e.g., breast augmentation), and consent communication (e.g., lack of explicitly asking for, or giving, consent to engage in different sex acts).

Sexual Media Effects Theories

While CT and SCT are broad media effects theories applicable to pornography as well as many other types of media content, Sexual Script Theory and the 3AM specifically address sexual media and their effects.

Sexual Script Theory (SST) was developed by sociologists John Gagnon and William Simon in the 1970s (Gagnon & Simon, 1973; Simon & Gagnon, 2003; Wiederman, 2015). SST argues that human sexuality is not merely a biological instinct, but a highly complex set of cognitions and behaviors shaped by symbolic, social and cultural factors: People develop ideas about how to have sex in terms of organized cognitive schemas or “scripts” that reflect intra-psychic desires (e.g., their sexual fantasies), social norms (e.g., peers’ and partners’ sexual expectations), and cultural influences (e.g., representations of sexuality in the media they consume). SST stresses that the intra-psychic, social, and cultural determinants of individuals’ sexual scripts mutually influence each other and can change over time (Simon & Gagnon, 2003). However, in pornography research, usually only the third element of the theory (cultural influences through media representations of sexuality) is considered. Applied to pornography, SST predicts that sexual scripts presented in pornographic material (e.g., spontaneous anal sex with strangers without condoms or overt consent communication) can shape individuals’ sexual scripts.
The Acquisition, Activation, and Application Model of Media Sexual Socialization (3AM) was developed more recently by communication researcher Paul Wright as a specification of SST regarding media influence (Wright, 2011). According to the 3AM, sexually explicit media content shapes cognitive schemas of sexuality in three ways: Pornography can foster the creation of new schemas (schema acquisition), it can prime extant schemas (schema activation), and it can facilitate the utilization of extant schemas to inform attitudes and behaviors (schema application). The 3AM differentiates between specific scripting effects of pornography (e.g., engaging in condom-free casual anal sex without sufficient consent communication after having observed this exact sexual script multiple times in pornography) versus abstract scripting effects (e.g., adopting a more permissive sexual worldview after having observed many people engaging in unrestricted sex in pornography).

The aforementioned general and sexuality-specific media effects theories have been used predominantly to predict negative (unwanted, harmful) effects such as dangerously distorted views of sexuality and gender roles as well as engagement in risky or violent sexual behaviors, while potential positive effects have been mostly ignored. Only recently, has serious consideration been given to the beneficial effects of pornography use (e.g., sexual identity validation, sexual empowerment, improved couple communication, sexual skill acquisition, etc.) in the research literature (e.g., Döring, 2021; Döring & Mohseni, 2018; Döring et al., 2021; Kohut & Fisher, 2013; Kohut et al., 2017; Miller et al., 2018; Tillmann & Wells, 2022). Depending on specific negative and/or positive effect assumptions, different aspects of the representation of sexuality will be measured (e.g., expressions of aggression during sex or different types of sexual stimulation techniques).

Gender Role, Feminist and Queer Theories

Typically, analysis of the ways in which sexuality is represented in pornography involves considerations of gender relations, therefore gender role theories and feminist theories of gender (in-)equality are frequently drawn upon (e.g., Eagly, 1987). There are two main reasons for this additional theoretical focus on gender: 1) Most SEM depicts heterosexual encounters, hence the portrayal of sexuality in pornography implies a portrayal of sexual gender relations (Williams, 1989). 2) Gender relations in the media are often asymmetrical, depicting men and women in superior and subordinate positions, respectively. Such patriarchal gender relations are expected to be reflected, or even exaggerated, in pornographic material. Radical feminist approaches in particular characterize pornography as a portrayal of sexual degradation of women by men, that is so harmful to society that it should be prohibited (e.g., MacKinnon, 1991). Other feminist approaches are also critical of asymmetric gender relations in traditional mainstream pornography and call for more gender equality in SEM, such as in feminist pornography (Williams, 1989). Feminist criticism of gender roles and relations in pornography does not address the demographic variable of sex/gender alone, but also covers other diversity dimensions such as age, race/ethnicity, or disability. According to the analytical framework of intersectionality, the subordination and discrimination of women in society and media representations particularly affect those women who have multiple marginalized demographic characteristics (e.g., the representation of white women in pornography differs from that of black or Asian women; Fritz et al. 2021). Queer theory is also concerned with different racial/ethnic and sexual identities of women and their participation and representation in pornography (Ingraham 2013).

Content analyses of pornography need to take into consideration that pornography is becoming increasingly diverse (Miller & McBain, 2022). Hence, content analyses need to differentiate between various pornographic sub-genres such as commercial heterosexual mainstream pornography (traditionally targeting men) versus, for example, women-friendly and couple-oriented pornography, feminist pornography, queer pornography, fetish and kink pornography, or authentic amateur and DIY (do it yourself) pornography in the form of visual or text pornography (Döring, 2021; McKee et al., 2008). Gender role, feminist, and queer theories predict that gender relations in mainstream pornography are more asymmetrical, stereotypical and patriarchal than in women- and couple-friendly, feminist and queer pornography.

Sexual Fantasy and Desire Theories

The above-mentioned effect theories do not address and explain the main intended effect of pornography, namely immediate sexual arousal, pleasure and satisfaction. The theories focus on linking the fictional pornographic content directly with real life opinions and behaviors, but mostly ignore the links between fictional pornographic content and sexual fantasies. Research shows that many sexual fantasies of people of all genders are unrealistic, extreme, clichéd, violent and norm-violating and that norm-violation is often what makes them arousing (e.g., Bivona et al., 2012; Critelli & Bivona, 2008; Joyal, 2015). The same might be true for pornographic content. Hence, measuring pornography, a fictional media genre, against standards of realism, health and morality might not always be in line with the main entertainment purpose of the genre. Erotizing the forbidden and dangerous (e.g, sex with family members, with mysterious strangers, with authority figures, with non-human creatures) is a common trope of sexual fantasies, hence meaningful variables to measure pornographic portrayals of sexuality could be derived from, and related to, theories of sexual fantasy and desire (e.g., Salmon et al., 2019; Stoller, 1985). Indulging in unrealistic and norm-violating fantasies and fictional media contents is part of media entertainment and may not necessarily lead to norm-violating behaviors. Competent media users should be able to differentiate between fiction and reality.

Mold Theories versus Mirror Theories

When analyzing and criticizing sexuality portrayals in pornography, it is important to realize that media do not just uni-directionally influence public opinions and behaviors (mold theory). Rather, media also bi-directionally reflect existing sexual relations and fantasies (mirror theory). Recent sex surveys, for example, demonstrate that engagement in consensual BDSM (Bondage/Discipline, Dominance/Submission, Sadism/Masochism) practices and rough sex (e.g., name calling, spanking, hair pulling) is fairly widespread in the general population and enjoyed by all genders (e.g., Burch & Salmon, 2019; Herbenick et al., 2021a, 2021b; Strizzi et al., 2022). Hence, it might not always be the adult industry that influences audiences’ sexualities, but also audiences’ sexual interests that influence porn production. Particularly in the digital pornography market, producers and vendors can easily analyze audience preferences through the analysis of search terms and download statistics and adopt their content accordingly. Furthermore, general beauty trends in society (e.g., regarding shaving of pubic and body hair, growing of beards, or multiple tattoos and other body art) might be mirrored in pornography (through its selection and presentation of performers) rather than of generated by it.

References/combination with other methods of data collection:

Manual quantitative content analyses of pornographic material can be combined with qualitative (e.g., Keft-Kennedy, 2008) as well as computational (e.g., Seehuus et al., 2019) content analyses. Furthermore, content analyses can be complemented with qualitative interviews and quantitative surveys to investigate perceptions and evaluations of the portrayals of sexuality in pornography among pornography’s creators and performers (e.g., West, 2019) and audiences (e.g., Cowan & Dunn, 1994; Hardy et al., 2022; Paasoonen, 2021; Shor, 2022). Additionally, experimental studies are helpful to measure directly how different dimensions of pornographic portrayals of sexuality are perceived and evaluated by recipients, and if and how these portrayals can affect audiences’ sexuality-related thoughts, feelings, and behaviors (e.g., Kohut & Fisher, 2013; Miller et al., 2019).

Example studies for manual quantitative content analyses:

Acknowledging the multidimensionality and complexity of portrayals of sexuality in pornography, a recent research review identified eight main dimensions of analysis (Miller & McBain, 2022) that are adopted and extended in this DOCA entry as: 1) violence, 2) degradation, 3) sex acts, 4) performer demographics (sex/gender, age, race/ethnicity), 5) performer bodily appearance, 6) safer sex practices, 7) relational context of sex, and 8) consent communication. Example studies and measures for all eight dimensions of pornographic portrayals of sexuality are presented in separate DOCA entries.

Eight Dimension of Portrayals of Sexuality in Pornography	DOCA entry
1) Violence	Portrayals of Sexuality in Pornography: Violence
2) Degradation	Portrayals of Sexuality in Pornography: Degradation
3) Sex Acts	Portrayals of Sexuality in Pornography: Sex Acts
4) Performer Demographics	Portrayals of Sexuality in Pornography: Performer Demographics
5) Performer Bodily Appearance	Portrayals of Sexuality in Pornography: Performer Bodily Appearance
6) Safer Sex Practices	Portrayals of Sexuality in Pornography: Safer Sex Practices
7) Relational Context of Sex	Portrayals of Sexuality in Pornography: Relational Context of Sex
8) Consent Communication	Portrayals of Sexuality in Pornography: Consent Communication

References

Bandura, A. (1971). Social learning theory. General Learning.

Bandura, A. (2001). Social cognitive theory of mass communication. Media Psychology, 3(3), 265–299. https://doi.org/10.1207/S1532785XMEP0303_03

Bivona, J. M., Critelli, J. W., & Clark, M. J. (2012). Women's rape fantasies: An empirical evaluation of the major explanations. Archives of Sexual Behavior, 41(5), 1107–1119. https://doi.org/10.1007/s10508-012-9934-6

Burch, R. L., & Salmon, C. (2019). The rough stuff: Understanding aggressive consensual sex. Evolutionary Psychological Science, 5(4), 383–393. https://doi.org/10.1007/s40806-019-00196-y

Carrotte, E. R., Davis, A. C., & Lim, M. S. (2020). Sexual behaviors and violence in pornography: Systematic review and narrative synthesis of video content analyses. Journal of Medical Internet Research, 22(5), Article e16702. https://doi.org/10.2196/16702

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Critelli, J. W., & Bivona, J. M. (2008). Women's erotic rape fantasies: An evaluation of theory and research. Journal of Sex Research, 45(1), 57–70. https://doi.org/10.1080/00224490701808191

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Döring, N. (2021). Erotic Fan Fiction. In A. D. Lykins (Ed.), Encyclopedia of sexuality and gender (pp. 1–8). Springer. https://doi.org/10.1007/978-3-319-59531-3_65-1

Döring, N., Krämer, N., Mikhailova, V., Brand, M., Krüger, T. H. C., & Vowe, G. (2021). Sexual interaction in digital contexts and its implications for sexual health: A conceptual analysis. Frontiers in Psychology, 12, Article 769732. https://doi.org/10.3389/fpsyg.2021.769732

Döring, N., & Mohseni, M. R. (2018). Are online sexual activities and sexting good for adults’ sexual well-being? Results from a national online survey. International Journal of Sexual Health, 30(3), 250–263. https://doi.org/10.1080/19317611.2018.1491921

Eagly, A. H. (1987). Sex differences in social behavior: A social-role interpretation. Erlbaum.

Fritz, N., Malic, V. [Vinny], Paul, B., & Zhou, Y. (2021). Worse than objects: The depiction of black women and men and their sexual relationship in pornography. Gender Issues 38, 100-120. https://doi.org/10.1007/s12147-020-09255-2

Gagnon, J. H., & Simon, W. (1973). Sexual conduct: The social sources of human sexuality (2. ed.). AldineTransaction.

Gerbner, G. (1998). Cultivation analysis: An overview. Mass Communication and Society, 1(3-4), 175–194. https://doi.org/10.1080/15205436.1998.9677855

Hardy, J., Kukkonen, T., & Milhausen, R. (2022). Examining sexually explicit material use in adults over the age of 65 years. The Canadian Journal of Human Sexuality, 31(1), 117–129. https://doi.org/10.3138/cjhs.2021-0047

Herbenick, D., Fu, T.‑C., Patterson, C., Rosenstock Gonzalez, Y. R., Luetke, M., Svetina Valdivia, D., Eastman-Mueller, H., Guerra-Reyes, L., & Rosenberg, M. (2021a). Prevalence and characteristics of choking/strangulation during sex: Findings from a probability survey of undergraduate students. Journal of American College Health, 1–15. https://doi.org/10.1080/07448481.2021.1920599

Herbenick, D., Patterson, C., Beckmeyer, J., Gonzalez, Y. R. R., Luetke, M., Guerra-Reyes, L., Eastman-Mueller, H., Valdivia, D. S., & Rosenberg, M. (2021b). Diverse sexual behaviors in undergraduate students: Findings from a campus probability survey. The Journal of Sexual Medicine, 18(6), 1024–1041. https://doi.org/10.1016/j.jsxm.2021.03.006

Ingraham, N. (2013). Queering pornography through qualitative method. International Journal of Multiple Research Approaches, 7(2), 158-159. https://doi.org/10.5172/mra.2013.7.2.218

Joyal, C. C. (2015). Defining "normophilic" and "paraphilic" sexual fantasies in a population-based sample: On the importance of considering subgroups. Sexual Medicine, 3(4), 321–330. https://doi.org/10.1002/sm2.96

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

Kohut, T., & Fisher, W. A. (2013). The impact of brief exposure to sexually explicit video clips on partnered female clitoral self-stimulation, orgasm and sexual satisfaction. The Canadian Journal of Human Sexuality, 22(1), 40–50. https://doi.org/10.3138/cjhs.935

Kohut, T., Fisher, W. A., & Campbell, L. (2017). Perceived effects of pornography on the couple relationship: Initial findings of open-ended, participant-informed, "bottom-up" research. Archives of Sexual Behavior, 46(2), 585–602. https://doi.org/10.1007/s10508-016-0783-6

MacKinnon, C. A. (1991). Pornography as defamation and discrimination. Boston University Law Review, 71, 793-818.

McKee, A., Albury, K., & Lumby, C. (2008). The porn report. Melbourne University Press.

McKee, A., Byron, P., Litsou, K., & Ingham, R. (2020). An interdisciplinary definition of pornography: Results from a global Delphi panel. Archives of Sexual Behavior, 49(3), 1085–1091. https://doi.org/10.1007/s10508-019-01554-4

Miller, D. J., Hald, G. M., & Kidd, G. (2018). Self-perceived effects of pornography consumption among heterosexual men. Psychology of Men & Masculinity, 19(3), 469–476. https://doi.org/10.1037/men0000112

Miller, D. J., & McBain, K. A. (2022). The content of contemporary, mainstream pornography: A literature review of content analytic studies. American Journal of Sexuality Education, 17(2), 219–256. https://doi.org/10.1080/15546128.2021.2019648

Miller, D. J., McBain, K. A., & Raggatt, P. T. F. (2019). An experimental investigation into pornography’s effect on men’s perceptions of the likelihood of women engaging in porn-like sex. Psychology of Popular Media Culture, 8(4), 365–375. https://doi.org/10.1037/ppm0000202

Miller, D. J., Raggatt, P. T. F., & McBain, K. (2020). A literature review of studies into the prevalence and frequency of men’s pornography use. American Journal of Sexuality Education, 15(4), 502–529. https://doi.org/10.1080/15546128.2020.1831676

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Salmon, C., Fisher, M. L., & Burch, R. L. (2019). Evolutionary approaches: Integrating pornography preferences, short-term mating, and infidelity. Personality and Individual Differences, 148, 45–49. https://doi.org/10.1016/j.paid.2019.05.030

Seehuus, M., Stanton, A. M., & Handy, A. B. (2019). On the content of "real-world" sexual fantasy: Results from an analysis of 250,000+ anonymous text-based erotic fantasies. Archives of Sexual Behavior, 48(3), 725–737. https://doi.org/10.1007/s10508-018-1334-0

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

Simon, W., & Gagnon, J. H. (2003). Sexual scripts: Origins, influences and changes. Qualitative Sociology, 26(4), 491–497. https://doi.org/10.1023/B:QUAS.0000005053.99846.e5

Stoller, R. J. (1985). Observing the erotic imagination. Yale University Press.

Strizzi, J. M., Øverup, C. S., Ciprić, A., Hald, G. M., & Træen, B. (2022). BDSM: Does it hurt or help sexual satisfaction, relationship satisfaction, and relationship closeness? Journal of Sex Research, 59(2), 248–257. https://doi.org/10.1080/00224499.2021.1950116

Tillman, M., & Wells, B. E. (2022). An intersectional feminist analysis of women's experiences of authenticity in pornography. Journal of Sex Research, 1–17. https://doi.org/10.1080/00224499.2021.2024489

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Wiederman, M. W. (2015). Sexual Script Theory: Past, present, and future. In J. DeLamater & R. F. Plante (Eds.), Handbook of the sociology of sexualities (pp. 7–22). Springer. https://doi.org/10.1007/978-3-319-17341-2_2

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.

Wright, P. J. (2011). Mass media effects on youth sexual behavior assessing the claim for causality. Annals of the International Communication Association, 35(1), 343–385. https://doi.org/10.1080/23808985.2011.11679121

Violence (Portrayals of Sexuality in Pornography)

Nicola Döring, Dan J. Miller — Mon, 24 Oct 2022 00:00:00 +0200

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This DOCA entry focuses on the representation of violence as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of violence as one dimension of the portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Example studies for manual quantitative content analyses:

Common research hypotheses state that pornography depicts sexuality as violent and shows violent acts such as verbal aggression, physical aggression or image-based abuse being predominately perpetrated by men targeting women. To test such hypotheses and code pornographic material accordingly, it is necessary to clarify the concept of “violence” and use valid and reliable measures for different types of violence. In addition, it is necessary to code the sex/gender of the person depicted as the source and/or the target of the respective violent act (e.g., in the context of verbal sexual aggression, the target of verbal aggression is coded as female or male and the source of verbal aggression is coded as female or male).

It is important to note that in the context of pornographic content research, researchers conceptualize violence differently. Also, it should be noted, that there is some overlap between the variable violence and the variable degradation in the context of pornographic portrayals of sexuality. For example, the depiction of “name calling” in a pornographic scene can be understood as an indicator of “violence” (namely verbal aggression) or of “degradation”. Name calling is covered here as verbal aggression (following Fritz et al., 2020), hence, it is not covered again as degradation, even though some authors do so (such as Gorman et al., 2010; see DOCA entry “Degradation (Portrayals of Sexuality in Pornography)”). In general, one can argue that all violent acts – apart from being potentially painful and harmful – have a component of degradation because they put the target of violence in a subordinate role. However, not all degrading acts are violent (e.g., degradation by systematic lack of sexual reciprocity does not entail overt aggression).

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Violence:* Usually, violence is defined as behavior directed toward the goal of harm or injury of another living being, who is motivated to avoid such treatment (McKee, 2015). However, in content analyses of pornography, violent behavior is often coded regardless of intention to harm or actual harm done. Instead, violence is coded with reference to the presence of prespecified behaviors (e.g., spanking, slapping, choking), even if these behaviors are presented as consensual and sexually arousing (Miller & McBain, 2022). Technology-facilitated sexual violence (image-based sexual abuse) addresses the illegal recording and dissemination of intimate imagery without consent, such as revenge porn, upskirting or spy cams (Henry & Powell, 2018). Mainstream pornography platforms partly disseminate illegal material and partly market some of their legal commercial pornography under these respective labels, hence pretending to provide non-consensual pornography (Vera-Gray et al., 2021). Apart from issues of performer health protection, violent acts are also regarded as relevant in terms of modelling behaviors for audiences.
N=4,009 heterosexual scenes from 3,767 pornographic videos sampled from PornHub.com (574 scenes) and and Xvideos. com (3,435 scenes)	Verbal aggression	“An action that clearly does or could reasonably be expected to cause psychological harm to oneself or another person through name calling or insulting”. Binary coding (1: present; 2: not present).	Percentage Agreement: 97.5% (PornHub) / 88.9% (Xvideos)	Fritz et al. (2020)
	Physical aggression	“Any action that clearly did or could reasonably be expected to cause physical harm to oneself or another person, regardless of the perpetrator’s intent and the target’s response”. Binary coding (1: present; 2: not present).	Percentage Agreement: 98.8% (Pornhub) / 97.6% (Xvideos)
	- Spanking (type of physical aggression)	“Striking on the buttocks with an open hand”. Binary coding (1: present; 2: not present).	Percentage Agreement: 94.2% (Pornhub) / 96.9% (Xvideos)
	- Slapping (type of physical aggression)	“Striking oneself or another with an entirely unclosed hand, group of fingers, or palm”. Binary coding (1: present; 2: not present).	Percentage Agreement: 99.2% (Pornhub) / 98.1% (Xvideos)
	- Gagging (type of physical aggression)	“Any instance in which an object (including the genitals) is inserted into a person’s mouth, such that it appears to cut off their ability to breathe freely and/or causes them to experience a throat spasm”. Binary coding (1: present; 2: not present).	Percentage Agreement: 99.2% (Pornhub) / 96.7% (Xvideos)
	- Pulling hair (type of physical aggression)	“Any instance where the hair on a person’s head is grasped or pulled on, such that the person’s head is pulled (even slightly) in a particular direction”. Binary coding (1: present; 2: not present).	Percentage Agreement: 100.0% (Pornhub) / 98.9% (Xvideos)
	- Choking (type of physical aggression)	“To cause another to stop breathing, if only for a moment, by grabbing the throat”. Binary coding (1: present; 2: not present).	Percentage Agreement: 98.3% (Pornhub) / 98.8% (Xvideos)
	- Pushing (type of physical aggression)	‘‘Use of one’s hands, arms, or other body parts to force another person’s body or part of their body to move in a particular manner or direction”. Binary coding (1: present; 2: not present).	Percentage Agreement: 95.0% (Pornhub) / 97.9% (Xvideos)
N=131,738 titles of pornographic videos presented on the landing pages of the three leading mainstream pornography video platforms in the UK: PornHub.com, Xhamster.com, Xvideos.com	Image-based sexual abuse	Pornographic video title includes keywords indicating image-based sexual abuse such as “spy”, “hidden”, “upskirting”, “leak” or “revenge”. Binary coding (1: present; 2: not present).	Not available	Vera-Gray et al. (2021)

At the same time, porn platforms may disseminate material without the consent of the depicted persons in such violence-indicating categories, but also in regular sub-genre categories (such as "Threesome", "Handjob"), making it impossible for coders to reliably detect all image-based violence.

References

Cowan, G., & Dunn, K. F. (1994). What themes in pornography lead to perceptions of the degradation of women? Journal of Sex Research, 31(1), 11–21. https://doi.org/10.1080/00224499409551726

Döring, N. (2009). The Internet’s impact on sexuality: A critical review of 15 years of research. Computers in Human Behavior, 25(5), 1089–1101. https://doi.org/10.1016/j.chb.2009.04.003

Fritz, N., Malic, V. [Vinny], Paul, B., & Zhou, Y. (2020). A descriptive analysis of the types, targets, and relative frequency of aggression in mainstream pornography. Archives of Sexual Behavior, 49(8), 3041–3053. https://doi.org/10.1007/s10508-020-01773-0

Gorman, S., Monk-Turner, E., & Fish, J. N. (2010). Free adult internet web sites: How prevalent are degrading acts? Gender Issues, 27(3-4), 131–145. https://doi.org/10.1007/s12147-010-9095-7

Henry, N., & Powell, A. (2018). Technology-facilitated sexual violence: A literature review of empirical research. Trauma, Violence & Abuse, 19(2), 195–208. https://doi.org/10.1177/1524838016650189

Keft-Kennedy, V. (2008). Fantasising masculinity in Buffyverse slash fiction: Sexuality, violence, and the vampire. Nordic Journal of English Studies, 7(1), 49–80.

McKee, A. (2015). Methodological issues in defining aggression for content analyses of sexually explicit material. Archives of Sexual Behavior, 44(1), 81–87. https://doi.org/10.1007/s10508-013-0253-3

Paasonen, S. (2021). “We watch porn for the fucking, not for romantic tiptoeing”: Extremity, fantasy and women’s porn use. Porn Studies, 1–14. https://doi.org/10.1080/23268743.2021.1956366

Shor, E. (2022). Who seeks aggression in pornography? Findings from interviews with viewers. Archives of Sexual Behavior, 51(2), 1237–1255. https://doi.org/10.1007/s10508-021-02053-1

Vera-Gray, F., McGlynn, C., Kureshi, I., & Butterby, K. (2021). Sexual violence as a sexual script in mainstream online pornography. The British Journal of Criminology, 61(5), 1243–1260. https://doi.org/10.1093/bjc/azab035

West, C. (2019). Pornography and ethics: An interview with porn performer Blath. Porn Studies, 6(2), 264–267. https://doi.org/10.1080/23268743.2018.1505540

Williams, L. (1989). Hard Core: Power, pleasure, and the frenzy of the visible. University of California Press.

Degradation (Portrayals of Sexuality in Pornography)

Nicola Döring, Dan J. Miller — Mon, 24 Oct 2022 00:00:00 +0200

Pornography is a fictional media genre that depicts sexual fantasies and explicitly presents naked bodies and sexual activities for the purpose of sexual arousal (Williams, 1989; McKee et al., 2020). Regarding media ethics and media effects, pornography has traditionally been viewed as highly problematic. Pornographic material has been accused of portraying sexuality in unhealthy, morally questionable and often sexist ways, thereby harming performers, audiences, and society at large. In the age of the Internet, pornography has become more diverse, accessible, and widespread than ever (Döring, 2009; Miller et al., 2020). Consequently, the depiction of sexuality in pornography is the focus of a growing number of content analyses of both mass media (e.g., erotic and pornographic novels and movies) and social media (e.g., erotic and pornographic stories, photos and videos shared via online platforms). Typically, pornography’s portrayals of sexuality are examined by measuring the prevalence and frequency of sexual practices or relational dynamics and related gender roles via quantitative content analysis (for research reviews see Carrotte et al., 2020; Miller & McBain, 2022). This entry focuses on the representation of degradation as one of eight important dimensions of the portrayals of sexuality in pornography.

Field of application/theoretical foundation:

In the field of pornographic media content research, different theories are used, mainly 1) general media effects theories, 2) sexual media effects theories, 3) gender role, feminist and queer theories, 4) sexual fantasy and desire theories, and different 5) mold theories versus mirror theories. The DOCA entry “Conceptual Overview (Portrayals of Sexuality in Pornography)” introduces all these theories and explains their application to pornography. The respective theories are applicable to the analysis of the depiction of degradation as one dimension of the portrayals of sexuality in pornography.

References/combination with other methods of data collection:

Example studies for manual quantitative content analyses:

A common research hypothesis states that pornography often depicts sexuality which is degrading towards women (by men). (Conversely, an indicator against degradation is the depiction of sexual agency of women, i.e., representations of women actively initiating and guiding sexual encounters, and enjoying self-determined and reciprocal sex acts.) To test such hypotheses and code pornographic material accordingly, it is necessary to clarify the concept of “degradation” and use valid and reliable measures of different types of degradation. In addition, it is necessary to code the sex/gender of the person depicted as the source and/or the target of the respective degrading act.

It is important to note that in the context of pornographic content research, researchers conceptualize degradation differently. Also, it should be noted, that there is some overlap between the variable degradation and the variable violence in the context of pornographic portrayals of sexuality. For example, the depiction of “name calling” in a pornographic scene can be understood as an indicator of “violence” (namely verbal aggression) or of “degradation”. Name calling is covered as verbal aggression (following Fritz et al., 2020; see DOCA entry “Violence (Portrayals of Sexuality in Pornography)”), hence, it is not covered here again as degradation, even though some authors do so (such as Gorman et al., 2010). In general, one can argue that all violent acts – apart from being potentially painful and harmful – have a component of degradation because they put the target of violence in a subordinate role. However, not all degrading acts are violent (e.g., degradation by systematic lack of sexual reciprocity does not entail overt aggression).

Coding Material	Measure	Operationalization (excerpt)	Reliability	Source
*Degradation:* Degradation in the context of pornography is defined as a depiction of sexuality that is not characterized by mutuality, respect and equal power but instead is characterized by non-reciprocity, inequality, dominance, objectification and dehumanization, usually with men in the superior role and women in the subordinate role (Cowan & Dunn, 1994). Several variables indicating degradation during sex have been developed and are measured together with the sex/gender of persons involved, such as unreciprocated sex (e.g., female performer gives oral sex but does not receive it; male performer orgasms but female performer does not), status inequality (e.g., male performer depicted as older, better educated, more affluent than female performer), expressions of dominance (e.g., male performer ties female performer up or orders her around), objectification (e.g., male performer ejaculates on female performer’s body or face; gaping of the vagina or anus; double penetration of vagina or anus of the female performer) and dehumanization (e.g., male performer urinates on female performer’s body). While consensus can be reached between some researchers and media users that respective sex acts appear degrading to them (Cowan & Dunn, 1994), others disagree and either do not find these acts inherently degrading or recognize that they may be part of sexual fantasies and role play of degradation (Miller & McBain, 2022). Apart from issues of performer health protection, degrading acts are also regarded as relevant in terms of modelling behaviors for audiences.
N=45 pornographic videos from 15 different adult websites (3 videos per website)	Display of body	Being degraded: Actor displayed showing a higher level of nudity in comparison to co-actor(s). Binary coding (1: yes; 2: no).	Percentage agreement 100% for all degradation variables in codebook	Gorman et al. (2010)
	Domination	Degrading another: Actor displayed showing control and being in the dominating position, i.e. directing the co-actor(s) and the sexual acts. Binary coding (1: yes; 2: no).
	Submission	Being degraded: Actor displayed in the submissive role, i.e. following demands, allowing to be moved in any position. Binary coding (1: yes; 2: no).
	Ejaculation onto the face	Being degraded: Actor’s face or mouth displayed as being ejaculated on. Binary coding (1: yes; 2: no).
	Exploitation	Degrading another: Actor displayed as using another with less power (e.g., due to age, social status, social role) as sexual object. Binary coding (1: yes; 2: no).
	Lack of reciprocity	Degrading another: Actor displayed as disregarding mutuality and reciprocity during sexual acts and focusing only on their own satisfaction. Binary coding (1: yes; 2: no).