Inaccuracies and exaggerations (Health Coverage)

Abstract

Exaggerated or simplistic media coverage on health issues is often blamed for affecting public health (Sumner et al., 2016). For example, MacDonald and Hoffman-Goetz (2002) have shown that cancer information in newspapers frequently contained inaccuracies in the past. However, more recent findings suggest that inaccuracies, like an oversimplified language, and exaggerations are already present in health news press releases (Brechman et al., 2009; Sumner et al., 2016). 

Field of application/theoretical founation:

Health communication, science communication

Example studies:

Brechman et al. (2009); MacDonald & Hoffman-Goetz (2002); Sumner et al. (2014); Sumner et al. (2016)

Information on Brechman et al., 2009

Authors: Jean M- Brechman, Chul-joo Lee, Joseph N. Cappella

Research question: The study explores the communication of genetic science to the lay public. To address this issue, this study compares the presentation of genetic research relating to cancer outcomes and behaviors (i.e., prostate cancer, breast cancer, colon cancer, smoking and obesity) in the press release (N = 23) to the presentation in the subsequent news coverage (N = 71).

Object of analysis: The total sample included N = 71 articles on gene/cancer-outcome discoveries from major U.S. newspapers (no further information) as well as all corresponding press releases (N  = 23) from institution web sites and EurekAlert! or PRNewswire (electronic archives of releases for science writers).

Time frame of analysis: July 2004 to June 2007

Info about variables

Variables: Coding schema to capture conceptual and contextual differences between information presented in the press release and information presented in related news coverage; codes used to make these distinctions included overgeneralization/ simplification, assimilation of speculation into fact, contradiction, and level of specificity/qualifying information.

Reliability: In order to assess reliability, five cases containing 109 claims were coded by two independent coders. Overall agreement was 79.8%.

Level of analysis: Central claims on genetic research relating to cancer outcomes and behaviors in press release and media articles

Information on MacDonald & Hoffman-Goetz, 2009

Authors: Megan M. MacDonald, Laurie Hoffman-Goetz

Research question: The purpose of this study was to determine whether cancer articles in Canadian newspapers provide accurate cancer information relative to the original scientific sources of the information and the extent of mobilizing information about cancer prevention and treatment. A second objective was to determine whether newspaper circulation size influenced the accuracy of reporting of cancer information.

Object of analysis: From a total of 38 newspapers serving Ontario, the top 5 and bottom 5 newspapers in terms of circulation were identified for extreme group comparisons. All articles including the term “cancer” in the headline were extracted and a random sampling led to a total sample of N = 306 articles, including The Toronto Star (n = 63), The Ottawa Citizen (n = 49), The Hamilton Spectator (n = 53), The London Free Press (n = 42) and The Windsor Star (n = 30) as top 5 newspapers as well as. the Pembroke Daily Observer (n = 12), Lindsay Daily Post (n = 20), Northern Daily News (Kirkland Lake) (n = 12), Cobourg Daily Star (n = 10) and The Daily Miner & News (Kenora) (n = 15) as bottom 5.

Time frame of analysis: 1991

Info about variables

Variables: The accuracy of each article was assessed using the following criteria: misleading title, treating speculation as fact, erroneous information, omitting important results and omitting qualifications or caveats to findings.

Reliability: The articles were coded separately by the researchers using the identified criteria. Where discrepancies occurred in coding results, these were discussed until a consensus was met. Consensus discussions occurred early in data collection to allow this process to inform and direct future coding (no further information provided).

Level of analysis: article

Information on Sumner et al., 2014

Authors: Petroc Sumner, Solveiga Vivian-Griffiths, Jacky Boivin, Andy Williams, Christos A Venetis, Aimée Davis, Jack Ogden, Leanne Whelan, Bethan Hughes, Bethan Dalton, Fred Boy, Christopher D Chambers

Research question: The study examines whether the press release or the news article are the source of distortions, exaggerations, or changes to the main conclusions drawn from research that could potentially influence a reader’s health related behaviour.

Object of analysis: Press releases (n = 462) on biomedical and health related science issued by 20 leading UK universities, alongside their associated peer reviewed research papers and news stories (n = 668).

Time frame of analysis: 2011

Info about variables

Variables: Taking the peer reviewed paper as a baseline, the authors sought cases where news stories offered advice to readers, made causal claims, or inferred relevance to humans beyond (or different to) that stated in the associated peer reviewed paper. Given the likelihood that some statements in journal articles themselves would be considered exaggerated by other scientists in the specialty, the overall levels of measured exaggeration are likely to be underestimates. The authors then asked whether such discrepancies were already present in the corresponding press release. For example, if a study reported a correlation between stress and wine consumption and the news story claimed that wine causes stress, what did the press release say? Similarly, if a news story claimed a new treatment for humans but the study was on rodents, what did the press release say?

Full coding guidelines: https://figshare.com/articles/InSciOut/903704

“Is there a generalisation?”: these variables provide information on whether exaggerations have occurred between the journal article and abstract, press release, or news report(s)

• No generalisation – yes/ no
• minor generalisation - yes/ no
• major generalisation - yes/ no
• Justification offered for generalisation between actual study and abstract / press release /news report - yes/ no

Reliability: no information provided

Level of analysis: article

Information on Sumner et al., 2016

Authors: Petroc Sumner, Solveiga Vivian-Griffiths, Jacky Boivin, Andrew Williams, Lewis Bott, Racel Adams, Christos A Venetis, Aimée Davis, Leanne Whelan, Bethan Hughes, Christopher D Chambers

Research question: Recent findings suggested many exaggerations in the portrayal of health information were already present in university press releases, which scientists approve. Surprisingly, these exaggerations were not associated with more news coverage. This study examines whether these two controversial results also arise in press releases from prominent science and medical journals.

Object of analysis: press releases (n = 534) on biomedical and health-related science issued by leading peer-reviewed journals. The authors similarly analysed the associated peer-reviewed papers (n = 534) and news stories (n = 582).

Time frame of analysis: 2011

Info about variables

Variables: The process of data extraction and analysis was identical to that in Sumner et al. (2014).

Full coding guidelines: https://figshare.com/articles/InSciOut/903704

References

Brechman, J. M., Lee, C.?J., & Cappella, J. N. (2009). Lost in Translation? A Comparison of Cancer-Genetics Reporting in the Press Release and its Subsequent Coverage in Lay Press. Science Communication, 30(4), 453–474. https://doi.org/10.1177/1075547009332649

MacDonald, M. M., & Hoffman-Goetz, L. (2002). A Retrospective Study of the Accuracy of Cancer Information in Ontario Daily Newspapers. Canadian Journal of Public Health / Revue Canadienne De Sante'e Publique, 93(2), 142–145. www.jstor.org/stable/41993460

Sumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Venetis, C. A., Davies, A., Ogden, J., Whelan, L., Hughes, B., Dalton, B., Boy, F., & Chambers, C. D. (2014). The association between exaggeration in health related science news and academic press releases: Retrospective observational study. BMJ (Clinical Research Ed.), 349, g7015. https://doi.org/10.1136/bmj.g7015

Sumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Bott, L., Adams, R., Venetis, C. A., Whelan, L., Hughes, B., & Chambers, C. D. (2016). Exaggerations and Caveats in Press Releases and Health-Related Science News. PloS One, 11(12), e0168217. https://doi.org/10.1371/journal.pone.0168217