Eye tracking data (Frequently Applied Designs)

Abstract

Eye tracking can be used to record individual search processes on the Internet and the eye movements of subjects searching for information. These search processes combined with the gaze data can be examined by means of standardized content analysis. 

Field of application/theoretical foundation:

Reception, perception, and selection behavior on the Internet; selective exposure and framing effects studies; role of news factors in selection on the Internet

References/combination with other methods of data collection:

Zillich and Kessler (2019) evaluate and compare the method combination with the advantages and disadvantages of established methods for measuring selective exposure processes with regard to online information. Kessler and Guenther (2017), Kessler and Langmann (2020), and Kessler and Zillich (2019) combined the content analysis of eye tracking data with an online pre- and post-survey. Kessler and Engelmann (2019) compare findings on the role of news factors in online news selection from three different methods: standardized content analysis of eye tracking data, qualitative open survey, and standardized closed survey.

Example studies:

Kessler & Langmann (2020); Kessler & Engelmann (2019); Kessler & Zillich (2019); Kessler & Guenther (2017); Zillich & Kessler (2019)

Information on Kessler and Guenther (2017)

Authors: Sabrina Heike Kessler & Lars Guenther                             

Research question: Does individual online behavior (i.e., searching for and reading information) conform to previously presented media frames? How much impact do presented media frames have on different levels of individuals’ online searching for and on reading information?

Object of analysis: In an experiment combining eye tracking and content analysis, participants (N = 72) were exposed to one of three TV clips with different media frames that focused on Alzheimer’s disease. After exposure, participants informed themselves about the issue online. The researchers examined the online search behavior via eye tracking while the participants searched for information, followed by a standardized content analysis of the eye tracking data.

Timeframe of analysis: 2014

Codebook: in the appendix (in German)

Info about variables

Construct: online behavior, the way people search for and read information on the Internet

Level of analysis: levels of online behavior: input words participants used for their online searches; search results from the online search engines that were both viewed and selected; search results that were viewed but not selected; and the content of viewed websites.

Variables: Duration of the search behavior; duration of the reception behavior; words typed into the search engine; website visited; categorization of the website; number of search results received and selected or received and not selected; main topic, problem, cause, evaluation, opportunities and risks, forecasts, proposed solution and demands of the individual search results; number and type of actors of the individual selected and unselected search results; number of contributions received; main topic, problem, cause, evaluation, opportunities and risks, forecasts, proposed solution and demands of the individual websites received; depicted controversy, images, explicit and implicit certainty and uncertainty at the individual websites received

Reliability: “Nine trained coders helped conduct the content analysis. In total, 12 clips of participants’ online behavior (17 percent of the total sample) were randomly selected for reliability testing. For intercoder reliability, Cohen’s ? for the formal variables was ?=0.98 (CR=0.99). Intercoder reliability for the variables of the frame elements had the following values: selected search results ?=0.77 (CR=0.97), search results that were viewed but not selected ?=0.81 (CR=0.97), and contents of the websites that were viewed ?=0.71 (CR=0.93).” (p. 316)

Information on Kessler and Langmann (2020)

Authors: Sabrina Heike Kessler & Klara Langmann                           

Research question: How does biological sex influence search behavior for political information on the Internet? Hypothesis: Prior political knowledge, political interest, and Internet skills mediate the influence of biological and social sex on search behavior for political information on the Internet.

Object of analysis: This study aimed to investigate how people (N = 44 students) search online for political information (N = 220 search tasks) and if gendered online search exist. We examined the online search behavior via eye tracking while the participants searched for information about political party positions on the Internet. A content analysis of the eye tracking data followed and was evaluated with a special focus on the role of biological sex and social gender and the relationship of both factors with other variables, such as self-reported prior political knowledge, political interest, and Internet skills (via online survey).

Timeframe of analysis: April 2017       

Codebook: in the appendix (in German)

Info about variables

Construct: online search behavior, the way people search for and read information on the Internet

Level of analysis: levels of online behavior: input words participants used for their online searches; search results on search engine result pages (SERPs) that were both viewed and selected, search results that were viewed but not selected; and the content of viewed websites. 

Variables and reliability: Four coders conducted the content analysis by satisfying reliability values (based on 11.4% of the total sample, randomly selected).

Variables	Measures	Reliability coefficient (Krippendorff's ?); n=25
Search task	5 values	1
Length of online search behavior	seconds	1
Number of search queries on SERPs	0 to x	1
Time on SERPs	seconds	.81 (10% tolerance)
Scanpath on SERPs	4 values	.78
Length of search queries	3 values	.96
Number of clicked search results	0 to x	.96
Number of viewed and unselected search results	0 to x	.67 (10% tolerance)
Number of selected search results position 1	0 to x	.98
Number of selected search results position 2–3	0 to x	.98
Number of perceived websites	0 to x	.93
Type of website accessed	19 values	.88
Website scanpaths	4 values	.81
Reception scope on website	3 values	.84
Time on websites	seconds	.81 (10% tolerance)

References

Kessler, S. H. & Langmann, K. (2020). The role of sex and gender on search behavior for political informationon the Internet. Communications: The European Journal of Communication Research. DOI: 10.1515/commun-2019-0137

Kessler, S. H. & Engelmann, I. (2019). Why do we click? Investigating reasons for user selection on a news aggregator website. Communications, 44(2), 225-247. DOI: 10.1515/commun-2018-2003

Kessler, S. H. & Zillich, A. F. (2019). Searching online for information about vaccination: Assessing the influence of user-specific cognitive factors using eye-tracking. Health Communication, 34(10), 1150-1158. DOI: 10.1080/10410236.2018.1465793

Kessler, S. H. & Guenther, L. (2017). Eyes on the frame: Explaining people's online searching behavior in response to TV consumption. Internet Research, 27(2), 303-320. DOI: 10.1108/IntR-01-2016-0015

Zillich, A. F., & Kessler, S. H. (2019). Measuring selective exposure to online information. Combining eye-tracking and content analysis of users’ actual search behavior. In C. Peter, T. Naab, & R. Kühne (eds.), Measuring media use and exposure: Recent developments and challenges (pp. 196-220). Köln, GER: Halem.