Everyday Research Methods

Training tools for information literacy

Photo: fotogestoeber/Shutterstock

More than ever, our citizenry needs skills for information literacy. A recent study of almost 4000 U.S. teens, conducted by Stanford's History Education group, concluded that teenagers' ability to think critically about online information is "troubling." Their report included the findings that: 

• Fifty-two percent of students believed a grainy video claiming to show ballot stuffing in the 2016 Democratic primaries (the video was actually shot in Russia) constituted “strong evidence” of voter fraud in the U.S. [...]
• Two-thirds of students couldn’t tell the difference between news stories and ads (set off by the words “Sponsored Content”) on Slate’s homepage.
• Ninety-six percent of students did not consider why ties between a climate change website and the fossil fuel industry might lessen that website’s credibility. [...]

If you're inspired to build skills in your own students, consider assigning an activity designed to develop students' critical reading skills as a supplement to Chapter 2. The following tools have been developed by experts in library science, communication, and social cognition. All of these raise our awareness of the information we may encounter online: 

Games that Teach

1. The free online game Bad News  trains people to spread false ideas with the goal of teaching them how propaganda works. It also includes an (optional) embedded test of people's ability to detect real and fake Twitter accounts. Users earn badges for impersonating experts, exploiting emotion, and polarization. It's a legitimately fun game! What's more, two recent studies have indicated that this game improves people's reasoning about the news (Roozenbeek et al., 2019;  Basol et al., 2020).

2. The Fake News Game (still in development) plans to teach about fake news by challenging people to create their own fake news, building in elements intended to make it go viral. 

Online Curricula

3. Stanford's Civic Online Reasoning (COR) course presents several planned sets of lessons:

• • An three-lesson introduction to skeptical reading, looking for evidence, and fact-checking
  • An six-lesson overview of lateral reading, the strategy used by professional fact-checkers (and a skill associated with better information literacy)
  • A combination package of six lessons offering a little bit of everything (lateral reading, introduction to evidence, and "click restraint"). This might be the package to assign if you're only assigning one set of lessons.

4. Carl Bergstrom and Jevin West's site, Calling Bullshit, offers a syllabus for a full semester college course on misinformation. You can also assign some of their "tools and tricks" to give people practice in the following skills:

• • Spotting misleading axes and formatting in graphs
  • Deciding if a scientific paper is legitimate
  • Deciding if an online face is real or computer-generated
  • They also have case studies in misinformation, sorted by difficulty. 

5. Morton Ann Gernsbacher has developed and shared her free, online methods course focused primarily on scientific information literacy. Check it out for a full set of resources!

My own students have benefitted from working through various modules from these resources, in both methods and content courses. 

I'm grateful to David Dunning (University of Michigan), who shared some of these resources at his NITOP general session. I'm also grateful to UD librarians Meg Grotti and Alison Wessel for sharing resources with me. 

Screen time (again!)

The effect of screen time on well-being is likely to depend on moderators such as what's on the screen, when it's used, and who's using it.
Photo: Eric Nathan / Alamy Stock Photo

I can't promise this is the last article on screen time research that I will blog about. But this long-form piece by journalist Kim Tingley was irresistable because it illustrates several of the concepts in the Research Methods course. (Instructors--it would make a nice final exam review.)

The questions here are set up as an end-of-semester scavenger hunt. After you read each question, read a bit further down in the article and find the quote or paragraph section that fits that question. 

Questions:

a) There are two frequency claims that open this piece. What are they? 

b) What's the first association claim presented in this article? What are the variables in the claim? 

c) The second association claim contradicts the first one. What is the second association claim and how does it differ? 

d) Reading on, you'll find some information relevant to the external validity of these studies. What is the relevant information? 

e) The journalist makes a claim about self-report data. What is it? What concept in Chapter 5 is being discussed here?

f) The journalist does a quick job of covering the three criteria for causation next. Which passage addresses that topic?

g)  Next the journalist walks us through the potential for p-hacking in the Monitoring the Future (M.T.F.) survey. Which paragraphs address this?

h) Next look for a discussion of effect size (and especially effect size in context) in these data. 

i) What about moderators? The article has those too. Which passages address moderators? Identify the core relationship of interest (the two variables) and identify at least one moderator. You can make a sentence of this format: _______ might moderate the relationship between _____ and ______.

j) Finally, ethics. Which passage addresses this issue in screen time? 

 

Suggested Answers

a) The two frequency claims are "95 percent of teenagers had access to a smartphone, and 45 percent said they were online 'almost constantly.'"

b) "the longer adolescents were engaged with screens, the greater their likelihood of having symptoms of depression or of attempting suicide."  The variables are Time engaged with screens  and Likelihood of depression or suicide attempts"

c) The second claim is about  "a tenuous relationship between adolescent well-being and the use of digital technology". 

d) The journalist mentions that epidemiological surveys involve "conducting phone interviews with thousands of randomly selected people," which would enhance external validity. 

e) She mentions "For starters, people are notoriously bad at self-reporting how often they do something or how they feel." This is probably about criterion validity. People's self reports (for example, of "how often they use their phones" may not correlate with their actual behavioral phone use.  (For the record, I disagree with the journalist's point that people are bad at self-reporting how they feel. How else will you find out about a person's subjective state?)

f) She writes, "there’s no way to say if the technology use caused their low mood or vice versa" (that's temporal precedence) and "or if other factors were involved" (That's internal validity.)

g) The paragraphs that start "A researcher must decide..." and "To show how many legitimate outcomes..." provide excellent overviews of the potential for unintentional p-hacking.

h) The journalist is discussing effect size when she describes "smoking marijuana and being bullied were more closely linked with decreased well-being than tech use was" and "the strength of the association screen time had with well-being was similar to neutral factors like wearing glasses or regularly eating potatoes."

i) The phrase "depends on" is a good signal that there's a moderator.  Your sentence might be:
The type of person (perhaps good student vs. poor student)  might moderate the relationship between screen time and well-being. 

j) Ethics come into play when we ask whether companies should be collecting data on our screen use, and if they are doing so, whether they have our permission and if they should share that data with researchers.