Everyday Research Methods

Claim: "Going to museums and concerts helps you live longer"

Does the study's method support the claim that attending arts events makes you live longer? Photo credit: agefotostock/Alamy

We'll never run out of material in the "correlation is not causation" department. 

This time, a New York Times  journalist led with the headline, "Another benefit of going to museums? You may live longer." 

Let's check out the study behind this causal claim. 

According to the journalist's summary of the empirical article, 

Researchers in London who followed thousands of people 50 and older over a 14-year period discovered that those who went to a museum or attended a concert just once or twice a year were 14 percent less likely to die during that period than those who didn’t.

The chances of living longer only went up the more frequently people engaged with the arts, according to the study, which was published this month in The BMJ, formerly The British Medical Journal. People who went to a museum or the theater once a month or even every few months had a 31 percent reduced risk of dying in that period, according to the study.

a) Starting just from here--what are the two variables they've mentioned so far? Were they manipulated or measured? 

b) Based on what you've read so far, is this study correlational or experimental? 

c) Without reading further, think of a third variable (a "C") that is likely to be correlated with both A (going to museums or concerts) and B (being less likely to die). Any good ideas? 

OK! Time for more information about the study from the journalist:

The study controlled for socioeconomic factors like a participant’s income, education level and mobility, said Andrew Steptoe, a co-author of the study and the head of University College London’s research department of behavioral science and health.

“Even if you take those things into account, we still found that there is a difference in the survival of people who are involved in the arts,” Professor Steptoe said in an interview on Friday.

 

Sounds like multiple regression was used here.

d) What was the criterion (dependent) variable in this study now?

e) What were the predictor ("independent") variables? 

f) Sketch a little regression table similar to those in Chapter 9--put the criterion variable at the top and the predictors at the bottom. Which betas do you think were significant?

g) More importantly, was the "C" variable you thought about in c), above, controlled in their regression?

h) Even though the authors controlled for income, education, and mobility, there are likely to be additional third variables that were NOT controlled for. Think of at least one more alternative explanation. 

 

As you learn in Chapter 9, regression does not support causation because even though it can control for alternative explanations you can think of (such as income, education, or mobility), it doesn't control for everything. If you read the Time's summary, you'll see even more variables they seem to have controlled for , such as marital status and profession. But there could still be unmeasured variables that are responsible for the relationship. 

 

But that doesn't stop the authors from making causal speculations. In fact, here's a mediator hypothesis. i) Sketch this mediator following the diagrams at the end of Chapter 9:

 

The study also noted that engaging in the arts can reduce loneliness, promote empathy and emotional intelligence, and keep people from becoming sedentary — all factors that contribute to a longer life.

 

What kind of study would you need to conduct to test this proposed mediator?

 

 

 

 

 

 

Does this font make me look conservative?

Of these three candidates, Yang is considered more liberal than Klobuchar. Do the fonts used in their posters convey that? Photo credit: Kevin McGovern/Shutterstock

Here's something I never considered--different fonts lean conservative or liberal. 

Serif fonts have little curves and edges (like the "Amy" in the green poster here). Sans-serif fonts are cleaner, without these extra flourishes. 

A study showed multiple fonts to a sample of people and found that people rated the serif fonts as looking more conservative and sans-serif fonts as more liberal.

This research was covered by CNN.com. They summarized the results like this: 

For example, study participants saw Times New Roman as more conservative than Gill Sans. Blackletter, which looks like it belongs on a newspaper masthead, was seen as the most conservative font, while Sunrise, a cartoonish-looking script, was seen as the most liberal.

When they interviewed the researcher, she suggested, 

"If you think about serifs being used in more formal types of print or communications, maybe they're viewed as more traditional and sans serifs are viewed as more modern," Katherine Haenschen, an assistant professor of communications at Virginia Tech and the lead author of the study, told CNN. "There's a small but significant difference in how people perceive these fonts."

The researchers showed each font using a simple sentence, "The quick brown fox jumped over the lazy dog" and asked online participants to rate the font's conservative (vs. liberal) nature. For example:

The quick brown fox jumped over the lazy dog

The quick brown fox jumped over the lazy dog

The quick brown fox jumped over the lazy dog

a) In this study, what was the independent variable (IV)? What were its levels? What was the dependent variable? 

Here is CNN's description:

...987 participants were shown the phrase "The quick brown fox jumped over the lazy dog" in one of six fonts and styles: Times New Roman regular, Times New Roman bold, Times New Roman italic, Gill Sans regular, Gill Sans bold or Gill Sans italic.

b) Based on this description, was the IV manipulated as independent groups or within-groups? 

c) You could do the study either way. What are the advantages and disadvantages of manipulating the font variable as independent groups?What are the advantages and disadvantages of manipulating the font variable as within-groups?

d) What does it mean when the author says "There's a small but significant difference in how people perceive these fonts."? 

At the end of the story, the CNN journalist wrote about the potential applications: 

For example, if Bernie Sanders changed the font on his campaign materials, there probably wouldn't be much of a difference in how people see him because he's already a widely known figure.

The choice of font could, however, make a difference for a new candidate, like someone running for school board, town council or state legislature, Haenschen said. While researchers don't know for sure whether a font would change people's perceptions of a candidate, that's another question that could be explored in the future.

e) How might you conduct an experimental test of the journalist's speculation, above? 

You can see the empirical journal article here. Their Figure 1 shows ratings of conservatism and the 95% CIs for six fonts. 

 

 

Training tools for information literacy

Photo: fotogestoeber/Shutterstock

More than ever, our citizenry needs skills for information literacy. A recent study of almost 4000 U.S. teens, conducted by Stanford's History Education group, concluded that teenagers' ability to think critically about online information is "troubling." Their report included the findings that: 

• Fifty-two percent of students believed a grainy video claiming to show ballot stuffing in the 2016 Democratic primaries (the video was actually shot in Russia) constituted “strong evidence” of voter fraud in the U.S. [...]
• Two-thirds of students couldn’t tell the difference between news stories and ads (set off by the words “Sponsored Content”) on Slate’s homepage.
• Ninety-six percent of students did not consider why ties between a climate change website and the fossil fuel industry might lessen that website’s credibility. [...]

If you're inspired to build skills in your own students, consider assigning an activity designed to develop students' critical reading skills as a supplement to Chapter 2. The following tools have been developed by experts in library science, communication, and social cognition. All of these raise our awareness of the information we may encounter online: 

Games that Teach

1. The free online game Bad News  trains people to spread false ideas with the goal of teaching them how propaganda works. It also includes an (optional) embedded test of people's ability to detect real and fake Twitter accounts. Users earn badges for impersonating experts, exploiting emotion, and polarization. It's a legitimately fun game! What's more, two recent studies have indicated that this game improves people's reasoning about the news (Roozenbeek et al., 2019;  Basol et al., 2020).

2. The Fake News Game (still in development) plans to teach about fake news by challenging people to create their own fake news, building in elements intended to make it go viral. 

Online Curricula

3. Stanford's Civic Online Reasoning (COR) course presents several planned sets of lessons:

• • An three-lesson introduction to skeptical reading, looking for evidence, and fact-checking
  • An six-lesson overview of lateral reading, the strategy used by professional fact-checkers (and a skill associated with better information literacy)
  • A combination package of six lessons offering a little bit of everything (lateral reading, introduction to evidence, and "click restraint"). This might be the package to assign if you're only assigning one set of lessons.

4. Carl Bergstrom and Jevin West's site, Calling Bullshit, offers a syllabus for a full semester college course on misinformation. You can also assign some of their "tools and tricks" to give people practice in the following skills:

• • Spotting misleading axes and formatting in graphs
  • Deciding if a scientific paper is legitimate
  • Deciding if an online face is real or computer-generated
  • They also have case studies in misinformation, sorted by difficulty. 

5. Morton Ann Gernsbacher has developed and shared her free, online methods course focused primarily on scientific information literacy. Check it out for a full set of resources!

My own students have benefitted from working through various modules from these resources, in both methods and content courses. 

I'm grateful to David Dunning (University of Michigan), who shared some of these resources at his NITOP general session. I'm also grateful to UD librarians Meg Grotti and Alison Wessel for sharing resources with me. 

Screen time (again!)

The effect of screen time on well-being is likely to depend on moderators such as what's on the screen, when it's used, and who's using it.
Photo: Eric Nathan / Alamy Stock Photo

I can't promise this is the last article on screen time research that I will blog about. But this long-form piece by journalist Kim Tingley was irresistable because it illustrates several of the concepts in the Research Methods course. (Instructors--it would make a nice final exam review.)

The questions here are set up as an end-of-semester scavenger hunt. After you read each question, read a bit further down in the article and find the quote or paragraph section that fits that question. 

Questions:

a) There are two frequency claims that open this piece. What are they? 

b) What's the first association claim presented in this article? What are the variables in the claim? 

c) The second association claim contradicts the first one. What is the second association claim and how does it differ? 

d) Reading on, you'll find some information relevant to the external validity of these studies. What is the relevant information? 

e) The journalist makes a claim about self-report data. What is it? What concept in Chapter 5 is being discussed here?

f) The journalist does a quick job of covering the three criteria for causation next. Which passage addresses that topic?

g)  Next the journalist walks us through the potential for p-hacking in the Monitoring the Future (M.T.F.) survey. Which paragraphs address this?

h) Next look for a discussion of effect size (and especially effect size in context) in these data. 

i) What about moderators? The article has those too. Which passages address moderators? Identify the core relationship of interest (the two variables) and identify at least one moderator. You can make a sentence of this format: _______ might moderate the relationship between _____ and ______.

j) Finally, ethics. Which passage addresses this issue in screen time? 

 

Suggested Answers

a) The two frequency claims are "95 percent of teenagers had access to a smartphone, and 45 percent said they were online 'almost constantly.'"

b) "the longer adolescents were engaged with screens, the greater their likelihood of having symptoms of depression or of attempting suicide."  The variables are Time engaged with screens  and Likelihood of depression or suicide attempts"

c) The second claim is about  "a tenuous relationship between adolescent well-being and the use of digital technology". 

d) The journalist mentions that epidemiological surveys involve "conducting phone interviews with thousands of randomly selected people," which would enhance external validity. 

e) She mentions "For starters, people are notoriously bad at self-reporting how often they do something or how they feel." This is probably about criterion validity. People's self reports (for example, of "how often they use their phones" may not correlate with their actual behavioral phone use.  (For the record, I disagree with the journalist's point that people are bad at self-reporting how they feel. How else will you find out about a person's subjective state?)

f) She writes, "there’s no way to say if the technology use caused their low mood or vice versa" (that's temporal precedence) and "or if other factors were involved" (That's internal validity.)

g) The paragraphs that start "A researcher must decide..." and "To show how many legitimate outcomes..." provide excellent overviews of the potential for unintentional p-hacking.

h) The journalist is discussing effect size when she describes "smoking marijuana and being bullied were more closely linked with decreased well-being than tech use was" and "the strength of the association screen time had with well-being was similar to neutral factors like wearing glasses or regularly eating potatoes."

i) The phrase "depends on" is a good signal that there's a moderator.  Your sentence might be:
The type of person (perhaps good student vs. poor student)  might moderate the relationship between screen time and well-being. 

j) Ethics come into play when we ask whether companies should be collecting data on our screen use, and if they are doing so, whether they have our permission and if they should share that data with researchers. 

 

Research on Native American mascots

The College of William and Mary has phased out its Indian-themed mascot over the years. In 2010, the College replaced its former mascot with "Griffin", above. Photo: Mitchell Layton/Getty Images

Since 2005, the NCAA has required colleges and universities to reflect upon, and if necessary, change sports team mascots that feature depictions of Native-Americans.  Now a team of psychology researchers have published an investigation about support for such mascots among different groups of people. The set of studies  includes both correlational and experimental designs, including one testing whether the mascots can affect people's donations to the universities. 

The lead researcher, Dr. Michael Kraus, told a journalist what led to his interest in the research question. 

“I was at the University of Illinois at the time, and despite their Native American mascot being officially discontinued in 2007, it was surprising to see the mascot on campus in 2013. The continued presence said something about the norms on campus, and what kinds of students feel safe and welcome there,” Kraus said.

The journalist summarized two studies conducted by Kraus and his team. Here's the first:

In a survey of 201 students, the researchers found that those with lower levels of explicit prejudice toward Native Americans reported lower levels of belonging at the University of Illinois.

In other words, students who disagreed with statements such as “It is now unnecessary for the U.S. government to honor their treaty obligations to Native tribes” also tended to disagree with statements such as “I identify with the university” and “I show school spirit regularly.”

In addition, students with higher levels of explicit prejudice, as well as white students, were more likely to have positive views of the retired “Chief” mascot.

Questions

a) This is a bivariate correlational study. What are its two main variables?

b) The journalist gives some clues about how each of these two variables was measured. What kinds of measurements were used--self-report? physiological? observational? What do you think of the items? 

c) Sketch a scatterplot of the results that are described. Label your axes carefully.

d) Can this study support the causal claim? Why or why not? (Apply the three causal criteria; when you come to internal validity, specify at least one third variable that might be associated with both measured variables.)

The empirical article contains several studies on this topic. One of them tested the impact of  the University of Illinois' retired mascot, Fighting Illini, on donations to the school. 

The experiments showed that stereotypic Native American imagery “reduced actual donations to the institution by 5.5%, and was particularly likely to decrease (increase) donations among those low (high) in prejudice,” the researchers wrote.

In the study, researchers had online participants read about different universities, one of which was the University of Illinois. Some participants  were randomly assigned to see the Native American mascot, Chief Illini, next to the school name, while some did not.  Then participants had the chance to distribute $2 in donations to the four universities. When people had been exposed to the  Native American mascot, donations to the University of Illinois decreased. 

But like all research, the study includes some caveats. “Probably the biggest one is that we really only studied these dynamics at the University of Illinois, but we expect these norms to operate similarly everywhere. Someone, maybe us, needs to do that additional work,” Kraus said.

More Questions

e) Is the donation study an experiment or a correlational study? What are its two main variables?

f) When Kraus states that "we expect these norms to operate similarly everywhere," to which of the four big validities is he referring?

You can access the empirical research here.

Instructors: The complete package of studies is a complete set with a frequency claim (via an observational study of mascot use on campus) , a correlational study, and an experimental study.

 

 

 

 

Interested students might learn from Dr. Stephanie Fryberg's work, showing how American Indian high school students respond to being reminded of professional sports mascots that feature American Indians.

Fidget spinners seem to cause worse memory and attention

The fidget spinner IV was manipulated as independent groups in the first study and as within groups in the second. Photo: Marcel Jancovic/Shutterstock

Fidget spinners were extremely popular a couple of years ago. When people promoted them, they often suggested that the toys could help people with ADHD focus and learn more. At the time, such claims were not scrutinized empirically (here's a nice summary of this lack of evidence). It takes time for empirical research to be conducted, analyzed, peer-reviewed, and published, and even though fidget spinners were hot in 2017, we now have a study that tested the effect of using a fidget spinner during a learning task. At least one press outlet covered this study. Here's their summary:

In the first experiment, 98 undergraduate students watched a 15-minute educational video lecture on baking bread while either using a fidget spinner or not using a fidget spinner [randomly assigned]. In addition, half of the participants not using a fidget spinner were randomly assigned to watch the lecture while sitting near someone using a fidget spinner.

A fill-in-the blank memory test showed that participants who used fidget spinners answered significantly fewer questions correctly about the material covered in the video. Participants who used fidget spinners were also more likely to report “zoning out” and having difficulty staying on task while watching the video.

Participants who were sitting near someone using a fidget spinner, however, did not appear to be affected.

Questions

a) This was an experiment. How do you know?

b) What were the IV and DV? How many levels were in the IV and what were they? Was the IV manipulated as independent groups or within groups? 

c) Which of the four simple experiments is this: postttest-only? Pretest-posttest? Repeated-measures? Concurrent measures? 

d) The journalist attached a causal claim to these results, saying "study finds using a fidget spinner can impair memory and attention." Apply the three criteria for causation to this claim to explain why the causal claim is probably valid. How does random assignment assist this study's internal validity?

e) Ask one question each about the study's statistical, construct, and external validities. 

The researchers replicated the effect in a second study. The wondered if fidget spinners might only work among people who believed in their powers: 

The researchers then conducted the experiment again. However, this time around they specifically recruited 48 undergraduates who believed that fidget spinners could help them focus in class.

But the results were the same.

According to the empirical journal article, the second study on 48 students was conducted as a repeated measures design, using two videos. Students used a fidget spinner during one video and no spinner during the other one. Students were tested on their recall for the videos after each one.

f) What are the IV and DV in the second study? How many levels were there in the IV?

g) How do you think the researchers used counterbalancing in this second study?  Why would that have been important? 

You can access the original empirical article here, though you may need to log in via your university library. 

 

 

 

 

 

Articles that are student-friendly

A list of recent articles that are student-accessible 

Compiled by Beth Morling and Jeong-Min Lee, Georgia State

Our students have used these articles in their final projects for Research Methods in the past. In this final project, students summarize and analyze the validities for two empirical studies. (I can provide the assignment upon request: morling@udel.edu.)

Most of these articles report more than one study, and at least one of the studies is simple enough for a student with basic skills in research methods to understand. The most complex experimental design would be a 3-way factorial design. The most complex correlational design uses multiple regression and (sometimes) mediation.

 

Brummelman, E., Thomaes, S., de Castro, B. O., Overbeek, G., & Bushman, B. J. (2014). “That’s not just beautiful—that’s incredibly beautiful!”: The adverse impact of inflated praise on children with low self-esteem. Psychological Science, 25, 728-735. doi:http://dx.doi.org/10.1177/0956797613514251

Bryan, C.J., Yeager, D.S., Hinojosa, C.P., Chabot, A., Bergen, H., Kawamura, M., & Steubing, F. (2016). Harnessing adolescent values to motivate healthier eating. PNAS, 113, 10830-10835. doi: 10.1073/pnas.1604586113

Cantú, S. M., Simpson, J. A., Griskevicius, V., Weisberg, Y. J., Durante, K. M., & Beal, D. J. (2014). Fertile and selectively flirty: Women’s behavior toward men changes across the ovulatory cycle. Psychological Science, 25, 431-438. Retrieved from http://search.proquest.com/docview/1501839110?accountid=10457

Chan, K. Q., Tong, E. M. W., & Tan, Y. L. (2014). Taking a leap of faith: Reminders of God lead to greater risk taking. Social Psychological and Personality Science, 5, 901-909. doi: 10.1177/1948550614537309

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257. https://doi.org/10.1073/pnas.1821936116

 Fraley, R.C., Griffin, B.N., Belsky, J., & Roisman, G.I. (2012). Developmental antecedents of political ideology: A longitudinal investigation from birth to age 18 years. Psychological Science, 23, 1425-1431. doi: 10.1177/0956797612440102

Francis, H. M., Stevenson, R. J., Chambers, J. R., Gupta, D., Newey, B., & Lim, C. K. (2019). A brief diet intervention can reduce symptoms of depression in young adults – A randomised controlled trial. PLOS ONE, 14(10), e0222768. https://doi.org/10.1371/journal.pone.0222768

 Frenda, S., Patihis, L., Loftus, E., Lewis, H., Fenn, K.  (2014) Sleep deprivation and false memories. Psychological Science, 25, 1674-1681. doi: 10.1177/0956797614534694

 Gino, F., & Wiltermuth, S. S. (2014). Evil genius? How dishonesty can lead to greater creativity. Psychological Science, 25, 973-981.doi: 10.1177/0956797614520714

Gino, F., Norton, M. I., & Ariely, D. (2010). The counterfeit self: The deceptive costs of faking it. Psychological Science, 21, 712-720. doi:http://dx.doi.org/10.1177/0956797610366545

Gunraj, D. N., Drumm-Hewitt, A. M., Dashow, E. M., Upadhyay, S. S. N., & Klin, C. M. (2016). Texting insincerely: The role of the period in text messaging. Computers in Human Behavior, 55, 1067–1075. https://doi.org/10.1016/j.chb.2015.11.003

House, J., DeVoe, S. E., & Zhong, C. (2014). Too impatient to smell the roses: Exposure to fast food impedes happiness. Social Psychological and Personality Science, 5, 534-541. doi:http://dx.doi.org/10.1177/1948550613511498 [use study 3]

Jirout, J. J., & Newcombe, N. S. (2015). Building blocks for developing spatial skills: Evidence from a large, representative U.S. sample. Psychological Science, 26, 302-310. doi:http://dx.doi.org/10.1177/0956797614563338

 Kruse, E., Chancellor, J., Ruberton, P. M., & Lyubomirsky, S. (2014). An upward spiral between gratitude and humility. Social Psychological and Personality Science, 5, 805-814. doi:http://dx.doi.org/10.1177/1948550614534700 [Study 2]

Lee, K., Talwar, V., McCarthy, A., Ross, I., Evans, A., & Arruda, C. (2014). Can classic moral stories promote honesty in children? Psychological Science, 25, 1630-1636. doi:10.1177/0956797614536401

Levy, K. N., & Kelly, K. M. (2010). Sex differences in jealousy: A contribution from attachment theory. Psychological Science, 21, 168-173. doi:http://dx.doi.org/10.1177/0956797609357708

Markey, P. M., Ivory, J. D., Slotter, E. B., Oliver, M. B., & Maglalang, O. (2019). He does not look like video games made him do it: Racial stereotypes and school shootings. Psychology of Popular Media Culture. https://doi.org/10.1037/ppm0000255

 Mrazek, M. D., Franklin, M. S., Phillips, D. T., Baird, B., & Schooler, J. W. (2009). Mindfulness training improves working memory capacity and GRE performance while reducing mind wandering. Psychological Science, 24, 776-781. DOI: 10.1177/0956797612459659.

Nelson, S. K., Kushlev, K., English, T., Dunn, E. W., & Lyubomirsky, S. (2013). In defense of parenthood: Children are associated with more joy than misery. Psychological Science, 24, 3-10. doi:http://dx.doi.org/10.1177/0956797612447798

 Nusbaum, E. C., & Silvia, P. J. (2011). Shivers and timbres: Personality and the experience of chills from music. Social Psychological and Personality Science, 2, 199-204. doi:http://dx.doi.org/10.1177/1948550610386810 [Focus on “personality and chills during music” and “do music preferences mediate effects of openness” results]

 Ronay, R., & von Hippel, W. (2010). The presence of an attractive woman elevates testosterone and physical risk taking in young men. Social Psychological and Personality Science, 1, 57-64. doi:http://dx.doi.org/10.1177/1948550609352807 [focus on “skateboarding” and “testosterone mediation” results]

 Saslow, L. R., Muise, A., Impett, E. A., & Dubin, M. (2013). Can you see how happy we are? facebook images and relationship satisfaction. Social Psychological and Personality Science, 4, 411-418. doi:http://dx.doi.org/10.1177/1948550612460059 [Study 2]

 Seder, J. P., & Oishi, S. (2012). Intensity of smiling in facebook photos predicts future life satisfaction. Social Psychological and Personality Science, 3, 407-413. doi:http://dx.doi.org/10.1177/1948550611424968 [focus on Study 1]

Tice, D. M., & Baumeister, R. F. (1997). Longitudinal study of procrastination, performance, stress, and health: The costs and benefits of dawdling. Psychological Science, 8, 454-458. Retrieved from http://search.proquest.com/docview/619221802?accountid=10457

 Valor-Segura, I., Expósito, F., Moya, M., & Kluwer, E. (2014). Don't leave me: The effect of dependency and emotions in relationship conflict. Journal of Applied Social Psychology, 44, 579-587. doi:http://dx.doi.org/10.1111/jasp.12250 [use Study 1]

 Vezzali, L., Stathi, S., Giovannini, D., Capozza, D., & Trifiletti, E. (2014). The greatest magic of harry potter: Reducing prejudice. Journal of Applied Social Psychology, 45, 105-121. doi:http://dx.doi.org/10.1111/jasp.12279  [Study 1]

Witkower, Z., & Tracy, J. L. (2019). A facial-action imposter: How head tilt influences perceptions of dominance from a neutral face. Psychological Science, 30(6), 893–906. https://doi.org/10.1177/0956797619838762

 Wong, E., Ormiston, M., Haselhuhn, M. (2011). A face only an investor could love: CEOs’ facial structures predict their firms’ financial performance. Psychological Science, 22, 1478-1483. doi: 10.1177/0956797611418838.

Cuter kids get higher grades

One of the proposed mediators of the effect was that teachers report better relationships with attractive students.Photo credit: FatCamera/Getty Images

File this one under: Depressing, but we suspected it all along. The Washington Post ran a summary of research that looked at the link between school achievement and physical appearance. 

A new study finds that good looking kids do better in school than their less striking peers.

The research, by Barnard College economist Daniel Hamermesh and colleagues, finds that people whose looks are “one standard deviation above average” attain nearly five more months of schooling than an “otherwise identical average-looking individual.”

The study in question found the effect in two large, national studies, one in the U.S. and one in the U.K. You might be wondering, how did they decide how good looking the kids were? What do you think of the construct validity of this operationalization: 

For each kid in the American study, looks were assessed by a panel of at least 10 undergraduates who watched segments of video interviews with the children collected over the course of the survey. The raters assessed each one’s looks on a scale from 1 (not at all cute/very unattractive) to 5 (very cute/very attractive).

For the U.K. study, attractiveness was assessed by the children’s teachers, who assigned each child to one of four categories: attractive, unattractive, “abnormal feature” or “underfed or scruffy and dirty.”

Here is a description of the results:

Ratings in hand, the researchers then analyzed the relationship between a child’s looks and academic achievement, as measured via various standardized tests administered throughout the two studies. The studies revealed the same general pattern: Better looking children performed better on tests of academic achievement, even when controlling for ethnicity, gender and parents’ education and income.

The language above should cue you to realize that this study evaluated the core relationship (between looks and achievement) but used multiple regression techniques to control for these potential internal validity threats. 

a) What is the criterion variable in this study? What are the predictor variables?

b) Sketch a little regression table with results that would be consistent with what is mentioned above. Put the criterion variable at the top, and list the predictor variables. Which betas would be significant here? 

(Note: You can see the actual results in Table 7 of the scientific article.)

The journalist continues the summary with this statement:

Economists who’ve studied academic performance have found that children who perform better on tests tend to get more schooling: A student who aces all her reading and math tests is much more likely to go on to college than one who does not. Hamermesh and his colleagues estimate that good-looking kids’ heightened test performance relative to their average-looking peers works out to about 0.4 years, or five months, of additional schooling.

c) The above is a statement about effect size. Do you think this effect is a large one? Is it important?  What more do you need to know to evaluate this effect? 

d) Next comes a statement about potential mediators of the relationship. Read the following and sketch out one of the mediation paths they are describing (use Figure 9.11 as a model)

To determine why, Hamermesh and his colleagues tested a number of theories. They found some evidence that teachers report better relationships with the more attractive students, which explains some of the achievement gap. They also found that youngsters rated as unattractive were somewhat more likely to report being bullied by their peers, with detrimental spillover on their academic performance.

Similarly, they found that there were fewer reports of behavioral problems at school among the highly rated children.

 

Are there other potential mediators you can think of?  

 

Guess what! This is my 200th post! Thanks for reading and making this blog possible. 

Dog people live longer

The study found an association between dog ownership and longevity. Which third variables were controlled for? Which third variables might still be responsible for the relationship? Photo: Shutterstock

I can't resist blogging about research on dogs. Here's a story on NPR that summarized several studies that have tested the correlation between longevity and dog ownership. The journalist introduces the topic this way: 

Dogs and humans that interact with one another get a jolt of oxytocin, the so-called "cuddle hormone." And, if you get to look at dogs and hug them every day, you just might live longer than people who don't have to clean animal hair off their clothes, according to a pair of studies out this month.

Here is a summary of one of the studies:

[researchers] looked over a 12-year period at dog owners in Sweden who have cardiovascular disease. They adjusted for the kinds of things we know affect cardiovascular health — age, demographics, socioeconomic status, marital status, number of children at home — and even after adjusting for all of that, they found a benefit of dog ownership [on longevity].

....they followed over 100,000 people and found a 21% reduction in deaths from any cause.

The study above used multiple regression. 

a) What was the dependent (criterion) variable in this study?  What were the predictor variables? 

b) Sketch a little regression table with results that would be consistent with what is mentioned above. Put the criterion variable at the top, and list the predictor variables. Which betas would be significant here? 

c) The researchers controlled for age, demographics, SES, marital status, and children--but regression doesn't control for all third variables. What else could be associated with both dog ownership and longevity? 

d) They mention a "21% reduction in deaths from any cause."  Which of the four big validities is this concerning? 

One of the experts who was interviewed for this story said:

In particular, I found it very convincing and striking that the benefits seems to be larger among individuals who live alone compared to multiperson households. That suggests the companionship of a dog is possibly very important to their heart health.

e) This is a moderator! Identify the variables in this relationship and complete this sentence: ____ moderates the relationship between ____ and _____.

 

In part of the article, the journalist presents quotes from an expert, who seems to equivocate on the causality issue:

"When you look at the big picture and look at all the evidence around dog ownership and cardiovascular health, it's pretty clear the signal is real and likely causal," he says.

In other words, he's convinced getting a dog improves your health — especially for those with heart disease.

The research is not definitive, though, Kazi adds. "They're not randomized trials, the gold standard for what we would do to evaluate a new drug," he says.

f) It is frustrating to read that the expert believes the "signal is real and likely causal" and then admits that "it's not definitive." You can set this expert straight. Why can't this correlational study support causation?  Apply covariance, temporal precedence, and internal validity. When you address internal validity, check out answer c) above!

 

 

Suggested answers

a) The criterion variable is longevity (being alive at the end of the study). The predictor variables are dog ownership, age, demographics, SES, marital status, and children living at home. 

b) The table might look like this:

DV: Longevity (or being alive after 12 years)

IV                            Beta (sig)
Dog ownership         .12*
Age                            -.11
SES                              .13*
Married                        .08*
Children at home        .07*

d) Current health and mobility might be associated with both dog ownership and longevity. After all, you  may need to have a certain level of health and mobility to own a dog, and healthier folks are more likely to be alive at the end of the study. 

e)  Living status (being alone or married) moderates the relationship between dog ownership and longevity.  

f) There is covariance--there is a link between owning a dog and living longer. 

There is temporal precedence-- the study took place over several years and dog ownership was measured first, and death rates later. 

There is not full internal validity. They controlled for age, demographics, SES, marital status, and children living at home, so they can rule those out. But they did not control for everything. One possibility is that current health and mobility might be associated with both--after all, you  may need to be at a certain level of health and mobility to own a dog, and those folks are more likely to be alive at the end of the study. 

 

 

 

Racial stereotypes behind how we explain school shootings

Participants and newspapers were more likely to blame video games when a school shooter was white than when he was black. Photo: Richard Levine / Alamy Stock Photo

What causes mass shootings? Of course, we have no experimental evidence on this question. Correlational evidence (both within the U.S. and evidence that compares the U.S. to other countries) is consistent with the argument that gun availability is a factor. 

However, this post isn't about what really causes school shootings. It's about what people think causes them. And it turns out that the race of a school shooter influences what people believe about the shooter. 

After some school shootings (a subset of mass shootings), people have sometimes looked for answers not in the availability of guns, but in video games. Now, there is actually no evidence linking video games to mass shootings. However, this ABC news story talks about when people are more likely to think that video games are responsible: 

There's little evidence showing a relationship between violent video games and school shootings, but video games are often blamed for overall violence in our communities. A recent study published in the Psychology of Popular Media Culture set out to find if we look for an external explanation for violence by whites more often than we do for African Americans.

"We try harder to make excuses for white perpetrators," said investigator Dr. James Ivory, a professor in the Department of Communications at Virginia Tech, in an interview with ABC News.

The first study gave people a mock news story about a school shooting. When asked what caused the shooting, people were more likely to blame video games if the shooter was white than if the shooter was black.

It turns out that in the original study (linked here;  if it's paywalled you may be able to access via your university library), participants read the mock news story and were randomly assigned to one of two photos of the perpetrator--a mugshot showing either a Black or a White man. 

a)  There were two variables in this study. Name each one. For each, was it manipulated or measured?

b) What kind of study is this, correlational or experimental?

c) Sketch a graph of the results of this study. What will you put on the y-axis? 

d) In the study above, what role does random assignment play? Why did the researchers randomly assign people to see either a Black or White perpetrator?  

 

Here is how ABC news described the second study:

The second study looked at thousands of news articles -- video games were mentioned more often in stories of white shooters [compared to black shooters]. 

e) What are the variables in the study above? Name each one; for each, was it manipulated or measured?

f) What kind of study is this, correlational or experimental?

g) Sketch a graph of the results of this study.

h) In the study above, the researchers could not randomly assign news article authors to write about a black vs. a white perpetrator. How does that change the impact of this study? 

i) Which of the two studies do you think provides the strongest evidence that race influences people's willingness to think video games are a cause of the shooting?

 

Suggested answers

a) One variable is race of the perpetrator (levels are Black or White), this was manipulated. The other variable is "likelihood to blame video games". This was measured.

b) This was an experiment because one variable was manipulated. 

d) Random assignment contributes to the study's internal validity. They can be sure that the same kinds of participants were in each group (reading about Black vs. White perpetrators). Therefore, people who play a lot of video games themselves were equally likely to be in the two groups. You might be interested to know that in the original study, the researchers also pretested the Black and White photos they used to be sure that the faces differed on race but not on perceived attractiveness or dangerousness. This pretesting also contributed to the experiment's internal validity.

e) The first variable was race of the perpetrator in the article (Black or White), which was measured. The second variable was whether or not the article mentioned video games (yes or no).

f) This was a correlational study because all variables were measured. 

h) Because this study used naturally-occurring crimes, the researchers may have an internal validity problem. It is possible that the Black and White perpetrated shootings differed not just on race but also on number of victims, age of victims, geography, or some other variable. This introduces an alternative explanation for the result--perhaps people blamed video games for White perpetrated shootings because they had more victims, had older victims, and so on.

i) You might have selected the first study as the strongest because it was an experiment and allows us to make a causal inference. Or you might have selected the second study because it was conducted on "real world" stimuli such as news stories--therefore it appears to have good ecological validity or external validity. 

Thanks to Marianne Lloyd for this idea!