On my first read of this headline, I was skeptical. "Probably a correlational study behind this one," I thought. As it turns out we are actually looking at an experiment. However, there's another problem with the journalist's coverage of this study. Read on.
The journalist writes for Inc.com, which summarizes research for the business world. The headline was "Want to raise kind, generous kids? Take them to an art museum" The original empirical article appeared in Psychological Science. Let's work through it. The introduction starts out:
Everyone wants to raise successful kids, but a big part of being successful is being a kind and generous person. So how can parents give their kids the best while also doing their best to ensure they raise decent, thoughtful little people?
They introduce the idea that art might make kids more generous:
You may be wondering what possible connection a day viewing Van Gogh or Picasso could have to raising an empathetic kid. Certainly, museums make children more worldly and cultured, but how do they make them kinder? The link, according to the new study, recently published in Psychological Science, is awe.
a) Before you read more, make a prediction. Given this introduction, what independent variable do you think the researchers probably manipulated? What variable do you think was measured?
Now let's read the details. There were two studies. In the first one,
..the research team asked 159 volunteers aged 8 to 13 to watch short movie clips. Some of these clips were neutral, others cheerful, and others awe-inspiring. .
In the second study, about 350 kids participated. The researchers used one of the same three clips for the awe variable (neutral, joy, and awe), but in this case, the generosity variable was operationalized differently:
The researchers then asked the kids [in Study 1] to determine how many items in a list of foods should be donated to a food drive for needy families. Alternately [in Study 2], the kids were given the option of donating their reward for participating in the study--a ticket to a local art museum--to a refugee family.
(By the way, the original article reports that "The awe-eliciting clip was from the film Song of the Sea, in which the main character turns into a seal and transforms the city while flying above it.")
Back to the Inc.com story for the results:
"Children who watched the awe-inspiring video chose to count 50 percent more items for the food drive than children who watched the joy-inspiring clip and more than twice as many items as children who watched the neutral clip. [in Study 2], children in the awe-inspiring condition were also 2 to 3 times more likely to donate their museum tickets than children in the joyful or neutral conditions," reports the Association for Psychological Science blog.
b) Take a moment to classify the two variables in Study 1:
Variable name | What are the variable's possible levels? |
Is this manipulated or measured? | Is this an IV or a DV? | For IVs: is it within groups or independent groups? |
c) Which kind of experiment is Study 1: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures?
d) And now for Study 2:
Variable name | What are the variable's possible levels? |
Is this manipulated or measured? | Is this an IV or a DV? | For IVs: is it within groups or independent groups? |
e) Which kind of experiment is Study 2: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures?
Look back at the description of the study. Did either Study 1 or Study 2 manipulate "taking kids to an art museum"? Nope. In fact, Study 1 took place online (kids watched video clips on their own computers). And Study 2 took place in a science museum. Neither of the studies manipulated art museum exposure. Art museum tickets were used in Study 2, but only as a way of operationalizing generosity.
That factual error didn't stop the journalist from asserting:
Parents ... can always go to their local art museum to give their kids a guaranteed dose of awe.
Guaranteed dose of awe? That conclusion does not really follow from this study! So here are a couple more questions.
f) What could be an alternative headline for the Inc.com story--one that is more accurate to what the study actually manipulated?
g) The journalist appears to have generalized from one setting (watching video clips) to another (going to an art museum). Which of the four big validities is the journalist treading on here?
h) Here's a construct validity question: The two studies operationalized the dependent variable, "generosity" in two different ways. Were each of them a good way to measure this variable? Why or why not? What are the strengths of operationalizing the variable more than one way?
i) Now for a question related to internal validity: The main independent variable was the content of the film clip the children saw. To ensure internal validity, what should the researchers hold constant about the film clips used in the three conditions?