Everyday Research Methods

Claim: "Going to museums and concerts helps you live longer"

Does the study's method support the claim that attending arts events makes you live longer? Photo credit: agefotostock/Alamy

We'll never run out of material in the "correlation is not causation" department. 

This time, a New York Times  journalist led with the headline, "Another benefit of going to museums? You may live longer." 

Let's check out the study behind this causal claim. 

According to the journalist's summary of the empirical article, 

Researchers in London who followed thousands of people 50 and older over a 14-year period discovered that those who went to a museum or attended a concert just once or twice a year were 14 percent less likely to die during that period than those who didn’t.

The chances of living longer only went up the more frequently people engaged with the arts, according to the study, which was published this month in The BMJ, formerly The British Medical Journal. People who went to a museum or the theater once a month or even every few months had a 31 percent reduced risk of dying in that period, according to the study.

a) Starting just from here--what are the two variables they've mentioned so far? Were they manipulated or measured? 

b) Based on what you've read so far, is this study correlational or experimental? 

c) Without reading further, think of a third variable (a "C") that is likely to be correlated with both A (going to museums or concerts) and B (being less likely to die). Any good ideas? 

OK! Time for more information about the study from the journalist:

The study controlled for socioeconomic factors like a participant’s income, education level and mobility, said Andrew Steptoe, a co-author of the study and the head of University College London’s research department of behavioral science and health.

“Even if you take those things into account, we still found that there is a difference in the survival of people who are involved in the arts,” Professor Steptoe said in an interview on Friday.

 

Sounds like multiple regression was used here.

d) What was the criterion (dependent) variable in this study now?

e) What were the predictor ("independent") variables? 

f) Sketch a little regression table similar to those in Chapter 9--put the criterion variable at the top and the predictors at the bottom. Which betas do you think were significant?

g) More importantly, was the "C" variable you thought about in c), above, controlled in their regression?

h) Even though the authors controlled for income, education, and mobility, there are likely to be additional third variables that were NOT controlled for. Think of at least one more alternative explanation. 

 

As you learn in Chapter 9, regression does not support causation because even though it can control for alternative explanations you can think of (such as income, education, or mobility), it doesn't control for everything. If you read the Time's summary, you'll see even more variables they seem to have controlled for , such as marital status and profession. But there could still be unmeasured variables that are responsible for the relationship. 

 

But that doesn't stop the authors from making causal speculations. In fact, here's a mediator hypothesis. i) Sketch this mediator following the diagrams at the end of Chapter 9:

 

The study also noted that engaging in the arts can reduce loneliness, promote empathy and emotional intelligence, and keep people from becoming sedentary — all factors that contribute to a longer life.

 

What kind of study would you need to conduct to test this proposed mediator?

 

 

 

 

 

 

Cuter kids get higher grades

One of the proposed mediators of the effect was that teachers report better relationships with attractive students.Photo credit: FatCamera/Getty Images

File this one under: Depressing, but we suspected it all along. The Washington Post ran a summary of research that looked at the link between school achievement and physical appearance. 

A new study finds that good looking kids do better in school than their less striking peers.

The research, by Barnard College economist Daniel Hamermesh and colleagues, finds that people whose looks are “one standard deviation above average” attain nearly five more months of schooling than an “otherwise identical average-looking individual.”

The study in question found the effect in two large, national studies, one in the U.S. and one in the U.K. You might be wondering, how did they decide how good looking the kids were? What do you think of the construct validity of this operationalization: 

For each kid in the American study, looks were assessed by a panel of at least 10 undergraduates who watched segments of video interviews with the children collected over the course of the survey. The raters assessed each one’s looks on a scale from 1 (not at all cute/very unattractive) to 5 (very cute/very attractive).

For the U.K. study, attractiveness was assessed by the children’s teachers, who assigned each child to one of four categories: attractive, unattractive, “abnormal feature” or “underfed or scruffy and dirty.”

Here is a description of the results:

Ratings in hand, the researchers then analyzed the relationship between a child’s looks and academic achievement, as measured via various standardized tests administered throughout the two studies. The studies revealed the same general pattern: Better looking children performed better on tests of academic achievement, even when controlling for ethnicity, gender and parents’ education and income.

The language above should cue you to realize that this study evaluated the core relationship (between looks and achievement) but used multiple regression techniques to control for these potential internal validity threats. 

a) What is the criterion variable in this study? What are the predictor variables?

b) Sketch a little regression table with results that would be consistent with what is mentioned above. Put the criterion variable at the top, and list the predictor variables. Which betas would be significant here? 

(Note: You can see the actual results in Table 7 of the scientific article.)

The journalist continues the summary with this statement:

Economists who’ve studied academic performance have found that children who perform better on tests tend to get more schooling: A student who aces all her reading and math tests is much more likely to go on to college than one who does not. Hamermesh and his colleagues estimate that good-looking kids’ heightened test performance relative to their average-looking peers works out to about 0.4 years, or five months, of additional schooling.

c) The above is a statement about effect size. Do you think this effect is a large one? Is it important?  What more do you need to know to evaluate this effect? 

d) Next comes a statement about potential mediators of the relationship. Read the following and sketch out one of the mediation paths they are describing (use Figure 9.11 as a model)

To determine why, Hamermesh and his colleagues tested a number of theories. They found some evidence that teachers report better relationships with the more attractive students, which explains some of the achievement gap. They also found that youngsters rated as unattractive were somewhat more likely to report being bullied by their peers, with detrimental spillover on their academic performance.

Similarly, they found that there were fewer reports of behavioral problems at school among the highly rated children.

 

Are there other potential mediators you can think of?  

 

Guess what! This is my 200th post! Thanks for reading and making this blog possible. 

Claim: Mothers' friendships are good for babies' brains

The study showed that moms with larger social support networks had kids with higher cognitive test scores. Photo: Noriko Cooper / Alamy Stock Photo

The  Washington Post headline reads, "Mothers' friendships may be good for babies' brains."  The story is about an empirical study on mothers' social networks. Although it says "babies' brains", the study measured the babies' cognitive skill (there were no baby brain scans). The empirical article came out in JAMA Network Open. Does the original study support the claim? Let's find out.Before reading on, answer these questions:

 

a) What are the two variables in the Washington Post headline claim? What makes the claim a causal one? 

b) Of course, we need to conduct an experiment to definitively support a causal claim. Would an experiment be possible in this particular case?  

Here are some details about the study, as summarized by the journalist (not the researcher):

For the current study, researchers examined data on 1,082 mother-child pairs. They questioned women about their family structure, friendships and relationships in their communities and also looked at test results from cognitive assessments done when children were 2 years old.

Overall, mothers had an average of 3.5 friends in their social support networks. The kids of mothers with more than that tended to have higher cognitive test scores than the kids of those who had fewer, suggesting “network conditions were significantly associated with early cognitive development in children,” the study authors wrote.

c) Sketch a  simple scatterplot of the result described above. What will you put on the two axes? How many dots should your scatterplot have? What does one dot represent? 

Read the following details if you've studied Chapter 9: 

The researchers accounted for some factors that might influence child cognitive development, including mother’s age and IQ, father’s education level, and the child’s birth weight. 

In addition, "researchers also accounted for poverty levels"

d) The paragraph above is describing a multiple regression analysis. What would be the dependent (criterion variable) in this analysis? What are the predictor variables? (Hint: There are six predictor variables).

e) Imagine that someone proposes income as a possible internal validity problem. Given what you know about the study, can we say that "Perhaps friendship network size and child cognitive skills are correlated because of the third variable of income: That is, higher-income mothers may have more friends and more cognitively skilled children"?  Would this criticism be reasonable? Why or why not?

f) The journalist writes: "...they did not have data to assess other personal differences, such as history of depression, between mothers with large networks of friends and those with fewer."
This means history of depression could be a possible third variable explaining the relationship. How could the researchers rule out this third variable? 

The journalist writes: 

All of the families in the study lived in the area around Memphis. It is possible that results might be different elsewhere.

g) Which big validity is the journalist addressing in the statement above? 

h) Do you think the correlation between friendship and cognitive test scores would actually come out differently in a different city or country? Why or why not? 

 

Suggested answers:

a) The two variables are Amount of mothers' friendships and Level of health of babies' brains. The language of "may be good for" makes this a causal headline (remember we ignore adverbs such as "may" when classifying claims). 

b) Probably not; you'd have to randomly assign some mothers to have more friends and others to have fewer friends.

c)  One axis should have "number of friends" and the other should have "cognitive test score". There should be about 1082 dots on the scatterplot (better count your dots now...), and each dot represents a mother-child dyad.

d) The criterion variable is cognitive test score. The predictor variables are number of friends, poverty levels, mother’s age, mother's IQ, father’s education level and the child’s birth weight. 

e) Since the researchers did control for poverty levels, we can actually rule out wealth as a possible third variable. 

f) the researchers could obtain history of depression data on each mother and then add this variable as a predictor variable to the regression analysis. If the beta associated with "number of friends" is still significantly predicting cognitive test score, then we can rule out this possible confound (/third variable/ internal validity problem).

g) external validity

h) In my opinion, it seems unlikely that such a relationship is limited only to Memphis residents. 

Note: You can read the original empirical article in JAMA, here. Note to savvy instructors: The p-curve on the results in this paper suggests that there may be little evidentiary value. 

Which comes first? Cross-lag study on depression and social media use

Which comes first, social media use or depression? Photo:Africa Studio/Shutterstock

We have seen multiple studies that have tried to estimate how strong the link is between social media use and depression among young adults. We've covered some of these studies on this blog, including the effect size of the correlation and even a brain imaging study that manipulated "likes."

One challenge with studying real-world links between social media use and depression is that most use correlational designs. Therefore, we can't be sure whether any link between social media use and depression means that social media leads to more depression, or if depression leads to more social media use. And of course, in correlational studies there may even be a third variable, such as age, which is associated with both variables and causing them to appear correlated. 

The latest study used a longitudinal, cross-lag panel design, such as those described in Chapter 9, to study this question. Here's a writeup of the study by a journalist at Earth.com.

First, here are some comments about the design: 

The researchers point out that previous studies that found links between depression and social media use among tweens are limited in their results because they only captured a snapshot of teen social media use.

“You have to follow the same people over time in order to draw the conclusion that social media use predicts greater depressive symptoms,” said Taylor Heffer, the lead author of the study. “By using two large longitudinal samples, we were able to empirically test that assumption.”

Here are some more details about the method. They actually used two longitudinal samples: 

For this study, the researchers conducted a two-year longitudinal study surveying 6th, 7th, and 8th graders in Canada. The team also surveyed a series of undergraduate students once a year for six years as they progressed through college.

What were the results? 

Higher depression symptoms predicted social media use among teen girls which indicates that unhealthy social media use may be a symptom depression, not a catalyst for it.

“This finding contrasts with the idea that people who use a lot of social media become more depressed over time. Instead, adolescent girls who are feeling down may turn to social media to try and make themselves feel better,” said Heffer.

Questions

a) In a cross-lag panel design, researchers measure the same two variables in the same sample of people over time. What are the two variables they focused on here?

b) Sketch a series of boxes depicting the design of the study, similar to the figures in Chapter 9's cross-lag panel section. How many time periods would you need to depict for the undergraduate sample? 

c) Focus only on the girls. What do you think the autocorrelations might have been in this design? Fill those in on your sketch.

d) Still focusing only on the girls, given the description of the results, estimate what the two cross-lag correlations might have been. Which correlation should be larger--the one from Time 1 depression to Time 2 social media, or the one from Time 1 social media to Time 2 depression?

 

Do kids do better in private schools?

What evidence would it take to convince us that it's the schools, not other factors, that are responsible for the outcomes of private school students? Photo: Image Source / Alamy Stock Photo

A large study has compared the outcomes of children who've attended private schools to those who've attended public schools.  A journalist summarized the report in the Washington Post. The study provides a nice example of how multivariate regression can be used to test third variable hypotheses. 

When we look simply at the educational acheivement of students in private schools vs. public schools, private school students have higher achievement scores. However, all such studies are correlational, because the two variables--Type of School and Level of Achievement--are measured. 

Therefore, such studies how covariance, because the results depict a relationship. The study may even show temporal precedence, because attending school presumably precedes the measure of achievement. However, such studies are weak on internal validity. We can think of several alternative explanations for why children in private schools are scoring higher. 

One major alternative explanation is socioeconomic status. Children from wealthier families are more likely to afford private schools. And in general, children from wealthier families tend to score higher on achievement tests. 

The Washington Post journalist quoted one of study's authors, Robert Pianta, who summed up the study's results this way:

“You only need to control for family income and there’s no advantage,” Pianta said in an interview. “So when you first look, without controlling for anything, the kids who go to private schools are far and away outperforming the public school kids. And as soon as you control for family income and parents’ education level, that difference is eliminated completely.”

Questions

a) Draw little diagrams similar to those in Figure 8.15 (in the 3rd ed.) to depict the arguments being made in this study. What would A be? What about B? In the quote from Pianta, above, what would the C variable(s) be? 

b) The researchers used type of school (private vs. public), which is a categorical variable. But in some analyses, the researchers also used "number of years in private school" as an alternative version of this variable. Is "number of years in private school" categorical, ordinal, interval, or ratio data? 

c) Sketch a mock-up regression table with the criterion variable  at the top and predictors below (Use Table 9.1 as a model). Which variable do you think the researchers selected as the criterion (dependent) variable in their analyses? Which variable(s) would have been the predictors? 

d) Now that you know what the results were, think about how the beta associated with "number of years in private school"  would change when parental SES is added and removed from the regression analyses. 

Original journal article seems to be open-access. Give it a try!

Suggested answers

a) A and B would be Type of School and Level of Acheivement. It doesn't really  matter which one is A and which one is called B. C would be Family Income and/or Parental Education.

b) Ratio data (zero is meaningful in this scale because you could attend zero years of private school)

c) The criterion variable would be Achievement, and the predictors would be Number of Years of Private School, Family Income, and Parental Education.

d)When the Number of Years of Private School is on the table (in the analysis) by itself, its beta is likely to be positve and significant (more years of private school goes with higher achievement). When Family Income and Parental Education are added to the table, the beta for Number of Years of Private School should drop to zero.  This pattern of results is consistent with the argument that Family Income and Parental Education are the alternative explanation for the original relationship .

Delay of gratification (marshmallow study replication)

Her school achievement later in life can be predicted from her ability to wait for a treat (or by her family's SES). Photo: Manley099/Getty Images

There's a new replication study about the famous "marshmallow study",  and it's all over the popular press. You've probably heard of the original research: Kids are asked to sit alone in a room with a single marshmallow (or some other treat they like, such as pretzels). If the child can wait for up to 15 minutes until the experimenter comes back, they receive two marshmallows. But if they eat the first one early, they don't. As part of the original study, kids were tracked over several years. One of the key findings was that the longer children were able to wait at age 4, the better they were doing in school as teenagers. Psychologists have often used this study as an illustration of how self-control is related to important life outcomes. 

The press coverage of this year's replication study illustrates at least two things. First, it's a nice example of multiple regression.  Second, it's an example of how different media outlets assign catchy--but sometimes erroneous--headlines on the same study.

First, let's talk about the multiple regression piece. Regression analyses often try to understand a core bivariate relationship more fully. In this case, the core relationship they start with is between the two variables, "length of time kids waited at age 4" and "test performance at age 15." Here's how it was described by Payne and Sheeran in the online magazine Behavioral Scientist:

The result? Kids who resisted temptation longer on the marshmallow test had higher achievement later in life. The correlation was in the same direction as in Mischel’s early study. It was statistically significant, like the original study. The correlation was somewhat smaller, and this smaller association is probably the more accurate estimate, because the sample size in the new study was larger than the original. Still, this finding says that observing a child for seven minutes with candy can tell you something remarkable about how well the child is likely to do in high school. 

a) Sketch a well-labelled scatterplot of the relationship described above. What direction will the dots slope? Will they be fairly tight to a straight line, or spread out?

b) The writers (Payne and Sheeran) suggest that a larger sample size leads to a more accurate estimate of a correlation. Can you explain why a large sample size might give a more accurate statistical estimate? (Hint: Chapter 8 talks about outliers and sample size--see Figures 8.10 and 8.11.)

Now here's more about the study: 

The researchers next added a series of “control variables” using regression analysis. This statistical technique removes whatever factors the control variables and the marshmallow test have in common. These controls included measures of the child’s socioeconomic status, intelligence, personality, and behavior problems. As more and more factors were controlled for, the association between marshmallow waiting and academic achievement as a teenager became nonsignificant. 

c) What's proposed above is that social class is a third variable ("C") that might be associated with both waiting time ("A") and school achievement ("B"). Using Figure 8.15. draw this proposal. Think about it, too: Why does it make sense that lower SES might go both with lower waiting time (A)? Why might lower SES go with lower school achievement (B)? 

d) Now create a mockup regression table that might fit the pattern of results being described above. Put the DV at the top (what is the DV?), then list the predictor variables underneath, starting with Waiting time at Age 4, and including things like Child's Socioeconomic Status and Intelligence. Which betas should be significant? Which should not?

Basically, here we have a core bivariate relationship (between wait time and later achievement), and then a critic suggests a possible third variable (SES). They used regression to see if the core relationship was still there when the third variable was controlled for. The core relationship went away, suggesting that SES was a third variable that can help explain why kids who wait longer do better in school later on. 

Next let's talk about some of the hype around this replication study. The Behavioral Scientist piece (quoted above) is one of the more balanced descriptions. Its headline was, Try to Resist Misinterpreting the Marshmallow Test. It emphasized that the core relationship was replicated. It also explains in some detail why SES is related to self-control, and how the two probably cannot be meaningfully separated--it's a nuanced report. But other press coverage had a doomsday feel:

Vox's subtitle read:
The famous psychology test gets roasted in the new era of replication.

The Guardian blared:
Famed impulse control 'marshmallow test' fails in new research

And the Quartz website said:
We learned the wrong lesson about self-control from the famous marshmallow test

One person on Twitter even wrote,
"The marshmallow/delayed gratification study always felt "wrong" to me - this year it was reported to be hopelessly flawed"

Are these headlines and comments fair? Probably not. As Payne and Sheeran write in Behavioral Scientist, 

The problem is that scholars have known for decades that affluence and poverty shape the ability to delay gratification. Writing in 1974, Mischel observed that waiting for the larger reward was not only a trait of the individual but also depended on people’s expectancies and experience. If researchers were unreliable in their promise to return with two marshmallows, anyone would soon learn to seize the moment and eat the treat. He illustrated this with an example of lower-class black residents in Trinidad who fared poorly on the test when it was administered by white people, who had a history of breaking their promises. Following this logic, multiple studies over the years have confirmed that people living in poverty or who experience chaotic futures tend to prefer the sure thing now over waiting for a larger reward that might never come. But if this has been known for years, where is the replication crisis?

e) Give at least 2 reasons why online sources might use incorrect headlines when they cover psychological science. 

f) Check out at least two different press stories about the marshmallow replication effect, and see how similar they are to each other and to the summary you analyzed above, by Payne and Sheeran. 

You can read the published paper for yourself! The full replication paper is in Psychological Science and is open access. 

Engagement with Disney Princesses

The study's results showed that the more children were engaged with Disney Princess media, the more they adhered to stereotypically female play later on. Photo: Shutterstock

There's a lot to think about in a recent study on Disney Princess Media. The study was covered by the Huffington Post here, and by another source in Oregon.  (You'll find other press coverage if you search for it).  The study illustrates some of the concepts in Chapter 8 and 9.

Let's point out the causal claims made by the journalists first. The Huffington Post journalist's headline reads:

Disney Princesses May Impact Gender Stereotypes For Girls (But Not Boys), Study Finds

And the Oregon paper's headline reads:

Disney 'princess culture' damaging to young girls, study says

Did you notice the causal verbs, "impact" and "damaging"? That's probably inappropriate, because this was a correlational study. In the first paragraph, the HuffPo journalist continues the dramatic causal language: 

Brigham Young University family life professor Sarah M. Coyne conducted a study with 198 preschoolers, both male and female, to rank their interaction with Disney Princess culture — including toys and movies — and how that affected their behavior one year later with reports from parents and teachers. 

a) Look back at that quote and find the verb that makes the causal claim. 

 

Now let's talk about the moderator in the headline, Disney Princesses May Impact Gender Stereotypes For Girls (But Not Boys).

b) This headline describes a moderator. Review what you learned about moderators in Chapter 8. What is the core relationship in this example (what are the two core variables)? And what is the moderator variable? How would you sketch this headline in a table or figure? 

 

Now, here are some more points made about the study, quoted from the Oregon news source:

Linder and her co-authors found that 96 percent of girls and 87 percent of boys had taken in some form of princess culture. Roughly 61 percent of girls reported playing with a princess doll in the last week, but only 4 percent of boys said they had played with the dolls.

When the subjects were examined a year after the initial analysis, both girls and boys who played with the dolls showed an increased likelihood to adhere to female gender stereotypes, a fact that could prove especially problematic for girls, said Sara Coyne, a co-author of the study and professor of family life at Brigham Young University.

Here's how the HuffPo journalist described this part of the results:

For both boys and girls, greater interactions with princesses predicted more gender-normative behavior — like wanting to play with traditional female stereotypical toys and activities (e.g., dolls, tea sets, playing house), and behaviors (e.g., avoiding getting dirty, avoiding taking risks)

c) What are the variables in this study, as described above? Are they measured or manipulated? And given your answer, does that make the study correlational or experimental? Did you notice that the design is longitudinal? (What signaled you to that?) Which design in Chapter 9 does this design seem closest to? 

d) Now, do these descriptions of the results actually show a relationship between Disney Princess media and gender-normative behavior that is different for boys and girls, as the one headline suggested? 

 

OK. If the relationship between Disney culture and stereotypical behavior was actually the same for boys and girls, where did the moderator in the headline come from? Turns out that when the researcher was interviewed about the results, she indicated that the gender stereotypes could, in the future, lead to different outcomes for boys and girls. Therefore, the moderator in the headline came more from the researcher's discussion than from the study itself. The researcher said, for example:

"We know that girls who strongly adhere to female gender stereotypes feel like they can't do some things," she told the university's news site. "They're not as confident that they can do well in math and science. They don't like getting dirty, so they're less likely to try and experiment with things."

And for boys:

The effects of princess culture were less troubling for boys, the authors said, as more stereotypically feminine behavior could serve as a counterbalance to the hyper-masculine superhero culture many young boys take in on a daily basis.

 

If your college or university subscribes to the journal, Child Development,you can see the full report of the study and learn more about the full pattern of results!

 

 

Social lives of the rich

The wealthier people were, the more likely they were to spend time alone. Photo: Shutterstock

In some earlier entries on this blog, I've posted stories about how money is related to (or can even change) people's social behavior, such as here and here. This week, the Washington Post ran a story about the association between money and socialization patterns. Let'sl= review the study, and you can have the chance to decide if the journalist made the right kinds of claims about the research.

The journalist first described how researchers Emily Bianchi of Emory University and Kathleen Vohs of the University of Minnesota tested the relationship between making money and socializing. People who make more money, they found, spend less time socializing with others:

 Bianchi and Vohs...looked at nearly 30,000 responses to the General Social Survey (GSS) to trace how household income affects respondents' social interactions. The GSS has repeatedly asked Americans how often they spend a "social evening" with relatives, neighbors and friends. Those questions draw a fairly robust portrait of Americans' social interactions. Bianchi and Vohs controlled for age, race, gender, marital status, household size, city size and hours worked to isolate the effect of money on relationships.

a) What are the two key variables in the basic association?

b) The language used above, "controlled for" should tell you that they also analyzed the data using multiple regression. First, why was it important to control for variables such as "hours worked"?

c) Sketch out what the regression table might have looked like in their study. What would the dependent, or criterion variable, have been? What would the predictor variables have been? What will the beta be for "income"? Will it be positive or negative? Which other betas do you think will be significant, and what will their direction be? (You can make up some betas for your table to match your predictions.)

Next, here's an indicator of the effect size of the relationship:

Compared to a low-income individual (earning $5,000 a year), a person from a higher-income household ($131,000 a year) spends, on average, 6.4 fewer evenings each year socializing with other people — even after controlling for differences in hours worked.

d) What do you think of the practically-stated effect size of "6.4 fewer evenings per year"? Does that seem small or large, in a practical sense?  

Now, here are some of the sentences included in the story that were used to describe the results. For each one, decide if it is a causal statement or an association statement (remember to attend to the verbs!):

e)  "How money changes our social networks"

f) "Inflation-adjusted household income has risen considerably since the World War II era (even if it's been fairly stagnant in the past decade). And if the relationship between income and social behavior they describe is true, then it's only natural that decreasing social interactions will be one consequence of that change."

g) "as people make more money, they spend less time socializing with others"

As you have learned, a correlational study like this one, even though it controlled for several possible third variables, cannot support a causal claim. For one, there may not be temporal precedence because in surveys like the General Social Survey, people usually answer all the questions at the same time. In addition, there are always uncontrolled third variables that the study did not measure or include in the regression. What possible third variables can you think of that might be associated with both having money and socializing less?

 

Happiness and longevity: What about mediators?

One hypothesis is that healthy habits mediate the relationship between happiness and longevity. Could you draw that mediator argument? Photo: Canon Boy/Shutterstock

When I teach multivariate correlational designs, students always end up asking, "What's the difference between multiple regression and mediation?" 

Here's the better question. What they should be asking is,  "What is the difference between using multiple regression analyses to rule out third variables, and using multiple regression analyses to test for mediation?"   This LA times editorial  provides an excellent example to illustrate the difference.

First things first:  Lots of people believe that happy people live longer, and happiness researchers have previously found that happiness does predict longevity.  However, a large-scale study published in the Lancet recently concluded that, in fact, happy people do not live longer. Their study was conducted on nearly a million women in the UK, so it's nicknamed "the million women study." The press covered the study by saying, "Good news for the grumpy."  You can see one story about the big Lancet study here.

However, this LA Times editorial, written by three happiness psychologists, explained a problem with the Lancet study. The Lancet study used regression to look at the relationship between happiness at age 59 and their life span, while holding constant a variety of third variables, such as income, being in a romantic relationship, being religious, smoking, and exercising. From one perspective, this makes sense--they wanted to look for some relatively "pure" relationship between happiness and longevity controlling for these factors, and they did not find one. That is, happiness did not predict longevity when these variables were controlled for.

a) Students: can you sketch a regression table of the pattern described above? Label the criterion variable, the predictor variables, and think about what betas might have been associated with each predictor variable.

Now for the problem: The three happiness psychologists writing the editorial argued that by controlling for such variables as relationship status, smoking, and exercise, the Lancet study obscured the important features of the relationship--the mediators. Here's how they put it.

We ... have concerns with the authors' choice to statistically control so many other variables. In our work, we hypothesize that the key ways positive feelings influence health are through changes in behaviors. For example, research has shown that positive individuals sleep better, smoke less and exercise more, all of which are known to predict longevity. Thus, when the authors control for all these variables, they are taking away a major path through which happiness is likely to influence longevity. This is like trying to determine whether a hurricane causes damage after controlling for its wind speed. Again, the question is changed, from “Will happy people live longer” to “Will happy people live longer if we take away their healthier behavior?”

b) The authors are stating a mediator argument in the above paragraph. Draw the mediator pattern they are describing, using Figures 9.11 or 9.13 as models. (Hint: Draw only one proposed mediator at a time.)

When the authors of the editorial write, "we hypothesize that the key ways positive feelings influence health are through changes in behaviors" they are talking about a mediator. The "through" part is a way of stating the mediator.

The authors of the editorial, therefore, end up describing how multiple regression can be used for at least two purposes. Regression is simply a statistical tool, and it can be used for both ruling out third variable problems and for testing mediation. The Lancet article used regression to control for what it saw as "pesky" third variables such as smoking and relationship status to see if happiness, on its own, predicted longevity. In contrast, the three happiness researchers have used regression to see if mediators, such as smoking and relationship status, explain why happiness predicts longevity.

You might recall in Chapter 9's discussion of mediation, that mediation tests are conducted in several steps. Step 4, in fact, uses multiple regression (the text states, for example, "if physical activity is the mediator of the relationship [between recess and behavior problems,] then the relationship between recess and behavior problems should drop when physical activity is controlled for").

So there you go. A great example of how a single data set can be viewed in two very different, but ultimately two very legitimate ways, depending on one's research goals and hypotheses.

By the way, the three happiness psychologists also criticized the construct validity of the Lancet study's happiness measure, saying this:

the happiness measure used in the Lancet study is weak. In well-crafted research, social scientists think carefully about question and answer design, the ordering of items and how many questions are too many before the participant gives up and stops answering or stops answering honestly...The Million Women Study asked just one happiness question: How often do you feel happy (from “rarely/never” to “most of the time”). A better approach would have been to ask the question in more ways, with a broader range of positive emotions for participants to choose from.

"Turn off the TV, save your brain?"

Photo: Shutterstock

Pacific Standard does a great job covering social science in the news. Here's a nice example of multiple regression analyses,  in a study they headlined, "Turn off the TV, Save your brain?"

The basic story comes from a large study that found  the amount of TV people watched in their young-adult years was linked to lower cognitive abilities in middle age.

a) So far, this is a simple bivariate correlation. Can you sketch a well-labelled scatterplot of this? Is the relationship positive or negative? 

b) This bivariate correlation apparently has temporal precedence, because the TV variable was measured in young adulthood, before the cognitive abilities (in middle age). Can we make a causal claim now? If not, what third variables might be responsible for this relationship?

Here are some methodological details about the study. You'll see that the story shows the telltale signs of controlling for some of these possible third variables:

The study featured 3,247 people who were recruited from four American cities (Chicago, Minneapolis, Oakland, and Birmingham) in the late 1980s. All were between the ages of 18 and 30 at the study's outset. They were tracked for the next quarter century, with follow-up examinations every two to five years....once every five years, [the participants] indicated "the average number of hours per day (they) spent watching television in the past 12 months."Those who reported watching an average of more than three hours per day during more than two-thirds of their visits were categorized as having a "long-term pattern of high television viewing."

At year 25, the participants took three well-known cognitive tests: the Digit Symbol Substitution Test, which measures speed of mental processing; the Stroop Test, which measures processing speed and attention; and the Rey Auditory Verbal Learning Test, which assesses how well we remember information delivered orally.

Participants who showed a long-term pattern of high TV viewing (approximately 11 percent of the total) scored significantly lower, on average, on [the] cognitive tests...This held true after taking into consideration a range of factors, including age, race, education, alcohol use, and body mass index.

 

c) Based on the description above, you could sketch a multiple regression table depicting this study's results. What would be the criterion variable? What would be the predictor variables? What betas do you think would be significant? Which  betas will be positive? Which will be negative?

d) The description above also tells you how some of the variables were operationalized. For example, how did they operationalize television viewing? How did they operationalize cognitive ability?

e) Even though this study sure feels like it could support a causal claim, it can't. (Remember, no correlational study can definitively support a causal claim.)  That's because even though it has temporal precedence (TV was measured before cognitive ability), and controls for several third variables (including age, race, education, alcohol use, and BMI), they didn't control for all possible third variables. What are some additional third variables that were not controlled for in this study?